2024-05-14T16:27:03.101 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-14T16:27:03.101 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_leonidus@teuthology --name leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} --verbose --archive /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174 -- /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174/orig.config.yaml 2024-05-14T16:27:03.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T16:27:03.259 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174 branch: wip-lusov-quiesce description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} email: null first_in_suite: false job_id: '7706174' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 710 sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiesce suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJ90eUpCJ6wuZXp6MAOhv+w6thOMP7d5tfpHr2kJOllAejmcDjei6RR6cAWZtok72fh5dzotJjz18ss+bKNHaI= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHiCDolLsnnAyQTGc2/+VkJf/cpW8ZyjHTchzfW7kEwYOZuaDq+ygUkRBxfqYoahbclH03H9cNOLguNfjx4NiMk= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZnvpHt//FMVuNIYvC3cHDn1f24SOomogUattDg1gTz01/KZ4UMHrZnCao3eXY99c3oxT7VKenGDLhB2+TU4dI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-14_16:11:40 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-14T16:27:03.259 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa; will attempt to use it 2024-05-14T16:27:03.260 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks 2024-05-14T16:27:03.260 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-14T16:27:03.262 INFO:teuthology.task.internal:Checking packages... 2024-05-14T16:27:03.285 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b' 2024-05-14T16:27:03.285 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-14T16:27:03.286 INFO:teuthology.packaging:ref: None 2024-05-14T16:27:03.286 INFO:teuthology.packaging:tag: None 2024-05-14T16:27:03.286 INFO:teuthology.packaging:branch: wip-lusov-quiesce 2024-05-14T16:27:03.286 INFO:teuthology.packaging:sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:27:03.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce 2024-05-14T16:27:03.443 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3687.g9b638668 2024-05-14T16:27:03.444 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-14T16:27:03.451 INFO:teuthology.task.internal:no buildpackages task found 2024-05-14T16:27:03.451 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-14T16:27:03.456 INFO:teuthology.task.internal:Saving configuration 2024-05-14T16:27:03.477 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-14T16:27:03.483 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T16:27:03.509 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:22:02.942460', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJ90eUpCJ6wuZXp6MAOhv+w6thOMP7d5tfpHr2kJOllAejmcDjei6RR6cAWZtok72fh5dzotJjz18ss+bKNHaI='} 2024-05-14T16:27:03.532 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:22:02.944237', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHiCDolLsnnAyQTGc2/+VkJf/cpW8ZyjHTchzfW7kEwYOZuaDq+ygUkRBxfqYoahbclH03H9cNOLguNfjx4NiMk='} 2024-05-14T16:27:03.554 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:22:02.945299', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZnvpHt//FMVuNIYvC3cHDn1f24SOomogUattDg1gTz01/KZ4UMHrZnCao3eXY99c3oxT7VKenGDLhB2+TU4dI='} 2024-05-14T16:27:03.555 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-14T16:27:03.561 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-14T16:27:03.561 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-14T16:27:03.561 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-14T16:27:03.561 INFO:teuthology.run_tasks:Running task console_log... 2024-05-14T16:27:03.652 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f38667c30d0>, signals=[15]) 2024-05-14T16:27:03.653 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-14T16:27:03.660 INFO:teuthology.task.internal:Opening connections... 2024-05-14T16:27:03.660 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T16:27:03.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:27:03.739 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-05-14T16:27:03.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:27:03.814 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-05-14T16:27:03.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:27:03.890 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-14T16:27:03.896 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-05-14T16:27:03.914 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-05-14T16:27:03.914 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-05-14T16:27:03.971 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-05-14T16:27:03.971 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-05-14T16:27:03.971 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-05-14T16:27:03.971 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-05-14T16:27:03.971 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-05-14T16:27:03.971 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-05-14T16:27:03.971 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T16:27:03.971 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-05-14T16:27:03.972 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-05-14T16:27:03.972 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T16:27:03.972 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-05-14T16:27:03.972 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T16:27:03.972 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T16:27:03.972 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T16:27:03.972 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-05-14T16:27:03.992 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-05-14T16:27:04.008 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-05-14T16:27:04.008 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-05-14T16:27:04.065 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-05-14T16:27:04.065 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-05-14T16:27:04.065 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-05-14T16:27:04.065 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-05-14T16:27:04.065 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-05-14T16:27:04.065 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-05-14T16:27:04.065 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T16:27:04.065 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-05-14T16:27:04.065 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-05-14T16:27:04.066 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T16:27:04.066 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-05-14T16:27:04.066 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T16:27:04.066 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T16:27:04.066 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T16:27:04.066 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-05-14T16:27:04.083 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-05-14T16:27:04.100 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-05-14T16:27:04.100 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-05-14T16:27:04.157 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-05-14T16:27:04.157 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-05-14T16:27:04.157 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-05-14T16:27:04.157 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-05-14T16:27:04.157 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-05-14T16:27:04.157 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-05-14T16:27:04.158 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T16:27:04.158 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-05-14T16:27:04.158 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-05-14T16:27:04.158 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T16:27:04.158 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-05-14T16:27:04.158 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T16:27:04.158 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T16:27:04.158 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T16:27:04.158 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-05-14T16:27:04.183 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-14T16:27:04.193 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-14T16:27:04.199 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-14T16:27:04.199 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2024-05-14T16:27:04.202 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-05-14T16:27:04.204 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-05-14T16:27:04.220 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-14T16:27:04.226 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-14T16:27:04.226 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2024-05-14T16:27:04.261 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-05-14T16:27:04.263 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-05-14T16:27:04.362 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-14T16:27:04.441 INFO:teuthology.run_tasks:Running task kernel... 2024-05-14T16:27:04.456 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-05-14T16:27:04.457 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-14T16:27:04.457 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-14T16:27:04.457 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T16:27:04.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:27:04.474 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-05-14T16:27:04.531 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-437.el9.x86_64 2024-05-14T16:27:04.531 INFO:teuthology.task.kernel:Running kernel on smithi069: 5.14.0-437.el9.x86_64 2024-05-14T16:27:04.531 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum install -y kernel 2024-05-14T16:27:08.544 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.1 MB 00:02 2024-05-14T16:27:11.153 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-05-14T16:27:19.528 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.2 MB 00:04 2024-05-14T16:27:21.395 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-05-14T16:27:22.582 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-14T16:27:27.705 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-05-14T16:27:29.694 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-14T16:27:29.694 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-14T16:27:29.783 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-05-14T16:27:29.793 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-05-14T16:27:29.794 INFO:teuthology.orchestra.run.smithi069.stdout:Install 4 Packages 2024-05-14T16:27:29.794 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:27:29.794 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 78 M 2024-05-14T16:27:29.794 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 123 M 2024-05-14T16:27:29.794 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-05-14T16:27:30.190 INFO:teuthology.orchestra.run.smithi069.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 2.0 MB/s | 550 kB 00:00 2024-05-14T16:27:30.608 INFO:teuthology.orchestra.run.smithi069.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-05-14T16:27:31.165 INFO:teuthology.orchestra.run.smithi069.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 27 MB/s | 34 MB 00:01 2024-05-14T16:27:31.590 INFO:teuthology.orchestra.run.smithi069.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 20 MB/s | 28 MB 00:01 2024-05-14T16:27:31.590 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:27:31.591 INFO:teuthology.orchestra.run.smithi069.stdout:Total 43 MB/s | 78 MB 00:01 2024-05-14T16:27:32.274 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-05-14T16:27:32.606 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-05-14T16:27:32.606 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-05-14T16:27:33.526 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-05-14T16:27:33.527 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-05-14T16:27:35.916 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-05-14T16:27:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-14T16:27:36.640 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T16:27:39.313 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T16:27:39.450 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T16:27:44.136 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T16:27:44.186 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:27:48.926 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:28:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:28:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:28:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:28:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-14T16:28:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T16:28:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T16:28:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:28:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:28:19.553 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-05-14T16:28:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-14T16:28:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-14T16:28:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-14T16:28:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-14T16:28:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:28:19.554 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-05-14T16:28:19.894 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q kernel | sort -rV | head -n 1 2024-05-14T16:28:19.967 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-14T16:28:19.967 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-14T16:28:19.968 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-14T16:28:19.968 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-14T16:28:19.968 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-14T16:28:19.968 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-14T16:28:19.968 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=None, version=distro) 2024-05-14T16:28:19.968 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q kernel | sort -rV | head -n 1 2024-05-14T16:28:20.017 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-14T16:28:20.017 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-14T16:28:20.018 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -qi grub2-tools 2024-05-14T16:28:20.048 INFO:teuthology.orchestra.run.smithi069.stdout:Name : grub2-tools 2024-05-14T16:28:20.048 INFO:teuthology.orchestra.run.smithi069.stdout:Epoch : 1 2024-05-14T16:28:20.048 INFO:teuthology.orchestra.run.smithi069.stdout:Version : 2.06 2024-05-14T16:28:20.048 INFO:teuthology.orchestra.run.smithi069.stdout:Release : 68.el9 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: x86_64 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Group : Unspecified 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Size : 8264254 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:License : GPLv3+ 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Packager : builder@centos.org 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Vendor : CentOS 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Summary : Support tools for GRUB. 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:Description : 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:28:20.049 INFO:teuthology.orchestra.run.smithi069.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-14T16:28:20.050 INFO:teuthology.orchestra.run.smithi069.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-14T16:28:20.050 INFO:teuthology.orchestra.run.smithi069.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-14T16:28:20.050 INFO:teuthology.orchestra.run.smithi069.stdout:hardware devices. 2024-05-14T16:28:20.050 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:28:20.050 INFO:teuthology.orchestra.run.smithi069.stdout:This subpackage provides tools for support of all platforms. 2024-05-14T16:28:20.051 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-14T16:28:20.051 INFO:teuthology.task.kernel:Updating grub on smithi069 to boot 5.14.0-446.el9.x86_64 2024-05-14T16:28:20.051 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-14T16:28:20.664 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2024-05-14T16:28:22.036 INFO:teuthology.orchestra.run.smithi069.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-14T16:28:22.068 INFO:teuthology.orchestra.run.smithi069.stderr:done 2024-05-14T16:28:22.071 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2024-05-14T16:28:22.086 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.RikWzHlOA0 2024-05-14T16:28:22.086 DEBUG:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RikWzHlOA0 2024-05-14T16:28:22.151 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.RikWzHlOA0 2024-05-14T16:28:22.281 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.RikWzHlOA0 is 6KB 2024-05-14T16:28:22.300 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.RikWzHlOA0 2024-05-14T16:28:22.314 DEBUG:teuthology.orchestra.run.smithi069:> sudo /bin/ls /boot/loader/entries || true 2024-05-14T16:28:22.381 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-14T16:28:22.381 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-14T16:28:22.381 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-14T16:28:22.381 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-14T16:28:22.383 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-14T16:28:22.547 DEBUG:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2024-05-14T16:28:52.550 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-14T16:28:52.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-14T16:28:52.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:29:52.553 DEBUG:teuthology.orchestra.remote:timed out 2024-05-14T16:30:01.554 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-14T16:30:01.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:30:02.925 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-05-14T16:30:03.412 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-14T16:30:03.412 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-446.el9.x86_64"... 2024-05-14T16:30:03.412 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-05-14T16:30:03.427 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-446.el9.x86_64 2024-05-14T16:30:03.427 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-14T16:30:03.427 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-14T16:30:03.427 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-14T16:30:04.428 DEBUG:teuthology.parallel:result is None 2024-05-14T16:30:04.428 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-14T16:30:04.435 INFO:teuthology.task.internal:Creating test directory... 2024-05-14T16:30:04.435 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T16:30:04.438 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T16:30:04.441 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T16:30:04.458 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-14T16:30:04.465 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-14T16:30:04.475 INFO:teuthology.task.internal:Creating archive directory... 2024-05-14T16:30:04.475 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T16:30:04.494 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T16:30:04.498 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T16:30:04.597 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-14T16:30:04.603 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-14T16:30:04.603 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T16:30:04.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:30:04.620 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T16:30:04.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:30:04.636 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T16:30:04.638 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T16:30:04.665 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T16:30:04.695 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:30:04.715 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:30:04.719 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:30:04.729 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:30:04.762 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:30:04.785 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:30:04.787 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-14T16:30:04.799 INFO:teuthology.task.internal:Configuring sudo... 2024-05-14T16:30:04.799 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T16:30:04.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T16:30:04.803 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T16:30:04.831 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-14T16:30:04.841 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-14T16:30:04.841 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T16:30:04.869 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T16:30:04.872 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T16:30:04.888 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:30:04.963 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:30:05.018 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:30:05.099 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:30:05.140 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:30:05.140 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T16:30:05.230 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:30:05.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:30:05.332 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:30:05.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:30:05.456 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:30:05.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T16:30:05.521 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:30:05.559 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:30:05.611 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:30:05.699 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:30:05.737 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:30:05.737 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T16:30:05.802 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2024-05-14T16:30:05.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-05-14T16:30:05.807 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-05-14T16:30:05.838 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T16:30:05.839 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T16:30:05.875 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T16:30:06.286 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-14T16:30:06.294 INFO:teuthology.task.internal:Starting timer... 2024-05-14T16:30:06.294 INFO:teuthology.run_tasks:Running task pcp... 2024-05-14T16:30:06.304 INFO:teuthology.run_tasks:Running task selinux... 2024-05-14T16:30:06.313 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-14T16:30:06.313 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2024-05-14T16:30:06.388 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2024-05-14T16:30:06.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-05-14T16:30:06.475 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-05-14T16:30:06.477 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-05-14T16:30:06.537 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-05-14T16:30:06.539 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-14T16:30:06.539 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2024-05-14T16:30:06.563 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2024-05-14T16:30:06.563 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-05-14T16:30:06.592 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-05-14T16:30:06.592 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-05-14T16:30:06.617 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-05-14T16:30:06.618 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi069.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-05-14T16:30:06.618 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T16:30:06.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:30:06.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T16:30:06.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:30:06.670 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T16:30:06.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:30:06.697 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-14T16:30:06.698 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2024-05-14T16:30:06.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-05-14T16:30:06.767 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-05-14T16:30:06.792 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-14T16:30:06.801 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-14T16:30:06.806 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-14T16:30:06.912 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-14T16:30:06.928 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-14T16:30:06.929 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi069.front.sepia.ceph.com,smithi092.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-14T16:35:21.715 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-05-14T16:35:21.716 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-14T16:35:21.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:35:21.792 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-05-14T16:35:21.831 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-14T16:35:21.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-14T16:35:21.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:35:21.901 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-05-14T16:35:21.992 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-14T16:35:21.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-14T16:35:21.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:35:22.107 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-05-14T16:35:22.143 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-14T16:35:22.143 INFO:teuthology.run_tasks:Running task clock... 2024-05-14T16:35:22.153 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-14T16:35:22.153 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T16:35:22.153 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T16:35:22.155 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T16:35:22.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T16:35:22.158 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T16:35:22.158 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T16:35:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T16:35:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T16:35:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T16:35:22.196 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T16:35:22.213 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T16:35:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-05-14T16:35:22.220 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2024-05-14T16:35:22.228 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T16:35:22.229 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-05-14T16:35:22.231 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2024-05-14T16:35:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T16:35:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T16:35:22.251 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-05-14T16:35:22.255 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T16:35:22.259 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T16:35:22.262 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-05-14T16:35:22.275 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T16:35:22.288 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T16:35:22.312 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-05-14T16:35:22.315 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T16:35:22.315 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-05-14T16:35:22.316 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.316 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.316 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.316 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-05-14T16:35:22.322 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T16:35:22.323 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-05-14T16:35:22.323 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.323 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.323 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.323 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.347 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-05-14T16:35:22.350 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T16:35:22.350 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-05-14T16:35:22.351 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.351 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.351 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.351 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:35:22.352 INFO:teuthology.run_tasks:Running task install... 2024-05-14T16:35:22.360 DEBUG:teuthology.task.install:project ceph 2024-05-14T16:35:22.360 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b'}} 2024-05-14T16:35:22.360 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b'} 2024-05-14T16:35:22.360 INFO:teuthology.task.install:Using flavor: default 2024-05-14T16:35:22.365 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-14T16:35:22.365 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-14T16:35:22.365 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-14T16:35:22.366 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:35:22.367 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-14T16:35:22.367 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:35:22.368 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-14T16:35:22.368 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:35:22.515 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-14T16:35:22.515 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-14T16:35:22.519 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-14T16:35:22.520 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-14T16:35:22.592 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-14T16:35:22.592 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-14T16:35:22.631 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T16:35:22.631 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:35:22.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T16:35:22.655 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T16:35:22.655 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T16:35:22.692 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T16:35:22.692 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:35:22.692 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T16:35:22.718 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T16:35:22.719 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T16:35:22.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T16:35:22.769 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T16:35:22.769 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:35:22.770 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T16:35:22.785 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T16:35:22.793 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T16:35:22.795 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T16:35:22.796 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T16:35:22.856 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-05-14T16:35:22.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-05-14T16:35:22.860 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T16:35:22.861 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T16:35:22.924 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2024-05-14T16:35:22.926 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2024-05-14T16:35:22.935 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T16:35:22.999 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-05-14T16:35:23.000 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-05-14T16:35:23.083 INFO:teuthology.orchestra.run.smithi092.stdout:57 files removed 2024-05-14T16:35:23.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T16:35:23.180 INFO:teuthology.orchestra.run.smithi069.stdout:57 files removed 2024-05-14T16:35:23.208 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T16:35:23.233 INFO:teuthology.orchestra.run.smithi203.stdout:57 files removed 2024-05-14T16:35:23.261 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T16:35:23.697 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 259 kB/s | 84 kB 00:00 2024-05-14T16:35:23.883 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 233 kB/s | 84 kB 00:00 2024-05-14T16:35:23.937 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 183 kB/s | 84 kB 00:00 2024-05-14T16:35:24.130 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-14T16:35:24.292 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-05-14T16:35:24.445 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-05-14T16:35:24.639 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-05-14T16:35:24.894 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-05-14T16:35:25.172 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-05-14T16:35:25.213 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-05-14T16:35:25.295 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 22 MB/s | 8.1 MB 00:00 2024-05-14T16:35:27.123 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.1 MB 00:01 2024-05-14T16:35:29.556 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 19 MB 00:03 2024-05-14T16:35:29.581 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 19 MB 00:03 2024-05-14T16:35:29.724 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-05-14T16:35:33.648 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-14T16:35:34.775 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.2 MB 00:01 2024-05-14T16:35:34.807 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-05-14T16:35:35.125 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-05-14T16:35:36.393 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-05-14T16:35:36.778 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-14T16:35:37.408 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-05-14T16:35:37.797 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-14T16:35:39.193 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-14T16:35:42.017 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-05-14T16:35:42.920 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-14T16:35:44.276 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:35:44.277 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:35:44.287 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T16:35:44.288 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T16:35:44.289 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T16:35:44.290 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T16:35:44.291 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T16:35:44.294 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T16:35:44.294 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T16:35:44.356 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-14T16:35:44.369 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-14T16:35:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-14T16:35:44.369 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-14T16:35:44.369 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-14T16:35:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T16:35:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-14T16:35:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:35:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-14T16:35:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-14T16:35:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-14T16:35:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-14T16:35:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-14T16:35:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-14T16:35:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-14T16:35:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-14T16:35:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-14T16:35:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-14T16:35:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-14T16:35:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-14T16:35:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-14T16:35:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-14T16:35:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T16:35:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T16:35:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-14T16:35:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-14T16:35:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-14T16:35:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-14T16:35:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-14T16:35:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:35:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-14T16:35:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T16:35:44.373 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-05-14T16:35:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T16:35:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T16:35:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T16:35:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T16:35:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-14T16:35:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:35:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T16:35:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T16:35:44.375 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-14T16:35:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T16:35:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T16:35:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T16:35:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-14T16:35:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-14T16:35:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-14T16:35:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-14T16:35:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-14T16:35:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-14T16:35:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-14T16:35:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-14T16:35:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T16:35:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T16:35:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T16:35:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T16:35:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T16:35:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T16:35:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T16:35:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T16:35:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-14T16:35:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T16:35:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T16:35:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T16:35:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T16:35:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T16:35:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-14T16:35:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T16:35:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:35:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T16:35:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T16:35:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T16:35:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T16:35:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T16:35:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T16:35:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T16:35:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T16:35:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T16:35:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T16:35:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T16:35:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T16:35:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T16:35:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T16:35:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T16:35:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T16:35:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T16:35:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-14T16:35:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-14T16:35:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T16:35:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T16:35:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T16:35:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T16:35:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T16:35:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T16:35:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T16:35:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T16:35:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T16:35:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T16:35:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T16:35:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T16:35:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout:Install 148 Packages 2024-05-14T16:35:44.387 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 8 Packages 2024-05-14T16:35:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:35:44.388 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 226 M 2024-05-14T16:35:44.388 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-14T16:35:44.418 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-14T16:35:45.026 INFO:teuthology.orchestra.run.smithi092.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-05-14T16:35:45.245 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:35:45.246 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:35:45.257 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T16:35:45.258 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T16:35:45.258 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T16:35:45.259 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T16:35:45.261 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T16:35:45.263 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T16:35:45.264 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T16:35:45.326 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-05-14T16:35:45.343 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-14T16:35:45.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T16:35:45.345 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-14T16:35:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T16:35:45.347 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T16:35:45.348 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T16:35:45.349 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T16:35:45.350 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T16:35:45.351 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T16:35:45.352 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T16:35:45.353 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T16:35:45.354 INFO:teuthology.orchestra.run.smithi069.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout:Installing weak dependencies: 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout:Install 148 Packages 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 8 Packages 2024-05-14T16:35:45.355 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:35:45.356 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 226 M 2024-05-14T16:35:45.356 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-05-14T16:35:45.360 INFO:teuthology.orchestra.run.smithi092.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 2.6 MB/s | 891 kB 00:00 2024-05-14T16:35:45.427 INFO:teuthology.orchestra.run.smithi092.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 2.1 MB/s | 145 kB 00:00 2024-05-14T16:35:45.802 INFO:teuthology.orchestra.run.smithi092.stdout:(4/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 6.3 MB/s | 2.4 MB 00:00 2024-05-14T16:35:45.886 INFO:teuthology.orchestra.run.smithi092.stdout:(5/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 5.2 MB/s | 5.4 MB 00:01 2024-05-14T16:35:46.020 INFO:teuthology.orchestra.run.smithi092.stdout:(6/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 7.0 MB/s | 1.5 MB 00:00 2024-05-14T16:35:46.045 INFO:teuthology.orchestra.run.smithi069.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-05-14T16:35:46.362 INFO:teuthology.orchestra.run.smithi092.stdout:(7/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 9.9 MB/s | 4.7 MB 00:00 2024-05-14T16:35:46.554 INFO:teuthology.orchestra.run.smithi069.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 1.7 MB/s | 891 kB 00:00 2024-05-14T16:35:46.704 INFO:teuthology.orchestra.run.smithi069.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 965 kB/s | 145 kB 00:00 2024-05-14T16:35:46.710 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:35:46.711 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:35:46.722 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T16:35:46.723 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T16:35:46.723 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T16:35:46.724 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T16:35:46.726 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T16:35:46.728 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T16:35:46.729 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T16:35:46.789 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-14T16:35:46.810 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-05-14T16:35:46.810 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-14T16:35:46.810 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-05-14T16:35:46.810 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-14T16:35:46.810 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T16:35:46.810 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-14T16:35:46.811 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-14T16:35:46.812 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:35:46.813 INFO:teuthology.orchestra.run.smithi203.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-14T16:35:46.814 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T16:35:46.815 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T16:35:46.816 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T16:35:46.817 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T16:35:46.817 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T16:35:46.817 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T16:35:46.817 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T16:35:46.817 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T16:35:46.817 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T16:35:46.817 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-14T16:35:46.817 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-14T16:35:46.817 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T16:35:46.817 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T16:35:46.818 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T16:35:46.818 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T16:35:46.818 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T16:35:46.818 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T16:35:46.818 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T16:35:46.818 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T16:35:46.818 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T16:35:46.818 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T16:35:46.818 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T16:35:46.819 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T16:35:46.820 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T16:35:46.820 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T16:35:46.820 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T16:35:46.820 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T16:35:46.820 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T16:35:46.820 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T16:35:46.820 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T16:35:46.821 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T16:35:46.822 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout:Install 148 Packages 2024-05-14T16:35:46.823 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 8 Packages 2024-05-14T16:35:46.824 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:35:46.824 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 226 M 2024-05-14T16:35:46.824 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-14T16:35:47.428 INFO:teuthology.orchestra.run.smithi092.stdout:(8/156): ceph-radosgw-19.0.0-3687.g9b638668.el9 9.9 MB/s | 11 MB 00:01 2024-05-14T16:35:47.520 INFO:teuthology.orchestra.run.smithi092.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 275 kB/s | 25 kB 00:00 2024-05-14T16:35:47.663 INFO:teuthology.orchestra.run.smithi069.stdout:(4/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 2.5 MB/s | 2.4 MB 00:00 2024-05-14T16:35:47.836 INFO:teuthology.orchestra.run.smithi203.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-05-14T16:35:47.921 INFO:teuthology.orchestra.run.smithi092.stdout:(10/156): ceph-osd-19.0.0-3687.g9b638668.el9.x8 8.9 MB/s | 17 MB 00:01 2024-05-14T16:35:48.021 INFO:teuthology.orchestra.run.smithi092.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 2.2 MB/s | 225 kB 00:00 2024-05-14T16:35:48.029 INFO:teuthology.orchestra.run.smithi069.stdout:(5/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-05-14T16:35:48.079 INFO:teuthology.orchestra.run.smithi092.stdout:(12/156): cephfs-shell-19.0.0-3687.g9b638668.el 463 kB/s | 27 kB 00:00 2024-05-14T16:35:48.155 INFO:teuthology.orchestra.run.smithi092.stdout:(13/156): libcephfs-devel-19.0.0-3687.g9b638668 422 kB/s | 32 kB 00:00 2024-05-14T16:35:48.388 INFO:teuthology.orchestra.run.smithi092.stdout:(14/156): libcephfs2-19.0.0-3687.g9b638668.el9. 3.1 MB/s | 748 kB 00:00 2024-05-14T16:35:48.530 INFO:teuthology.orchestra.run.smithi092.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 1.1 MB/s | 163 kB 00:00 2024-05-14T16:35:48.596 INFO:teuthology.orchestra.run.smithi069.stdout:(6/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-05-14T16:35:48.605 INFO:teuthology.orchestra.run.smithi092.stdout:(16/156): librados-devel-19.0.0-3687.g9b638668. 1.6 MB/s | 127 kB 00:00 2024-05-14T16:35:48.814 INFO:teuthology.orchestra.run.smithi092.stdout:(17/156): libradosstriper1-19.0.0-3687.g9b63866 2.3 MB/s | 501 kB 00:00 2024-05-14T16:35:48.886 INFO:teuthology.orchestra.run.smithi203.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 848 kB/s | 891 kB 00:01 2024-05-14T16:35:49.186 INFO:teuthology.orchestra.run.smithi203.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 483 kB/s | 145 kB 00:00 2024-05-14T16:35:49.280 INFO:teuthology.orchestra.run.smithi069.stdout:(7/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 3.8 MB/s | 4.7 MB 00:01 2024-05-14T16:35:50.653 INFO:teuthology.orchestra.run.smithi203.stdout:(4/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-05-14T16:35:51.314 INFO:teuthology.orchestra.run.smithi092.stdout:(18/156): librgw2-19.0.0-3687.g9b638668.el9.x86 2.1 MB/s | 5.4 MB 00:02 2024-05-14T16:35:51.329 INFO:teuthology.orchestra.run.smithi203.stdout:(5/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-05-14T16:35:51.406 INFO:teuthology.orchestra.run.smithi092.stdout:(19/156): python3-ceph-argparse-19.0.0-3687.g9b 494 kB/s | 45 kB 00:00 2024-05-14T16:35:51.506 INFO:teuthology.orchestra.run.smithi092.stdout:(20/156): python3-ceph-common-19.0.0-3687.g9b63 1.3 MB/s | 136 kB 00:00 2024-05-14T16:35:51.606 INFO:teuthology.orchestra.run.smithi092.stdout:(21/156): python3-cephfs-19.0.0-3687.g9b638668. 1.6 MB/s | 161 kB 00:00 2024-05-14T16:35:51.790 INFO:teuthology.orchestra.run.smithi092.stdout:(22/156): python3-rados-19.0.0-3687.g9b638668.e 1.7 MB/s | 321 kB 00:00 2024-05-14T16:35:51.804 INFO:teuthology.orchestra.run.smithi203.stdout:(6/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 926 kB/s | 2.4 MB 00:02 2024-05-14T16:35:52.099 INFO:teuthology.orchestra.run.smithi092.stdout:(23/156): ceph-common-19.0.0-3687.g9b638668.el9 2.9 MB/s | 21 MB 00:07 2024-05-14T16:35:52.124 INFO:teuthology.orchestra.run.smithi092.stdout:(24/156): python3-rbd-19.0.0-3687.g9b638668.el9 896 kB/s | 299 kB 00:00 2024-05-14T16:35:52.216 INFO:teuthology.orchestra.run.smithi092.stdout:(25/156): python3-rgw-19.0.0-3687.g9b638668.el9 854 kB/s | 100 kB 00:00 2024-05-14T16:35:52.323 INFO:teuthology.orchestra.run.smithi092.stdout:(26/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 428 kB/s | 85 kB 00:00 2024-05-14T16:35:52.475 INFO:teuthology.orchestra.run.smithi092.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 1.1 MB/s | 170 kB 00:00 2024-05-14T16:35:52.566 INFO:teuthology.orchestra.run.smithi092.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 374 kB/s | 34 kB 00:00 2024-05-14T16:35:52.650 INFO:teuthology.orchestra.run.smithi092.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.7 MB/s | 143 kB 00:00 2024-05-14T16:35:52.813 INFO:teuthology.orchestra.run.smithi069.stdout:(8/156): ceph-radosgw-19.0.0-3687.g9b638668.el9 3.0 MB/s | 11 MB 00:03 2024-05-14T16:35:52.880 INFO:teuthology.orchestra.run.smithi069.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 378 kB/s | 25 kB 00:00 2024-05-14T16:35:53.054 INFO:teuthology.orchestra.run.smithi203.stdout:(7/156): ceph-common-19.0.0-3687.g9b638668.el9. 3.8 MB/s | 21 MB 00:05 2024-05-14T16:35:53.255 INFO:teuthology.orchestra.run.smithi203.stdout:(8/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 2.4 MB/s | 4.7 MB 00:01 2024-05-14T16:35:53.321 INFO:teuthology.orchestra.run.smithi203.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 378 kB/s | 25 kB 00:00 2024-05-14T16:35:53.442 INFO:teuthology.orchestra.run.smithi092.stdout:(30/156): ceph-test-19.0.0-3687.g9b638668.el9.x 8.3 MB/s | 49 MB 00:05 2024-05-14T16:35:53.526 INFO:teuthology.orchestra.run.smithi092.stdout:(31/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 2.4 MB/s | 3.1 MB 00:01 2024-05-14T16:35:53.726 INFO:teuthology.orchestra.run.smithi092.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3687.g9b 1.3 MB/s | 274 kB 00:00 2024-05-14T16:35:53.851 INFO:teuthology.orchestra.run.smithi092.stdout:(33/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 395 kB/s | 49 kB 00:00 2024-05-14T16:35:53.968 INFO:teuthology.orchestra.run.smithi092.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3687.g9 141 kB/s | 16 kB 00:00 2024-05-14T16:35:54.118 INFO:teuthology.orchestra.run.smithi092.stdout:(35/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 2.4 MB/s | 3.5 MB 00:01 2024-05-14T16:35:54.177 INFO:teuthology.orchestra.run.smithi092.stdout:(36/156): ceph-volume-19.0.0-3687.g9b638668.el9 1.3 MB/s | 273 kB 00:00 2024-05-14T16:35:54.286 INFO:teuthology.orchestra.run.smithi092.stdout:(37/156): cephfs-top-19.0.0-3687.g9b638668.el9. 168 kB/s | 18 kB 00:00 2024-05-14T16:35:54.364 INFO:teuthology.orchestra.run.smithi203.stdout:(10/156): ceph-radosgw-19.0.0-3687.g9b638668.el 8.1 MB/s | 11 MB 00:01 2024-05-14T16:35:54.428 INFO:teuthology.orchestra.run.smithi092.stdout:(38/156): cephadm-19.0.0-3687.g9b638668.el9.noa 2.4 MB/s | 763 kB 00:00 2024-05-14T16:35:54.456 INFO:teuthology.orchestra.run.smithi203.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 2.4 MB/s | 225 kB 00:00 2024-05-14T16:35:54.503 INFO:teuthology.orchestra.run.smithi092.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 370 kB/s | 80 kB 00:00 2024-05-14T16:35:54.539 INFO:teuthology.orchestra.run.smithi203.stdout:(12/156): cephfs-shell-19.0.0-3687.g9b638668.el 324 kB/s | 27 kB 00:00 2024-05-14T16:35:54.595 INFO:teuthology.orchestra.run.smithi092.stdout:(40/156): ceph-mgr-diskprediction-local-19.0.0- 6.4 MB/s | 7.4 MB 00:01 2024-05-14T16:35:54.606 INFO:teuthology.orchestra.run.smithi203.stdout:(13/156): libcephfs-devel-19.0.0-3687.g9b638668 475 kB/s | 32 kB 00:00 2024-05-14T16:35:54.620 INFO:teuthology.orchestra.run.smithi092.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-05-14T16:35:54.748 INFO:teuthology.orchestra.run.smithi203.stdout:(14/156): libcephfs2-19.0.0-3687.g9b638668.el9. 5.2 MB/s | 748 kB 00:00 2024-05-14T16:35:54.762 INFO:teuthology.orchestra.run.smithi092.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.2 MB 00:00 2024-05-14T16:35:54.815 INFO:teuthology.orchestra.run.smithi203.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 2.4 MB/s | 163 kB 00:00 2024-05-14T16:35:54.821 INFO:teuthology.orchestra.run.smithi092.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-05-14T16:35:54.846 INFO:teuthology.orchestra.run.smithi092.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-05-14T16:35:54.888 INFO:teuthology.orchestra.run.smithi092.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-14T16:35:54.890 INFO:teuthology.orchestra.run.smithi203.stdout:(16/156): librados-devel-19.0.0-3687.g9b638668. 1.6 MB/s | 127 kB 00:00 2024-05-14T16:35:54.913 INFO:teuthology.orchestra.run.smithi092.stdout:(46/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 796 kB/s | 253 kB 00:00 2024-05-14T16:35:54.938 INFO:teuthology.orchestra.run.smithi092.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-14T16:35:54.964 INFO:teuthology.orchestra.run.smithi092.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-14T16:35:54.974 INFO:teuthology.orchestra.run.smithi203.stdout:(17/156): libradosstriper1-19.0.0-3687.g9b63866 5.9 MB/s | 501 kB 00:00 2024-05-14T16:35:55.114 INFO:teuthology.orchestra.run.smithi092.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-05-14T16:35:55.139 INFO:teuthology.orchestra.run.smithi092.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 611 kB/s | 107 kB 00:00 2024-05-14T16:35:55.181 INFO:teuthology.orchestra.run.smithi092.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-05-14T16:35:55.207 INFO:teuthology.orchestra.run.smithi092.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 233 kB/s | 21 kB 00:00 2024-05-14T16:35:55.240 INFO:teuthology.orchestra.run.smithi092.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-05-14T16:35:55.265 INFO:teuthology.orchestra.run.smithi092.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-05-14T16:35:55.299 INFO:teuthology.orchestra.run.smithi092.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-05-14T16:35:55.349 INFO:teuthology.orchestra.run.smithi092.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-05-14T16:35:55.374 INFO:teuthology.orchestra.run.smithi092.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-14T16:35:55.400 INFO:teuthology.orchestra.run.smithi092.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-05-14T16:35:55.459 INFO:teuthology.orchestra.run.smithi092.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-14T16:35:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-14T16:35:55.507 INFO:teuthology.orchestra.run.smithi203.stdout:(18/156): librgw2-19.0.0-3687.g9b638668.el9.x86 10 MB/s | 5.4 MB 00:00 2024-05-14T16:35:55.526 INFO:teuthology.orchestra.run.smithi092.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-05-14T16:35:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:(10/156): ceph-osd-19.0.0-3687.g9b638668.el9.x8 2.4 MB/s | 17 MB 00:06 2024-05-14T16:35:55.574 INFO:teuthology.orchestra.run.smithi203.stdout:(19/156): python3-ceph-argparse-19.0.0-3687.g9b 678 kB/s | 45 kB 00:00 2024-05-14T16:35:55.610 INFO:teuthology.orchestra.run.smithi092.stdout:(62/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 169 kB/s | 25 kB 00:00 2024-05-14T16:35:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 2.4 MB/s | 225 kB 00:00 2024-05-14T16:35:55.635 INFO:teuthology.orchestra.run.smithi092.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 333 kB/s | 47 kB 00:00 2024-05-14T16:35:55.641 INFO:teuthology.orchestra.run.smithi203.stdout:(20/156): python3-ceph-common-19.0.0-3687.g9b63 2.0 MB/s | 136 kB 00:00 2024-05-14T16:35:55.660 INFO:teuthology.orchestra.run.smithi092.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-05-14T16:35:55.702 INFO:teuthology.orchestra.run.smithi092.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-14T16:35:55.708 INFO:teuthology.orchestra.run.smithi203.stdout:(21/156): python3-cephfs-19.0.0-3687.g9b638668. 2.4 MB/s | 161 kB 00:00 2024-05-14T16:35:55.727 INFO:teuthology.orchestra.run.smithi092.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 382 kB/s | 35 kB 00:00 2024-05-14T16:35:55.784 INFO:teuthology.orchestra.run.smithi203.stdout:(22/156): python3-rados-19.0.0-3687.g9b638668.e 4.2 MB/s | 321 kB 00:00 2024-05-14T16:35:55.822 INFO:teuthology.orchestra.run.smithi069.stdout:(12/156): ceph-common-19.0.0-3687.g9b638668.el9 2.1 MB/s | 21 MB 00:09 2024-05-14T16:35:55.828 INFO:teuthology.orchestra.run.smithi092.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-14T16:35:55.848 INFO:teuthology.orchestra.run.smithi069.stdout:(13/156): cephfs-shell-19.0.0-3687.g9b638668.el 120 kB/s | 27 kB 00:00 2024-05-14T16:35:55.859 INFO:teuthology.orchestra.run.smithi203.stdout:(23/156): python3-rbd-19.0.0-3687.g9b638668.el9 3.9 MB/s | 299 kB 00:00 2024-05-14T16:35:55.881 INFO:teuthology.orchestra.run.smithi069.stdout:(14/156): libcephfs-devel-19.0.0-3687.g9b638668 546 kB/s | 32 kB 00:00 2024-05-14T16:35:55.895 INFO:teuthology.orchestra.run.smithi092.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-05-14T16:35:55.934 INFO:teuthology.orchestra.run.smithi203.stdout:(24/156): python3-rgw-19.0.0-3687.g9b638668.el9 1.3 MB/s | 100 kB 00:00 2024-05-14T16:35:55.953 INFO:teuthology.orchestra.run.smithi092.stdout:(69/156): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-05-14T16:35:55.956 INFO:teuthology.orchestra.run.smithi069.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 2.1 MB/s | 163 kB 00:00 2024-05-14T16:35:55.979 INFO:teuthology.orchestra.run.smithi092.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-05-14T16:35:55.993 INFO:teuthology.orchestra.run.smithi203.stdout:(25/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-14T16:35:56.013 INFO:teuthology.orchestra.run.smithi092.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 4.8 MB/s | 911 kB 00:00 2024-05-14T16:35:56.015 INFO:teuthology.orchestra.run.smithi069.stdout:(16/156): libcephfs2-19.0.0-3687.g9b638668.el9. 4.4 MB/s | 748 kB 00:00 2024-05-14T16:35:56.041 INFO:teuthology.orchestra.run.smithi069.stdout:(17/156): librados-devel-19.0.0-3687.g9b638668. 1.5 MB/s | 127 kB 00:00 2024-05-14T16:35:56.046 INFO:teuthology.orchestra.run.smithi092.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-14T16:35:56.202 INFO:teuthology.orchestra.run.smithi069.stdout:(18/156): libradosstriper1-19.0.0-3687.g9b63866 2.7 MB/s | 501 kB 00:00 2024-05-14T16:35:56.204 INFO:teuthology.orchestra.run.smithi092.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 456 kB/s | 87 kB 00:00 2024-05-14T16:35:56.258 INFO:teuthology.orchestra.run.smithi069.stdout:(19/156): python3-ceph-argparse-19.0.0-3687.g9b 773 kB/s | 45 kB 00:00 2024-05-14T16:35:56.271 INFO:teuthology.orchestra.run.smithi092.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-05-14T16:35:56.308 INFO:teuthology.orchestra.run.smithi069.stdout:(20/156): python3-ceph-common-19.0.0-3687.g9b63 2.7 MB/s | 136 kB 00:00 2024-05-14T16:35:56.318 INFO:teuthology.orchestra.run.smithi203.stdout:(26/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 9.5 MB/s | 3.1 MB 00:00 2024-05-14T16:35:56.346 INFO:teuthology.orchestra.run.smithi092.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-05-14T16:35:56.367 INFO:teuthology.orchestra.run.smithi069.stdout:(21/156): python3-cephfs-19.0.0-3687.g9b638668. 2.7 MB/s | 161 kB 00:00 2024-05-14T16:35:56.372 INFO:teuthology.orchestra.run.smithi092.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-14T16:35:56.393 INFO:teuthology.orchestra.run.smithi203.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 2.2 MB/s | 170 kB 00:00 2024-05-14T16:35:56.397 INFO:teuthology.orchestra.run.smithi092.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-14T16:35:56.448 INFO:teuthology.orchestra.run.smithi092.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-05-14T16:35:56.450 INFO:teuthology.orchestra.run.smithi069.stdout:(22/156): python3-rados-19.0.0-3687.g9b638668.e 3.8 MB/s | 321 kB 00:00 2024-05-14T16:35:56.461 INFO:teuthology.orchestra.run.smithi203.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 511 kB/s | 34 kB 00:00 2024-05-14T16:35:56.464 INFO:teuthology.orchestra.run.smithi092.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-05-14T16:35:56.498 INFO:teuthology.orchestra.run.smithi092.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-05-14T16:35:56.536 INFO:teuthology.orchestra.run.smithi203.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.9 MB/s | 143 kB 00:00 2024-05-14T16:35:56.542 INFO:teuthology.orchestra.run.smithi069.stdout:(23/156): python3-rbd-19.0.0-3687.g9b638668.el9 3.2 MB/s | 299 kB 00:00 2024-05-14T16:35:56.631 INFO:teuthology.orchestra.run.smithi092.stdout:(81/156): postgresql-server-13.14-1.el9.x86_64. 8.5 MB/s | 5.8 MB 00:00 2024-05-14T16:35:56.634 INFO:teuthology.orchestra.run.smithi069.stdout:(24/156): python3-rgw-19.0.0-3687.g9b638668.el9 1.1 MB/s | 100 kB 00:00 2024-05-14T16:35:56.656 INFO:teuthology.orchestra.run.smithi092.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-05-14T16:35:56.699 INFO:teuthology.orchestra.run.smithi092.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-05-14T16:35:56.709 INFO:teuthology.orchestra.run.smithi069.stdout:(25/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-14T16:35:56.757 INFO:teuthology.orchestra.run.smithi092.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 725 kB/s | 42 kB 00:00 2024-05-14T16:35:56.782 INFO:teuthology.orchestra.run.smithi092.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 615 kB/s | 77 kB 00:00 2024-05-14T16:35:56.808 INFO:teuthology.orchestra.run.smithi092.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-14T16:35:56.858 INFO:teuthology.orchestra.run.smithi092.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-05-14T16:35:56.894 INFO:teuthology.orchestra.run.smithi203.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 9.9 MB/s | 3.5 MB 00:00 2024-05-14T16:35:56.917 INFO:teuthology.orchestra.run.smithi092.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-05-14T16:35:56.942 INFO:teuthology.orchestra.run.smithi092.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 995 kB/s | 159 kB 00:00 2024-05-14T16:35:56.976 INFO:teuthology.orchestra.run.smithi092.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 914 kB/s | 54 kB 00:00 2024-05-14T16:35:57.035 INFO:teuthology.orchestra.run.smithi092.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.9 MB/s | 5.1 MB 00:00 2024-05-14T16:35:57.060 INFO:teuthology.orchestra.run.smithi092.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 498 kB/s | 42 kB 00:00 2024-05-14T16:35:57.118 INFO:teuthology.orchestra.run.smithi092.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-14T16:35:57.143 INFO:teuthology.orchestra.run.smithi092.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 406 kB/s | 44 kB 00:00 2024-05-14T16:35:57.169 INFO:teuthology.orchestra.run.smithi092.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-05-14T16:35:57.194 INFO:teuthology.orchestra.run.smithi092.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-05-14T16:35:57.219 INFO:teuthology.orchestra.run.smithi092.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-14T16:35:57.261 INFO:teuthology.orchestra.run.smithi092.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 864 kB/s | 58 kB 00:00 2024-05-14T16:35:57.295 INFO:teuthology.orchestra.run.smithi092.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-05-14T16:35:57.320 INFO:teuthology.orchestra.run.smithi092.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-14T16:35:57.346 INFO:teuthology.orchestra.run.smithi092.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-05-14T16:35:57.513 INFO:teuthology.orchestra.run.smithi092.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-05-14T16:35:57.576 INFO:teuthology.orchestra.run.smithi069.stdout:(26/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 3.6 MB/s | 3.1 MB 00:00 2024-05-14T16:35:57.579 INFO:teuthology.orchestra.run.smithi092.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-05-14T16:35:57.645 INFO:teuthology.orchestra.run.smithi203.stdout:(31/156): ceph-mgr-diskprediction-local-19.0.0- 9.9 MB/s | 7.4 MB 00:00 2024-05-14T16:35:57.702 INFO:teuthology.orchestra.run.smithi069.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 1.3 MB/s | 170 kB 00:00 2024-05-14T16:35:57.705 INFO:teuthology.orchestra.run.smithi092.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-05-14T16:35:57.762 INFO:teuthology.orchestra.run.smithi203.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3687.g9b 2.3 MB/s | 274 kB 00:00 2024-05-14T16:35:57.810 INFO:teuthology.orchestra.run.smithi069.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 316 kB/s | 34 kB 00:00 2024-05-14T16:35:57.887 INFO:teuthology.orchestra.run.smithi203.stdout:(33/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 393 kB/s | 49 kB 00:00 2024-05-14T16:35:57.927 INFO:teuthology.orchestra.run.smithi069.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.2 MB/s | 143 kB 00:00 2024-05-14T16:35:57.947 INFO:teuthology.orchestra.run.smithi092.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-05-14T16:35:58.012 INFO:teuthology.orchestra.run.smithi203.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3687.g9 131 kB/s | 16 kB 00:00 2024-05-14T16:35:58.047 INFO:teuthology.orchestra.run.smithi092.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-14T16:35:58.072 INFO:teuthology.orchestra.run.smithi092.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 627 kB/s | 309 kB 00:00 2024-05-14T16:35:58.113 INFO:teuthology.orchestra.run.smithi203.stdout:(35/156): ceph-volume-19.0.0-3687.g9b638668.el9 2.7 MB/s | 273 kB 00:00 2024-05-14T16:35:58.139 INFO:teuthology.orchestra.run.smithi092.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-05-14T16:35:58.144 INFO:teuthology.orchestra.run.smithi069.stdout:(30/156): librgw2-19.0.0-3687.g9b638668.el9.x86 2.6 MB/s | 5.4 MB 00:02 2024-05-14T16:35:58.164 INFO:teuthology.orchestra.run.smithi092.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-05-14T16:35:58.231 INFO:teuthology.orchestra.run.smithi092.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 734 kB/s | 67 kB 00:00 2024-05-14T16:35:58.238 INFO:teuthology.orchestra.run.smithi203.stdout:(36/156): cephadm-19.0.0-3687.g9b638668.el9.noa 6.0 MB/s | 763 kB 00:00 2024-05-14T16:35:58.265 INFO:teuthology.orchestra.run.smithi092.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-05-14T16:35:58.330 INFO:teuthology.orchestra.run.smithi203.stdout:(37/156): cephfs-top-19.0.0-3687.g9b638668.el9. 199 kB/s | 18 kB 00:00 2024-05-14T16:35:58.415 INFO:teuthology.orchestra.run.smithi092.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-05-14T16:35:58.440 INFO:teuthology.orchestra.run.smithi092.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-05-14T16:35:58.447 INFO:teuthology.orchestra.run.smithi203.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 688 kB/s | 80 kB 00:00 2024-05-14T16:35:58.499 INFO:teuthology.orchestra.run.smithi092.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 521 kB/s | 43 kB 00:00 2024-05-14T16:35:58.566 INFO:teuthology.orchestra.run.smithi092.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-05-14T16:35:58.572 INFO:teuthology.orchestra.run.smithi203.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-05-14T16:35:58.591 INFO:teuthology.orchestra.run.smithi092.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 213 kB/s | 32 kB 00:00 2024-05-14T16:35:58.608 INFO:teuthology.orchestra.run.smithi092.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 137 kB/s | 15 kB 00:00 2024-05-14T16:35:58.623 INFO:teuthology.orchestra.run.smithi203.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-05-14T16:35:58.659 INFO:teuthology.orchestra.run.smithi092.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-14T16:35:58.673 INFO:teuthology.orchestra.run.smithi203.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-05-14T16:35:58.692 INFO:teuthology.orchestra.run.smithi092.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-14T16:35:58.718 INFO:teuthology.orchestra.run.smithi092.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-05-14T16:35:58.751 INFO:teuthology.orchestra.run.smithi092.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 382 kB/s | 35 kB 00:00 2024-05-14T16:35:58.773 INFO:teuthology.orchestra.run.smithi203.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-05-14T16:35:58.793 INFO:teuthology.orchestra.run.smithi092.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-14T16:35:58.824 INFO:teuthology.orchestra.run.smithi203.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-14T16:35:58.843 INFO:teuthology.orchestra.run.smithi092.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-14T16:35:58.874 INFO:teuthology.orchestra.run.smithi203.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-14T16:35:58.893 INFO:teuthology.orchestra.run.smithi092.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-05-14T16:35:58.924 INFO:teuthology.orchestra.run.smithi203.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-05-14T16:35:58.944 INFO:teuthology.orchestra.run.smithi092.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-05-14T16:35:58.969 INFO:teuthology.orchestra.run.smithi092.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-05-14T16:35:58.974 INFO:teuthology.orchestra.run.smithi203.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-14T16:35:58.994 INFO:teuthology.orchestra.run.smithi092.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-05-14T16:35:59.037 INFO:teuthology.orchestra.run.smithi092.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-05-14T16:35:59.062 INFO:teuthology.orchestra.run.smithi092.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-05-14T16:35:59.104 INFO:teuthology.orchestra.run.smithi092.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-05-14T16:35:59.111 INFO:teuthology.orchestra.run.smithi069.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 3.0 MB/s | 3.5 MB 00:01 2024-05-14T16:35:59.129 INFO:teuthology.orchestra.run.smithi092.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-05-14T16:35:59.142 INFO:teuthology.orchestra.run.smithi203.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-05-14T16:35:59.154 INFO:teuthology.orchestra.run.smithi092.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 854 kB/s | 79 kB 00:00 2024-05-14T16:35:59.204 INFO:teuthology.orchestra.run.smithi092.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-05-14T16:35:59.217 INFO:teuthology.orchestra.run.smithi203.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-05-14T16:35:59.230 INFO:teuthology.orchestra.run.smithi092.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-14T16:35:59.255 INFO:teuthology.orchestra.run.smithi092.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-05-14T16:35:59.259 INFO:teuthology.orchestra.run.smithi203.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-05-14T16:35:59.278 INFO:teuthology.orchestra.run.smithi069.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3687.g9b 1.6 MB/s | 274 kB 00:00 2024-05-14T16:35:59.297 INFO:teuthology.orchestra.run.smithi092.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-05-14T16:35:59.301 INFO:teuthology.orchestra.run.smithi203.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 517 kB/s | 21 kB 00:00 2024-05-14T16:35:59.322 INFO:teuthology.orchestra.run.smithi092.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-05-14T16:35:59.347 INFO:teuthology.orchestra.run.smithi092.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-05-14T16:35:59.351 INFO:teuthology.orchestra.run.smithi203.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-14T16:35:59.370 INFO:teuthology.orchestra.run.smithi069.stdout:(33/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 539 kB/s | 49 kB 00:00 2024-05-14T16:35:59.393 INFO:teuthology.orchestra.run.smithi203.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 783 kB/s | 33 kB 00:00 2024-05-14T16:35:59.398 INFO:teuthology.orchestra.run.smithi092.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-14T16:35:59.423 INFO:teuthology.orchestra.run.smithi092.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-05-14T16:35:59.449 INFO:teuthology.orchestra.run.smithi092.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-05-14T16:35:59.478 INFO:teuthology.orchestra.run.smithi069.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3687.g9 152 kB/s | 16 kB 00:00 2024-05-14T16:35:59.485 INFO:teuthology.orchestra.run.smithi203.stdout:(53/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-05-14T16:35:59.490 INFO:teuthology.orchestra.run.smithi092.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-05-14T16:35:59.524 INFO:teuthology.orchestra.run.smithi092.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-14T16:35:59.527 INFO:teuthology.orchestra.run.smithi203.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-05-14T16:35:59.549 INFO:teuthology.orchestra.run.smithi092.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 893 kB/s | 90 kB 00:00 2024-05-14T16:35:59.569 INFO:teuthology.orchestra.run.smithi203.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-05-14T16:35:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-14T16:35:59.611 INFO:teuthology.orchestra.run.smithi203.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-05-14T16:35:59.641 INFO:teuthology.orchestra.run.smithi092.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-05-14T16:35:59.646 INFO:teuthology.orchestra.run.smithi069.stdout:(35/156): ceph-volume-19.0.0-3687.g9b638668.el9 1.6 MB/s | 273 kB 00:00 2024-05-14T16:35:59.653 INFO:teuthology.orchestra.run.smithi203.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-14T16:35:59.675 INFO:teuthology.orchestra.run.smithi092.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-14T16:35:59.704 INFO:teuthology.orchestra.run.smithi203.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-05-14T16:35:59.763 INFO:teuthology.orchestra.run.smithi203.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 582 kB 00:00 2024-05-14T16:35:59.805 INFO:teuthology.orchestra.run.smithi203.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 612 kB/s | 25 kB 00:00 2024-05-14T16:35:59.847 INFO:teuthology.orchestra.run.smithi203.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-14T16:35:59.888 INFO:teuthology.orchestra.run.smithi203.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-05-14T16:35:59.931 INFO:teuthology.orchestra.run.smithi203.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-05-14T16:35:59.942 INFO:teuthology.orchestra.run.smithi092.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-05-14T16:35:59.962 INFO:teuthology.orchestra.run.smithi069.stdout:(36/156): cephadm-19.0.0-3687.g9b638668.el9.noa 2.4 MB/s | 763 kB 00:00 2024-05-14T16:35:59.972 INFO:teuthology.orchestra.run.smithi203.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 847 kB/s | 35 kB 00:00 2024-05-14T16:36:00.054 INFO:teuthology.orchestra.run.smithi069.stdout:(37/156): cephfs-top-19.0.0-3687.g9b638668.el9. 199 kB/s | 18 kB 00:00 2024-05-14T16:36:00.073 INFO:teuthology.orchestra.run.smithi203.stdout:(65/156): openblas-openmp-0.3.21-2.el9.x86_64.r 47 MB/s | 4.7 MB 00:00 2024-05-14T16:36:00.173 INFO:teuthology.orchestra.run.smithi203.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-05-14T16:36:00.197 INFO:teuthology.orchestra.run.smithi069.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 566 kB/s | 80 kB 00:00 2024-05-14T16:36:00.240 INFO:teuthology.orchestra.run.smithi203.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-05-14T16:36:00.276 INFO:teuthology.orchestra.run.smithi092.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 6.9 MB/s | 2.3 MB 00:00 2024-05-14T16:36:00.299 INFO:teuthology.orchestra.run.smithi203.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 15 MB/s | 911 kB 00:00 2024-05-14T16:36:00.313 INFO:teuthology.orchestra.run.smithi069.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-05-14T16:36:00.341 INFO:teuthology.orchestra.run.smithi203.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 133 kB 00:00 2024-05-14T16:36:00.364 INFO:teuthology.orchestra.run.smithi069.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-05-14T16:36:00.392 INFO:teuthology.orchestra.run.smithi092.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 15 MB/s | 1.8 MB 00:00 2024-05-14T16:36:00.414 INFO:teuthology.orchestra.run.smithi069.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-14T16:36:00.451 INFO:teuthology.orchestra.run.smithi092.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 710 kB/s | 41 kB 00:00 2024-05-14T16:36:00.491 INFO:teuthology.orchestra.run.smithi203.stdout:(70/156): postgresql-server-13.14-1.el9.x86_64. 39 MB/s | 5.8 MB 00:00 2024-05-14T16:36:00.498 INFO:teuthology.orchestra.run.smithi069.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-05-14T16:36:00.510 INFO:teuthology.orchestra.run.smithi092.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 4.1 MB/s | 248 kB 00:00 2024-05-14T16:36:00.548 INFO:teuthology.orchestra.run.smithi069.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-14T16:36:00.549 INFO:teuthology.orchestra.run.smithi203.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-05-14T16:36:00.591 INFO:teuthology.orchestra.run.smithi203.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-05-14T16:36:00.598 INFO:teuthology.orchestra.run.smithi069.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-14T16:36:00.610 INFO:teuthology.orchestra.run.smithi092.stdout:(153/156): librados2-19.0.0-3687.g9b638668.el9. 3.5 MB/s | 3.4 MB 00:00 2024-05-14T16:36:00.643 INFO:teuthology.orchestra.run.smithi092.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-05-14T16:36:00.648 INFO:teuthology.orchestra.run.smithi069.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-05-14T16:36:00.699 INFO:teuthology.orchestra.run.smithi069.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-14T16:36:00.709 INFO:teuthology.orchestra.run.smithi203.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 51 MB/s | 6.0 MB 00:00 2024-05-14T16:36:00.750 INFO:teuthology.orchestra.run.smithi203.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-05-14T16:36:00.793 INFO:teuthology.orchestra.run.smithi203.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-05-14T16:36:00.834 INFO:teuthology.orchestra.run.smithi203.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-05-14T16:36:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-05-14T16:36:00.919 INFO:teuthology.orchestra.run.smithi092.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.8 MB/s | 2.4 MB 00:00 2024-05-14T16:36:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-05-14T16:36:00.982 INFO:teuthology.orchestra.run.smithi069.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-05-14T16:36:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 833 kB/s | 35 kB 00:00 2024-05-14T16:36:01.085 INFO:teuthology.orchestra.run.smithi092.stdout:(156/156): librbd1-19.0.0-3687.g9b638668.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-05-14T16:36:01.088 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:36:01.088 INFO:teuthology.orchestra.run.smithi092.stdout:Total 14 MB/s | 226 MB 00:16 2024-05-14T16:36:01.103 INFO:teuthology.orchestra.run.smithi203.stdout:(80/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 47 MB/s | 5.1 MB 00:00 2024-05-14T16:36:01.241 INFO:teuthology.orchestra.run.smithi069.stdout:(48/156): ceph-mgr-diskprediction-local-19.0.0- 2.4 MB/s | 7.4 MB 00:03 2024-05-14T16:36:01.270 INFO:teuthology.orchestra.run.smithi203.stdout:(81/156): ceph-osd-19.0.0-3687.g9b638668.el9.x8 1.8 MB/s | 17 MB 00:09 2024-05-14T16:36:01.275 INFO:teuthology.orchestra.run.smithi069.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-05-14T16:36:01.303 INFO:teuthology.orchestra.run.smithi203.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-05-14T16:36:01.317 INFO:teuthology.orchestra.run.smithi069.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 518 kB/s | 21 kB 00:00 2024-05-14T16:36:01.337 INFO:teuthology.orchestra.run.smithi203.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 2.3 MB/s | 77 kB 00:00 2024-05-14T16:36:01.379 INFO:teuthology.orchestra.run.smithi203.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-14T16:36:01.413 INFO:teuthology.orchestra.run.smithi203.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-05-14T16:36:01.438 INFO:teuthology.orchestra.run.smithi203.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-14T16:36:01.464 INFO:teuthology.orchestra.run.smithi203.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-14T16:36:01.489 INFO:teuthology.orchestra.run.smithi203.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-14T16:36:01.514 INFO:teuthology.orchestra.run.smithi203.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-14T16:36:01.539 INFO:teuthology.orchestra.run.smithi203.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-14T16:36:01.581 INFO:teuthology.orchestra.run.smithi203.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 999 kB/s | 42 kB 00:00 2024-05-14T16:36:01.624 INFO:teuthology.orchestra.run.smithi203.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-14T16:36:01.666 INFO:teuthology.orchestra.run.smithi203.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-05-14T16:36:01.707 INFO:teuthology.orchestra.run.smithi203.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-05-14T16:36:01.842 INFO:teuthology.orchestra.run.smithi069.stdout:(51/156): ceph-test-19.0.0-3687.g9b638668.el9.x 5.5 MB/s | 49 MB 00:08 2024-05-14T16:36:01.867 INFO:teuthology.orchestra.run.smithi069.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 171 kB/s | 107 kB 00:00 2024-05-14T16:36:01.875 INFO:teuthology.orchestra.run.smithi203.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-05-14T16:36:01.892 INFO:teuthology.orchestra.run.smithi069.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 545 kB/s | 313 kB 00:00 2024-05-14T16:36:01.900 INFO:teuthology.orchestra.run.smithi203.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-05-14T16:36:01.918 INFO:teuthology.orchestra.run.smithi069.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-05-14T16:36:01.925 INFO:teuthology.orchestra.run.smithi203.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-14T16:36:01.950 INFO:teuthology.orchestra.run.smithi203.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-14T16:36:01.968 INFO:teuthology.orchestra.run.smithi069.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-05-14T16:36:01.976 INFO:teuthology.orchestra.run.smithi203.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-14T16:36:01.993 INFO:teuthology.orchestra.run.smithi069.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 181 kB/s | 18 kB 00:00 2024-05-14T16:36:02.001 INFO:teuthology.orchestra.run.smithi203.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-14T16:36:02.026 INFO:teuthology.orchestra.run.smithi203.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-05-14T16:36:02.052 INFO:teuthology.orchestra.run.smithi069.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-05-14T16:36:02.077 INFO:teuthology.orchestra.run.smithi203.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-05-14T16:36:02.077 INFO:teuthology.orchestra.run.smithi069.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-05-14T16:36:02.152 INFO:teuthology.orchestra.run.smithi203.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-05-14T16:36:02.227 INFO:teuthology.orchestra.run.smithi069.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-05-14T16:36:02.261 INFO:teuthology.orchestra.run.smithi069.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 582 kB 00:00 2024-05-14T16:36:02.276 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-14T16:36:02.303 INFO:teuthology.orchestra.run.smithi069.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-05-14T16:36:02.328 INFO:teuthology.orchestra.run.smithi069.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-05-14T16:36:02.425 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-14T16:36:02.426 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-14T16:36:02.445 INFO:teuthology.orchestra.run.smithi069.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-14T16:36:02.470 INFO:teuthology.orchestra.run.smithi069.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-05-14T16:36:02.521 INFO:teuthology.orchestra.run.smithi069.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 467 kB/s | 35 kB 00:00 2024-05-14T16:36:02.611 INFO:teuthology.orchestra.run.smithi203.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-14T16:36:02.636 INFO:teuthology.orchestra.run.smithi203.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 676 kB/s | 309 kB 00:00 2024-05-14T16:36:02.686 INFO:teuthology.orchestra.run.smithi203.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-05-14T16:36:02.762 INFO:teuthology.orchestra.run.smithi203.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-05-14T16:36:02.771 INFO:teuthology.orchestra.run.smithi069.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.3 MB/s | 3.0 MB 00:00 2024-05-14T16:36:03.155 INFO:teuthology.orchestra.run.smithi069.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-05-14T16:36:03.254 INFO:teuthology.orchestra.run.smithi203.stdout:(108/156): ceph-test-19.0.0-3687.g9b638668.el9. 4.9 MB/s | 49 MB 00:09 2024-05-14T16:36:03.279 INFO:teuthology.orchestra.run.smithi203.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 94 kB/s | 49 kB 00:00 2024-05-14T16:36:03.313 INFO:teuthology.orchestra.run.smithi069.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-05-14T16:36:03.371 INFO:teuthology.orchestra.run.smithi203.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-05-14T16:36:03.372 INFO:teuthology.orchestra.run.smithi069.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-14T16:36:03.421 INFO:teuthology.orchestra.run.smithi203.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-05-14T16:36:03.497 INFO:teuthology.orchestra.run.smithi203.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.3 MB/s | 839 kB 00:00 2024-05-14T16:36:03.530 INFO:teuthology.orchestra.run.smithi203.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-05-14T16:36:03.572 INFO:teuthology.orchestra.run.smithi203.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 578 kB/s | 43 kB 00:00 2024-05-14T16:36:03.580 INFO:teuthology.orchestra.run.smithi069.stdout:(70/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-05-14T16:36:03.606 INFO:teuthology.orchestra.run.smithi203.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 427 kB/s | 32 kB 00:00 2024-05-14T16:36:03.648 INFO:teuthology.orchestra.run.smithi203.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-14T16:36:03.665 INFO:teuthology.orchestra.run.smithi203.stdout:(117/156): libunwind-1.6.2-1.el9.x86_64.rpm 164 kB/s | 67 kB 00:00 2024-05-14T16:36:03.690 INFO:teuthology.orchestra.run.smithi203.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-14T16:36:03.706 INFO:teuthology.orchestra.run.smithi069.stdout:(71/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.8 MB/s | 4.7 MB 00:01 2024-05-14T16:36:03.732 INFO:teuthology.orchestra.run.smithi203.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-14T16:36:03.758 INFO:teuthology.orchestra.run.smithi203.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 528 kB/s | 35 kB 00:00 2024-05-14T16:36:03.781 INFO:teuthology.orchestra.run.smithi069.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-05-14T16:36:03.808 INFO:teuthology.orchestra.run.smithi203.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-14T16:36:03.867 INFO:teuthology.orchestra.run.smithi203.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-05-14T16:36:03.900 INFO:teuthology.orchestra.run.smithi203.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-14T16:36:03.948 INFO:teuthology.orchestra.run.smithi069.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-05-14T16:36:03.951 INFO:teuthology.orchestra.run.smithi203.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-14T16:36:03.976 INFO:teuthology.orchestra.run.smithi203.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-05-14T16:36:04.001 INFO:teuthology.orchestra.run.smithi203.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-14T16:36:04.026 INFO:teuthology.orchestra.run.smithi203.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-05-14T16:36:04.032 INFO:teuthology.orchestra.run.smithi069.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-14T16:36:04.052 INFO:teuthology.orchestra.run.smithi203.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-05-14T16:36:04.077 INFO:teuthology.orchestra.run.smithi203.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-14T16:36:04.090 INFO:teuthology.orchestra.run.smithi069.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-05-14T16:36:04.141 INFO:teuthology.orchestra.run.smithi069.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-05-14T16:36:04.208 INFO:teuthology.orchestra.run.smithi069.stdout:(77/156): postgresql-server-13.14-1.el9.x86_64. 6.9 MB/s | 5.8 MB 00:00 2024-05-14T16:36:04.224 INFO:teuthology.orchestra.run.smithi069.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 229 kB/s | 19 kB 00:00 2024-05-14T16:36:04.227 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-14T16:36:04.228 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-14T16:36:04.258 INFO:teuthology.orchestra.run.smithi069.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-14T16:36:04.269 INFO:teuthology.orchestra.run.smithi203.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-05-14T16:36:04.275 INFO:teuthology.orchestra.run.smithi069.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-05-14T16:36:04.294 INFO:teuthology.orchestra.run.smithi203.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 191 kB/s | 46 kB 00:00 2024-05-14T16:36:04.320 INFO:teuthology.orchestra.run.smithi203.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 325 kB/s | 79 kB 00:00 2024-05-14T16:36:04.334 INFO:teuthology.orchestra.run.smithi069.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-14T16:36:04.347 INFO:teuthology.orchestra.run.smithi203.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 743 kB/s | 58 kB 00:00 2024-05-14T16:36:04.373 INFO:teuthology.orchestra.run.smithi203.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.4 MB/s | 272 kB 00:00 2024-05-14T16:36:04.392 INFO:teuthology.orchestra.run.smithi069.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-14T16:36:04.398 INFO:teuthology.orchestra.run.smithi203.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 211 kB/s | 16 kB 00:00 2024-05-14T16:36:04.423 INFO:teuthology.orchestra.run.smithi203.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-14T16:36:04.449 INFO:teuthology.orchestra.run.smithi203.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-14T16:36:04.451 INFO:teuthology.orchestra.run.smithi069.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-14T16:36:04.474 INFO:teuthology.orchestra.run.smithi203.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-05-14T16:36:04.499 INFO:teuthology.orchestra.run.smithi203.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-14T16:36:04.501 INFO:teuthology.orchestra.run.smithi069.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-05-14T16:36:04.524 INFO:teuthology.orchestra.run.smithi203.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-05-14T16:36:04.558 INFO:teuthology.orchestra.run.smithi203.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-05-14T16:36:04.560 INFO:teuthology.orchestra.run.smithi069.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-14T16:36:04.583 INFO:teuthology.orchestra.run.smithi203.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 610 kB/s | 51 kB 00:00 2024-05-14T16:36:04.608 INFO:teuthology.orchestra.run.smithi203.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-14T16:36:04.619 INFO:teuthology.orchestra.run.smithi069.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-14T16:36:04.659 INFO:teuthology.orchestra.run.smithi203.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-05-14T16:36:04.677 INFO:teuthology.orchestra.run.smithi069.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-05-14T16:36:04.684 INFO:teuthology.orchestra.run.smithi203.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-05-14T16:36:04.709 INFO:teuthology.orchestra.run.smithi203.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-14T16:36:04.735 INFO:teuthology.orchestra.run.smithi069.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 876 kB/s | 51 kB 00:00 2024-05-14T16:36:04.751 INFO:teuthology.orchestra.run.smithi203.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-14T16:36:04.803 INFO:teuthology.orchestra.run.smithi203.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-14T16:36:04.837 INFO:teuthology.orchestra.run.smithi069.stdout:(89/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.7 MB/s | 6.0 MB 00:01 2024-05-14T16:36:04.862 INFO:teuthology.orchestra.run.smithi069.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 428 kB/s | 54 kB 00:00 2024-05-14T16:36:04.929 INFO:teuthology.orchestra.run.smithi069.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.6 MB/s | 5.1 MB 00:00 2024-05-14T16:36:04.954 INFO:teuthology.orchestra.run.smithi069.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-05-14T16:36:04.980 INFO:teuthology.orchestra.run.smithi069.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-05-14T16:36:05.021 INFO:teuthology.orchestra.run.smithi069.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-05-14T16:36:05.055 INFO:teuthology.orchestra.run.smithi069.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 475 kB/s | 36 kB 00:00 2024-05-14T16:36:05.060 INFO:teuthology.orchestra.run.smithi203.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 8.9 MB/s | 2.3 MB 00:00 2024-05-14T16:36:05.080 INFO:teuthology.orchestra.run.smithi069.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 807 kB/s | 47 kB 00:00 2024-05-14T16:36:05.110 INFO:teuthology.orchestra.run.smithi203.stdout:(150/156): librbd1-19.0.0-3687.g9b638668.el9.x8 8.8 MB/s | 3.1 MB 00:00 2024-05-14T16:36:05.122 INFO:teuthology.orchestra.run.smithi069.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-05-14T16:36:05.147 INFO:teuthology.orchestra.run.smithi069.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 865 kB/s | 58 kB 00:00 2024-05-14T16:36:05.161 INFO:teuthology.orchestra.run.smithi203.stdout:(151/156): NetworkManager-libnm-1.47.90-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-05-14T16:36:05.181 INFO:teuthology.orchestra.run.smithi069.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-05-14T16:36:05.206 INFO:teuthology.orchestra.run.smithi069.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-14T16:36:05.211 INFO:teuthology.orchestra.run.smithi203.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 4.9 MB/s | 248 kB 00:00 2024-05-14T16:36:05.240 INFO:teuthology.orchestra.run.smithi069.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-05-14T16:36:05.245 INFO:teuthology.orchestra.run.smithi203.stdout:(153/156): NetworkManager-team-1.47.90-1.el9.x8 310 kB/s | 41 kB 00:00 2024-05-14T16:36:05.286 INFO:teuthology.orchestra.run.smithi203.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-05-14T16:36:05.307 INFO:teuthology.orchestra.run.smithi069.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-05-14T16:36:05.441 INFO:teuthology.orchestra.run.smithi069.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-05-14T16:36:05.453 INFO:teuthology.orchestra.run.smithi203.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-14T16:36:05.512 INFO:teuthology.orchestra.run.smithi203.stdout:(156/156): librados2-19.0.0-3687.g9b638668.el9. 4.2 MB/s | 3.4 MB 00:00 2024-05-14T16:36:05.517 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:36:05.517 INFO:teuthology.orchestra.run.smithi203.stdout:Total 12 MB/s | 226 MB 00:18 2024-05-14T16:36:05.616 INFO:teuthology.orchestra.run.smithi069.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-05-14T16:36:05.675 INFO:teuthology.orchestra.run.smithi069.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-05-14T16:36:05.750 INFO:teuthology.orchestra.run.smithi069.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-05-14T16:36:05.808 INFO:teuthology.orchestra.run.smithi069.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-05-14T16:36:05.892 INFO:teuthology.orchestra.run.smithi069.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-14T16:36:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-05-14T16:36:05.959 INFO:teuthology.orchestra.run.smithi069.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-14T16:36:05.984 INFO:teuthology.orchestra.run.smithi069.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-05-14T16:36:06.035 INFO:teuthology.orchestra.run.smithi069.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-05-14T16:36:06.102 INFO:teuthology.orchestra.run.smithi069.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 653 kB/s | 43 kB 00:00 2024-05-14T16:36:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-05-14T16:36:06.311 INFO:teuthology.orchestra.run.smithi069.stdout:(115/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-05-14T16:36:06.327 INFO:teuthology.orchestra.run.smithi069.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 75 kB/s | 15 kB 00:00 2024-05-14T16:36:06.344 INFO:teuthology.orchestra.run.smithi069.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 133 kB/s | 32 kB 00:00 2024-05-14T16:36:06.369 INFO:teuthology.orchestra.run.smithi069.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-14T16:36:06.411 INFO:teuthology.orchestra.run.smithi069.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-05-14T16:36:06.428 INFO:teuthology.orchestra.run.smithi069.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 602 kB/s | 35 kB 00:00 2024-05-14T16:36:06.471 INFO:teuthology.orchestra.run.smithi069.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-05-14T16:36:06.529 INFO:teuthology.orchestra.run.smithi069.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-14T16:36:06.571 INFO:teuthology.orchestra.run.smithi069.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-14T16:36:06.588 INFO:teuthology.orchestra.run.smithi069.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 181 kB/s | 11 kB 00:00 2024-05-14T16:36:06.614 INFO:teuthology.orchestra.run.smithi069.stdout:(125/156): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-05-14T16:36:06.631 INFO:teuthology.orchestra.run.smithi069.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 300 kB/s | 18 kB 00:00 2024-05-14T16:36:06.653 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-14T16:36:06.656 INFO:teuthology.orchestra.run.smithi069.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 345 kB/s | 23 kB 00:00 2024-05-14T16:36:06.681 INFO:teuthology.orchestra.run.smithi069.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-05-14T16:36:06.706 INFO:teuthology.orchestra.run.smithi069.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-14T16:36:06.740 INFO:teuthology.orchestra.run.smithi069.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-05-14T16:36:06.765 INFO:teuthology.orchestra.run.smithi069.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 552 kB/s | 46 kB 00:00 2024-05-14T16:36:06.790 INFO:teuthology.orchestra.run.smithi069.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 941 kB/s | 79 kB 00:00 2024-05-14T16:36:06.805 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-14T16:36:06.805 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-14T16:36:06.816 INFO:teuthology.orchestra.run.smithi069.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-05-14T16:36:06.841 INFO:teuthology.orchestra.run.smithi069.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-14T16:36:06.866 INFO:teuthology.orchestra.run.smithi069.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-05-14T16:36:06.891 INFO:teuthology.orchestra.run.smithi069.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-14T16:36:06.917 INFO:teuthology.orchestra.run.smithi069.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-05-14T16:36:06.942 INFO:teuthology.orchestra.run.smithi069.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-14T16:36:06.976 INFO:teuthology.orchestra.run.smithi069.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-14T16:36:07.001 INFO:teuthology.orchestra.run.smithi069.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-05-14T16:36:07.026 INFO:teuthology.orchestra.run.smithi069.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-05-14T16:36:07.051 INFO:teuthology.orchestra.run.smithi069.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-05-14T16:36:07.077 INFO:teuthology.orchestra.run.smithi069.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-14T16:36:07.102 INFO:teuthology.orchestra.run.smithi069.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-14T16:36:07.136 INFO:teuthology.orchestra.run.smithi069.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-05-14T16:36:07.161 INFO:teuthology.orchestra.run.smithi069.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-14T16:36:07.186 INFO:teuthology.orchestra.run.smithi069.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-14T16:36:07.237 INFO:teuthology.orchestra.run.smithi069.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-14T16:36:07.495 INFO:teuthology.orchestra.run.smithi069.stdout:(149/156): librados2-19.0.0-3687.g9b638668.el9. 10 MB/s | 3.4 MB 00:00 2024-05-14T16:36:07.545 INFO:teuthology.orchestra.run.smithi069.stdout:(150/156): NetworkManager-1.47.90-1.el9.x86_64. 7.5 MB/s | 2.3 MB 00:00 2024-05-14T16:36:07.595 INFO:teuthology.orchestra.run.smithi069.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 834 kB/s | 41 kB 00:00 2024-05-14T16:36:07.646 INFO:teuthology.orchestra.run.smithi069.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 4.8 MB/s | 248 kB 00:00 2024-05-14T16:36:07.747 INFO:teuthology.orchestra.run.smithi069.stdout:(153/156): librbd1-19.0.0-3687.g9b638668.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-05-14T16:36:07.788 INFO:teuthology.orchestra.run.smithi069.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 9.5 MB/s | 1.3 MB 00:00 2024-05-14T16:36:07.830 INFO:teuthology.orchestra.run.smithi069.stdout:(155/156): NetworkManager-libnm-1.47.90-1.el9.x 5.4 MB/s | 1.8 MB 00:00 2024-05-14T16:36:08.014 INFO:teuthology.orchestra.run.smithi069.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-05-14T16:36:08.017 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:36:08.017 INFO:teuthology.orchestra.run.smithi069.stdout:Total 10 MB/s | 226 MB 00:22 2024-05-14T16:36:08.609 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-14T16:36:08.610 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-14T16:36:09.171 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-05-14T16:36:09.319 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-05-14T16:36:09.319 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-05-14T16:36:10.484 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-14T16:36:10.589 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T16:36:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T16:36:10.840 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T16:36:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T16:36:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T16:36:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T16:36:11.263 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T16:36:11.366 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:36:11.469 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-05-14T16:36:11.470 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-05-14T16:36:11.488 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:36:11.509 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:36:11.599 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:36:11.621 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:36:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T16:36:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T16:36:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:36:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:36:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:36:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:36:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:36:12.115 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:36:12.291 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:36:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:36:12.563 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:36:12.624 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:36:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:36:12.833 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:36:13.060 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:36:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T16:36:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T16:36:13.238 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T16:36:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T16:36:13.509 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T16:36:13.665 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T16:36:13.697 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T16:36:13.726 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T16:36:13.794 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T16:36:13.847 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T16:36:13.878 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T16:36:13.913 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T16:36:13.989 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T16:36:14.024 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-14T16:36:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-14T16:36:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-14T16:36:14.165 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T16:36:14.211 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T16:36:14.285 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T16:36:14.417 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T16:36:14.472 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T16:36:14.762 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-14T16:36:14.850 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T16:36:14.888 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T16:36:14.950 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T16:36:14.984 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T16:36:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T16:36:15.009 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T16:36:15.053 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T16:36:15.076 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T16:36:15.105 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T16:36:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T16:36:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-14T16:36:15.228 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T16:36:15.374 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T16:36:15.390 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T16:36:15.447 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T16:36:15.510 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:36:15.536 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T16:36:15.605 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T16:36:15.632 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:36:15.669 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:36:15.678 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T16:36:15.709 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T16:36:15.760 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:36:15.765 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T16:36:15.799 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:36:15.817 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T16:36:15.853 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T16:36:15.952 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T16:36:15.980 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T16:36:16.004 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T16:36:16.023 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T16:36:16.033 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:36:16.144 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:36:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T16:36:16.167 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:36:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T16:36:16.229 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:36:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T16:36:16.251 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:36:16.274 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:36:16.274 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T16:36:16.360 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T16:36:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T16:36:16.435 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:36:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T16:36:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T16:36:16.490 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T16:36:16.563 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T16:36:16.605 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:36:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T16:36:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T16:36:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T16:36:16.766 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:36:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T16:36:16.827 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:36:16.832 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T16:36:16.863 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T16:36:16.894 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T16:36:16.922 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:36:16.928 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T16:36:16.962 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T16:36:17.003 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T16:36:17.044 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:36:17.046 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T16:36:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T16:36:17.176 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T16:36:17.208 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T16:36:17.252 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:36:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T16:36:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T16:36:17.369 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T16:36:17.398 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T16:36:17.401 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T16:36:17.433 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T16:36:17.507 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T16:36:17.549 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T16:36:17.593 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T16:36:17.625 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T16:36:17.654 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T16:36:17.724 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T16:36:17.727 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-05-14T16:36:17.784 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T16:36:17.815 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T16:36:17.818 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T16:36:17.850 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T16:36:17.856 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T16:36:17.926 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T16:36:17.927 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T16:36:17.961 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-14T16:36:17.968 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T16:36:17.994 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T16:36:18.004 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-14T16:36:18.033 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-14T16:36:18.045 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T16:36:18.105 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T16:36:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T16:36:18.148 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T16:36:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T16:36:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T16:36:18.223 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T16:36:18.314 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T16:36:18.356 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T16:36:18.365 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T16:36:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T16:36:18.409 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T16:36:18.452 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T16:36:18.510 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T16:36:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:36:18.642 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:36:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:36:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T16:36:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T16:36:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:36:18.784 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:36:18.793 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T16:36:18.939 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T16:36:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T16:36:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T16:36:19.021 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T16:36:19.026 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:36:19.049 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T16:36:19.092 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T16:36:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:36:19.130 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-14T16:36:19.151 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:36:19.172 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T16:36:19.214 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:36:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:36:19.250 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:36:19.323 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T16:36:19.384 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T16:36:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T16:36:19.446 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:36:19.475 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T16:36:19.555 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T16:36:19.556 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T16:36:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:36:19.629 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T16:36:19.660 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T16:36:19.696 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T16:36:19.733 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:36:19.743 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T16:36:19.779 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T16:36:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:36:19.831 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T16:36:19.873 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T16:36:19.890 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:36:19.970 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:36:20.011 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T16:36:20.061 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T16:36:20.092 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T16:36:20.133 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T16:36:20.180 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:36:20.229 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T16:36:20.266 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T16:36:20.285 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T16:36:20.307 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T16:36:20.316 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T16:36:20.340 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T16:36:20.350 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T16:36:20.365 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T16:36:20.423 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T16:36:20.440 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T16:36:20.467 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T16:36:20.510 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T16:36:20.543 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T16:36:20.568 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T16:36:20.571 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T16:36:20.597 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T16:36:20.625 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T16:36:20.638 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T16:36:20.660 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T16:36:20.692 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T16:36:20.698 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T16:36:20.723 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T16:36:20.739 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T16:36:20.758 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T16:36:20.768 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T16:36:20.774 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T16:36:20.803 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T16:36:20.834 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T16:36:20.836 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T16:36:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T16:36:20.878 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T16:36:20.878 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-14T16:36:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T16:36:20.913 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T16:36:20.917 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-14T16:36:20.931 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T16:36:20.950 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-14T16:36:21.000 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T16:36:21.020 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T16:36:21.043 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T16:36:21.064 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T16:36:21.082 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T16:36:21.129 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T16:36:21.139 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T16:36:21.228 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T16:36:21.254 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T16:36:21.273 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T16:36:21.283 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T16:36:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:36:21.325 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T16:36:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:36:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-14T16:36:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T16:36:21.855 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T16:36:21.937 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T16:36:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T16:36:21.967 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T16:36:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:36:22.010 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T16:36:22.012 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T16:36:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:36:22.056 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-14T16:36:22.064 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T16:36:22.075 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-14T16:36:22.098 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T16:36:22.098 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T16:36:22.198 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T16:36:22.248 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T16:36:22.255 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T16:36:22.300 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T16:36:22.336 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T16:36:22.375 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T16:36:22.390 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T16:36:22.456 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T16:36:22.528 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T16:36:22.554 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T16:36:22.560 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T16:36:22.589 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T16:36:22.596 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T16:36:22.643 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T16:36:22.668 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T16:36:22.679 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T16:36:22.723 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T16:36:22.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T16:36:22.899 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T16:36:22.944 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T16:36:22.975 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T16:36:23.016 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T16:36:23.109 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T16:36:23.142 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T16:36:23.183 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T16:36:23.215 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T16:36:23.240 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T16:36:23.323 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T16:36:23.341 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T16:36:23.447 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T16:36:23.465 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T16:36:23.497 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T16:36:23.525 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T16:36:23.560 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T16:36:23.598 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T16:36:23.630 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T16:36:23.660 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T16:36:23.694 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T16:36:23.728 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T16:36:23.770 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T16:36:23.804 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T16:36:23.890 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T16:36:23.934 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T16:36:23.966 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T16:36:24.012 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T16:36:24.109 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T16:36:24.167 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T16:36:24.295 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:36:24.346 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:36:24.687 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T16:36:24.777 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T16:36:24.805 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T16:36:24.838 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T16:36:24.886 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T16:36:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:36:24.930 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T16:36:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:36:24.964 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T16:36:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:36:25.023 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T16:36:25.023 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T16:36:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:25.051 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:36:25.061 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T16:36:25.122 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T16:36:25.150 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T16:36:25.201 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:36:25.208 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T16:36:25.241 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T16:36:25.275 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:36:25.306 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-14T16:36:25.342 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T16:36:25.418 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T16:36:25.447 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T16:36:25.525 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T16:36:25.819 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T16:36:25.845 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:36:25.916 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:36:25.947 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-14T16:36:26.167 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T16:36:26.279 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T16:36:27.521 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T16:36:27.676 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T16:36:27.703 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T16:36:27.737 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T16:36:28.048 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T16:36:28.091 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:36:28.188 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:36:28.229 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-14T16:36:28.252 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:36:28.266 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T16:36:28.285 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:36:28.743 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T16:36:28.778 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:36:28.827 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:36:28.832 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:36:28.863 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-14T16:36:28.863 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:36:28.942 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:36:28.942 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T16:36:28.942 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T16:36:28.943 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:28.990 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:36:31.186 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:36:31.219 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:36:31.774 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:36:31.814 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:36:31.925 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:36:31.926 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T16:36:31.926 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T16:36:31.926 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:31.957 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:36:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:36:36.497 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-05-14T16:36:36.497 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-05-14T16:36:36.497 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-05-14T16:36:36.497 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-05-14T16:36:36.497 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-05-14T16:36:36.497 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-05-14T16:36:36.497 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-05-14T16:36:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:36.675 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:36:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:36:36.691 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:36.691 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T16:36:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:36:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:36:38.171 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:38.171 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T16:36:38.171 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:36:38.171 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:36:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:36:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:36:38.426 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:38.426 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T16:36:38.426 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:36:38.426 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:36:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:38.871 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:36:38.893 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:36:38.894 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:38.894 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T16:36:38.894 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:36:38.894 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:36:38.894 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T16:36:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T16:36:39.126 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T16:36:39.314 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T16:36:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T16:36:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T16:36:39.562 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T16:36:39.710 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T16:36:39.746 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T16:36:39.786 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T16:36:39.814 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T16:36:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:36:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:36:40.312 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-14T16:36:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-14T16:36:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:36:40.771 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:36:40.771 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-05-14T16:36:40.771 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-05-14T16:36:40.771 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-05-14T16:36:40.771 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-05-14T16:36:40.771 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-05-14T16:36:40.771 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-05-14T16:36:40.771 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-05-14T16:36:40.772 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:40.936 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:36:40.951 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:36:40.951 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:40.951 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T16:36:40.952 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:36:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:36:42.430 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:36:42.444 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:36:42.456 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:36:42.456 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:42.457 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T16:36:42.457 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:36:42.457 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:36:42.457 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:42.471 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:36:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:36:42.715 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:36:42.738 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:36:42.738 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:42.738 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T16:36:42.738 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:36:42.738 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:36:42.738 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-14T16:36:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:36:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:36:42.806 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:42.806 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T16:36:42.806 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:36:42.806 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:36:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:36:42.861 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:36:42.962 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-14T16:36:43.014 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-14T16:36:43.201 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:36:43.224 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:36:43.224 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:43.224 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T16:36:43.224 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:36:43.224 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:36:43.224 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:43.329 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T16:36:43.386 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T16:36:43.442 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T16:36:43.629 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T16:36:43.683 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T16:36:43.771 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T16:36:43.868 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T16:36:43.938 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T16:36:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:36:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:36:43.999 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:43.999 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T16:36:43.999 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:36:43.999 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:36:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:44.008 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T16:36:44.039 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:36:44.039 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2024-05-14T16:36:44.039 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2024-05-14T16:36:44.039 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2024-05-14T16:36:44.040 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2024-05-14T16:36:44.040 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2024-05-14T16:36:44.040 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /root 2024-05-14T16:36:44.040 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2024-05-14T16:36:44.040 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:36:44.050 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T16:36:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:36:44.066 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:44.066 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T16:36:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:44.082 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T16:36:44.110 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T16:36:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:36:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:36:44.151 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:44.151 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T16:36:44.151 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:36:44.151 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:36:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:44.220 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:36:44.235 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:36:44.235 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:44.235 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T16:36:44.235 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:36:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:36:44.493 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:44.493 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T16:36:44.493 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:36:44.493 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:36:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:44.504 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:36:44.534 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:36:44.567 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-14T16:36:44.629 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-14T16:36:44.658 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:36:45.702 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:36:45.723 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:36:45.743 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:36:45.749 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:36:45.749 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:45.749 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T16:36:45.749 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:36:45.749 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:36:45.749 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:45.974 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:36:45.996 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:36:45.996 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:45.996 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T16:36:45.997 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:36:45.997 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:36:45.997 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:46.441 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:36:46.464 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:36:46.464 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:46.464 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T16:36:46.464 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:36:46.464 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:36:46.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:46.562 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T16:36:46.618 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T16:36:46.679 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T16:36:46.708 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:36:46.735 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:36:46.846 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:36:46.865 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T16:36:46.924 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T16:36:47.002 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T16:36:47.005 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-14T16:36:47.037 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:36:47.060 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:36:47.060 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:47.061 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T16:36:47.061 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:36:47.061 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:36:47.061 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:47.090 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T16:36:47.105 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:36:47.116 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:36:47.216 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-14T16:36:47.228 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T16:36:47.262 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-14T16:36:47.282 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T16:36:47.315 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T16:36:47.342 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T16:36:47.737 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:36:47.774 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:36:47.814 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-14T16:36:47.875 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-14T16:36:47.906 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:36:48.191 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T16:36:48.224 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:36:48.247 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:36:48.247 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:48.247 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T16:36:48.247 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:36:48.247 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:36:48.247 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:48.292 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:36:48.314 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:36:48.314 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:48.314 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T16:36:48.314 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:48.359 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:36:48.382 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:36:48.382 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:48.382 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T16:36:48.382 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:36:48.382 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:36:48.382 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:48.693 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:36:48.715 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:36:48.716 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:48.716 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T16:36:48.716 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:36:48.716 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:36:48.716 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:36:48.943 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:36:48.991 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:36:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-14T16:36:49.123 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-14T16:36:49.163 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-14T16:36:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T16:36:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T16:36:49.406 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-14T16:36:49.452 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T16:36:49.483 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-14T16:36:49.520 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T16:36:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T16:36:49.956 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:36:49.983 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:36:50.091 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:36:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:36:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:36:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:36:50.260 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-14T16:36:50.293 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:36:50.317 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:36:50.317 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:50.317 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T16:36:50.317 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:36:50.317 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:36:50.318 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:50.369 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:36:50.380 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:36:50.479 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-14T16:36:50.543 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-14T16:36:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-14T16:36:51.154 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T16:36:51.194 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T16:36:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T16:36:51.346 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T16:36:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T16:36:51.372 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:36:51.457 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:36:51.479 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T16:36:51.490 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T16:36:51.490 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:36:51.505 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:36:51.528 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:36:51.528 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:51.528 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T16:36:51.529 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:36:51.529 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:36:51.529 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:51.573 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:36:51.574 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:36:51.595 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:36:51.595 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:51.595 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T16:36:51.595 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T16:36:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:36:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:36:51.648 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:36:51.671 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:36:51.671 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:51.671 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T16:36:51.671 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:36:51.671 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:36:51.671 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:36:51.793 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:36:51.886 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:36:51.948 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:36:51.974 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:36:51.997 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:36:51.997 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:36:51.997 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T16:36:51.997 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:36:51.997 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:36:51.997 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:36:53.323 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-14T16:36:53.364 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-14T16:36:53.405 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-14T16:36:53.437 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T16:36:53.464 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T16:36:53.652 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-14T16:36:53.703 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T16:36:53.733 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-14T16:36:53.762 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T16:36:53.801 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T16:36:54.415 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:36:54.457 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:36:54.477 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:36:55.314 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-14T16:36:55.397 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T16:36:55.438 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T16:36:55.521 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T16:36:55.559 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T16:36:55.585 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T16:36:55.585 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:36:55.653 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:36:55.686 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T16:36:55.686 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:36:55.770 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:36:55.795 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T16:36:55.800 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:36:55.800 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:36:56.048 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:36:56.048 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:36:56.115 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:36:56.180 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:36:56.585 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-14T16:36:56.638 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-14T16:36:56.684 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-14T16:36:56.725 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T16:36:56.753 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T16:36:56.943 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-14T16:36:56.994 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T16:36:57.024 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-14T16:36:57.053 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T16:36:57.091 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T16:36:57.499 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:36:57.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-14T16:36:57.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-14T16:36:57.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-14T16:36:57.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-14T16:36:57.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-14T16:36:57.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-14T16:36:57.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-14T16:36:57.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:36:57.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-14T16:36:57.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-14T16:36:57.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-14T16:36:57.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-14T16:36:57.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-14T16:36:57.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-14T16:36:57.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-14T16:36:57.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-14T16:36:57.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-14T16:36:57.514 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-14T16:36:57.514 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-14T16:36:57.514 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-14T16:36:57.514 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-14T16:36:57.514 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-14T16:36:57.514 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-14T16:36:57.514 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-14T16:36:57.514 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-14T16:36:57.514 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-14T16:36:57.514 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T16:36:57.515 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T16:36:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T16:36:57.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T16:36:57.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T16:36:57.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T16:36:57.520 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T16:36:57.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T16:36:57.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T16:36:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T16:36:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:36:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:36:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:36:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-14T16:36:58.610 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T16:36:58.664 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T16:36:58.768 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T16:36:58.805 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T16:36:58.831 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T16:36:58.831 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:36:58.915 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:36:58.940 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T16:36:58.940 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:36:59.033 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:36:59.058 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T16:36:59.063 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:36:59.063 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:36:59.251 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:36:59.252 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:36:59.345 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:36:59.417 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:36:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T16:36:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:36:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T16:36:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T16:36:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T16:36:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T16:36:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T16:36:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T16:36:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T16:36:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T16:37:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T16:37:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T16:37:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T16:37:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-14T16:37:00.002 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-14T16:37:00.410 DEBUG:teuthology.parallel:result is None 2024-05-14T16:37:01.879 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-14T16:37:01.880 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-14T16:37:01.881 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-14T16:37:01.894 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-14T16:37:01.894 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-14T16:37:01.894 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-14T16:37:01.894 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-14T16:37:01.894 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-14T16:37:01.894 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-14T16:37:01.894 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-14T16:37:01.894 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T16:37:01.895 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T16:37:01.896 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T16:37:01.897 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T16:37:01.898 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T16:37:01.899 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T16:37:01.900 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T16:37:01.901 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T16:37:01.902 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T16:37:01.903 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T16:37:01.904 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T16:37:01.904 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T16:37:01.904 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T16:37:01.904 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T16:37:01.904 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T16:37:01.904 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T16:37:01.904 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T16:37:04.560 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:37:04.560 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T16:37:04.561 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:04.562 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T16:37:04.563 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T16:37:04.564 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T16:37:04.565 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T16:37:04.566 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T16:37:04.567 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T16:37:04.568 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.569 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T16:37:04.570 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T16:37:04.571 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T16:37:04.572 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T16:37:04.572 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:37:04.572 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-14T16:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-14T16:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-14T16:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-14T16:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-14T16:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-14T16:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-14T16:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-14T16:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-14T16:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-14T16:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-14T16:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T16:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T16:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T16:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T16:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T16:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T16:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T16:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T16:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T16:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T16:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T16:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T16:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T16:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T16:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T16:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T16:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T16:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T16:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T16:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T16:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T16:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T16:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T16:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T16:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T16:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T16:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T16:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T16:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T16:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T16:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T16:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T16:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T16:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T16:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T16:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T16:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T16:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T16:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T16:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T16:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T16:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T16:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T16:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T16:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T16:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T16:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T16:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T16:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T16:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T16:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T16:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T16:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T16:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T16:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-14T16:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T16:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-14T16:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T16:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T16:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T16:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T16:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T16:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T16:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T16:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T16:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T16:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T16:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T16:37:04.983 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T16:37:04.986 DEBUG:teuthology.parallel:result is None 2024-05-14T16:37:07.587 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:37:07.587 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:37:07.587 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-05-14T16:37:07.587 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T16:37:07.587 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T16:37:07.587 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T16:37:07.587 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T16:37:07.587 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.587 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.588 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.589 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T16:37:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.591 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T16:37:07.592 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T16:37:07.593 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T16:37:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T16:37:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T16:37:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T16:37:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T16:37:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T16:37:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T16:37:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T16:37:07.595 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T16:37:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T16:37:07.598 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T16:37:07.599 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.599 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.599 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:37:07.599 INFO:teuthology.orchestra.run.smithi069.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T16:37:07.599 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T16:37:07.599 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T16:37:07.599 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T16:37:07.599 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T16:37:07.599 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T16:37:07.600 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T16:37:07.600 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:37:07.600 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-05-14T16:37:08.005 DEBUG:teuthology.parallel:result is None 2024-05-14T16:37:08.005 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:37:08.151 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T16:37:08.173 INFO:teuthology.orchestra.run.smithi069.stdout:19.0.0-3687.g9b638668.el9 2024-05-14T16:37:08.174 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-14T16:37:08.174 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-14T16:37:08.176 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:37:08.388 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T16:37:08.413 INFO:teuthology.orchestra.run.smithi092.stdout:19.0.0-3687.g9b638668.el9 2024-05-14T16:37:08.413 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-14T16:37:08.413 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-14T16:37:08.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:37:08.641 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T16:37:08.666 INFO:teuthology.orchestra.run.smithi203.stdout:19.0.0-3687.g9b638668.el9 2024-05-14T16:37:08.667 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-14T16:37:08.667 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-14T16:37:08.668 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-14T16:37:08.668 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:37:08.669 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T16:37:08.698 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:37:08.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T16:37:08.729 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:37:08.730 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T16:37:08.760 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-14T16:37:08.761 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:37:08.761 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T16:37:08.787 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T16:37:08.853 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:37:08.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T16:37:08.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T16:37:08.947 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:37:08.947 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T16:37:08.975 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T16:37:09.043 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-14T16:37:09.043 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:37:09.044 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T16:37:09.069 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T16:37:09.134 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:37:09.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T16:37:09.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T16:37:09.227 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:37:09.227 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T16:37:09.256 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T16:37:09.322 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-14T16:37:09.322 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:37:09.323 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T16:37:09.348 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T16:37:09.412 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:37:09.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T16:37:09.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T16:37:09.505 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:37:09.505 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T16:37:09.534 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T16:37:09.601 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-14T16:37:09.695 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-14T16:37:09.695 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:37:09.695 INFO:tasks.cephadm:Cluster fsid is 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:37:09.696 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-14T16:37:09.696 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.69', 'mon.b': '172.21.15.92', 'mon.c': '172.21.15.203'} 2024-05-14T16:37:09.696 INFO:tasks.cephadm:First mon is mon.a on smithi069 2024-05-14T16:37:09.696 INFO:tasks.cephadm:First mgr is x 2024-05-14T16:37:09.696 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-14T16:37:09.696 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2024-05-14T16:37:09.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-05-14T16:37:09.750 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname $(hostname -s) 2024-05-14T16:37:09.778 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-14T16:37:09.779 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:37:09.997 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79371/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3687-g9b638668', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3687.g9b638668'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-14 15:14:57.349842', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiesce', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-14T16:37:10.213 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-quiesce, sha1 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:37:10.215 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm 2024-05-14T16:37:10.215 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm 2024-05-14T16:37:10.215 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T16:37:10.908 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 14 16:37 /home/ubuntu/cephtest/cephadm 2024-05-14T16:37:10.908 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T16:37:11.495 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 14 16:37 /home/ubuntu/cephtest/cephadm 2024-05-14T16:37:11.495 DEBUG:teuthology.orchestra.run.smithi203:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T16:37:11.973 INFO:teuthology.orchestra.run.smithi203.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 14 16:37 /home/ubuntu/cephtest/cephadm 2024-05-14T16:37:11.974 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T16:37:11.993 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T16:37:12.012 DEBUG:teuthology.orchestra.run.smithi203:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T16:37:12.045 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b on all hosts... 2024-05-14T16:37:12.045 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-14T16:37:12.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-14T16:37:12.055 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-14T16:37:12.921 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:37:12.936 INFO:teuthology.orchestra.run.smithi203.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:37:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:37:42.604 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-14T16:37:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-14T16:37:42.605 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-14T16:37:42.605 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-05-14T16:37:42.605 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-14T16:37:42.605 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-05-14T16:37:42.605 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-14T16:38:19.687 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:38:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-14T16:38:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-14T16:38:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2024-05-14T16:38:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-14T16:38:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T16:38:19.688 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:38:24.061 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-05-14T16:38:24.061 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-14T16:38:24.061 INFO:teuthology.orchestra.run.smithi203.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-14T16:38:24.061 INFO:teuthology.orchestra.run.smithi203.stdout: "repo_digests": [ 2024-05-14T16:38:24.061 INFO:teuthology.orchestra.run.smithi203.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-14T16:38:24.061 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2024-05-14T16:38:24.061 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-05-14T16:38:24.087 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2024-05-14T16:38:24.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-05-14T16:38:24.151 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph 2024-05-14T16:38:24.179 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2024-05-14T16:38:24.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-05-14T16:38:24.234 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /etc/ceph 2024-05-14T16:38:24.258 INFO:tasks.cephadm:Writing seed config... 2024-05-14T16:38:24.259 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-14T16:38:24.259 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-14T16:38:24.259 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-14T16:38:24.259 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-14T16:38:24.259 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-14T16:38:24.259 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-14T16:38:24.259 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-14T16:38:24.259 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-14T16:38:24.259 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-14T16:38:24.260 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-14T16:38:24.261 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-14T16:38:24.262 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-14T16:38:24.262 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-14T16:38:24.262 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-14T16:38:24.262 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-14T16:38:24.262 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-14T16:38:24.262 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-14T16:38:24.262 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-14T16:38:24.262 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-14T16:38:24.263 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:38:24.263 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-14T16:38:24.281 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 356e185e-1210-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-14T16:38:24.281 DEBUG:teuthology.orchestra.run.smithi069:mon.a> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.a.service 2024-05-14T16:38:24.323 DEBUG:teuthology.orchestra.run.smithi069:mgr.x> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.x.service 2024-05-14T16:38:24.366 INFO:tasks.cephadm:Bootstrapping... 2024-05-14T16:38:24.366 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b -v bootstrap --fsid 356e185e-1210-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.69 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-14T16:38:24.561 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:38:24.561 INFO:teuthology.orchestra.run.smithi069.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', '-v', 'bootstrap', '--fsid', '356e185e-1210-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.69', '--skip-admin-label'] 2024-05-14T16:38:24.561 INFO:teuthology.orchestra.run.smithi069.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-14T16:38:24.562 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying podman|docker is present... 2024-05-14T16:38:24.586 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 5.0.2 2024-05-14T16:38:24.586 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying lvm2 is present... 2024-05-14T16:38:24.587 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying time synchronization is in place... 2024-05-14T16:38:24.594 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-14T16:38:24.594 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-14T16:38:24.600 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-14T16:38:24.600 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-05-14T16:38:24.607 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2024-05-14T16:38:24.613 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2024-05-14T16:38:24.614 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2024-05-14T16:38:24.614 INFO:teuthology.orchestra.run.smithi069.stdout:Repeating the final host check... 2024-05-14T16:38:24.637 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 5.0.2 2024-05-14T16:38:24.637 INFO:teuthology.orchestra.run.smithi069.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-14T16:38:24.637 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl is present 2024-05-14T16:38:24.637 INFO:teuthology.orchestra.run.smithi069.stdout:lvcreate is present 2024-05-14T16:38:24.643 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-14T16:38:24.643 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-14T16:38:24.649 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-14T16:38:24.649 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-05-14T16:38:24.656 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2024-05-14T16:38:24.663 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2024-05-14T16:38:24.663 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2024-05-14T16:38:24.663 INFO:teuthology.orchestra.run.smithi069.stdout:Host looks OK 2024-05-14T16:38:24.663 INFO:teuthology.orchestra.run.smithi069.stdout:Cluster fsid: 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:38:24.663 INFO:teuthology.orchestra.run.smithi069.stdout:Acquiring lock 140115289832992 on /run/cephadm/356e185e-1210-11ef-bc98-c7b262605968.lock 2024-05-14T16:38:24.663 INFO:teuthology.orchestra.run.smithi069.stdout:Lock 140115289832992 acquired on /run/cephadm/356e185e-1210-11ef-bc98-c7b262605968.lock 2024-05-14T16:38:24.663 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 3300 ... 2024-05-14T16:38:24.664 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 6789 ... 2024-05-14T16:38:24.664 INFO:teuthology.orchestra.run.smithi069.stdout:Base mon IP(s) is [172.21.15.69:3300, 172.21.15.69:6789], mon addrv is [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-05-14T16:38:24.667 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.69 metric 100 2024-05-14T16:38:24.667 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.69 metric 100 2024-05-14T16:38:24.669 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-14T16:38:24.669 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-14T16:38:24.669 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1746sec hoplimit 64 pref medium 2024-05-14T16:38:24.672 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-14T16:38:24.672 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-14T16:38:24.672 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-14T16:38:24.672 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-14T16:38:24.672 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-05-14T16:38:24.672 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-14T16:38:24.673 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2024-05-14T16:38:24.673 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2024-05-14T16:38:24.673 INFO:teuthology.orchestra.run.smithi069.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-14T16:38:24.673 INFO:teuthology.orchestra.run.smithi069.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-14T16:38:24.674 INFO:teuthology.orchestra.run.smithi069.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:38:25.773 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a 2024-05-14T16:38:25.773 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:38:25.773 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Getting image source signatures 2024-05-14T16:38:25.773 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:959e8422c821254139eba7e305bc0e5ccc9179a9c74388bbc27d9cf0e33e5891 2024-05-14T16:38:25.774 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-14T16:38:25.774 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying config sha256:c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a 2024-05-14T16:38:25.774 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-14T16:38:26.719 INFO:teuthology.orchestra.run.smithi069.stdout:ceph: stdout ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T16:38:26.719 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph version: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T16:38:26.719 INFO:teuthology.orchestra.run.smithi069.stdout:Extracting ceph user uid/gid from container image... 2024-05-14T16:38:27.616 INFO:teuthology.orchestra.run.smithi069.stdout:stat: stdout 167 167 2024-05-14T16:38:27.616 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial keys... 2024-05-14T16:38:28.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQCDk0NmZqM4NxAAEAZ5sCV20gTKsr9xb62SLQ== 2024-05-14T16:38:29.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQCEk0Nmbo6tMhAAVKXrq/FTVJqDEgTIBWTo4g== 2024-05-14T16:38:30.416 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQCFk0NmtfJsKxAAIm7KF/ygrIKvwWsfqVfqVQ== 2024-05-14T16:38:30.417 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial monmap... 2024-05-14T16:38:31.305 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-14T16:38:31.305 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-14T16:38:31.305 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:38:31.305 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-14T16:38:31.305 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool for a [v2:172.21.15.69:3300,v1:172.21.15.69:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-14T16:38:31.305 INFO:teuthology.orchestra.run.smithi069.stdout:setting min_mon_release = quincy 2024-05-14T16:38:31.305 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: set fsid to 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:38:31.305 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-14T16:38:31.306 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:38:31.306 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mon... 2024-05-14T16:38:32.236 INFO:teuthology.orchestra.run.smithi069.stdout:create mon.a on 2024-05-14T16:38:32.430 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-14T16:38:32.584 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-14T16:38:32.757 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-356e185e-1210-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-356e185e-1210-11ef-bc98-c7b262605968.target. 2024-05-14T16:38:32.758 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-356e185e-1210-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-356e185e-1210-11ef-bc98-c7b262605968.target. 2024-05-14T16:38:33.003 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.a 2024-05-14T16:38:33.003 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-14T16:38:33.174 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-356e185e-1210-11ef-bc98-c7b262605968.target.wants/ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-356e185e-1210-11ef-bc98-c7b262605968@.service. 2024-05-14T16:38:33.736 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:38:33.736 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-05-14T16:38:33.743 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:38:33.743 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-05-14T16:38:33.744 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-05-14T16:38:33.744 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2024-05-14T16:38:33.744 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon to start... 2024-05-14T16:38:33.744 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon... 2024-05-14T16:38:33.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:33 smithi069 systemd[1]: Started Ceph mon.a for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:38:34.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout cluster: 2024-05-14T16:38:34.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout id: 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:38:34.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-14T16:38:34.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout services: 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.567772s) 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout data: 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pgs: 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.173+0000 7fd4de5cc700 1 Processor -- start 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.173+0000 7fd4de5cc700 1 -- start start 2024-05-14T16:38:34.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.174+0000 7fd4de5cc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d8109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.174+0000 7fd4de5cc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81097c0 con 0x7fd4d8108e60 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.174+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d8109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.174+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d8109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35290/0 (socket says 172.21.15.69:35290) 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.174+0000 7fd4d7fff700 1 -- 172.21.15.69:0/4005073549 learned_addr learned my addr 172.21.15.69:0/4005073549 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.174+0000 7fd4d7fff700 1 -- 172.21.15.69:0/4005073549 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d810a050 con 0x7fd4d8108e60 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.174+0000 7fd4d7fff700 1 --2- 172.21.15.69:0/4005073549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d8109280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098a0 tx=0x7fd4c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4d6e3059c86dfe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.175+0000 7fd4d6ffd700 1 -- 172.21.15.69:0/4005073549 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c0004160 con 0x7fd4d8108e60 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.175+0000 7fd4d6ffd700 1 -- 172.21.15.69:0/4005073549 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0004300 con 0x7fd4d8108e60 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.175+0000 7fd4d6ffd700 1 -- 172.21.15.69:0/4005073549 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c00036a0 con 0x7fd4d8108e60 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.175+0000 7fd4de5cc700 1 -- 172.21.15.69:0/4005073549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 msgr2=0x7fd4d8109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.175+0000 7fd4de5cc700 1 --2- 172.21.15.69:0/4005073549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d8109280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098a0 tx=0x7fd4c0009bb0 comp rx=0 tx=0).stop 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.175+0000 7fd4de5cc700 1 -- 172.21.15.69:0/4005073549 shutdown_connections 2024-05-14T16:38:34.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.175+0000 7fd4de5cc700 1 --2- 172.21.15.69:0/4005073549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d8109280 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.175+0000 7fd4de5cc700 1 -- 172.21.15.69:0/4005073549 >> 172.21.15.69:0/4005073549 conn(0x7fd4d8104450 msgr2=0x7fd4d8106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.175+0000 7fd4de5cc700 1 -- 172.21.15.69:0/4005073549 shutdown_connections 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.175+0000 7fd4de5cc700 1 -- 172.21.15.69:0/4005073549 wait complete. 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.176+0000 7fd4de5cc700 1 Processor -- start 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.176+0000 7fd4de5cc700 1 -- start start 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.176+0000 7fd4de5cc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.176+0000 7fd4de5cc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d810a520 con 0x7fd4d8108e60 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.176+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.176+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35302/0 (socket says 172.21.15.69:35302) 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.176+0000 7fd4d7fff700 1 -- 172.21.15.69:0/729470848 learned_addr learned my addr 172.21.15.69:0/729470848 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.177+0000 7fd4d7fff700 1 -- 172.21.15.69:0/729470848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d819cb70 con 0x7fd4d8108e60 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.177+0000 7fd4d7fff700 1 --2- 172.21.15.69:0/729470848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d819c630 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0000c00 tx=0x7fd4c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.177+0000 7fd4d57fa700 1 -- 172.21.15.69:0/729470848 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c0003db0 con 0x7fd4d8108e60 2024-05-14T16:38:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.177+0000 7fd4d57fa700 1 -- 172.21.15.69:0/729470848 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0011630 con 0x7fd4d8108e60 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.177+0000 7fd4d57fa700 1 -- 172.21.15.69:0/729470848 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c0011950 con 0x7fd4d8108e60 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.177+0000 7fd4de5cc700 1 -- 172.21.15.69:0/729470848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d819ce00 con 0x7fd4d8108e60 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.177+0000 7fd4de5cc700 1 -- 172.21.15.69:0/729470848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d819faf0 con 0x7fd4d8108e60 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.177+0000 7fd4d57fa700 1 -- 172.21.15.69:0/729470848 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4c001a460 con 0x7fd4d8108e60 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.177+0000 7fd4d57fa700 1 -- 172.21.15.69:0/729470848 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4c001b780 con 0x7fd4d8108e60 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.178+0000 7fd4de5cc700 1 -- 172.21.15.69:0/729470848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d8064ba0 con 0x7fd4d8108e60 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.180+0000 7fd4d57fa700 1 -- 172.21.15.69:0/729470848 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd4c0031020 con 0x7fd4d8108e60 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.250+0000 7fd4de5cc700 1 -- 172.21.15.69:0/729470848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd4d802e860 con 0x7fd4d8108e60 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.250+0000 7fd4d57fa700 1 -- 172.21.15.69:0/729470848 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd4c001b9d0 con 0x7fd4d8108e60 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.251+0000 7fd4de5cc700 1 -- 172.21.15.69:0/729470848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 msgr2=0x7fd4d819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.251+0000 7fd4de5cc700 1 --2- 172.21.15.69:0/729470848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d819c630 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0000c00 tx=0x7fd4c0004750 comp rx=0 tx=0).stop 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.251+0000 7fd4de5cc700 1 -- 172.21.15.69:0/729470848 shutdown_connections 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.251+0000 7fd4de5cc700 1 --2- 172.21.15.69:0/729470848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d8108e60 0x7fd4d819c630 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.251+0000 7fd4de5cc700 1 -- 172.21.15.69:0/729470848 >> 172.21.15.69:0/729470848 conn(0x7fd4d8104450 msgr2=0x7fd4d81931f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:34.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.251+0000 7fd4de5cc700 1 -- 172.21.15.69:0/729470848 shutdown_connections 2024-05-14T16:38:34.800 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:34.251+0000 7fd4de5cc700 1 -- 172.21.15.69:0/729470848 wait complete. 2024-05-14T16:38:34.800 INFO:teuthology.orchestra.run.smithi069.stdout:mon is available 2024-05-14T16:38:34.800 INFO:teuthology.orchestra.run.smithi069.stdout:Assimilating anything we can from ceph.conf... 2024-05-14T16:38:34.903 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: monmap epoch 1 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: last_changed 2024-05-14T16:38:30.757480+0000 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: created 2024-05-14T16:38:30.757480+0000 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: min_mon_release 19 (squid) 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: election_strategy: 1 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: fsmap 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: osdmap e1: 0 total, 0 up, 0 in 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: mgrmap e1: no daemons active 2024-05-14T16:38:34.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:34 smithi069 ceph-mon[31136]: from='client.? 172.21.15.69:0/729470848' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:35.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-14T16:38:35.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-14T16:38:35.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:35.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2024-05-14T16:38:35.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-14T16:38:35.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-14T16:38:35.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-14T16:38:35.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-14T16:38:35.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-14T16:38:35.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.212+0000 7fe444779700 1 Processor -- start 2024-05-14T16:38:35.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.212+0000 7fe444779700 1 -- start start 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.212+0000 7fe444779700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c1092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.212+0000 7fe444779700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c109830 con 0x7fe43c108ed0 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.212+0000 7fe442515700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c1092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.212+0000 7fe442515700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c1092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34780/0 (socket says 172.21.15.69:34780) 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.212+0000 7fe442515700 1 -- 172.21.15.69:0/1438871646 learned_addr learned my addr 172.21.15.69:0/1438871646 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.213+0000 7fe442515700 1 -- 172.21.15.69:0/1438871646 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c109fe0 con 0x7fe43c108ed0 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.213+0000 7fe442515700 1 --2- 172.21.15.69:0/1438871646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c1092f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe4380098a0 tx=0x7fe438009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba722285ccab592c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.213+0000 7fe441513700 1 -- 172.21.15.69:0/1438871646 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe438004160 con 0x7fe43c108ed0 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.213+0000 7fe441513700 1 -- 172.21.15.69:0/1438871646 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe438004300 con 0x7fe43c108ed0 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.213+0000 7fe441513700 1 -- 172.21.15.69:0/1438871646 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4380036a0 con 0x7fe43c108ed0 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.213+0000 7fe444779700 1 -- 172.21.15.69:0/1438871646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 msgr2=0x7fe43c1092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.213+0000 7fe444779700 1 --2- 172.21.15.69:0/1438871646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c1092f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe4380098a0 tx=0x7fe438009bb0 comp rx=0 tx=0).stop 2024-05-14T16:38:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.213+0000 7fe444779700 1 -- 172.21.15.69:0/1438871646 shutdown_connections 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.213+0000 7fe444779700 1 --2- 172.21.15.69:0/1438871646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c1092f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.213+0000 7fe444779700 1 -- 172.21.15.69:0/1438871646 >> 172.21.15.69:0/1438871646 conn(0x7fe43c1044b0 msgr2=0x7fe43c1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.214+0000 7fe444779700 1 -- 172.21.15.69:0/1438871646 shutdown_connections 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.214+0000 7fe444779700 1 -- 172.21.15.69:0/1438871646 wait complete. 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.214+0000 7fe444779700 1 Processor -- start 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.214+0000 7fe444779700 1 -- start start 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.214+0000 7fe444779700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.214+0000 7fe444779700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c10a4b0 con 0x7fe43c108ed0 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.214+0000 7fe442515700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.214+0000 7fe442515700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34790/0 (socket says 172.21.15.69:34790) 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.214+0000 7fe442515700 1 -- 172.21.15.69:0/2875583600 learned_addr learned my addr 172.21.15.69:0/2875583600 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:35.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.215+0000 7fe442515700 1 -- 172.21.15.69:0/2875583600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c19cac0 con 0x7fe43c108ed0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.215+0000 7fe442515700 1 --2- 172.21.15.69:0/2875583600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c19c580 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe438000c00 tx=0x7fe438004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.215+0000 7fe4337fe700 1 -- 172.21.15.69:0/2875583600 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe438003db0 con 0x7fe43c108ed0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.215+0000 7fe4337fe700 1 -- 172.21.15.69:0/2875583600 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe438011630 con 0x7fe43c108ed0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.215+0000 7fe4337fe700 1 -- 172.21.15.69:0/2875583600 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe438011950 con 0x7fe43c108ed0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.215+0000 7fe444779700 1 -- 172.21.15.69:0/2875583600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c19cd50 con 0x7fe43c108ed0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.215+0000 7fe444779700 1 -- 172.21.15.69:0/2875583600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c19fa40 con 0x7fe43c108ed0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.215+0000 7fe4337fe700 1 -- 172.21.15.69:0/2875583600 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe43801a460 con 0x7fe43c108ed0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.215+0000 7fe4337fe700 1 -- 172.21.15.69:0/2875583600 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe43801b780 con 0x7fe43c108ed0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.216+0000 7fe444779700 1 -- 172.21.15.69:0/2875583600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43c064ba0 con 0x7fe43c108ed0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.218+0000 7fe4337fe700 1 -- 172.21.15.69:0/2875583600 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe43801b980 con 0x7fe43c108ed0 2024-05-14T16:38:35.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.286+0000 7fe444779700 1 -- 172.21.15.69:0/2875583600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe43c02e7b0 con 0x7fe43c108ed0 2024-05-14T16:38:35.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.289+0000 7fe4337fe700 1 -- 172.21.15.69:0/2875583600 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fe43801bb60 con 0x7fe43c108ed0 2024-05-14T16:38:35.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.289+0000 7fe4337fe700 1 -- 172.21.15.69:0/2875583600 <== mon.0 v2:172.21.15.69:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe43801ad70 con 0x7fe43c108ed0 2024-05-14T16:38:35.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.290+0000 7fe444779700 1 -- 172.21.15.69:0/2875583600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 msgr2=0x7fe43c19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:35.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.290+0000 7fe444779700 1 --2- 172.21.15.69:0/2875583600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c19c580 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe438000c00 tx=0x7fe438004750 comp rx=0 tx=0).stop 2024-05-14T16:38:35.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.290+0000 7fe444779700 1 -- 172.21.15.69:0/2875583600 shutdown_connections 2024-05-14T16:38:35.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.290+0000 7fe444779700 1 --2- 172.21.15.69:0/2875583600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe43c108ed0 0x7fe43c19c580 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:35.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.290+0000 7fe444779700 1 -- 172.21.15.69:0/2875583600 >> 172.21.15.69:0/2875583600 conn(0x7fe43c1044b0 msgr2=0x7fe43c193180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:35.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.290+0000 7fe444779700 1 -- 172.21.15.69:0/2875583600 shutdown_connections 2024-05-14T16:38:35.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:35.290+0000 7fe444779700 1 -- 172.21.15.69:0/2875583600 wait complete. 2024-05-14T16:38:35.731 INFO:teuthology.orchestra.run.smithi069.stdout:Generating new minimal ceph.conf... 2024-05-14T16:38:35.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:35 smithi069 ceph-mon[31136]: from='client.? 172.21.15.69:0/2875583600' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-14T16:38:35.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:35 smithi069 ceph-mon[31136]: from='client.? 172.21.15.69:0/2875583600' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-14T16:38:36.765 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:36 smithi069 ceph-mon[31136]: from='client.? 172.21.15.69:0/1031866511' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:38:36.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.135+0000 7f1566ee6700 1 Processor -- start 2024-05-14T16:38:36.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.135+0000 7f1566ee6700 1 -- start start 2024-05-14T16:38:36.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.135+0000 7f1566ee6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f15601092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:36.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.135+0000 7f1566ee6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1560109830 con 0x7f1560108ed0 2024-05-14T16:38:36.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.135+0000 7f1564c82700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f15601092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:36.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.135+0000 7f1564c82700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f15601092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34802/0 (socket says 172.21.15.69:34802) 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.135+0000 7f1564c82700 1 -- 172.21.15.69:0/1805771348 learned_addr learned my addr 172.21.15.69:0/1805771348 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.136+0000 7f1564c82700 1 -- 172.21.15.69:0/1805771348 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1560109fe0 con 0x7f1560108ed0 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.136+0000 7f1564c82700 1 --2- 172.21.15.69:0/1805771348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f15601092f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f15540098a0 tx=0x7f1554009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3052dd01df0c9d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.136+0000 7f155f7fe700 1 -- 172.21.15.69:0/1805771348 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1554004160 con 0x7f1560108ed0 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.136+0000 7f155f7fe700 1 -- 172.21.15.69:0/1805771348 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1554004300 con 0x7f1560108ed0 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.136+0000 7f155f7fe700 1 -- 172.21.15.69:0/1805771348 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1554011af0 con 0x7f1560108ed0 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.136+0000 7f1566ee6700 1 -- 172.21.15.69:0/1805771348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 msgr2=0x7f15601092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.136+0000 7f1566ee6700 1 --2- 172.21.15.69:0/1805771348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f15601092f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f15540098a0 tx=0x7f1554009bb0 comp rx=0 tx=0).stop 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.136+0000 7f1566ee6700 1 -- 172.21.15.69:0/1805771348 shutdown_connections 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.136+0000 7f1566ee6700 1 --2- 172.21.15.69:0/1805771348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f15601092f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.136+0000 7f1566ee6700 1 -- 172.21.15.69:0/1805771348 >> 172.21.15.69:0/1805771348 conn(0x7f15601044b0 msgr2=0x7f15601068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.137+0000 7f1566ee6700 1 -- 172.21.15.69:0/1805771348 shutdown_connections 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.137+0000 7f1566ee6700 1 -- 172.21.15.69:0/1805771348 wait complete. 2024-05-14T16:38:36.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.137+0000 7f1566ee6700 1 Processor -- start 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.137+0000 7f1566ee6700 1 -- start start 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.137+0000 7f1566ee6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f156019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.137+0000 7f1566ee6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f156010d420 con 0x7f1560108ed0 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.137+0000 7f1564c82700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f156019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.137+0000 7f1564c82700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f156019ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34812/0 (socket says 172.21.15.69:34812) 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.137+0000 7f1564c82700 1 -- 172.21.15.69:0/1031866511 learned_addr learned my addr 172.21.15.69:0/1031866511 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.138+0000 7f1564c82700 1 -- 172.21.15.69:0/1031866511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156019d370 con 0x7f1560108ed0 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.138+0000 7f1564c82700 1 --2- 172.21.15.69:0/1031866511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f156019ce30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1554009fb0 tx=0x7f1554011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.138+0000 7f155dffb700 1 -- 172.21.15.69:0/1031866511 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15540036a0 con 0x7f1560108ed0 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.138+0000 7f155dffb700 1 -- 172.21.15.69:0/1031866511 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f155401a430 con 0x7f1560108ed0 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.138+0000 7f1566ee6700 1 -- 172.21.15.69:0/1031866511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156019d600 con 0x7f1560108ed0 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.138+0000 7f155dffb700 1 -- 172.21.15.69:0/1031866511 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1554019550 con 0x7f1560108ed0 2024-05-14T16:38:36.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.138+0000 7f1566ee6700 1 -- 172.21.15.69:0/1031866511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15601a02f0 con 0x7f1560108ed0 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.138+0000 7f155dffb700 1 -- 172.21.15.69:0/1031866511 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1554019b90 con 0x7f1560108ed0 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.138+0000 7f155dffb700 1 -- 172.21.15.69:0/1031866511 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1554024730 con 0x7f1560108ed0 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.139+0000 7f1566ee6700 1 -- 172.21.15.69:0/1031866511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1560064ba0 con 0x7f1560108ed0 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.141+0000 7f155dffb700 1 -- 172.21.15.69:0/1031866511 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1554043ec0 con 0x7f1560108ed0 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.208+0000 7f1566ee6700 1 -- 172.21.15.69:0/1031866511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f15601a0840 con 0x7f1560108ed0 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.208+0000 7f155dffb700 1 -- 172.21.15.69:0/1031866511 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f1554032030 con 0x7f1560108ed0 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.209+0000 7f1566ee6700 1 -- 172.21.15.69:0/1031866511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 msgr2=0x7f156019ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.209+0000 7f1566ee6700 1 --2- 172.21.15.69:0/1031866511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f156019ce30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1554009fb0 tx=0x7f1554011940 comp rx=0 tx=0).stop 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.209+0000 7f1566ee6700 1 -- 172.21.15.69:0/1031866511 shutdown_connections 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.209+0000 7f1566ee6700 1 --2- 172.21.15.69:0/1031866511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1560108ed0 0x7f156019ce30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.209+0000 7f1566ee6700 1 -- 172.21.15.69:0/1031866511 >> 172.21.15.69:0/1031866511 conn(0x7f15601044b0 msgr2=0x7f1560104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.209+0000 7f1566ee6700 1 -- 172.21.15.69:0/1031866511 shutdown_connections 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:36.209+0000 7f1566ee6700 1 -- 172.21.15.69:0/1031866511 wait complete. 2024-05-14T16:38:36.775 INFO:teuthology.orchestra.run.smithi069.stdout:Restarting the monitor... 2024-05-14T16:38:37.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:36 smithi069 systemd[1]: Stopping Ceph mon.a for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:38:37.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:37 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a[31132]: 2024-05-14T16:38:37.035+0000 7ff3c311b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T16:38:37.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:37 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a[31132]: 2024-05-14T16:38:37.035+0000 7ff3c311b700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-14T16:38:37.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:37 smithi069 podman[31382]: 2024-05-14 16:38:37.199280855 +0000 UTC m=+0.264993999 container died 6039f1581c5cedf4b691f3bbf519fd3a9a9801b4cf8ad6d901e1bde927a50996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True) 2024-05-14T16:38:37.818 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:37 smithi069 podman[31382]: 2024-05-14 16:38:37.568335146 +0000 UTC m=+0.634048292 container cleanup 6039f1581c5cedf4b691f3bbf519fd3a9a9801b4cf8ad6d901e1bde927a50996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:38:37.818 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:37 smithi069 bash[31382]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a 2024-05-14T16:38:38.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:37 smithi069 podman[31394]: 2024-05-14 16:38:37.819188201 +0000 UTC m=+0.609642423 container remove 6039f1581c5cedf4b691f3bbf519fd3a9a9801b4cf8ad6d901e1bde927a50996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-14T16:38:38.170 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:37 smithi069 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-14T16:38:38.170 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:37 smithi069 systemd[1]: Stopped Ceph mon.a for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:38:38.170 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:37 smithi069 systemd[1]: Starting Ceph mon.a for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:38:38.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 podman[31488]: 2024-05-14 16:38:38.170287562 +0000 UTC m=+0.100307831 container create 3fcc6b76fa602cf3d2d4361bb39e6d84ea3c6d1620c3d241d095e46da6ef2989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-05-14T16:38:38.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 podman[31488]: 2024-05-14 16:38:38.087194113 +0000 UTC m=+0.017214340 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:38:38.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 podman[31488]: 2024-05-14 16:38:38.380721597 +0000 UTC m=+0.310741824 container init 3fcc6b76fa602cf3d2d4361bb39e6d84ea3c6d1620c3d241d095e46da6ef2989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True) 2024-05-14T16:38:38.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 podman[31488]: 2024-05-14 16:38:38.383877796 +0000 UTC m=+0.313898026 container start 3fcc6b76fa602cf3d2d4361bb39e6d84ea3c6d1620c3d241d095e46da6ef2989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: set uid:gid to 167:167 (ceph:ceph) 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev), process ceph-mon, pid 2 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: pidfile_write: ignore empty --pid-file 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: load: jerasure load: lrc 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: RocksDB version: 7.9.2 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Git sha 0 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Compile date 2024-05-14 14:11:45 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: DB SUMMARY 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: DB Session ID: 1KPMIL3PXG21TAKAMDUY 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: CURRENT file: CURRENT 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: IDENTITY file: IDENTITY 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104530 ; 2024-05-14T16:38:38.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.error_if_exists: 0 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.create_if_missing: 0 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.paranoid_checks: 1 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.env: 0x55da48ef0cc0 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.fs: PosixFileSystem 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.info_log: 0x55da49b01da0 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_file_opening_threads: 16 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.statistics: (nil) 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.use_fsync: 0 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_log_file_size: 0 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-14T16:38:38.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.keep_log_file_num: 1000 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.recycle_log_file_num: 0 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.allow_fallocate: 1 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.allow_mmap_reads: 0 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.allow_mmap_writes: 0 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.use_direct_reads: 0 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.create_missing_column_families: 0 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.db_log_dir: 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.wal_dir: 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-14T16:38:38.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-14T16:38:38.458 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.advise_random_on_open: 1 2024-05-14T16:38:38.458 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.db_write_buffer_size: 0 2024-05-14T16:38:38.458 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.write_buffer_manager: 0x55da49ad30e0 2024-05-14T16:38:38.458 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-14T16:38:38.458 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-14T16:38:38.458 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-14T16:38:38.458 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.rate_limiter: (nil) 2024-05-14T16:38:38.458 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-14T16:38:38.459 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.wal_recovery_mode: 2 2024-05-14T16:38:38.459 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.enable_thread_tracking: 0 2024-05-14T16:38:38.459 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.enable_pipelined_write: 0 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.unordered_write: 0 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.row_cache: None 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.wal_filter: None 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.allow_ingest_behind: 0 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.two_write_queues: 0 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.manual_wal_flush: 0 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.wal_compression: 0 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.atomic_flush: 0 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-14T16:38:38.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.log_readahead_size: 0 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.best_efforts_recovery: 0 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.allow_data_in_errors: 0 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.db_host_id: __hostname__ 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_background_jobs: 2 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_background_compactions: -1 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_subcompactions: 1 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-14T16:38:38.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_total_wal_size: 0 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_open_files: -1 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bytes_per_sync: 0 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_readahead_size: 0 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_background_flushes: -1 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Compression algorithms supported: 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-14T16:38:38.462 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: kZSTD supported: 0 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: kXpressCompression supported: 0 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: kLZ4HCCompression supported: 1 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: kZlibCompression supported: 1 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: kSnappyCompression supported: 1 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: kLZ4Compression supported: 1 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: kBZip2Compression supported: 0 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.merge_operator: 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_filter: None 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_filter_factory: None 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.sst_partitioner_factory: None 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-14T16:38:38.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55da49b01f60) 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks: 1 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_top_level_index_and_filter: 1 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_type: 0 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_index_type: 0 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_shortening: 1 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: checksum: 4 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: no_block_cache: 0 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache: 0x55da49b051f0 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_name: BinnedLRUCache 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_options: 2024-05-14T16:38:38.464 INFO:journalctl@ceph.mon.a.smithi069.stdout: capacity : 536870912 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_shard_bits : 4 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: strict_capacity_limit : 0 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: high_pri_pool_ratio: 0.000 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_compressed: (nil) 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: persistent_cache: (nil) 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size: 4096 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size_deviation: 10 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_restart_interval: 16 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_block_restart_interval: 1 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: metadata_block_size: 4096 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: partition_filters: 0 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: use_delta_encoding: 1 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: filter_policy: bloomfilter 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: whole_key_filtering: 1 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: verify_compression: 0 2024-05-14T16:38:38.465 INFO:journalctl@ceph.mon.a.smithi069.stdout: read_amp_bytes_per_bit: 0 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout: format_version: 5 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_index_compression: 1 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_align: 0 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout: max_auto_readahead_size: 262144 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout: prepopulate_block_cache: 0 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout: initial_auto_readahead_size: 8192 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_file_reads_for_auto_readahead: 2 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.write_buffer_size: 33554432 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_write_buffer_number: 2 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compression: NoCompression 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bottommost_compression: Disabled 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.prefix_extractor: nullptr 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-14T16:38:38.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.num_levels: 7 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compression_opts.level: 32767 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compression_opts.strategy: 0 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-14T16:38:38.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compression_opts.enabled: false 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.target_file_size_base: 67108864 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-14T16:38:38.468 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.arena_block_size: 1048576 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.disable_auto_compactions: 0 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-14T16:38:38.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.inplace_update_support: 0 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-14T16:38:38.470 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.bloom_locality: 0 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.max_successive_merges: 0 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.paranoid_file_checks: 0 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.force_consistency_checks: 1 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.report_bg_io_stats: 0 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.ttl: 2592000 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.enable_blob_files: false 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.min_blob_size: 0 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.blob_file_size: 268435456 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-14T16:38:38.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.blob_file_starting_level: 0 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c51a5782-12f0-4f8a-afa5-c2025ac56039 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715704718414001, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715704718415187, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99757, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97639, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14028, "raw_average_key_size": 49, "raw_value_size": 89997, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715704718, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c51a5782-12f0-4f8a-afa5-c2025ac56039", "db_session_id": "1KPMIL3PXG21TAKAMDUY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715704718415268, "job": 1, "event": "recovery_finished"} 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55da49c22000 2024-05-14T16:38:38.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: DB pointer 0x55da49c0e000 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** DB Stats ** 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: L0 2/0 99.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: Sum 2/0 99.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:38:38.473 INFO:journalctl@ceph.mon.a.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative compaction: 0.00 GB write, 29.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T16:38:38.474 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval compaction: 0.00 GB write, 29.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache BinnedLRUCache@0x55da49b051f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: starting mon.a rank 0 at public addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] at bind addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: mon.a@-1(???) e1 preinit fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: mon.a@-1(???).mds e1 new map 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: mon.a@-1(???).mds e1 print_map 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: e1 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: legacy client fscid: -1 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout: No filesystems configured 2024-05-14T16:38:38.475 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T16:38:38.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:38:38.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:38:38.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:38:38.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-14T16:38:38.476 INFO:teuthology.orchestra.run.smithi069.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-14T16:38:38.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 bash[31488]: 3fcc6b76fa602cf3d2d4361bb39e6d84ea3c6d1620c3d241d095e46da6ef2989 2024-05-14T16:38:38.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-14T16:38:38.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: monmap epoch 1 2024-05-14T16:38:38.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:38:38.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: last_changed 2024-05-14T16:38:30.757480+0000 2024-05-14T16:38:38.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: created 2024-05-14T16:38:30.757480+0000 2024-05-14T16:38:38.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: min_mon_release 19 (squid) 2024-05-14T16:38:38.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: election_strategy: 1 2024-05-14T16:38:38.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-05-14T16:38:38.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: fsmap 2024-05-14T16:38:38.776 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 systemd[1]: Started Ceph mon.a for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:38:38.776 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: osdmap e1: 0 total, 0 up, 0 in 2024-05-14T16:38:38.776 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:38 smithi069 ceph-mon[31507]: mgrmap e1: no daemons active 2024-05-14T16:38:39.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.879+0000 7fd9881be700 1 Processor -- start 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.879+0000 7fd9881be700 1 -- start start 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.879+0000 7fd9881be700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd980107c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.880+0000 7fd9881be700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd98007d5d0 con 0x7fd980107800 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.880+0000 7fd985f5a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd980107c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.880+0000 7fd985f5a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd980107c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34826/0 (socket says 172.21.15.69:34826) 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.880+0000 7fd985f5a700 1 -- 172.21.15.69:0/712402765 learned_addr learned my addr 172.21.15.69:0/712402765 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.880+0000 7fd985f5a700 1 -- 172.21.15.69:0/712402765 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd980108160 con 0x7fd980107800 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.880+0000 7fd985f5a700 1 --2- 172.21.15.69:0/712402765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd980107c20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0098a0 tx=0x7fd97c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc24702d881e1d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.880+0000 7fd984f58700 1 -- 172.21.15.69:0/712402765 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd97c004160 con 0x7fd980107800 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.880+0000 7fd984f58700 1 -- 172.21.15.69:0/712402765 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd97c004300 con 0x7fd980107800 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.880+0000 7fd984f58700 1 -- 172.21.15.69:0/712402765 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd97c011af0 con 0x7fd980107800 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.881+0000 7fd9881be700 1 -- 172.21.15.69:0/712402765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 msgr2=0x7fd980107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.881+0000 7fd9881be700 1 --2- 172.21.15.69:0/712402765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd980107c20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0098a0 tx=0x7fd97c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:38:39.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.881+0000 7fd9881be700 1 -- 172.21.15.69:0/712402765 shutdown_connections 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.881+0000 7fd9881be700 1 --2- 172.21.15.69:0/712402765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd980107c20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.881+0000 7fd9881be700 1 -- 172.21.15.69:0/712402765 >> 172.21.15.69:0/712402765 conn(0x7fd9801033e0 msgr2=0x7fd980105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.881+0000 7fd9881be700 1 -- 172.21.15.69:0/712402765 shutdown_connections 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.881+0000 7fd9881be700 1 -- 172.21.15.69:0/712402765 wait complete. 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.882+0000 7fd9881be700 1 Processor -- start 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.882+0000 7fd9881be700 1 -- start start 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.882+0000 7fd9881be700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd98019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.882+0000 7fd9881be700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd980109190 con 0x7fd980107800 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.882+0000 7fd985f5a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd98019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.882+0000 7fd985f5a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd98019eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34838/0 (socket says 172.21.15.69:34838) 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.882+0000 7fd985f5a700 1 -- 172.21.15.69:0/2079422003 learned_addr learned my addr 172.21.15.69:0/2079422003 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.882+0000 7fd985f5a700 1 -- 172.21.15.69:0/2079422003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98019f530 con 0x7fd980107800 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.882+0000 7fd985f5a700 1 --2- 172.21.15.69:0/2079422003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd98019eff0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009fb0 tx=0x7fd97c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.883+0000 7fd976ffd700 1 -- 172.21.15.69:0/2079422003 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd97c0036a0 con 0x7fd980107800 2024-05-14T16:38:39.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.883+0000 7fd976ffd700 1 -- 172.21.15.69:0/2079422003 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd97c01a430 con 0x7fd980107800 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.883+0000 7fd9881be700 1 -- 172.21.15.69:0/2079422003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98019f7c0 con 0x7fd980107800 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.883+0000 7fd976ffd700 1 -- 172.21.15.69:0/2079422003 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd97c019550 con 0x7fd980107800 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.883+0000 7fd9881be700 1 -- 172.21.15.69:0/2079422003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9801a24b0 con 0x7fd980107800 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.883+0000 7fd976ffd700 1 -- 172.21.15.69:0/2079422003 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd97c019b90 con 0x7fd980107800 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.883+0000 7fd976ffd700 1 -- 172.21.15.69:0/2079422003 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd97c024730 con 0x7fd980107800 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.883+0000 7fd9881be700 1 -- 172.21.15.69:0/2079422003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd980064ba0 con 0x7fd980107800 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.886+0000 7fd976ffd700 1 -- 172.21.15.69:0/2079422003 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd97c043ec0 con 0x7fd980107800 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.951+0000 7fd9881be700 1 -- 172.21.15.69:0/2079422003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fd98002e7e0 con 0x7fd980107800 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.953+0000 7fd976ffd700 1 -- 172.21.15.69:0/2079422003 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fd97c032030 con 0x7fd980107800 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.954+0000 7fd9881be700 1 -- 172.21.15.69:0/2079422003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 msgr2=0x7fd98019eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.954+0000 7fd9881be700 1 --2- 172.21.15.69:0/2079422003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd98019eff0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009fb0 tx=0x7fd97c011940 comp rx=0 tx=0).stop 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.954+0000 7fd9881be700 1 -- 172.21.15.69:0/2079422003 shutdown_connections 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.954+0000 7fd9881be700 1 --2- 172.21.15.69:0/2079422003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd980107800 0x7fd98019eff0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.954+0000 7fd9881be700 1 -- 172.21.15.69:0/2079422003 >> 172.21.15.69:0/2079422003 conn(0x7fd9801033e0 msgr2=0x7fd980195c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.954+0000 7fd9881be700 1 -- 172.21.15.69:0/2079422003 shutdown_connections 2024-05-14T16:38:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:38.954+0000 7fd9881be700 1 -- 172.21.15.69:0/2079422003 wait complete. 2024-05-14T16:38:39.505 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-14T16:38:39.519 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-14T16:38:39.519 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mgr... 2024-05-14T16:38:39.520 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-14T16:38:39.520 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-14T16:38:39.741 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.x 2024-05-14T16:38:39.741 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-14T16:38:39.883 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-356e185e-1210-11ef-bc98-c7b262605968.target.wants/ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-356e185e-1210-11ef-bc98-c7b262605968@.service. 2024-05-14T16:38:40.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:39 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2079422003' entity='client.admin' 2024-05-14T16:38:40.389 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:38:40.389 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-05-14T16:38:40.397 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:38:40.397 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-05-14T16:38:40.397 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-05-14T16:38:40.397 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2024-05-14T16:38:40.406 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:38:40.406 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-05-14T16:38:40.415 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:38:40.415 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-05-14T16:38:40.415 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-05-14T16:38:40.415 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-14T16:38:40.415 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr to start... 2024-05-14T16:38:40.415 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr... 2024-05-14T16:38:40.453 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:40 smithi069 systemd[1]: Started Ceph mgr.x for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:38:40.777 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:40 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:40.484+0000 7f162d823200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:38:40.777 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:40 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:40.672+0000 7f162d823200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:38:40.777 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:40 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:40.776+0000 7f162d823200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:38:41.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:41 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1907591058' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T16:38:41.103 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:41 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:41.102+0000 7f162d823200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:38:41.453 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:41 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:41.193+0000 7f162d823200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:38:41.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:41.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T16:38:41.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T16:38:41.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T16:38:41.529 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T16:38:41.529 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T16:38:41.529 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T16:38:41.529 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:41.529 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T16:38:41.529 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T16:38:41.529 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T16:38:41.529 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T16:38:41.529 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T16:38:41.529 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T16:38:41.530 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T16:38:33.685055+0000", 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:38:41.531 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.886+0000 7f75f6f26700 1 Processor -- start 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.886+0000 7f75f6f26700 1 -- start start 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.887+0000 7f75f6f26700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f007dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.887+0000 7f75f6f26700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75f007e010 con 0x7f75f007f620 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.887+0000 7f75f4cc2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f007dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.887+0000 7f75f4cc2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f007dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34852/0 (socket says 172.21.15.69:34852) 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.887+0000 7f75f4cc2700 1 -- 172.21.15.69:0/1381846987 learned_addr learned my addr 172.21.15.69:0/1381846987 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.887+0000 7f75f4cc2700 1 -- 172.21.15.69:0/1381846987 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f007e190 con 0x7f75f007f620 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.887+0000 7f75f4cc2700 1 --2- 172.21.15.69:0/1381846987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f007dad0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f75e40098a0 tx=0x7f75e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facc3032fbb3db22 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.888+0000 7f75ef7fe700 1 -- 172.21.15.69:0/1381846987 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75e4004160 con 0x7f75f007f620 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.888+0000 7f75ef7fe700 1 -- 172.21.15.69:0/1381846987 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f75e4004300 con 0x7f75f007f620 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.888+0000 7f75ef7fe700 1 -- 172.21.15.69:0/1381846987 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75e4011af0 con 0x7f75f007f620 2024-05-14T16:38:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.888+0000 7f75f6f26700 1 -- 172.21.15.69:0/1381846987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 msgr2=0x7f75f007dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.888+0000 7f75f6f26700 1 --2- 172.21.15.69:0/1381846987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f007dad0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f75e40098a0 tx=0x7f75e4009bb0 comp rx=0 tx=0).stop 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.888+0000 7f75f6f26700 1 -- 172.21.15.69:0/1381846987 shutdown_connections 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.888+0000 7f75f6f26700 1 --2- 172.21.15.69:0/1381846987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f007dad0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.888+0000 7f75f6f26700 1 -- 172.21.15.69:0/1381846987 >> 172.21.15.69:0/1381846987 conn(0x7f75f0104500 msgr2=0x7f75f0106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.888+0000 7f75f6f26700 1 -- 172.21.15.69:0/1381846987 shutdown_connections 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.888+0000 7f75f6f26700 1 -- 172.21.15.69:0/1381846987 wait complete. 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.889+0000 7f75f6f26700 1 Processor -- start 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.889+0000 7f75f6f26700 1 -- start start 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.889+0000 7f75f6f26700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.889+0000 7f75f6f26700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75f010f4b0 con 0x7f75f007f620 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.889+0000 7f75f4cc2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.889+0000 7f75f4cc2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34862/0 (socket says 172.21.15.69:34862) 2024-05-14T16:38:41.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.889+0000 7f75f4cc2700 1 -- 172.21.15.69:0/1907591058 learned_addr learned my addr 172.21.15.69:0/1907591058 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.890+0000 7f75f4cc2700 1 -- 172.21.15.69:0/1907591058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f019f570 con 0x7f75f007f620 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.890+0000 7f75f4cc2700 1 --2- 172.21.15.69:0/1907591058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f019f030 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f75e4011ac0 tx=0x7f75e4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.890+0000 7f75edffb700 1 -- 172.21.15.69:0/1907591058 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75e4003970 con 0x7f75f007f620 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.890+0000 7f75f6f26700 1 -- 172.21.15.69:0/1907591058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f019f800 con 0x7f75f007f620 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.890+0000 7f75edffb700 1 -- 172.21.15.69:0/1907591058 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f75e4003b10 con 0x7f75f007f620 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.890+0000 7f75edffb700 1 -- 172.21.15.69:0/1907591058 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75e4024610 con 0x7f75f007f620 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.890+0000 7f75f6f26700 1 -- 172.21.15.69:0/1907591058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f01a24f0 con 0x7f75f007f620 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.890+0000 7f75edffb700 1 -- 172.21.15.69:0/1907591058 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f75e4024c20 con 0x7f75f007f620 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.890+0000 7f75edffb700 1 -- 172.21.15.69:0/1907591058 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f75e4019700 con 0x7f75f007f620 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.891+0000 7f75f6f26700 1 -- 172.21.15.69:0/1907591058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f0064ba0 con 0x7f75f007f620 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.895+0000 7f75edffb700 1 -- 172.21.15.69:0/1907591058 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f75e401a9e0 con 0x7f75f007f620 2024-05-14T16:38:41.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.969+0000 7f75f6f26700 1 -- 172.21.15.69:0/1907591058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f75f01a2a40 con 0x7f75f007f620 2024-05-14T16:38:41.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.970+0000 7f75edffb700 1 -- 172.21.15.69:0/1907591058 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f75e401a430 con 0x7f75f007f620 2024-05-14T16:38:41.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.971+0000 7f75f6f26700 1 -- 172.21.15.69:0/1907591058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 msgr2=0x7f75f019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:41.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.971+0000 7f75f6f26700 1 --2- 172.21.15.69:0/1907591058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f019f030 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f75e4011ac0 tx=0x7f75e4011ba0 comp rx=0 tx=0).stop 2024-05-14T16:38:41.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.971+0000 7f75f6f26700 1 -- 172.21.15.69:0/1907591058 shutdown_connections 2024-05-14T16:38:41.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.971+0000 7f75f6f26700 1 --2- 172.21.15.69:0/1907591058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f007f620 0x7f75f019f030 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:41.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.971+0000 7f75f6f26700 1 -- 172.21.15.69:0/1907591058 >> 172.21.15.69:0/1907591058 conn(0x7f75f0104500 msgr2=0x7f75f0105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:41.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.971+0000 7f75f6f26700 1 -- 172.21.15.69:0/1907591058 shutdown_connections 2024-05-14T16:38:41.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:40.971+0000 7f75f6f26700 1 -- 172.21.15.69:0/1907591058 wait complete. 2024-05-14T16:38:41.535 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (1/15)... 2024-05-14T16:38:42.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:42 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:42.391+0000 7f162d823200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:38:42.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:42 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:42.558+0000 7f162d823200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:38:43.014 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:42 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:42.728+0000 7f162d823200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:38:43.014 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:43 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:43.014+0000 7f162d823200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:38:43.309 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:43 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:43.105+0000 7f162d823200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:38:43.309 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:43 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:43.309+0000 7f162d823200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:38:43.631 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:43 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:43.502+0000 7f162d823200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:38:44.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:44 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/445358770' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T16:38:44.703 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:44 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:44.165+0000 7f162d823200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:38:44.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:44 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:44.397+0000 7f162d823200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:38:45.203 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:44 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:44.824+0000 7f162d823200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:38:45.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:44 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:44.920+0000 7f162d823200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:38:45.647 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:45 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:45.293+0000 7f162d823200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:38:45.953 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:45 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:45.647+0000 7f162d823200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:38:45.954 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:45 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:45.739+0000 7f162d823200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:38:46.453 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:45 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:45.984+0000 7f162d823200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:38:46.454 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:46 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:46.122+0000 7f162d823200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:38:46.454 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:46 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:46.218+0000 7f162d823200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:38:46.911 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:46 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:46.547+0000 7f162d823200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:38:46.911 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:46 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:46.639+0000 7f162d823200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:38:47.531 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:47.255+0000 7f162d823200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:38:47.531 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:47.476+0000 7f162d823200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:38:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: Activating manager daemon x 2024-05-14T16:38:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: mgrmap e2: x(active, starting, since 0.00347477s) 2024-05-14T16:38:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: from='mgr.14100 172.21.15.69:0/3155295840' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:38:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: from='mgr.14100 172.21.15.69:0/3155295840' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:38:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: from='mgr.14100 172.21.15.69:0/3155295840' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:38:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: from='mgr.14100 172.21.15.69:0/3155295840' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:38:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: from='mgr.14100 172.21.15.69:0/3155295840' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:38:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: Manager daemon x is now available 2024-05-14T16:38:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: from='mgr.14100 172.21.15.69:0/3155295840' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:38:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: from='mgr.14100 172.21.15.69:0/3155295840' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:38:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: from='mgr.14100 172.21.15.69:0/3155295840' entity='mgr.x' 2024-05-14T16:38:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: from='mgr.14100 172.21.15.69:0/3155295840' entity='mgr.x' 2024-05-14T16:38:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:47 smithi069 ceph-mon[31507]: from='mgr.14100 172.21.15.69:0/3155295840' entity='mgr.x' 2024-05-14T16:38:49.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:49 smithi069 ceph-mon[31507]: mgrmap e3: x(active, since 1.00802s) 2024-05-14T16:38:49.801 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:49.801 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:38:49.801 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:38:49.801 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T16:38:49.801 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T16:38:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T16:38:49.803 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:49.803 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T16:38:49.803 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:49.803 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T16:38:49.803 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T16:38:49.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T16:38:49.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T16:38:33.685055+0000", 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.937+0000 7f0efe845700 1 Processor -- start 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.937+0000 7f0efe845700 1 -- start start 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.937+0000 7f0efe845700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef8107be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:49.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.937+0000 7f0efe845700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef807d5d0 con 0x7f0ef81077c0 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.937+0000 7f0ef7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef8107be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.937+0000 7f0ef7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef8107be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34864/0 (socket says 172.21.15.69:34864) 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.937+0000 7f0ef7fff700 1 -- 172.21.15.69:0/264115498 learned_addr learned my addr 172.21.15.69:0/264115498 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.938+0000 7f0ef7fff700 1 -- 172.21.15.69:0/264115498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef8108120 con 0x7f0ef81077c0 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.938+0000 7f0ef7fff700 1 --2- 172.21.15.69:0/264115498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef8107be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00098a0 tx=0x7f0ee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e750953f1c3f62 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.938+0000 7f0ef6ffd700 1 -- 172.21.15.69:0/264115498 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee0004160 con 0x7f0ef81077c0 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.938+0000 7f0ef6ffd700 1 -- 172.21.15.69:0/264115498 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0ee0004300 con 0x7f0ef81077c0 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.938+0000 7f0ef6ffd700 1 -- 172.21.15.69:0/264115498 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee0011af0 con 0x7f0ef81077c0 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.939+0000 7f0efe845700 1 -- 172.21.15.69:0/264115498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 msgr2=0x7f0ef8107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.939+0000 7f0efe845700 1 --2- 172.21.15.69:0/264115498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef8107be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00098a0 tx=0x7f0ee0009bb0 comp rx=0 tx=0).stop 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.939+0000 7f0efe845700 1 -- 172.21.15.69:0/264115498 shutdown_connections 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.939+0000 7f0efe845700 1 --2- 172.21.15.69:0/264115498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef8107be0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:49.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.939+0000 7f0efe845700 1 -- 172.21.15.69:0/264115498 >> 172.21.15.69:0/264115498 conn(0x7f0ef8103380 msgr2=0x7f0ef81057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.939+0000 7f0efe845700 1 -- 172.21.15.69:0/264115498 shutdown_connections 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.939+0000 7f0efe845700 1 -- 172.21.15.69:0/264115498 wait complete. 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.939+0000 7f0efe845700 1 Processor -- start 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.939+0000 7f0efe845700 1 -- start start 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0efe845700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0efe845700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8109150 con 0x7f0ef81077c0 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0ef7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0ef7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34866/0 (socket says 172.21.15.69:34866) 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0ef7fff700 1 -- 172.21.15.69:0/445358770 learned_addr learned my addr 172.21.15.69:0/445358770 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0ef7fff700 1 -- 172.21.15.69:0/445358770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef819f550 con 0x7f0ef81077c0 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0ef7fff700 1 --2- 172.21.15.69:0/445358770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef819f010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0011ac0 tx=0x7f0ee0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0ef57fa700 1 -- 172.21.15.69:0/445358770 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee0003970 con 0x7f0ef81077c0 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0ef57fa700 1 -- 172.21.15.69:0/445358770 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0ee0003b10 con 0x7f0ef81077c0 2024-05-14T16:38:49.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0efe845700 1 -- 172.21.15.69:0/445358770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef819f7e0 con 0x7f0ef81077c0 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.940+0000 7f0ef57fa700 1 -- 172.21.15.69:0/445358770 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee0024610 con 0x7f0ef81077c0 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.941+0000 7f0efe845700 1 -- 172.21.15.69:0/445358770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef81a24d0 con 0x7f0ef81077c0 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.941+0000 7f0ef57fa700 1 -- 172.21.15.69:0/445358770 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ee0024bd0 con 0x7f0ef81077c0 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.941+0000 7f0ef57fa700 1 -- 172.21.15.69:0/445358770 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ee0019700 con 0x7f0ef81077c0 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.941+0000 7f0efe845700 1 -- 172.21.15.69:0/445358770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef8064ba0 con 0x7f0ef81077c0 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:43.944+0000 7f0ef57fa700 1 -- 172.21.15.69:0/445358770 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0ee0043ec0 con 0x7f0ef81077c0 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:44.017+0000 7f0efe845700 1 -- 172.21.15.69:0/445358770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0ef80008d0 con 0x7f0ef81077c0 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:44.018+0000 7f0ef57fa700 1 -- 172.21.15.69:0/445358770 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0ee0032030 con 0x7f0ef81077c0 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:44.019+0000 7f0efe845700 1 -- 172.21.15.69:0/445358770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 msgr2=0x7f0ef819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:44.019+0000 7f0efe845700 1 --2- 172.21.15.69:0/445358770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef819f010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0011ac0 tx=0x7f0ee0011ba0 comp rx=0 tx=0).stop 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:44.019+0000 7f0efe845700 1 -- 172.21.15.69:0/445358770 shutdown_connections 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:44.019+0000 7f0efe845700 1 --2- 172.21.15.69:0/445358770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef81077c0 0x7f0ef819f010 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:44.019+0000 7f0efe845700 1 -- 172.21.15.69:0/445358770 >> 172.21.15.69:0/445358770 conn(0x7f0ef8103380 msgr2=0x7f0ef8195c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:49.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:44.019+0000 7f0efe845700 1 -- 172.21.15.69:0/445358770 shutdown_connections 2024-05-14T16:38:49.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:44.019+0000 7f0efe845700 1 -- 172.21.15.69:0/445358770 wait complete. 2024-05-14T16:38:49.817 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (2/15)... 2024-05-14T16:38:50.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:50 smithi069 ceph-mon[31507]: mgrmap e4: x(active, since 2s) 2024-05-14T16:38:55.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:55 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3266463625' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-05-14T16:38:56.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T16:38:56.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T16:38:56.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T16:38:56.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T16:38:56.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T16:38:56.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:56.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T16:38:56.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T16:38:56.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T16:38:33.685055+0000", 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.232+0000 7fa48a53c700 1 Processor -- start 2024-05-14T16:38:56.153 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.232+0000 7fa48a53c700 1 -- start start 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.232+0000 7fa48a53c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48410bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.232+0000 7fa48a53c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa48407d610 con 0x7fa4841099d0 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.232+0000 7fa483fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48410bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.232+0000 7fa483fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48410bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47494/0 (socket says 172.21.15.69:47494) 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.232+0000 7fa483fff700 1 -- 172.21.15.69:0/2174843990 learned_addr learned my addr 172.21.15.69:0/2174843990 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.233+0000 7fa483fff700 1 -- 172.21.15.69:0/2174843990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48410c300 con 0x7fa4841099d0 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.233+0000 7fa483fff700 1 --2- 172.21.15.69:0/2174843990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48410bdc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa4740098a0 tx=0x7fa474009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7256d481fbd9b11b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.233+0000 7fa4837fe700 1 -- 172.21.15.69:0/2174843990 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa474004160 con 0x7fa4841099d0 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.233+0000 7fa4837fe700 1 -- 172.21.15.69:0/2174843990 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa474004300 con 0x7fa4841099d0 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.233+0000 7fa4837fe700 1 -- 172.21.15.69:0/2174843990 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa474011af0 con 0x7fa4841099d0 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.234+0000 7fa48a53c700 1 -- 172.21.15.69:0/2174843990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 msgr2=0x7fa48410bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.234+0000 7fa48a53c700 1 --2- 172.21.15.69:0/2174843990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48410bdc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa4740098a0 tx=0x7fa474009bb0 comp rx=0 tx=0).stop 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.234+0000 7fa48a53c700 1 -- 172.21.15.69:0/2174843990 shutdown_connections 2024-05-14T16:38:56.154 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.234+0000 7fa48a53c700 1 --2- 172.21.15.69:0/2174843990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48410bdc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.234+0000 7fa48a53c700 1 -- 172.21.15.69:0/2174843990 >> 172.21.15.69:0/2174843990 conn(0x7fa4841033e0 msgr2=0x7fa484105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.234+0000 7fa48a53c700 1 -- 172.21.15.69:0/2174843990 shutdown_connections 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.234+0000 7fa48a53c700 1 -- 172.21.15.69:0/2174843990 wait complete. 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa48a53c700 1 Processor -- start 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa48a53c700 1 -- start start 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa48a53c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa48a53c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa48410d260 con 0x7fa4841099d0 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa483fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa483fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47508/0 (socket says 172.21.15.69:47508) 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa483fff700 1 -- 172.21.15.69:0/3266463625 learned_addr learned my addr 172.21.15.69:0/3266463625 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa483fff700 1 -- 172.21.15.69:0/3266463625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48419d340 con 0x7fa4841099d0 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa483fff700 1 --2- 172.21.15.69:0/3266463625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48419ce00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa474011ac0 tx=0x7fa474011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa481ffb700 1 -- 172.21.15.69:0/3266463625 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa474003970 con 0x7fa4841099d0 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa481ffb700 1 -- 172.21.15.69:0/3266463625 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa474003b10 con 0x7fa4841099d0 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.235+0000 7fa48a53c700 1 -- 172.21.15.69:0/3266463625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48419d5d0 con 0x7fa4841099d0 2024-05-14T16:38:56.155 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.236+0000 7fa481ffb700 1 -- 172.21.15.69:0/3266463625 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa474024610 con 0x7fa4841099d0 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.236+0000 7fa48a53c700 1 -- 172.21.15.69:0/3266463625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841a02c0 con 0x7fa4841099d0 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.236+0000 7fa481ffb700 1 -- 172.21.15.69:0/3266463625 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fa474024890 con 0x7fa4841099d0 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.237+0000 7fa48a53c700 1 -- 172.21.15.69:0/3266463625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa484064ba0 con 0x7fa4841099d0 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.237+0000 7fa481ffb700 1 --2- 172.21.15.69:0/3266463625 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fa47003e420 0x7fa4700408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.237+0000 7fa481ffb700 1 -- 172.21.15.69:0/3266463625 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa474053840 con 0x7fa4841099d0 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.237+0000 7fa47bfff700 1 --2- 172.21.15.69:0/3266463625 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fa47003e420 0x7fa4700408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.237+0000 7fa47bfff700 1 --2- 172.21.15.69:0/3266463625 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fa47003e420 0x7fa4700408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa46c006fd0 tx=0x7fa46c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.242+0000 7fa481ffb700 1 -- 172.21.15.69:0/3266463625 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa47402c350 con 0x7fa4841099d0 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.538+0000 7fa48a53c700 1 -- 172.21.15.69:0/3266463625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa4841a0490 con 0x7fa4841099d0 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.539+0000 7fa481ffb700 1 -- 172.21.15.69:0/3266463625 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa4841a0490 con 0x7fa4841099d0 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.541+0000 7fa48a53c700 1 -- 172.21.15.69:0/3266463625 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fa47003e420 msgr2=0x7fa4700408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.541+0000 7fa48a53c700 1 --2- 172.21.15.69:0/3266463625 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fa47003e420 0x7fa4700408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa46c006fd0 tx=0x7fa46c006e40 comp rx=0 tx=0).stop 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.541+0000 7fa48a53c700 1 -- 172.21.15.69:0/3266463625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 msgr2=0x7fa48419ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.541+0000 7fa48a53c700 1 --2- 172.21.15.69:0/3266463625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48419ce00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa474011ac0 tx=0x7fa474011ba0 comp rx=0 tx=0).stop 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.541+0000 7fa48a53c700 1 -- 172.21.15.69:0/3266463625 shutdown_connections 2024-05-14T16:38:56.156 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.541+0000 7fa48a53c700 1 --2- 172.21.15.69:0/3266463625 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fa47003e420 0x7fa4700408e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:56.157 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.541+0000 7fa48a53c700 1 --2- 172.21.15.69:0/3266463625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4841099d0 0x7fa48419ce00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:56.157 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.541+0000 7fa48a53c700 1 -- 172.21.15.69:0/3266463625 >> 172.21.15.69:0/3266463625 conn(0x7fa4841033e0 msgr2=0x7fa484103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:56.157 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.541+0000 7fa48a53c700 1 -- 172.21.15.69:0/3266463625 shutdown_connections 2024-05-14T16:38:56.157 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:55.541+0000 7fa48a53c700 1 -- 172.21.15.69:0/3266463625 wait complete. 2024-05-14T16:38:56.157 INFO:teuthology.orchestra.run.smithi069.stdout:mgr is available 2024-05-14T16:38:57.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:56 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2795571952' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-14T16:38:57.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:57.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2024-05-14T16:38:57.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:38:57.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-14T16:38:57.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-14T16:38:57.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.544+0000 7f51eb80d700 1 Processor -- start 2024-05-14T16:38:57.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.544+0000 7f51eb80d700 1 -- start start 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.544+0000 7f51eb80d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e41081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.544+0000 7f51eb80d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f51e4108780 con 0x7f51e4107d90 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.545+0000 7f51e95a9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e41081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.545+0000 7f51e95a9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e41081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47510/0 (socket says 172.21.15.69:47510) 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.545+0000 7f51e95a9700 1 -- 172.21.15.69:0/3858269393 learned_addr learned my addr 172.21.15.69:0/3858269393 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.545+0000 7f51e95a9700 1 -- 172.21.15.69:0/3858269393 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e4108f90 con 0x7f51e4107d90 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.545+0000 7f51e95a9700 1 --2- 172.21.15.69:0/3858269393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e41081b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009e80 tx=0x7f51d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fe83e94314b25d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.545+0000 7f51e3fff700 1 -- 172.21.15.69:0/3858269393 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51d4017070 con 0x7f51e4107d90 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.545+0000 7f51e3fff700 1 -- 172.21.15.69:0/3858269393 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f51d400b8c0 con 0x7f51e4107d90 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.545+0000 7f51e3fff700 1 -- 172.21.15.69:0/3858269393 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51d4004490 con 0x7f51e4107d90 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.546+0000 7f51eb80d700 1 -- 172.21.15.69:0/3858269393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 msgr2=0x7f51e41081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.546+0000 7f51eb80d700 1 --2- 172.21.15.69:0/3858269393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e41081b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009e80 tx=0x7f51d400b330 comp rx=0 tx=0).stop 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.546+0000 7f51eb80d700 1 -- 172.21.15.69:0/3858269393 shutdown_connections 2024-05-14T16:38:57.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.546+0000 7f51eb80d700 1 --2- 172.21.15.69:0/3858269393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e41081b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.546+0000 7f51eb80d700 1 -- 172.21.15.69:0/3858269393 >> 172.21.15.69:0/3858269393 conn(0x7f51e4103330 msgr2=0x7f51e4105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.546+0000 7f51eb80d700 1 -- 172.21.15.69:0/3858269393 shutdown_connections 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.546+0000 7f51eb80d700 1 -- 172.21.15.69:0/3858269393 wait complete. 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51eb80d700 1 Processor -- start 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51eb80d700 1 -- start start 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51eb80d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51eb80d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f51e410c340 con 0x7f51e4107d90 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51e95a9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51e95a9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47516/0 (socket says 172.21.15.69:47516) 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51e95a9700 1 -- 172.21.15.69:0/2795571952 learned_addr learned my addr 172.21.15.69:0/2795571952 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51e95a9700 1 -- 172.21.15.69:0/2795571952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e419d340 con 0x7f51e4107d90 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51e95a9700 1 --2- 172.21.15.69:0/2795571952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e419ce00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f51d4004a60 tx=0x7f51d400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51e27fc700 1 -- 172.21.15.69:0/2795571952 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51d4020020 con 0x7f51e4107d90 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51e27fc700 1 -- 172.21.15.69:0/2795571952 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f51d4012430 con 0x7f51e4107d90 2024-05-14T16:38:57.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51eb80d700 1 -- 172.21.15.69:0/2795571952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e419d5d0 con 0x7f51e4107d90 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51e27fc700 1 -- 172.21.15.69:0/2795571952 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51d4017040 con 0x7f51e4107d90 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.547+0000 7f51eb80d700 1 -- 172.21.15.69:0/2795571952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e41a02c0 con 0x7f51e4107d90 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.548+0000 7f51e27fc700 1 -- 172.21.15.69:0/2795571952 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f51d4023020 con 0x7f51e4107d90 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.548+0000 7f51eb80d700 1 -- 172.21.15.69:0/2795571952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e4064ba0 con 0x7f51e4107d90 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.549+0000 7f51e27fc700 1 --2- 172.21.15.69:0/2795571952 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f51cc03e420 0x7f51cc0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.549+0000 7f51e27fc700 1 -- 172.21.15.69:0/2795571952 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f51d400f050 con 0x7f51e4107d90 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.549+0000 7f51e8da8700 1 --2- 172.21.15.69:0/2795571952 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f51cc03e420 0x7f51cc0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.549+0000 7f51e8da8700 1 --2- 172.21.15.69:0/2795571952 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f51cc03e420 0x7f51cc0408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f51d8006fd0 tx=0x7f51d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.554+0000 7f51e27fc700 1 -- 172.21.15.69:0/2795571952 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51d402a020 con 0x7f51e4107d90 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.757+0000 7f51eb80d700 1 -- 172.21.15.69:0/2795571952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f51e402e8d0 con 0x7f51e4107d90 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.758+0000 7f51e27fc700 1 -- 172.21.15.69:0/2795571952 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f51d402b260 con 0x7f51e4107d90 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.761+0000 7f51eb80d700 1 -- 172.21.15.69:0/2795571952 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f51cc03e420 msgr2=0x7f51cc0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.761+0000 7f51eb80d700 1 --2- 172.21.15.69:0/2795571952 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f51cc03e420 0x7f51cc0408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f51d8006fd0 tx=0x7f51d8006e40 comp rx=0 tx=0).stop 2024-05-14T16:38:57.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.761+0000 7f51eb80d700 1 -- 172.21.15.69:0/2795571952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 msgr2=0x7f51e419ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:57.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.761+0000 7f51eb80d700 1 --2- 172.21.15.69:0/2795571952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e419ce00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f51d4004a60 tx=0x7f51d400bd50 comp rx=0 tx=0).stop 2024-05-14T16:38:57.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.761+0000 7f51eb80d700 1 -- 172.21.15.69:0/2795571952 shutdown_connections 2024-05-14T16:38:57.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.761+0000 7f51eb80d700 1 --2- 172.21.15.69:0/2795571952 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f51cc03e420 0x7f51cc0408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:57.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.761+0000 7f51eb80d700 1 --2- 172.21.15.69:0/2795571952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4107d90 0x7f51e419ce00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:57.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.761+0000 7f51eb80d700 1 -- 172.21.15.69:0/2795571952 >> 172.21.15.69:0/2795571952 conn(0x7f51e4103330 msgr2=0x7f51e41041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:57.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.761+0000 7f51eb80d700 1 -- 172.21.15.69:0/2795571952 shutdown_connections 2024-05-14T16:38:57.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:56.761+0000 7f51eb80d700 1 -- 172.21.15.69:0/2795571952 wait complete. 2024-05-14T16:38:57.790 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling cephadm module... 2024-05-14T16:38:58.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:58 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/120233977' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-14T16:38:58.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:58 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: ignoring --setuser ceph since I am not root 2024-05-14T16:38:58.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:58 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: ignoring --setgroup ceph since I am not root 2024-05-14T16:38:58.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:58 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:58.664+0000 7ff01c0bf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:38:59.056 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.177+0000 7f9cceee3700 1 Processor -- start 2024-05-14T16:38:59.056 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.177+0000 7f9cceee3700 1 -- start start 2024-05-14T16:38:59.056 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.177+0000 7f9cceee3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc81081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:59.056 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.177+0000 7f9cceee3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8108780 con 0x7f9cc8107d90 2024-05-14T16:38:59.056 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.178+0000 7f9cccc7f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc81081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:59.056 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.178+0000 7f9cccc7f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc81081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47526/0 (socket says 172.21.15.69:47526) 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.178+0000 7f9cccc7f700 1 -- 172.21.15.69:0/213233032 learned_addr learned my addr 172.21.15.69:0/213233032 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.178+0000 7f9cccc7f700 1 -- 172.21.15.69:0/213233032 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc8108f90 con 0x7f9cc8107d90 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.178+0000 7f9cccc7f700 1 --2- 172.21.15.69:0/213233032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc81081b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0098a0 tx=0x7f9cbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ad9d53e9b3321b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.178+0000 7f9cc77fe700 1 -- 172.21.15.69:0/213233032 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cbc004160 con 0x7f9cc8107d90 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.178+0000 7f9cc77fe700 1 -- 172.21.15.69:0/213233032 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9cbc004300 con 0x7f9cc8107d90 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.178+0000 7f9cc77fe700 1 -- 172.21.15.69:0/213233032 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cbc011af0 con 0x7f9cc8107d90 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.179+0000 7f9cceee3700 1 -- 172.21.15.69:0/213233032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 msgr2=0x7f9cc81081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.179+0000 7f9cceee3700 1 --2- 172.21.15.69:0/213233032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc81081b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0098a0 tx=0x7f9cbc009bb0 comp rx=0 tx=0).stop 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.179+0000 7f9cceee3700 1 -- 172.21.15.69:0/213233032 shutdown_connections 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.179+0000 7f9cceee3700 1 --2- 172.21.15.69:0/213233032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc81081b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.179+0000 7f9cceee3700 1 -- 172.21.15.69:0/213233032 >> 172.21.15.69:0/213233032 conn(0x7f9cc8103330 msgr2=0x7f9cc8105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.179+0000 7f9cceee3700 1 -- 172.21.15.69:0/213233032 shutdown_connections 2024-05-14T16:38:59.057 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.179+0000 7f9cceee3700 1 -- 172.21.15.69:0/213233032 wait complete. 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cceee3700 1 Processor -- start 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cceee3700 1 -- start start 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cceee3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc807d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cceee3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc810c340 con 0x7f9cc8107d90 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cccc7f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc807d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cccc7f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc807d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47538/0 (socket says 172.21.15.69:47538) 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cccc7f700 1 -- 172.21.15.69:0/120233977 learned_addr learned my addr 172.21.15.69:0/120233977 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cccc7f700 1 -- 172.21.15.69:0/120233977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc819fb90 con 0x7f9cc8107d90 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cccc7f700 1 --2- 172.21.15.69:0/120233977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc807d260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0117b0 tx=0x7f9cbc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cc5ffb700 1 -- 172.21.15.69:0/120233977 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cbc003af0 con 0x7f9cc8107d90 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cc5ffb700 1 -- 172.21.15.69:0/120233977 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9cbc020070 con 0x7f9cc8107d90 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cceee3700 1 -- 172.21.15.69:0/120233977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc819fd60 con 0x7f9cc8107d90 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.180+0000 7f9cc5ffb700 1 -- 172.21.15.69:0/120233977 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cbc019630 con 0x7f9cc8107d90 2024-05-14T16:38:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.181+0000 7f9cceee3700 1 -- 172.21.15.69:0/120233977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc81a0100 con 0x7f9cc8107d90 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.182+0000 7f9cc5ffb700 1 -- 172.21.15.69:0/120233977 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f9cbc01a430 con 0x7f9cc8107d90 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.182+0000 7f9cceee3700 1 -- 172.21.15.69:0/120233977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc8064ba0 con 0x7f9cc8107d90 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.182+0000 7f9cc5ffb700 1 --2- 172.21.15.69:0/120233977 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f9cb803e470 0x7f9cb8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.182+0000 7f9cc5ffb700 1 -- 172.21.15.69:0/120233977 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9cbc053a20 con 0x7f9cc8107d90 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.182+0000 7f9cc7fff700 1 --2- 172.21.15.69:0/120233977 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f9cb803e470 0x7f9cb8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.182+0000 7f9cc7fff700 1 --2- 172.21.15.69:0/120233977 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f9cb803e470 0x7f9cb8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4006fd0 tx=0x7f9cb4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.187+0000 7f9cc5ffb700 1 -- 172.21.15.69:0/120233977 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9cbc021e00 con 0x7f9cc8107d90 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.444+0000 7f9cceee3700 1 -- 172.21.15.69:0/120233977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f9cc81a09b0 con 0x7f9cc8107d90 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.495+0000 7f9cc5ffb700 1 -- 172.21.15.69:0/120233977 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f9cbc00be00 con 0x7f9cc8107d90 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.497+0000 7f9cc5ffb700 1 -- 172.21.15.69:0/120233977 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f9cbc02a060 con 0x7f9cc8107d90 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.498+0000 7f9cceee3700 1 -- 172.21.15.69:0/120233977 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f9cb803e470 msgr2=0x7f9cb8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.498+0000 7f9cceee3700 1 --2- 172.21.15.69:0/120233977 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f9cb803e470 0x7f9cb8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4006fd0 tx=0x7f9cb4006e40 comp rx=0 tx=0).stop 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.498+0000 7f9cceee3700 1 -- 172.21.15.69:0/120233977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 msgr2=0x7f9cc807d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:38:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.498+0000 7f9cceee3700 1 --2- 172.21.15.69:0/120233977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc807d260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0117b0 tx=0x7f9cbc011b70 comp rx=0 tx=0).stop 2024-05-14T16:38:59.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.498+0000 7f9cceee3700 1 -- 172.21.15.69:0/120233977 shutdown_connections 2024-05-14T16:38:59.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.498+0000 7f9cceee3700 1 --2- 172.21.15.69:0/120233977 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f9cb803e470 0x7f9cb8040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:59.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.498+0000 7f9cceee3700 1 --2- 172.21.15.69:0/120233977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cc8107d90 0x7f9cc807d260 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:38:59.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.498+0000 7f9cceee3700 1 -- 172.21.15.69:0/120233977 >> 172.21.15.69:0/120233977 conn(0x7f9cc8103330 msgr2=0x7f9cc8103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:38:59.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.498+0000 7f9cceee3700 1 -- 172.21.15.69:0/120233977 shutdown_connections 2024-05-14T16:38:59.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:58.498+0000 7f9cceee3700 1 -- 172.21.15.69:0/120233977 wait complete. 2024-05-14T16:38:59.157 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:58 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:58.851+0000 7ff01c0bf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:38:59.157 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:58 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:58.955+0000 7ff01c0bf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:38:59.451 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:59 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:59.272+0000 7ff01c0bf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:38:59.451 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:38:59 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:38:59.360+0000 7ff01c0bf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:38:59.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:59 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/120233977' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-14T16:38:59.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:38:59 smithi069 ceph-mon[31507]: mgrmap e5: x(active, since 11s) 2024-05-14T16:39:00.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:39:00.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-14T16:39:00.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T16:39:00.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-14T16:39:00.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.476+0000 7f6fb0c35700 1 Processor -- start 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.476+0000 7f6fb0c35700 1 -- start start 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.477+0000 7f6fb0c35700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac07db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.477+0000 7f6fb0c35700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac07e0e0 con 0x7f6fac07f660 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.477+0000 7f6faa59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac07db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.477+0000 7f6faa59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac07db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47566/0 (socket says 172.21.15.69:47566) 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.477+0000 7f6faa59c700 1 -- 172.21.15.69:0/1726238372 learned_addr learned my addr 172.21.15.69:0/1726238372 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.477+0000 7f6faa59c700 1 -- 172.21.15.69:0/1726238372 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac07e260 con 0x7f6fac07f660 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.477+0000 7f6faa59c700 1 --2- 172.21.15.69:0/1726238372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac07db10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0098a0 tx=0x7f6f9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95f439135207c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.477+0000 7f6fa959a700 1 -- 172.21.15.69:0/1726238372 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f9c004160 con 0x7f6fac07f660 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.477+0000 7f6fa959a700 1 -- 172.21.15.69:0/1726238372 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6f9c004300 con 0x7f6fac07f660 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.477+0000 7f6fa959a700 1 -- 172.21.15.69:0/1726238372 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f9c011af0 con 0x7f6fac07f660 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.478+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1726238372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 msgr2=0x7f6fac07db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.478+0000 7f6fb0c35700 1 --2- 172.21.15.69:0/1726238372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac07db10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0098a0 tx=0x7f6f9c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.478+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1726238372 shutdown_connections 2024-05-14T16:39:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.478+0000 7f6fb0c35700 1 --2- 172.21.15.69:0/1726238372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac07db10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.478+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1726238372 >> 172.21.15.69:0/1726238372 conn(0x7f6fac104480 msgr2=0x7f6fac1068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.478+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1726238372 shutdown_connections 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.478+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1726238372 wait complete. 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.479+0000 7f6fb0c35700 1 Processor -- start 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.479+0000 7f6fb0c35700 1 -- start start 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.479+0000 7f6fb0c35700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.479+0000 7f6fb0c35700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac10f430 con 0x7f6fac07f660 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.479+0000 7f6faa59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.480+0000 7f6faa59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47572/0 (socket says 172.21.15.69:47572) 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.480+0000 7f6faa59c700 1 -- 172.21.15.69:0/1512765664 learned_addr learned my addr 172.21.15.69:0/1512765664 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.480+0000 7f6faa59c700 1 -- 172.21.15.69:0/1512765664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac19f590 con 0x7f6fac07f660 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.480+0000 7f6faa59c700 1 --2- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac19f050 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009fb0 tx=0x7f6f9c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.480+0000 7f6f9b7fe700 1 -- 172.21.15.69:0/1512765664 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f9c0036a0 con 0x7f6fac07f660 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.480+0000 7f6f9b7fe700 1 -- 172.21.15.69:0/1512765664 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6f9c01a430 con 0x7f6fac07f660 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.480+0000 7f6f9b7fe700 1 -- 172.21.15.69:0/1512765664 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f9c019550 con 0x7f6fac07f660 2024-05-14T16:39:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.480+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1512765664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fac19f820 con 0x7f6fac07f660 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.480+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1512765664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fac1a2510 con 0x7f6fac07f660 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.481+0000 7f6f9b7fe700 1 -- 172.21.15.69:0/1512765664 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f6f9c019800 con 0x7f6fac07f660 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.481+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1512765664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fac064ba0 con 0x7f6fac07f660 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.481+0000 7f6f9b7fe700 1 --2- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f6f9403e470 0x7f6f94040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.481+0000 7f6f9b7fe700 1 -- 172.21.15.69:0/1512765664 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6f9c053a60 con 0x7f6fac07f660 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.481+0000 7f6fa9d9b700 1 -- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f6f9403e470 msgr2=0x7f6f94040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4209517756 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.481+0000 7f6fa9d9b700 1 --2- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f6f9403e470 0x7f6f94040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.486+0000 7f6f9b7fe700 1 -- 172.21.15.69:0/1512765664 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f9c023390 con 0x7f6fac07f660 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.682+0000 7f6fa9d9b700 1 -- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f6f9403e470 msgr2=0x7f6f94040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4209517756 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.682+0000 7f6fa9d9b700 1 --2- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f6f9403e470 0x7f6f94040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.769+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1512765664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6fac1a26e0 con 0x7f6fac07f660 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.769+0000 7f6f9b7fe700 1 -- 172.21.15.69:0/1512765664 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f6fac1a26e0 con 0x7f6fac07f660 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.773+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f6f9403e470 msgr2=0x7f6f94040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.773+0000 7f6fb0c35700 1 --2- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f6f9403e470 0x7f6f94040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:00.374 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.773+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 msgr2=0x7f6fac19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:00.375 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.773+0000 7f6fb0c35700 1 --2- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac19f050 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009fb0 tx=0x7f6f9c011940 comp rx=0 tx=0).stop 2024-05-14T16:39:00.375 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.773+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1512765664 shutdown_connections 2024-05-14T16:39:00.375 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.773+0000 7f6fb0c35700 1 --2- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7f6f9403e470 0x7f6f94040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:00.375 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.773+0000 7f6fb0c35700 1 --2- 172.21.15.69:0/1512765664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac07f660 0x7f6fac19f050 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:00.375 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.773+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1512765664 >> 172.21.15.69:0/1512765664 conn(0x7f6fac104480 msgr2=0x7f6fac105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:00.375 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.773+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1512765664 shutdown_connections 2024-05-14T16:39:00.375 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:38:59.773+0000 7f6fb0c35700 1 -- 172.21.15.69:0/1512765664 wait complete. 2024-05-14T16:39:00.375 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2024-05-14T16:39:00.375 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 5... 2024-05-14T16:39:00.766 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:00 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1512765664' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-14T16:39:00.766 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:00.593+0000 7ff01c0bf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:39:01.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:00.771+0000 7ff01c0bf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:39:01.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:00.943+0000 7ff01c0bf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:39:01.537 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:01 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:01.235+0000 7ff01c0bf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:39:01.537 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:01 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:01.328+0000 7ff01c0bf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:39:01.537 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:01 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:01.536+0000 7ff01c0bf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:39:01.953 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:01 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:01.730+0000 7ff01c0bf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:39:02.453 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:02 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:02.329+0000 7ff01c0bf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:39:02.953 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:02 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:02.553+0000 7ff01c0bf200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:39:03.449 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:02 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:02.981+0000 7ff01c0bf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:39:03.449 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:03 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:03.078+0000 7ff01c0bf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:39:03.449 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:03 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:03.449+0000 7ff01c0bf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:39:04.145 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:03 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:03.807+0000 7ff01c0bf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:39:04.145 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:03 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:03.901+0000 7ff01c0bf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:39:04.453 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:04.145+0000 7ff01c0bf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:39:04.454 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:04.279+0000 7ff01c0bf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:39:04.454 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:04.378+0000 7ff01c0bf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:39:05.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:04.734+0000 7ff01c0bf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:39:05.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:04.829+0000 7ff01c0bf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:39:05.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:05 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:05.435+0000 7ff01c0bf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:39:05.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:05 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:05.655+0000 7ff01c0bf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: Active manager daemon x restarted 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: Activating manager daemon x 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: osdmap e2: 0 total, 0 up, 0 in 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: mgrmap e6: x(active, starting, since 0.00344034s) 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: Manager daemon x is now available 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:06.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:39:06.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:39:06.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:05 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:39:06.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:06 smithi069 ceph-mon[31507]: Found migration_current of "None". Setting to last migration. 2024-05-14T16:39:06.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:06 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:39:06.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:06 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:06.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:06 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:06.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:06 smithi069 ceph-mon[31507]: [14/May/2024:16:39:06] ENGINE Bus STARTING 2024-05-14T16:39:06.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:06 smithi069 ceph-mon[31507]: mgrmap e7: x(active, since 1.00562s) 2024-05-14T16:39:07.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:39:07.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-14T16:39:07.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-14T16:39:07.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:39:07.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.768+0000 7fb20a729700 1 Processor -- start 2024-05-14T16:39:07.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.768+0000 7fb20a729700 1 -- start start 2024-05-14T16:39:07.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.768+0000 7fb20a729700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20410bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.768+0000 7fb20a729700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb20407d6c0 con 0x7fb204109900 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.768+0000 7fb203fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20410bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.768+0000 7fb203fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20410bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47578/0 (socket says 172.21.15.69:47578) 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.768+0000 7fb203fff700 1 -- 172.21.15.69:0/557421045 learned_addr learned my addr 172.21.15.69:0/557421045 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.769+0000 7fb203fff700 1 -- 172.21.15.69:0/557421045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20410c230 con 0x7fb204109900 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.769+0000 7fb203fff700 1 --2- 172.21.15.69:0/557421045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20410bcf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40098a0 tx=0x7fb1f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58565f4bbbbcd12e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.769+0000 7fb202ffd700 1 -- 172.21.15.69:0/557421045 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f4004160 con 0x7fb204109900 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.769+0000 7fb202ffd700 1 -- 172.21.15.69:0/557421045 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb1f4004300 con 0x7fb204109900 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.769+0000 7fb202ffd700 1 -- 172.21.15.69:0/557421045 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f4011af0 con 0x7fb204109900 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.769+0000 7fb20a729700 1 -- 172.21.15.69:0/557421045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 msgr2=0x7fb20410bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.769+0000 7fb20a729700 1 --2- 172.21.15.69:0/557421045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20410bcf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40098a0 tx=0x7fb1f4009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.770+0000 7fb20a729700 1 -- 172.21.15.69:0/557421045 shutdown_connections 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.770+0000 7fb20a729700 1 --2- 172.21.15.69:0/557421045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20410bcf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.770+0000 7fb20a729700 1 -- 172.21.15.69:0/557421045 >> 172.21.15.69:0/557421045 conn(0x7fb204103330 msgr2=0x7fb204105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.770+0000 7fb20a729700 1 -- 172.21.15.69:0/557421045 shutdown_connections 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.770+0000 7fb20a729700 1 -- 172.21.15.69:0/557421045 wait complete. 2024-05-14T16:39:07.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.770+0000 7fb20a729700 1 Processor -- start 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.770+0000 7fb20a729700 1 -- start start 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb20a729700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb20a729700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb20410d200 con 0x7fb204109900 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb203fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb203fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47590/0 (socket says 172.21.15.69:47590) 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb203fff700 1 -- 172.21.15.69:0/3183962193 learned_addr learned my addr 172.21.15.69:0/3183962193 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb203fff700 1 -- 172.21.15.69:0/3183962193 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20419d340 con 0x7fb204109900 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb203fff700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20419ce00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009fb0 tx=0x7fb1f4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f40036a0 con 0x7fb204109900 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb20a729700 1 -- 172.21.15.69:0/3183962193 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20419d5d0 con 0x7fb204109900 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb1f401a430 con 0x7fb204109900 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f4019550 con 0x7fb204109900 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.771+0000 7fb20a729700 1 -- 172.21.15.69:0/3183962193 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2041a02c0 con 0x7fb204109900 2024-05-14T16:39:07.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.772+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fb1f4019850 con 0x7fb204109900 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.773+0000 7fb2017fa700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 0x7fb1ec0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.773+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 --> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb1ec040ff0 con 0x7fb1ec03e420 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.773+0000 7fb2037fe700 1 -- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 msgr2=0x7fb1ec0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4209517756 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.773+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb1f4053890 con 0x7fb204109900 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.773+0000 7fb2037fe700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 0x7fb1ec0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.974+0000 7fb2037fe700 1 -- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 msgr2=0x7fb1ec0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4209517756 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:00.974+0000 7fb2037fe700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 0x7fb1ec0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:01.375+0000 7fb2037fe700 1 -- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 msgr2=0x7fb1ec0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4209517756 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:01.375+0000 7fb2037fe700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 0x7fb1ec0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:02.176+0000 7fb2037fe700 1 -- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 msgr2=0x7fb1ec0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4209517756 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:02.176+0000 7fb2037fe700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 0x7fb1ec0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:03.777+0000 7fb2037fe700 1 -- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 msgr2=0x7fb1ec0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4209517756 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:03.777+0000 7fb2037fe700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 0x7fb1ec0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-14T16:39:07.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:05.659+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fb1f400fd80 con 0x7fb204109900 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:05.659+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 msgr2=0x7fb1ec0408e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:05.659+0000 7fb2017fa700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 0x7fb1ec0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.660+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fb1f40556b0 con 0x7fb204109900 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.660+0000 7fb2017fa700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb1ec041d10 0x7fb1ec044100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.660+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 --> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb1f4016e70 con 0x7fb1ec041d10 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.661+0000 7fb2037fe700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb1ec041d10 0x7fb1ec044100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.661+0000 7fb2037fe700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb1ec041d10 0x7fb1ec044100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8003a10 tx=0x7fb1f8009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.662+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 <== mgr.14118 v2:172.21.15.69:6800/421328414 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb1f4016e70 con 0x7fb1ec041d10 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.670+0000 7fb20a729700 1 -- 172.21.15.69:0/3183962193 --> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb20406a650 con 0x7fb1ec041d10 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.670+0000 7fb2017fa700 1 -- 172.21.15.69:0/3183962193 <== mgr.14118 v2:172.21.15.69:6800/421328414 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb20406a650 con 0x7fb1ec041d10 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 -- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb1ec041d10 msgr2=0x7fb1ec044100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb1ec041d10 0x7fb1ec044100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8003a10 tx=0x7fb1f8009290 comp rx=0 tx=0).stop 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 -- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 msgr2=0x7fb20419ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20419ce00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009fb0 tx=0x7fb1f4011940 comp rx=0 tx=0).stop 2024-05-14T16:39:07.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 -- 172.21.15.69:0/3183962193 shutdown_connections 2024-05-14T16:39:07.235 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb1ec041d10 0x7fb1ec044100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:07.235 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb204109900 0x7fb20419ce00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:07.235 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 --2- 172.21.15.69:0/3183962193 >> [v2:172.21.15.69:6800/4209517756,v1:172.21.15.69:6801/4209517756] conn(0x7fb1ec03e420 0x7fb1ec0408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:07.235 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 -- 172.21.15.69:0/3183962193 >> 172.21.15.69:0/3183962193 conn(0x7fb204103330 msgr2=0x7fb204103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:07.235 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 -- 172.21.15.69:0/3183962193 shutdown_connections 2024-05-14T16:39:07.235 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:06.671+0000 7fb20a729700 1 -- 172.21.15.69:0/3183962193 wait complete. 2024-05-14T16:39:07.235 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 5 is available 2024-05-14T16:39:07.235 INFO:teuthology.orchestra.run.smithi069.stdout:Setting orchestrator backend to cephadm... 2024-05-14T16:39:07.839 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:07 smithi069 ceph-mon[31507]: [14/May/2024:16:39:06] ENGINE Serving on http://172.21.15.69:8765 2024-05-14T16:39:07.839 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:07 smithi069 ceph-mon[31507]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-14T16:39:07.839 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:07 smithi069 ceph-mon[31507]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-14T16:39:07.839 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:07 smithi069 ceph-mon[31507]: [14/May/2024:16:39:06] ENGINE Serving on https://172.21.15.69:7150 2024-05-14T16:39:07.839 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:07 smithi069 ceph-mon[31507]: [14/May/2024:16:39:06] ENGINE Bus STARTED 2024-05-14T16:39:07.839 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:07 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:39:08.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.611+0000 7fceaf5ab700 1 Processor -- start 2024-05-14T16:39:08.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.611+0000 7fceaf5ab700 1 -- start start 2024-05-14T16:39:08.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.611+0000 7fceaf5ab700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea810bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.611+0000 7fceaf5ab700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcea807d610 con 0x7fcea81099a0 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.611+0000 7fcead347700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea810bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.611+0000 7fcead347700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea810bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56706/0 (socket says 172.21.15.69:56706) 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.611+0000 7fcead347700 1 -- 172.21.15.69:0/878869231 learned_addr learned my addr 172.21.15.69:0/878869231 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.611+0000 7fcead347700 1 -- 172.21.15.69:0/878869231 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea810c2d0 con 0x7fcea81099a0 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.612+0000 7fcead347700 1 --2- 172.21.15.69:0/878869231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea810bd90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcea40098a0 tx=0x7fcea4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f18fb3205c53eb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.612+0000 7fce9ffff700 1 -- 172.21.15.69:0/878869231 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea4004160 con 0x7fcea81099a0 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.612+0000 7fce9ffff700 1 -- 172.21.15.69:0/878869231 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcea4004300 con 0x7fcea81099a0 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.612+0000 7fce9ffff700 1 -- 172.21.15.69:0/878869231 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea4011af0 con 0x7fcea81099a0 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.612+0000 7fceaf5ab700 1 -- 172.21.15.69:0/878869231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 msgr2=0x7fcea810bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:08.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.612+0000 7fceaf5ab700 1 --2- 172.21.15.69:0/878869231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea810bd90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcea40098a0 tx=0x7fcea4009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.612+0000 7fceaf5ab700 1 -- 172.21.15.69:0/878869231 shutdown_connections 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.612+0000 7fceaf5ab700 1 --2- 172.21.15.69:0/878869231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea810bd90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.612+0000 7fceaf5ab700 1 -- 172.21.15.69:0/878869231 >> 172.21.15.69:0/878869231 conn(0x7fcea8103350 msgr2=0x7fcea81057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.612+0000 7fceaf5ab700 1 -- 172.21.15.69:0/878869231 shutdown_connections 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.613+0000 7fceaf5ab700 1 -- 172.21.15.69:0/878869231 wait complete. 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.613+0000 7fceaf5ab700 1 Processor -- start 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.613+0000 7fceaf5ab700 1 -- start start 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.613+0000 7fceaf5ab700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea8109220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.613+0000 7fceaf5ab700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcea810d230 con 0x7fcea81099a0 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.613+0000 7fcead347700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea8109220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.613+0000 7fcead347700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea8109220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56708/0 (socket says 172.21.15.69:56708) 2024-05-14T16:39:08.393 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.613+0000 7fcead347700 1 -- 172.21.15.69:0/1934888928 learned_addr learned my addr 172.21.15.69:0/1934888928 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.614+0000 7fcead347700 1 -- 172.21.15.69:0/1934888928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea8109760 con 0x7fcea81099a0 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.614+0000 7fcead347700 1 --2- 172.21.15.69:0/1934888928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea8109220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009fb0 tx=0x7fcea4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.614+0000 7fce9e7fc700 1 -- 172.21.15.69:0/1934888928 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea40036a0 con 0x7fcea81099a0 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.614+0000 7fce9e7fc700 1 -- 172.21.15.69:0/1934888928 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcea401a430 con 0x7fcea81099a0 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.614+0000 7fceaf5ab700 1 -- 172.21.15.69:0/1934888928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea8107950 con 0x7fcea81099a0 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.614+0000 7fce9e7fc700 1 -- 172.21.15.69:0/1934888928 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea4019550 con 0x7fcea81099a0 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.614+0000 7fceaf5ab700 1 -- 172.21.15.69:0/1934888928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea8107e90 con 0x7fcea81099a0 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.615+0000 7fce9e7fc700 1 -- 172.21.15.69:0/1934888928 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fcea4003840 con 0x7fcea81099a0 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.615+0000 7fceaf5ab700 1 -- 172.21.15.69:0/1934888928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea8064ba0 con 0x7fcea81099a0 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.615+0000 7fce9e7fc700 1 --2- 172.21.15.69:0/1934888928 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fce9403e340 0x7fce94040800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.615+0000 7fce9e7fc700 1 -- 172.21.15.69:0/1934888928 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcea4053650 con 0x7fcea81099a0 2024-05-14T16:39:08.394 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.615+0000 7fceacb46700 1 --2- 172.21.15.69:0/1934888928 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fce9403e340 0x7fce94040800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.616+0000 7fceacb46700 1 --2- 172.21.15.69:0/1934888928 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fce9403e340 0x7fce94040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce98006fd0 tx=0x7fce98006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.620+0000 7fce9e7fc700 1 -- 172.21.15.69:0/1934888928 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcea400b3e0 con 0x7fcea81099a0 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.716+0000 7fce9e7fc700 1 -- 172.21.15.69:0/1934888928 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fcea4055b60 con 0x7fcea81099a0 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.839+0000 7fceaf5ab700 1 -- 172.21.15.69:0/1934888928 --> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fcea81045c0 con 0x7fce9403e340 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.844+0000 7fce9e7fc700 1 -- 172.21.15.69:0/1934888928 <== mgr.14118 v2:172.21.15.69:6800/421328414 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcea81045c0 con 0x7fce9403e340 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.846+0000 7fceaf5ab700 1 -- 172.21.15.69:0/1934888928 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fce9403e340 msgr2=0x7fce94040800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.846+0000 7fceaf5ab700 1 --2- 172.21.15.69:0/1934888928 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fce9403e340 0x7fce94040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce98006fd0 tx=0x7fce98006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.846+0000 7fceaf5ab700 1 -- 172.21.15.69:0/1934888928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 msgr2=0x7fcea8109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.846+0000 7fceaf5ab700 1 --2- 172.21.15.69:0/1934888928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea8109220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009fb0 tx=0x7fcea4011940 comp rx=0 tx=0).stop 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.846+0000 7fceaf5ab700 1 -- 172.21.15.69:0/1934888928 shutdown_connections 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.846+0000 7fceaf5ab700 1 --2- 172.21.15.69:0/1934888928 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fce9403e340 0x7fce94040800 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.846+0000 7fceaf5ab700 1 --2- 172.21.15.69:0/1934888928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea81099a0 0x7fcea8109220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.846+0000 7fceaf5ab700 1 -- 172.21.15.69:0/1934888928 >> 172.21.15.69:0/1934888928 conn(0x7fcea8103350 msgr2=0x7fcea8103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:08.395 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.846+0000 7fceaf5ab700 1 -- 172.21.15.69:0/1934888928 shutdown_connections 2024-05-14T16:39:08.396 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:07.846+0000 7fceaf5ab700 1 -- 172.21.15.69:0/1934888928 wait complete. 2024-05-14T16:39:08.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:08 smithi069 ceph-mon[31507]: mgrmap e8: x(active, since 2s) 2024-05-14T16:39:08.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:08 smithi069 ceph-mon[31507]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:08.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:08 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:08.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:08 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:39:09.581 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-14T16:39:09.581 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.798+0000 7fde10d63700 1 Processor -- start 2024-05-14T16:39:09.581 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.798+0000 7fde10d63700 1 -- start start 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.799+0000 7fde10d63700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c10bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.799+0000 7fde10d63700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c07d5f0 con 0x7fde0c1099d0 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.799+0000 7fde0a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c10bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.799+0000 7fde0a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c10bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56716/0 (socket says 172.21.15.69:56716) 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.799+0000 7fde0a59c700 1 -- 172.21.15.69:0/3091726990 learned_addr learned my addr 172.21.15.69:0/3091726990 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.799+0000 7fde0a59c700 1 -- 172.21.15.69:0/3091726990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c10c300 con 0x7fde0c1099d0 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.799+0000 7fde0a59c700 1 --2- 172.21.15.69:0/3091726990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c10bdc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0098a0 tx=0x7fddfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f962e25030686eb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.799+0000 7fde0959a700 1 -- 172.21.15.69:0/3091726990 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddfc004160 con 0x7fde0c1099d0 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.799+0000 7fde0959a700 1 -- 172.21.15.69:0/3091726990 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fddfc004300 con 0x7fde0c1099d0 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.799+0000 7fde0959a700 1 -- 172.21.15.69:0/3091726990 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddfc011af0 con 0x7fde0c1099d0 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.800+0000 7fde10d63700 1 -- 172.21.15.69:0/3091726990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 msgr2=0x7fde0c10bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:09.582 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.800+0000 7fde10d63700 1 --2- 172.21.15.69:0/3091726990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c10bdc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0098a0 tx=0x7fddfc009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.800+0000 7fde10d63700 1 -- 172.21.15.69:0/3091726990 shutdown_connections 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.800+0000 7fde10d63700 1 --2- 172.21.15.69:0/3091726990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c10bdc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.800+0000 7fde10d63700 1 -- 172.21.15.69:0/3091726990 >> 172.21.15.69:0/3091726990 conn(0x7fde0c1033e0 msgr2=0x7fde0c105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.800+0000 7fde10d63700 1 -- 172.21.15.69:0/3091726990 shutdown_connections 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.800+0000 7fde10d63700 1 -- 172.21.15.69:0/3091726990 wait complete. 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.801+0000 7fde10d63700 1 Processor -- start 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.801+0000 7fde10d63700 1 -- start start 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.801+0000 7fde10d63700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.801+0000 7fde10d63700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c10d200 con 0x7fde0c1099d0 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.801+0000 7fde0a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.801+0000 7fde0a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56722/0 (socket says 172.21.15.69:56722) 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.801+0000 7fde0a59c700 1 -- 172.21.15.69:0/1899230981 learned_addr learned my addr 172.21.15.69:0/1899230981 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.801+0000 7fde0a59c700 1 -- 172.21.15.69:0/1899230981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c19d3c0 con 0x7fde0c1099d0 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.801+0000 7fde0a59c700 1 --2- 172.21.15.69:0/1899230981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c19ce80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009fb0 tx=0x7fddfc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:09.583 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.802+0000 7fddfb7fe700 1 -- 172.21.15.69:0/1899230981 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddfc0036a0 con 0x7fde0c1099d0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.802+0000 7fddfb7fe700 1 -- 172.21.15.69:0/1899230981 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fddfc01a430 con 0x7fde0c1099d0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.802+0000 7fde10d63700 1 -- 172.21.15.69:0/1899230981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c19d650 con 0x7fde0c1099d0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.802+0000 7fddfb7fe700 1 -- 172.21.15.69:0/1899230981 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddfc019550 con 0x7fde0c1099d0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.802+0000 7fde10d63700 1 -- 172.21.15.69:0/1899230981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c1a0340 con 0x7fde0c1099d0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.803+0000 7fddfb7fe700 1 -- 172.21.15.69:0/1899230981 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fddfc019850 con 0x7fde0c1099d0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.803+0000 7fde10d63700 1 -- 172.21.15.69:0/1899230981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0c06a650 con 0x7fde0c1099d0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.803+0000 7fddfb7fe700 1 --2- 172.21.15.69:0/1899230981 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fddf403e420 0x7fddf40408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.803+0000 7fddfb7fe700 1 -- 172.21.15.69:0/1899230981 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fddfc0539c0 con 0x7fde0c1099d0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.803+0000 7fde09d9b700 1 --2- 172.21.15.69:0/1899230981 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fddf403e420 0x7fddf40408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.804+0000 7fde09d9b700 1 --2- 172.21.15.69:0/1899230981 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fddf403e420 0x7fddf40408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde00006fd0 tx=0x7fde00006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:08.808+0000 7fddfb7fe700 1 -- 172.21.15.69:0/1899230981 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fddfc055e20 con 0x7fde0c1099d0 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.017+0000 7fde10d63700 1 -- 172.21.15.69:0/1899230981 --> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fde0c104600 con 0x7fddf403e420 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.017+0000 7fddfb7fe700 1 -- 172.21.15.69:0/1899230981 <== mgr.14118 v2:172.21.15.69:6800/421328414 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fde0c104600 con 0x7fddf403e420 2024-05-14T16:39:09.584 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.019+0000 7fde10d63700 1 -- 172.21.15.69:0/1899230981 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fddf403e420 msgr2=0x7fddf40408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:09.585 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.020+0000 7fde10d63700 1 --2- 172.21.15.69:0/1899230981 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fddf403e420 0x7fddf40408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde00006fd0 tx=0x7fde00006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:09.585 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.020+0000 7fde10d63700 1 -- 172.21.15.69:0/1899230981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 msgr2=0x7fde0c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:09.585 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.020+0000 7fde10d63700 1 --2- 172.21.15.69:0/1899230981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c19ce80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009fb0 tx=0x7fddfc011940 comp rx=0 tx=0).stop 2024-05-14T16:39:09.585 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.020+0000 7fde10d63700 1 -- 172.21.15.69:0/1899230981 shutdown_connections 2024-05-14T16:39:09.585 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.020+0000 7fde10d63700 1 --2- 172.21.15.69:0/1899230981 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fddf403e420 0x7fddf40408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:09.585 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.020+0000 7fde10d63700 1 --2- 172.21.15.69:0/1899230981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde0c1099d0 0x7fde0c19ce80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:09.585 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.020+0000 7fde10d63700 1 -- 172.21.15.69:0/1899230981 >> 172.21.15.69:0/1899230981 conn(0x7fde0c1033e0 msgr2=0x7fde0c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:09.585 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.020+0000 7fde10d63700 1 -- 172.21.15.69:0/1899230981 shutdown_connections 2024-05-14T16:39:09.585 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:09.020+0000 7fde10d63700 1 -- 172.21.15.69:0/1899230981 wait complete. 2024-05-14T16:39:09.585 INFO:teuthology.orchestra.run.smithi069.stdout:Generating ssh key... 2024-05-14T16:39:09.901 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:09 smithi069 ceph-mon[31507]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:10.638 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: Generating public/private rsa key pair. 2024-05-14T16:39:10.638 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: Your identification has been saved in /tmp/tmpoctfvq7j/key. 2024-05-14T16:39:10.638 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: Your public key has been saved in /tmp/tmpoctfvq7j/key.pub. 2024-05-14T16:39:10.638 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: The key fingerprint is: 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: SHA256:KURlVW9uVMJR+p49nNkeLk/IHNTITwHeBJDDTimXFMQ ceph-356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: The key's randomart image is: 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: +---[RSA 3072]----+ 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: | ..o.**B+==o| 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: | . . . Eo.*+.| 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: | . = .==+ | 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: | . . ..+o. | 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: | . S .o..| 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: | . o.+.*| 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: | + O+| 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: | .o.o| 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: | oo.| 2024-05-14T16:39:10.639 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: +----[SHA256]-----+ 2024-05-14T16:39:10.913 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.003+0000 7f4862a8e700 1 Processor -- start 2024-05-14T16:39:10.913 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.003+0000 7f4862a8e700 1 -- start start 2024-05-14T16:39:10.913 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.003+0000 7f4862a8e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:10.913 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.003+0000 7f4862a8e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1097c0 con 0x7f485c108e60 2024-05-14T16:39:10.913 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.004+0000 7f486082a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:10.913 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.004+0000 7f486082a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56724/0 (socket says 172.21.15.69:56724) 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.004+0000 7f486082a700 1 -- 172.21.15.69:0/333455185 learned_addr learned my addr 172.21.15.69:0/333455185 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.004+0000 7f486082a700 1 -- 172.21.15.69:0/333455185 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c10a050 con 0x7f485c108e60 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.004+0000 7f486082a700 1 --2- 172.21.15.69:0/333455185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c109280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4858009e80 tx=0x7f485800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a649abcb44374170 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.004+0000 7f48537fe700 1 -- 172.21.15.69:0/333455185 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4858017070 con 0x7f485c108e60 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.004+0000 7f48537fe700 1 -- 172.21.15.69:0/333455185 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f485800b8c0 con 0x7f485c108e60 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.004+0000 7f48537fe700 1 -- 172.21.15.69:0/333455185 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4858004490 con 0x7f485c108e60 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.005+0000 7f4862a8e700 1 -- 172.21.15.69:0/333455185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 msgr2=0x7f485c109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.005+0000 7f4862a8e700 1 --2- 172.21.15.69:0/333455185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c109280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4858009e80 tx=0x7f485800b330 comp rx=0 tx=0).stop 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.005+0000 7f4862a8e700 1 -- 172.21.15.69:0/333455185 shutdown_connections 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.005+0000 7f4862a8e700 1 --2- 172.21.15.69:0/333455185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c109280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.005+0000 7f4862a8e700 1 -- 172.21.15.69:0/333455185 >> 172.21.15.69:0/333455185 conn(0x7f485c104450 msgr2=0x7f485c106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.005+0000 7f4862a8e700 1 -- 172.21.15.69:0/333455185 shutdown_connections 2024-05-14T16:39:10.914 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.005+0000 7f4862a8e700 1 -- 172.21.15.69:0/333455185 wait complete. 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.005+0000 7f4862a8e700 1 Processor -- start 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.005+0000 7f4862a8e700 1 -- start start 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f4862a8e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c07f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f4862a8e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f485c10d310 con 0x7f485c108e60 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f486082a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c07f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f486082a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c07f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56740/0 (socket says 172.21.15.69:56740) 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f486082a700 1 -- 172.21.15.69:0/4269882047 learned_addr learned my addr 172.21.15.69:0/4269882047 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f486082a700 1 -- 172.21.15.69:0/4269882047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c07f8c0 con 0x7f485c108e60 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f486082a700 1 --2- 172.21.15.69:0/4269882047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c07f380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f485800b7b0 tx=0x7f485800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f4851ffb700 1 -- 172.21.15.69:0/4269882047 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4858020020 con 0x7f485c108e60 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f4862a8e700 1 -- 172.21.15.69:0/4269882047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c07dab0 con 0x7f485c108e60 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f4851ffb700 1 -- 172.21.15.69:0/4269882047 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4858012430 con 0x7f485c108e60 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f4851ffb700 1 -- 172.21.15.69:0/4269882047 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4858017040 con 0x7f485c108e60 2024-05-14T16:39:10.915 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.006+0000 7f4862a8e700 1 -- 172.21.15.69:0/4269882047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c07dff0 con 0x7f485c108e60 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.007+0000 7f4862a8e700 1 -- 172.21.15.69:0/4269882047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f485c064ba0 con 0x7f485c108e60 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.007+0000 7f4851ffb700 1 -- 172.21.15.69:0/4269882047 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4858023020 con 0x7f485c108e60 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.008+0000 7f4851ffb700 1 --2- 172.21.15.69:0/4269882047 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f484403e420 0x7f48440408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.008+0000 7f4851ffb700 1 -- 172.21.15.69:0/4269882047 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f485800f050 con 0x7f485c108e60 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.008+0000 7f4853fff700 1 --2- 172.21.15.69:0/4269882047 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f484403e420 0x7f48440408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.008+0000 7f4853fff700 1 --2- 172.21.15.69:0/4269882047 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f484403e420 0x7f48440408e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f484c006fd0 tx=0x7f484c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.013+0000 7f4851ffb700 1 -- 172.21.15.69:0/4269882047 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f485802a020 con 0x7f485c108e60 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.215+0000 7f4862a8e700 1 -- 172.21.15.69:0/4269882047 --> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f485c1056a0 con 0x7f484403e420 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.351+0000 7f4851ffb700 1 -- 172.21.15.69:0/4269882047 <== mgr.14118 v2:172.21.15.69:6800/421328414 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f485c1056a0 con 0x7f484403e420 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.353+0000 7f4862a8e700 1 -- 172.21.15.69:0/4269882047 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f484403e420 msgr2=0x7f48440408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.353+0000 7f4862a8e700 1 --2- 172.21.15.69:0/4269882047 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f484403e420 0x7f48440408e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f484c006fd0 tx=0x7f484c006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.353+0000 7f4862a8e700 1 -- 172.21.15.69:0/4269882047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 msgr2=0x7f485c07f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.353+0000 7f4862a8e700 1 --2- 172.21.15.69:0/4269882047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c07f380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f485800b7b0 tx=0x7f485800be20 comp rx=0 tx=0).stop 2024-05-14T16:39:10.916 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.354+0000 7f4862a8e700 1 -- 172.21.15.69:0/4269882047 shutdown_connections 2024-05-14T16:39:10.917 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.354+0000 7f4862a8e700 1 --2- 172.21.15.69:0/4269882047 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f484403e420 0x7f48440408e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:10.917 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.354+0000 7f4862a8e700 1 --2- 172.21.15.69:0/4269882047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f485c108e60 0x7f485c07f380 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:10.917 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.354+0000 7f4862a8e700 1 -- 172.21.15.69:0/4269882047 >> 172.21.15.69:0/4269882047 conn(0x7f485c104450 msgr2=0x7f485c104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:10.917 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.354+0000 7f4862a8e700 1 -- 172.21.15.69:0/4269882047 shutdown_connections 2024-05-14T16:39:10.917 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:10.354+0000 7f4862a8e700 1 -- 172.21.15.69:0/4269882047 wait complete. 2024-05-14T16:39:11.520 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:11 smithi069 ceph-mon[31507]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:11.520 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:11 smithi069 ceph-mon[31507]: Generating ssh key... 2024-05-14T16:39:11.521 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:11 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:11.521 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:11 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:12.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJrE/J1SQekF8P0jH8uWb/HFR6iud/QCSeyVtuN8BsYabSgD5NLA6R5U2vdBIGh/V/mZ1TGT33DXeN7tEIIQwL/iqj5VPQR93fDxCm8tv3TDwfaQoK+4XvHIrWr4OywDiSG/YuouEbbRg0bdSks6ZYJxzW5uCYRy99r2N+K7lP+YtxjBR4JZZv9LO4r1O2rgaxre/m6PgxJTQFMSnoDNP1jEa7k852zvH79cmh0bHYLHcAeZQEoSS/YrWzNFQ0KNpZ36YKP22xxgQdsu3k6I0yb/ksGaJUsG2a+H8B/XY+iIvOqx+oMcGK+sFIZOKq25onFdZh0Eoaq0a/M8p+CgjTO+oX3r7IWhRvPX4g1xbiXz4eDYCvoXXDJCFTtRfNGApUnYwo95YpTQedMTpLm4ORxGWtbzqUhyRI1PYUkovI7f0fhQUqpnxSJTS5E5fCPlS/kHX/o+odeAx1Fh7Nj0JR7migU5S2GMzZQaYfujGHvOteqwrrWwXBBUVfW2oCI40= ceph-356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.306+0000 7fb353315700 1 Processor -- start 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.307+0000 7fb353315700 1 -- start start 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.307+0000 7fb353315700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c07daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.307+0000 7fb353315700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c07e030 con 0x7fb34c07f640 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.307+0000 7fb3510b1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c07daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.307+0000 7fb3510b1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c07daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56756/0 (socket says 172.21.15.69:56756) 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.307+0000 7fb3510b1700 1 -- 172.21.15.69:0/1281157421 learned_addr learned my addr 172.21.15.69:0/1281157421 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.307+0000 7fb3510b1700 1 -- 172.21.15.69:0/1281157421 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c07e1b0 con 0x7fb34c07f640 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.307+0000 7fb3510b1700 1 --2- 172.21.15.69:0/1281157421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c07daf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0098a0 tx=0x7fb33c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d43366e7a48f92b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.308+0000 7fb343fff700 1 -- 172.21.15.69:0/1281157421 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb33c004160 con 0x7fb34c07f640 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.308+0000 7fb343fff700 1 -- 172.21.15.69:0/1281157421 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb33c004300 con 0x7fb34c07f640 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.308+0000 7fb343fff700 1 -- 172.21.15.69:0/1281157421 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb33c011af0 con 0x7fb34c07f640 2024-05-14T16:39:12.097 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.308+0000 7fb353315700 1 -- 172.21.15.69:0/1281157421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 msgr2=0x7fb34c07daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.308+0000 7fb353315700 1 --2- 172.21.15.69:0/1281157421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c07daf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0098a0 tx=0x7fb33c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.308+0000 7fb353315700 1 -- 172.21.15.69:0/1281157421 shutdown_connections 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.308+0000 7fb353315700 1 --2- 172.21.15.69:0/1281157421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c07daf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.308+0000 7fb353315700 1 -- 172.21.15.69:0/1281157421 >> 172.21.15.69:0/1281157421 conn(0x7fb34c1044b0 msgr2=0x7fb34c1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.308+0000 7fb353315700 1 -- 172.21.15.69:0/1281157421 shutdown_connections 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.308+0000 7fb353315700 1 -- 172.21.15.69:0/1281157421 wait complete. 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.309+0000 7fb353315700 1 Processor -- start 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.309+0000 7fb353315700 1 -- start start 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.309+0000 7fb353315700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.309+0000 7fb353315700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c10f440 con 0x7fb34c07f640 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.309+0000 7fb3510b1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.309+0000 7fb3510b1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56772/0 (socket says 172.21.15.69:56772) 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.309+0000 7fb3510b1700 1 -- 172.21.15.69:0/2837276034 learned_addr learned my addr 172.21.15.69:0/2837276034 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.309+0000 7fb3510b1700 1 -- 172.21.15.69:0/2837276034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c19f560 con 0x7fb34c07f640 2024-05-14T16:39:12.098 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.310+0000 7fb3510b1700 1 --2- 172.21.15.69:0/2837276034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c19f020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb33c011ac0 tx=0x7fb33c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.310+0000 7fb3427fc700 1 -- 172.21.15.69:0/2837276034 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb33c003970 con 0x7fb34c07f640 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.310+0000 7fb3427fc700 1 -- 172.21.15.69:0/2837276034 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb33c003b10 con 0x7fb34c07f640 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.310+0000 7fb3427fc700 1 -- 172.21.15.69:0/2837276034 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb33c024610 con 0x7fb34c07f640 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.310+0000 7fb353315700 1 -- 172.21.15.69:0/2837276034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34c19f7f0 con 0x7fb34c07f640 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.310+0000 7fb353315700 1 -- 172.21.15.69:0/2837276034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34c1a24e0 con 0x7fb34c07f640 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.311+0000 7fb3427fc700 1 -- 172.21.15.69:0/2837276034 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb33c01a430 con 0x7fb34c07f640 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.311+0000 7fb353315700 1 -- 172.21.15.69:0/2837276034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34c064ba0 con 0x7fb34c07f640 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.311+0000 7fb3427fc700 1 --2- 172.21.15.69:0/2837276034 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb33803e470 0x7fb338040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.311+0000 7fb3427fc700 1 -- 172.21.15.69:0/2837276034 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb33c053850 con 0x7fb34c07f640 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.311+0000 7fb3508b0700 1 --2- 172.21.15.69:0/2837276034 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb33803e470 0x7fb338040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.311+0000 7fb3508b0700 1 --2- 172.21.15.69:0/2837276034 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb33803e470 0x7fb338040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb348006fd0 tx=0x7fb348006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.316+0000 7fb3427fc700 1 -- 172.21.15.69:0/2837276034 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb33c02de10 con 0x7fb34c07f640 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.520+0000 7fb353315700 1 -- 172.21.15.69:0/2837276034 --> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb34c105710 con 0x7fb33803e470 2024-05-14T16:39:12.099 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.521+0000 7fb3427fc700 1 -- 172.21.15.69:0/2837276034 <== mgr.14118 v2:172.21.15.69:6800/421328414 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb34c105710 con 0x7fb33803e470 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.523+0000 7fb353315700 1 -- 172.21.15.69:0/2837276034 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb33803e470 msgr2=0x7fb338040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.523+0000 7fb353315700 1 --2- 172.21.15.69:0/2837276034 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb33803e470 0x7fb338040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb348006fd0 tx=0x7fb348006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.523+0000 7fb353315700 1 -- 172.21.15.69:0/2837276034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 msgr2=0x7fb34c19f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.523+0000 7fb353315700 1 --2- 172.21.15.69:0/2837276034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c19f020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb33c011ac0 tx=0x7fb33c011ba0 comp rx=0 tx=0).stop 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.523+0000 7fb353315700 1 -- 172.21.15.69:0/2837276034 shutdown_connections 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.523+0000 7fb353315700 1 --2- 172.21.15.69:0/2837276034 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fb33803e470 0x7fb338040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.523+0000 7fb353315700 1 --2- 172.21.15.69:0/2837276034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb34c07f640 0x7fb34c19f020 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.523+0000 7fb353315700 1 -- 172.21.15.69:0/2837276034 >> 172.21.15.69:0/2837276034 conn(0x7fb34c1044b0 msgr2=0x7fb34c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.523+0000 7fb353315700 1 -- 172.21.15.69:0/2837276034 shutdown_connections 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:11.523+0000 7fb353315700 1 -- 172.21.15.69:0/2837276034 wait complete. 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:Adding key to root@localhost authorized_keys... 2024-05-14T16:39:12.100 INFO:teuthology.orchestra.run.smithi069.stdout:Adding host smithi069... 2024-05-14T16:39:12.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:12 smithi069 ceph-mon[31507]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:13.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:13 smithi069 ceph-mon[31507]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:13 smithi069 ceph-mon[31507]: Deploying cephadm binary to smithi069 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Added host 'smithi069' with addr '172.21.15.69' 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.484+0000 7fd85adaf700 1 Processor -- start 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.484+0000 7fd85adaf700 1 -- start start 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.484+0000 7fd85adaf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd854107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.484+0000 7fd85adaf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd85407d610 con 0x7fd8541077a0 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.484+0000 7fd858b4b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd854107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.484+0000 7fd858b4b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd854107bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56778/0 (socket says 172.21.15.69:56778) 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.484+0000 7fd858b4b700 1 -- 172.21.15.69:0/2257918893 learned_addr learned my addr 172.21.15.69:0/2257918893 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.485+0000 7fd858b4b700 1 -- 172.21.15.69:0/2257918893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd854108100 con 0x7fd8541077a0 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.485+0000 7fd858b4b700 1 --2- 172.21.15.69:0/2257918893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd854107bc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd8480098a0 tx=0x7fd848009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2397bfa370ee75ec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:17.287 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.485+0000 7fd8537fe700 1 -- 172.21.15.69:0/2257918893 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd848004160 con 0x7fd8541077a0 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.485+0000 7fd8537fe700 1 -- 172.21.15.69:0/2257918893 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd848004300 con 0x7fd8541077a0 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.485+0000 7fd8537fe700 1 -- 172.21.15.69:0/2257918893 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd848011af0 con 0x7fd8541077a0 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.486+0000 7fd85adaf700 1 -- 172.21.15.69:0/2257918893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 msgr2=0x7fd854107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.486+0000 7fd85adaf700 1 --2- 172.21.15.69:0/2257918893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd854107bc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd8480098a0 tx=0x7fd848009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.486+0000 7fd85adaf700 1 -- 172.21.15.69:0/2257918893 shutdown_connections 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.486+0000 7fd85adaf700 1 --2- 172.21.15.69:0/2257918893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd854107bc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.486+0000 7fd85adaf700 1 -- 172.21.15.69:0/2257918893 >> 172.21.15.69:0/2257918893 conn(0x7fd8541033e0 msgr2=0x7fd854105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.486+0000 7fd85adaf700 1 -- 172.21.15.69:0/2257918893 shutdown_connections 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.486+0000 7fd85adaf700 1 -- 172.21.15.69:0/2257918893 wait complete. 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.486+0000 7fd85adaf700 1 Processor -- start 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.486+0000 7fd85adaf700 1 -- start start 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd85adaf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd85419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd85adaf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd854109130 con 0x7fd8541077a0 2024-05-14T16:39:17.288 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd858b4b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd85419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd858b4b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd85419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56790/0 (socket says 172.21.15.69:56790) 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd858b4b700 1 -- 172.21.15.69:0/2939264718 learned_addr learned my addr 172.21.15.69:0/2939264718 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd858b4b700 1 -- 172.21.15.69:0/2939264718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85419f540 con 0x7fd8541077a0 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd858b4b700 1 --2- 172.21.15.69:0/2939264718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd85419f000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd848009fb0 tx=0x7fd848011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd851ffb700 1 -- 172.21.15.69:0/2939264718 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8480036a0 con 0x7fd8541077a0 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd851ffb700 1 -- 172.21.15.69:0/2939264718 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd84801a430 con 0x7fd8541077a0 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd85adaf700 1 -- 172.21.15.69:0/2939264718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85419f7d0 con 0x7fd8541077a0 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd851ffb700 1 -- 172.21.15.69:0/2939264718 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd848019550 con 0x7fd8541077a0 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.487+0000 7fd85adaf700 1 -- 172.21.15.69:0/2939264718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8541a24c0 con 0x7fd8541077a0 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.488+0000 7fd851ffb700 1 -- 172.21.15.69:0/2939264718 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd848003840 con 0x7fd8541077a0 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.488+0000 7fd85adaf700 1 -- 172.21.15.69:0/2939264718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd854064ba0 con 0x7fd8541077a0 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.488+0000 7fd851ffb700 1 --2- 172.21.15.69:0/2939264718 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fd84403e420 0x7fd8440408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:17.289 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.489+0000 7fd851ffb700 1 -- 172.21.15.69:0/2939264718 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd848053860 con 0x7fd8541077a0 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.489+0000 7fd853fff700 1 --2- 172.21.15.69:0/2939264718 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fd84403e420 0x7fd8440408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.489+0000 7fd853fff700 1 --2- 172.21.15.69:0/2939264718 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fd84403e420 0x7fd8440408e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd840006fd0 tx=0x7fd840006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.494+0000 7fd851ffb700 1 -- 172.21.15.69:0/2939264718 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd84800b3e0 con 0x7fd8541077a0 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:12.700+0000 7fd85adaf700 1 -- 172.21.15.69:0/2939264718 --> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}) -- 0x7fd8540008d0 con 0x7fd84403e420 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.758+0000 7fd851ffb700 1 -- 172.21.15.69:0/2939264718 <== mgr.14118 v2:172.21.15.69:6800/421328414 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fd8540008d0 con 0x7fd84403e420 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.761+0000 7fd85adaf700 1 -- 172.21.15.69:0/2939264718 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fd84403e420 msgr2=0x7fd8440408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.761+0000 7fd85adaf700 1 --2- 172.21.15.69:0/2939264718 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fd84403e420 0x7fd8440408e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd840006fd0 tx=0x7fd840006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.761+0000 7fd85adaf700 1 -- 172.21.15.69:0/2939264718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 msgr2=0x7fd85419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.761+0000 7fd85adaf700 1 --2- 172.21.15.69:0/2939264718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd85419f000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd848009fb0 tx=0x7fd848011940 comp rx=0 tx=0).stop 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.761+0000 7fd85adaf700 1 -- 172.21.15.69:0/2939264718 shutdown_connections 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.761+0000 7fd85adaf700 1 --2- 172.21.15.69:0/2939264718 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fd84403e420 0x7fd8440408e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.761+0000 7fd85adaf700 1 --2- 172.21.15.69:0/2939264718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8541077a0 0x7fd85419f000 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.761+0000 7fd85adaf700 1 -- 172.21.15.69:0/2939264718 >> 172.21.15.69:0/2939264718 conn(0x7fd8541033e0 msgr2=0x7fd854195c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:17.290 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.761+0000 7fd85adaf700 1 -- 172.21.15.69:0/2939264718 shutdown_connections 2024-05-14T16:39:17.291 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:16.761+0000 7fd85adaf700 1 -- 172.21.15.69:0/2939264718 wait complete. 2024-05-14T16:39:17.291 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mon service... 2024-05-14T16:39:17.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:17 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:17.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:17 smithi069 ceph-mon[31507]: Added host smithi069 2024-05-14T16:39:17.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:17 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.727+0000 7f4855566700 1 Processor -- start 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.727+0000 7f4855566700 1 -- start start 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.728+0000 7f4855566700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485010bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.728+0000 7f4855566700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f485007cdf0 con 0x7f4850109990 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.728+0000 7f484effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485010bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.728+0000 7f484effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485010bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40452/0 (socket says 172.21.15.69:40452) 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.728+0000 7f484effd700 1 -- 172.21.15.69:0/2009045282 learned_addr learned my addr 172.21.15.69:0/2009045282 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.728+0000 7f484effd700 1 -- 172.21.15.69:0/2009045282 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485010c2c0 con 0x7f4850109990 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.728+0000 7f484effd700 1 --2- 172.21.15.69:0/2009045282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485010bd80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f48400098a0 tx=0x7f4840009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c3e6fff5d92fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:18.586 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.728+0000 7f484e7fc700 1 -- 172.21.15.69:0/2009045282 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4840004160 con 0x7f4850109990 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.728+0000 7f484e7fc700 1 -- 172.21.15.69:0/2009045282 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4840004300 con 0x7f4850109990 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.728+0000 7f484e7fc700 1 -- 172.21.15.69:0/2009045282 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4840011af0 con 0x7f4850109990 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.729+0000 7f4855566700 1 -- 172.21.15.69:0/2009045282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 msgr2=0x7f485010bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.729+0000 7f4855566700 1 --2- 172.21.15.69:0/2009045282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485010bd80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f48400098a0 tx=0x7f4840009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.729+0000 7f4855566700 1 -- 172.21.15.69:0/2009045282 shutdown_connections 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.729+0000 7f4855566700 1 --2- 172.21.15.69:0/2009045282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485010bd80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.729+0000 7f4855566700 1 -- 172.21.15.69:0/2009045282 >> 172.21.15.69:0/2009045282 conn(0x7f48501033a0 msgr2=0x7f48501057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.729+0000 7f4855566700 1 -- 172.21.15.69:0/2009045282 shutdown_connections 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.729+0000 7f4855566700 1 -- 172.21.15.69:0/2009045282 wait complete. 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.730+0000 7f4855566700 1 Processor -- start 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.730+0000 7f4855566700 1 -- start start 2024-05-14T16:39:18.587 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.730+0000 7f4855566700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.730+0000 7f4855566700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f485010d280 con 0x7f4850109990 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.730+0000 7f484effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.730+0000 7f484effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40462/0 (socket says 172.21.15.69:40462) 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.730+0000 7f484effd700 1 -- 172.21.15.69:0/2359678120 learned_addr learned my addr 172.21.15.69:0/2359678120 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.730+0000 7f484effd700 1 -- 172.21.15.69:0/2359678120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485019d3c0 con 0x7f4850109990 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.731+0000 7f484effd700 1 --2- 172.21.15.69:0/2359678120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485019ce80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4840009fb0 tx=0x7f4840011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.731+0000 7f484cff9700 1 -- 172.21.15.69:0/2359678120 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48400036a0 con 0x7f4850109990 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.731+0000 7f484cff9700 1 -- 172.21.15.69:0/2359678120 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f484001a430 con 0x7f4850109990 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.731+0000 7f4855566700 1 -- 172.21.15.69:0/2359678120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485019d650 con 0x7f4850109990 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.731+0000 7f484cff9700 1 -- 172.21.15.69:0/2359678120 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4840019550 con 0x7f4850109990 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.731+0000 7f4855566700 1 -- 172.21.15.69:0/2359678120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48501a0340 con 0x7f4850109990 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.732+0000 7f484cff9700 1 -- 172.21.15.69:0/2359678120 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4840019800 con 0x7f4850109990 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.732+0000 7f4855566700 1 -- 172.21.15.69:0/2359678120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4850064ba0 con 0x7f4850109990 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.732+0000 7f484cff9700 1 --2- 172.21.15.69:0/2359678120 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f483c03e470 0x7f483c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:18.588 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.732+0000 7f484cff9700 1 -- 172.21.15.69:0/2359678120 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f48400537f0 con 0x7f4850109990 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.732+0000 7f4847fff700 1 --2- 172.21.15.69:0/2359678120 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f483c03e470 0x7f483c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.732+0000 7f4847fff700 1 --2- 172.21.15.69:0/2359678120 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f483c03e470 0x7f483c040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4838006fd0 tx=0x7f4838006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.737+0000 7f484cff9700 1 -- 172.21.15.69:0/2359678120 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f484001a690 con 0x7f4850109990 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.957+0000 7f4855566700 1 -- 172.21.15.69:0/2359678120 --> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f48501045f0 con 0x7f483c03e470 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.965+0000 7f484cff9700 1 -- 172.21.15.69:0/2359678120 <== mgr.14118 v2:172.21.15.69:6800/421328414 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f48501045f0 con 0x7f483c03e470 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.967+0000 7f4855566700 1 -- 172.21.15.69:0/2359678120 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f483c03e470 msgr2=0x7f483c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.967+0000 7f4855566700 1 --2- 172.21.15.69:0/2359678120 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f483c03e470 0x7f483c040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4838006fd0 tx=0x7f4838006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.967+0000 7f4855566700 1 -- 172.21.15.69:0/2359678120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 msgr2=0x7f485019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.967+0000 7f4855566700 1 --2- 172.21.15.69:0/2359678120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485019ce80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4840009fb0 tx=0x7f4840011940 comp rx=0 tx=0).stop 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.967+0000 7f4855566700 1 -- 172.21.15.69:0/2359678120 shutdown_connections 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.967+0000 7f4855566700 1 --2- 172.21.15.69:0/2359678120 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f483c03e470 0x7f483c040930 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.967+0000 7f4855566700 1 --2- 172.21.15.69:0/2359678120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4850109990 0x7f485019ce80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.967+0000 7f4855566700 1 -- 172.21.15.69:0/2359678120 >> 172.21.15.69:0/2359678120 conn(0x7f48501033a0 msgr2=0x7f4850103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:18.589 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.968+0000 7f4855566700 1 -- 172.21.15.69:0/2359678120 shutdown_connections 2024-05-14T16:39:18.590 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:17.968+0000 7f4855566700 1 -- 172.21.15.69:0/2359678120 wait complete. 2024-05-14T16:39:18.590 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mgr service... 2024-05-14T16:39:19.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:18 smithi069 ceph-mon[31507]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:19.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:18 smithi069 ceph-mon[31507]: Saving service mon spec with placement count:5 2024-05-14T16:39:19.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:18 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:19.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-14T16:39:19.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.969+0000 7f4a0e6f8700 1 Processor -- start 2024-05-14T16:39:19.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.969+0000 7f4a0e6f8700 1 -- start start 2024-05-14T16:39:19.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.969+0000 7f4a0e6f8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a08109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:19.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.969+0000 7f4a0e6f8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a081097c0 con 0x7f4a08108e60 2024-05-14T16:39:19.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.969+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a08109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:19.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.969+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a08109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40468/0 (socket says 172.21.15.69:40468) 2024-05-14T16:39:19.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.969+0000 7f4a07fff700 1 -- 172.21.15.69:0/3357447604 learned_addr learned my addr 172.21.15.69:0/3357447604 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:19.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.969+0000 7f4a07fff700 1 -- 172.21.15.69:0/3357447604 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0810a050 con 0x7f4a08108e60 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.969+0000 7f4a07fff700 1 --2- 172.21.15.69:0/3357447604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a08109280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f49f80098a0 tx=0x7f49f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c059571552b9c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.970+0000 7f4a077fe700 1 -- 172.21.15.69:0/3357447604 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49f8004160 con 0x7f4a08108e60 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.970+0000 7f4a077fe700 1 -- 172.21.15.69:0/3357447604 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49f8004300 con 0x7f4a08108e60 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.970+0000 7f4a077fe700 1 -- 172.21.15.69:0/3357447604 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49f8011af0 con 0x7f4a08108e60 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.970+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/3357447604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 msgr2=0x7f4a08109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.970+0000 7f4a0e6f8700 1 --2- 172.21.15.69:0/3357447604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a08109280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f49f80098a0 tx=0x7f49f8009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.970+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/3357447604 shutdown_connections 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.970+0000 7f4a0e6f8700 1 --2- 172.21.15.69:0/3357447604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a08109280 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.970+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/3357447604 >> 172.21.15.69:0/3357447604 conn(0x7f4a08104450 msgr2=0x7f4a08106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.970+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/3357447604 shutdown_connections 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.970+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/3357447604 wait complete. 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.971+0000 7f4a0e6f8700 1 Processor -- start 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.971+0000 7f4a0e6f8700 1 -- start start 2024-05-14T16:39:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.971+0000 7f4a0e6f8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a0819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.971+0000 7f4a0e6f8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0810d310 con 0x7f4a08108e60 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.971+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a0819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.971+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a0819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40484/0 (socket says 172.21.15.69:40484) 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.971+0000 7f4a07fff700 1 -- 172.21.15.69:0/513855250 learned_addr learned my addr 172.21.15.69:0/513855250 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.971+0000 7f4a07fff700 1 -- 172.21.15.69:0/513855250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0819d3d0 con 0x7f4a08108e60 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.972+0000 7f4a07fff700 1 --2- 172.21.15.69:0/513855250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a0819ce90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f49f80117b0 tx=0x7f49f8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.972+0000 7f4a05ffb700 1 -- 172.21.15.69:0/513855250 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49f8003af0 con 0x7f4a08108e60 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.972+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/513855250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0819d660 con 0x7f4a08108e60 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.972+0000 7f4a05ffb700 1 -- 172.21.15.69:0/513855250 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49f8020070 con 0x7f4a08108e60 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.972+0000 7f4a05ffb700 1 -- 172.21.15.69:0/513855250 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49f8019630 con 0x7f4a08108e60 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.972+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/513855250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a081a0350 con 0x7f4a08108e60 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.973+0000 7f4a05ffb700 1 -- 172.21.15.69:0/513855250 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f49f80198f0 con 0x7f4a08108e60 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.973+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/513855250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a08064ba0 con 0x7f4a08108e60 2024-05-14T16:39:19.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.973+0000 7f4a05ffb700 1 --2- 172.21.15.69:0/513855250 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f49f403e470 0x7f49f4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.973+0000 7f4a05ffb700 1 -- 172.21.15.69:0/513855250 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f49f8053ae0 con 0x7f4a08108e60 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.973+0000 7f49fffff700 1 --2- 172.21.15.69:0/513855250 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f49f403e470 0x7f49f4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.973+0000 7f49fffff700 1 --2- 172.21.15.69:0/513855250 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f49f403e470 0x7f49f4040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f49f0006fd0 tx=0x7f49f0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:18.978+0000 7f4a05ffb700 1 -- 172.21.15.69:0/513855250 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49f802b260 con 0x7f4a08108e60 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.198+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/513855250 --> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4a081056e0 con 0x7f49f403e470 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.205+0000 7f4a05ffb700 1 -- 172.21.15.69:0/513855250 <== mgr.14118 v2:172.21.15.69:6800/421328414 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4a081056e0 con 0x7f49f403e470 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.207+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/513855250 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f49f403e470 msgr2=0x7f49f4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.207+0000 7f4a0e6f8700 1 --2- 172.21.15.69:0/513855250 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f49f403e470 0x7f49f4040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f49f0006fd0 tx=0x7f49f0006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.207+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/513855250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 msgr2=0x7f4a0819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.207+0000 7f4a0e6f8700 1 --2- 172.21.15.69:0/513855250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a0819ce90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f49f80117b0 tx=0x7f49f8011b70 comp rx=0 tx=0).stop 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.208+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/513855250 shutdown_connections 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.208+0000 7f4a0e6f8700 1 --2- 172.21.15.69:0/513855250 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f49f403e470 0x7f49f4040930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.208+0000 7f4a0e6f8700 1 --2- 172.21.15.69:0/513855250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a08108e60 0x7f4a0819ce90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:19.979 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.208+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/513855250 >> 172.21.15.69:0/513855250 conn(0x7f4a08104450 msgr2=0x7f4a08104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:19.979 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.208+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/513855250 shutdown_connections 2024-05-14T16:39:19.979 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:19.208+0000 7f4a0e6f8700 1 -- 172.21.15.69:0/513855250 wait complete. 2024-05-14T16:39:20.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:20 smithi069 ceph-mon[31507]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:20.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:20 smithi069 ceph-mon[31507]: Saving service mgr spec with placement count:2 2024-05-14T16:39:20.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:20 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:21.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:21 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:21.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:21 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:21.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:21 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1694013823' entity='client.admin' 2024-05-14T16:39:21.806 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.001+0000 7f0a39f5e700 1 Processor -- start 2024-05-14T16:39:21.806 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.001+0000 7f0a39f5e700 1 -- start start 2024-05-14T16:39:21.806 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.001+0000 7f0a39f5e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3410bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:21.806 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.001+0000 7f0a39f5e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3407d600 con 0x7f0a341099c0 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.002+0000 7f0a337fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3410bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.002+0000 7f0a337fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3410bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40490/0 (socket says 172.21.15.69:40490) 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.002+0000 7f0a337fe700 1 -- 172.21.15.69:0/3692868062 learned_addr learned my addr 172.21.15.69:0/3692868062 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.002+0000 7f0a337fe700 1 -- 172.21.15.69:0/3692868062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3410c2f0 con 0x7f0a341099c0 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.002+0000 7f0a337fe700 1 --2- 172.21.15.69:0/3692868062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3410bdb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a240098a0 tx=0x7f0a24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e86cb12ed0b68f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.002+0000 7f0a327fc700 1 -- 172.21.15.69:0/3692868062 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a24004160 con 0x7f0a341099c0 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.002+0000 7f0a327fc700 1 -- 172.21.15.69:0/3692868062 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a24004300 con 0x7f0a341099c0 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.002+0000 7f0a327fc700 1 -- 172.21.15.69:0/3692868062 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a24011a20 con 0x7f0a341099c0 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.003+0000 7f0a39f5e700 1 -- 172.21.15.69:0/3692868062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 msgr2=0x7f0a3410bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.003+0000 7f0a39f5e700 1 --2- 172.21.15.69:0/3692868062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3410bdb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a240098a0 tx=0x7f0a24009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.003+0000 7f0a39f5e700 1 -- 172.21.15.69:0/3692868062 shutdown_connections 2024-05-14T16:39:21.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.003+0000 7f0a39f5e700 1 --2- 172.21.15.69:0/3692868062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3410bdb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.003+0000 7f0a39f5e700 1 -- 172.21.15.69:0/3692868062 >> 172.21.15.69:0/3692868062 conn(0x7f0a34103330 msgr2=0x7f0a34105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.003+0000 7f0a39f5e700 1 -- 172.21.15.69:0/3692868062 shutdown_connections 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.003+0000 7f0a39f5e700 1 -- 172.21.15.69:0/3692868062 wait complete. 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.004+0000 7f0a39f5e700 1 Processor -- start 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.004+0000 7f0a39f5e700 1 -- start start 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.004+0000 7f0a39f5e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.004+0000 7f0a39f5e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3410d240 con 0x7f0a341099c0 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.004+0000 7f0a337fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.004+0000 7f0a337fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40502/0 (socket says 172.21.15.69:40502) 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.004+0000 7f0a337fe700 1 -- 172.21.15.69:0/1694013823 learned_addr learned my addr 172.21.15.69:0/1694013823 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.005+0000 7f0a337fe700 1 -- 172.21.15.69:0/1694013823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3419d2f0 con 0x7f0a341099c0 2024-05-14T16:39:21.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.005+0000 7f0a337fe700 1 --2- 172.21.15.69:0/1694013823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3419cdb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0a24004660 tx=0x7f0a24011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.005+0000 7f0a30ff9700 1 -- 172.21.15.69:0/1694013823 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a24003aa0 con 0x7f0a341099c0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.005+0000 7f0a39f5e700 1 -- 172.21.15.69:0/1694013823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3419d580 con 0x7f0a341099c0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.005+0000 7f0a39f5e700 1 -- 172.21.15.69:0/1694013823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a341a0230 con 0x7f0a341099c0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.005+0000 7f0a30ff9700 1 -- 172.21.15.69:0/1694013823 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a2401b8f0 con 0x7f0a341099c0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.005+0000 7f0a30ff9700 1 -- 172.21.15.69:0/1694013823 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a24023900 con 0x7f0a341099c0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.006+0000 7f0a30ff9700 1 -- 172.21.15.69:0/1694013823 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0a24023b70 con 0x7f0a341099c0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.006+0000 7f0a39f5e700 1 -- 172.21.15.69:0/1694013823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a34064ba0 con 0x7f0a341099c0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.006+0000 7f0a30ff9700 1 --2- 172.21.15.69:0/1694013823 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f0a1c03e4c0 0x7f0a1c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.006+0000 7f0a30ff9700 1 -- 172.21.15.69:0/1694013823 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0a24053db0 con 0x7f0a341099c0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.006+0000 7f0a32ffd700 1 --2- 172.21.15.69:0/1694013823 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f0a1c03e4c0 0x7f0a1c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.007+0000 7f0a32ffd700 1 --2- 172.21.15.69:0/1694013823 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f0a1c03e4c0 0x7f0a1c040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a28006fd0 tx=0x7f0a28006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:21.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.011+0000 7f0a30ff9700 1 -- 172.21.15.69:0/1694013823 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a2401f030 con 0x7f0a341099c0 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.225+0000 7f0a39f5e700 1 -- 172.21.15.69:0/1694013823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f0a341a08b0 con 0x7f0a341099c0 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.231+0000 7f0a30ff9700 1 -- 172.21.15.69:0/1694013823 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f0a2401bde0 con 0x7f0a341099c0 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.234+0000 7f0a39f5e700 1 -- 172.21.15.69:0/1694013823 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f0a1c03e4c0 msgr2=0x7f0a1c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.234+0000 7f0a39f5e700 1 --2- 172.21.15.69:0/1694013823 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f0a1c03e4c0 0x7f0a1c040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a28006fd0 tx=0x7f0a28006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.234+0000 7f0a39f5e700 1 -- 172.21.15.69:0/1694013823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 msgr2=0x7f0a3419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.234+0000 7f0a39f5e700 1 --2- 172.21.15.69:0/1694013823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3419cdb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0a24004660 tx=0x7f0a24011aa0 comp rx=0 tx=0).stop 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.234+0000 7f0a39f5e700 1 -- 172.21.15.69:0/1694013823 shutdown_connections 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.234+0000 7f0a39f5e700 1 --2- 172.21.15.69:0/1694013823 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f0a1c03e4c0 0x7f0a1c040980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.234+0000 7f0a39f5e700 1 --2- 172.21.15.69:0/1694013823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a341099c0 0x7f0a3419cdb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.234+0000 7f0a39f5e700 1 -- 172.21.15.69:0/1694013823 >> 172.21.15.69:0/1694013823 conn(0x7f0a34103330 msgr2=0x7f0a34103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.234+0000 7f0a39f5e700 1 -- 172.21.15.69:0/1694013823 shutdown_connections 2024-05-14T16:39:21.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:21.234+0000 7f0a39f5e700 1 -- 172.21.15.69:0/1694013823 wait complete. 2024-05-14T16:39:23.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:23 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/4009426081' entity='client.admin' 2024-05-14T16:39:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:23 smithi069 ceph-mon[31507]: from='mgr.14118 172.21.15.69:0/3081219767' entity='mgr.x' 2024-05-14T16:39:23.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.269+0000 7fc0e0b36700 1 Processor -- start 2024-05-14T16:39:23.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.269+0000 7fc0e0b36700 1 -- start start 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.269+0000 7fc0e0b36700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc07db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.269+0000 7fc0e0b36700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc07e050 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.269+0000 7fc0da59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc07db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.269+0000 7fc0da59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc07db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40506/0 (socket says 172.21.15.69:40506) 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.269+0000 7fc0da59c700 1 -- 172.21.15.69:0/3460790470 learned_addr learned my addr 172.21.15.69:0/3460790470 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.269+0000 7fc0da59c700 1 -- 172.21.15.69:0/3460790470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0dc07e1d0 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0da59c700 1 --2- 172.21.15.69:0/3460790470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc07db10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0098a0 tx=0x7fc0cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fd3b393a81e45b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0d959a700 1 -- 172.21.15.69:0/3460790470 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0cc004160 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0d959a700 1 -- 172.21.15.69:0/3460790470 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0cc004300 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0d959a700 1 -- 172.21.15.69:0/3460790470 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0cc011a20 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0e0b36700 1 -- 172.21.15.69:0/3460790470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 msgr2=0x7fc0dc07db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0e0b36700 1 --2- 172.21.15.69:0/3460790470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc07db10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0098a0 tx=0x7fc0cc009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0e0b36700 1 -- 172.21.15.69:0/3460790470 shutdown_connections 2024-05-14T16:39:23.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0e0b36700 1 --2- 172.21.15.69:0/3460790470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc07db10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0e0b36700 1 -- 172.21.15.69:0/3460790470 >> 172.21.15.69:0/3460790470 conn(0x7fc0dc104520 msgr2=0x7fc0dc106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0e0b36700 1 -- 172.21.15.69:0/3460790470 shutdown_connections 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.270+0000 7fc0e0b36700 1 -- 172.21.15.69:0/3460790470 wait complete. 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.271+0000 7fc0e0b36700 1 Processor -- start 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.271+0000 7fc0e0b36700 1 -- start start 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.271+0000 7fc0e0b36700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.271+0000 7fc0e0b36700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc10f350 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.271+0000 7fc0da59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.271+0000 7fc0da59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40516/0 (socket says 172.21.15.69:40516) 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.271+0000 7fc0da59c700 1 -- 172.21.15.69:0/4009426081 learned_addr learned my addr 172.21.15.69:0/4009426081 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.272+0000 7fc0da59c700 1 -- 172.21.15.69:0/4009426081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0dc19f4e0 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.272+0000 7fc0da59c700 1 --2- 172.21.15.69:0/4009426081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc19efa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0116e0 tx=0x7fc0cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.272+0000 7fc0cb7fe700 1 -- 172.21.15.69:0/4009426081 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0cc003b20 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.272+0000 7fc0cb7fe700 1 -- 172.21.15.69:0/4009426081 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0cc01b8f0 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.272+0000 7fc0e0b36700 1 -- 172.21.15.69:0/4009426081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0dc19f770 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.272+0000 7fc0cb7fe700 1 -- 172.21.15.69:0/4009426081 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0cc0238c0 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.272+0000 7fc0e0b36700 1 -- 172.21.15.69:0/4009426081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0dc1a2460 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.273+0000 7fc0e0b36700 1 -- 172.21.15.69:0/4009426081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0dc064ba0 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.273+0000 7fc0cb7fe700 1 -- 172.21.15.69:0/4009426081 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc0cc023a60 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.273+0000 7fc0cb7fe700 1 --2- 172.21.15.69:0/4009426081 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fc0c403e470 0x7fc0c4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.273+0000 7fc0cb7fe700 1 -- 172.21.15.69:0/4009426081 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc0cc0540a0 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.273+0000 7fc0d9d9b700 1 --2- 172.21.15.69:0/4009426081 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fc0c403e470 0x7fc0c4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.274+0000 7fc0d9d9b700 1 --2- 172.21.15.69:0/4009426081 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fc0c403e470 0x7fc0c4040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0006fd0 tx=0x7fc0d0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.278+0000 7fc0cb7fe700 1 -- 172.21.15.69:0/4009426081 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0cc01f030 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.503+0000 7fc0e0b36700 1 -- 172.21.15.69:0/4009426081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc0dc1a2a80 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.506+0000 7fc0cb7fe700 1 -- 172.21.15.69:0/4009426081 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fc0cc012040 con 0x7fc0dc07f5a0 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.508+0000 7fc0e0b36700 1 -- 172.21.15.69:0/4009426081 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fc0c403e470 msgr2=0x7fc0c4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:23.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.508+0000 7fc0e0b36700 1 --2- 172.21.15.69:0/4009426081 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fc0c403e470 0x7fc0c4040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0006fd0 tx=0x7fc0d0006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:23.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.508+0000 7fc0e0b36700 1 -- 172.21.15.69:0/4009426081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 msgr2=0x7fc0dc19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:23.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.508+0000 7fc0e0b36700 1 --2- 172.21.15.69:0/4009426081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc19efa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0116e0 tx=0x7fc0cc011aa0 comp rx=0 tx=0).stop 2024-05-14T16:39:23.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.508+0000 7fc0e0b36700 1 -- 172.21.15.69:0/4009426081 shutdown_connections 2024-05-14T16:39:23.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.508+0000 7fc0e0b36700 1 --2- 172.21.15.69:0/4009426081 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7fc0c403e470 0x7fc0c4040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:23.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.508+0000 7fc0e0b36700 1 --2- 172.21.15.69:0/4009426081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc07f5a0 0x7fc0dc19efa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:23.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.508+0000 7fc0e0b36700 1 -- 172.21.15.69:0/4009426081 >> 172.21.15.69:0/4009426081 conn(0x7fc0dc104520 msgr2=0x7fc0dc105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:23.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.508+0000 7fc0e0b36700 1 -- 172.21.15.69:0/4009426081 shutdown_connections 2024-05-14T16:39:23.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:22.508+0000 7fc0e0b36700 1 -- 172.21.15.69:0/4009426081 wait complete. 2024-05-14T16:39:23.793 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling the dashboard module... 2024-05-14T16:39:24.846 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:24 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1643202477' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-14T16:39:24.846 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:24 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: ignoring --setuser ceph since I am not root 2024-05-14T16:39:24.847 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:24 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: ignoring --setgroup ceph since I am not root 2024-05-14T16:39:24.847 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:24 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:24.759+0000 7fbcd08f0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:39:25.139 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:24 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:24.952+0000 7fbcd08f0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:39:25.139 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:25 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:25.056+0000 7fbcd08f0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:39:25.147 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.243+0000 7f6b3dc45700 1 Processor -- start 2024-05-14T16:39:25.147 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.243+0000 7f6b3dc45700 1 -- start start 2024-05-14T16:39:25.147 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.243+0000 7f6b3dc45700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3810bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:25.147 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.243+0000 7f6b3dc45700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3807cdf0 con 0x7f6b38109990 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.243+0000 7f6b377fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3810bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.243+0000 7f6b377fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3810bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40524/0 (socket says 172.21.15.69:40524) 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.243+0000 7f6b377fe700 1 -- 172.21.15.69:0/67894425 learned_addr learned my addr 172.21.15.69:0/67894425 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.243+0000 7f6b377fe700 1 -- 172.21.15.69:0/67894425 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3810c2c0 con 0x7f6b38109990 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.244+0000 7f6b377fe700 1 --2- 172.21.15.69:0/67894425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3810bd80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b280098a0 tx=0x7f6b28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd424a7a4149584c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.244+0000 7f6b367fc700 1 -- 172.21.15.69:0/67894425 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b28004160 con 0x7f6b38109990 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.244+0000 7f6b367fc700 1 -- 172.21.15.69:0/67894425 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b28004300 con 0x7f6b38109990 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.244+0000 7f6b367fc700 1 -- 172.21.15.69:0/67894425 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b28011a20 con 0x7f6b38109990 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.245+0000 7f6b3dc45700 1 -- 172.21.15.69:0/67894425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 msgr2=0x7f6b3810bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.245+0000 7f6b3dc45700 1 --2- 172.21.15.69:0/67894425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3810bd80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b280098a0 tx=0x7f6b28009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.245+0000 7f6b3dc45700 1 -- 172.21.15.69:0/67894425 shutdown_connections 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.245+0000 7f6b3dc45700 1 --2- 172.21.15.69:0/67894425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3810bd80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.245+0000 7f6b3dc45700 1 -- 172.21.15.69:0/67894425 >> 172.21.15.69:0/67894425 conn(0x7f6b381033a0 msgr2=0x7f6b381057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.245+0000 7f6b3dc45700 1 -- 172.21.15.69:0/67894425 shutdown_connections 2024-05-14T16:39:25.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.245+0000 7f6b3dc45700 1 -- 172.21.15.69:0/67894425 wait complete. 2024-05-14T16:39:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.245+0000 7f6b3dc45700 1 Processor -- start 2024-05-14T16:39:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.246+0000 7f6b3dc45700 1 -- start start 2024-05-14T16:39:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.246+0000 7f6b3dc45700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.246+0000 7f6b3dc45700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3810d2c0 con 0x7f6b38109990 2024-05-14T16:39:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.246+0000 7f6b377fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.246+0000 7f6b377fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40528/0 (socket says 172.21.15.69:40528) 2024-05-14T16:39:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.246+0000 7f6b377fe700 1 -- 172.21.15.69:0/1643202477 learned_addr learned my addr 172.21.15.69:0/1643202477 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.246+0000 7f6b377fe700 1 -- 172.21.15.69:0/1643202477 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3819d330 con 0x7f6b38109990 2024-05-14T16:39:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.246+0000 7f6b377fe700 1 --2- 172.21.15.69:0/1643202477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3819cdf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6b280116e0 tx=0x7f6b28011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.247+0000 7f6b34ff9700 1 -- 172.21.15.69:0/1643202477 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b28003b20 con 0x7f6b38109990 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.247+0000 7f6b34ff9700 1 -- 172.21.15.69:0/1643202477 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b2801b8f0 con 0x7f6b38109990 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.247+0000 7f6b3dc45700 1 -- 172.21.15.69:0/1643202477 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3819d5c0 con 0x7f6b38109990 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.247+0000 7f6b34ff9700 1 -- 172.21.15.69:0/1643202477 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b280238c0 con 0x7f6b38109990 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.247+0000 7f6b3dc45700 1 -- 172.21.15.69:0/1643202477 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b381a02b0 con 0x7f6b38109990 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.248+0000 7f6b3dc45700 1 -- 172.21.15.69:0/1643202477 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b38064ba0 con 0x7f6b38109990 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.248+0000 7f6b34ff9700 1 -- 172.21.15.69:0/1643202477 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6b28023a60 con 0x7f6b38109990 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.248+0000 7f6b34ff9700 1 --2- 172.21.15.69:0/1643202477 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f6b2003e470 0x7f6b20040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.248+0000 7f6b34ff9700 1 -- 172.21.15.69:0/1643202477 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b28053f00 con 0x7f6b38109990 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.248+0000 7f6b36ffd700 1 --2- 172.21.15.69:0/1643202477 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f6b2003e470 0x7f6b20040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.249+0000 7f6b36ffd700 1 --2- 172.21.15.69:0/1643202477 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f6b2003e470 0x7f6b20040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c006fd0 tx=0x7f6b2c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.253+0000 7f6b34ff9700 1 -- 172.21.15.69:0/1643202477 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b2801f030 con 0x7f6b38109990 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.526+0000 7f6b3dc45700 1 -- 172.21.15.69:0/1643202477 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f6b381a0860 con 0x7f6b38109990 2024-05-14T16:39:25.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.578+0000 7f6b34ff9700 1 -- 172.21.15.69:0/1643202477 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f6b2800b3e0 con 0x7f6b38109990 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.580+0000 7f6b34ff9700 1 -- 172.21.15.69:0/1643202477 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f6b2801baa0 con 0x7f6b38109990 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.582+0000 7f6b3dc45700 1 -- 172.21.15.69:0/1643202477 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f6b2003e470 msgr2=0x7f6b20040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.582+0000 7f6b3dc45700 1 --2- 172.21.15.69:0/1643202477 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f6b2003e470 0x7f6b20040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c006fd0 tx=0x7f6b2c006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.582+0000 7f6b3dc45700 1 -- 172.21.15.69:0/1643202477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 msgr2=0x7f6b3819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.582+0000 7f6b3dc45700 1 --2- 172.21.15.69:0/1643202477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3819cdf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6b280116e0 tx=0x7f6b28011aa0 comp rx=0 tx=0).stop 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.582+0000 7f6b3dc45700 1 -- 172.21.15.69:0/1643202477 shutdown_connections 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.582+0000 7f6b3dc45700 1 --2- 172.21.15.69:0/1643202477 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f6b2003e470 0x7f6b20040930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.582+0000 7f6b3dc45700 1 --2- 172.21.15.69:0/1643202477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b38109990 0x7f6b3819cdf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.582+0000 7f6b3dc45700 1 -- 172.21.15.69:0/1643202477 >> 172.21.15.69:0/1643202477 conn(0x7f6b381033a0 msgr2=0x7f6b38103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.583+0000 7f6b3dc45700 1 -- 172.21.15.69:0/1643202477 shutdown_connections 2024-05-14T16:39:25.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:24.583+0000 7f6b3dc45700 1 -- 172.21.15.69:0/1643202477 wait complete. 2024-05-14T16:39:25.453 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:25 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:25.385+0000 7fbcd08f0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:39:25.817 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:25 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:25.473+0000 7fbcd08f0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:39:25.817 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:25 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1643202477' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-14T16:39:25.817 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:25 smithi069 ceph-mon[31507]: mgrmap e9: x(active, since 18s) 2024-05-14T16:39:26.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:39:26.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.611+0000 7f59062e4700 1 Processor -- start 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.611+0000 7f59062e4700 1 -- start start 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.611+0000 7f59062e4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f5900108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.611+0000 7f59062e4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59001087d0 con 0x7f5900107de0 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.611+0000 7f58fffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f5900108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.612+0000 7f58fffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f5900108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41326/0 (socket says 172.21.15.69:41326) 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.612+0000 7f58fffff700 1 -- 172.21.15.69:0/3911548923 learned_addr learned my addr 172.21.15.69:0/3911548923 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:26.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.612+0000 7f58fffff700 1 -- 172.21.15.69:0/3911548923 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5900108fe0 con 0x7f5900107de0 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.612+0000 7f58fffff700 1 --2- 172.21.15.69:0/3911548923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f5900108200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58f40098a0 tx=0x7f58f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a954ab65463bba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.612+0000 7f58feffd700 1 -- 172.21.15.69:0/3911548923 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f4004160 con 0x7f5900107de0 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.612+0000 7f58feffd700 1 -- 172.21.15.69:0/3911548923 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f4004300 con 0x7f5900107de0 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.612+0000 7f58feffd700 1 -- 172.21.15.69:0/3911548923 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f4011a20 con 0x7f5900107de0 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.613+0000 7f59062e4700 1 -- 172.21.15.69:0/3911548923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 msgr2=0x7f5900108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.613+0000 7f59062e4700 1 --2- 172.21.15.69:0/3911548923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f5900108200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58f40098a0 tx=0x7f58f4009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.613+0000 7f59062e4700 1 -- 172.21.15.69:0/3911548923 shutdown_connections 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.613+0000 7f59062e4700 1 --2- 172.21.15.69:0/3911548923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f5900108200 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.613+0000 7f59062e4700 1 -- 172.21.15.69:0/3911548923 >> 172.21.15.69:0/3911548923 conn(0x7f59001033a0 msgr2=0x7f59001057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.613+0000 7f59062e4700 1 -- 172.21.15.69:0/3911548923 shutdown_connections 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.613+0000 7f59062e4700 1 -- 172.21.15.69:0/3911548923 wait complete. 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.613+0000 7f59062e4700 1 Processor -- start 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.613+0000 7f59062e4700 1 -- start start 2024-05-14T16:39:26.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f59062e4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f590019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f59062e4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f590010c390 con 0x7f5900107de0 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f58fffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f590019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f58fffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f590019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41336/0 (socket says 172.21.15.69:41336) 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f58fffff700 1 -- 172.21.15.69:0/2052592716 learned_addr learned my addr 172.21.15.69:0/2052592716 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f58fffff700 1 -- 172.21.15.69:0/2052592716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590019c270 con 0x7f5900107de0 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f58fffff700 1 --2- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f590019bd30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58f4000c00 tx=0x7f58f4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f58fd7fa700 1 -- 172.21.15.69:0/2052592716 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f4003aa0 con 0x7f5900107de0 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f58fd7fa700 1 -- 172.21.15.69:0/2052592716 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f401aa10 con 0x7f5900107de0 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f59062e4700 1 -- 172.21.15.69:0/2052592716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590019c500 con 0x7f5900107de0 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f58fd7fa700 1 -- 172.21.15.69:0/2052592716 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f4023a10 con 0x7f5900107de0 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.614+0000 7f59062e4700 1 -- 172.21.15.69:0/2052592716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590007bf50 con 0x7f5900107de0 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.615+0000 7f59062e4700 1 -- 172.21.15.69:0/2052592716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5900064ba0 con 0x7f5900107de0 2024-05-14T16:39:26.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.616+0000 7f58fd7fa700 1 -- 172.21.15.69:0/2052592716 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f58f4023c50 con 0x7f5900107de0 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.616+0000 7f58fd7fa700 1 --2- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f58e803e4c0 0x7f58e8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.616+0000 7f58fd7fa700 1 -- 172.21.15.69:0/2052592716 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f58f4055770 con 0x7f5900107de0 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.616+0000 7f58ff7fe700 1 -- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f58e803e4c0 msgr2=0x7f58e8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/421328414 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.616+0000 7f58ff7fe700 1 --2- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f58e803e4c0 0x7f58e8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.621+0000 7f58fd7fa700 1 -- 172.21.15.69:0/2052592716 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f58f40554d0 con 0x7f5900107de0 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.817+0000 7f58ff7fe700 1 -- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f58e803e4c0 msgr2=0x7f58e8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/421328414 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.817+0000 7f58ff7fe700 1 --2- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f58e803e4c0 0x7f58e8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.924+0000 7f59062e4700 1 -- 172.21.15.69:0/2052592716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f590007c540 con 0x7f5900107de0 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.925+0000 7f58fd7fa700 1 -- 172.21.15.69:0/2052592716 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f58f4025db0 con 0x7f5900107de0 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.929+0000 7f59062e4700 1 -- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f58e803e4c0 msgr2=0x7f58e8040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.929+0000 7f59062e4700 1 --2- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f58e803e4c0 0x7f58e8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.929+0000 7f59062e4700 1 -- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 msgr2=0x7f590019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.929+0000 7f59062e4700 1 --2- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f590019bd30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58f4000c00 tx=0x7f58f4011aa0 comp rx=0 tx=0).stop 2024-05-14T16:39:26.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.930+0000 7f59062e4700 1 -- 172.21.15.69:0/2052592716 shutdown_connections 2024-05-14T16:39:26.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.930+0000 7f59062e4700 1 --2- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f58e803e4c0 0x7f58e8040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:26.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.930+0000 7f59062e4700 1 --2- 172.21.15.69:0/2052592716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5900107de0 0x7f590019bd30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:26.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.930+0000 7f59062e4700 1 -- 172.21.15.69:0/2052592716 >> 172.21.15.69:0/2052592716 conn(0x7f59001033a0 msgr2=0x7f5900103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:26.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.930+0000 7f59062e4700 1 -- 172.21.15.69:0/2052592716 shutdown_connections 2024-05-14T16:39:26.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:25.930+0000 7f59062e4700 1 -- 172.21.15.69:0/2052592716 wait complete. 2024-05-14T16:39:26.492 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2024-05-14T16:39:26.492 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 9... 2024-05-14T16:39:26.881 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:26 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2052592716' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-14T16:39:26.881 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:26 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:26.712+0000 7fbcd08f0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:39:27.203 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:26 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:26.891+0000 7fbcd08f0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:39:27.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:27 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:27.063+0000 7fbcd08f0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:39:27.652 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:27 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:27.354+0000 7fbcd08f0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:39:27.652 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:27 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:27.447+0000 7fbcd08f0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:39:27.652 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:27 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:27.652+0000 7fbcd08f0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:39:27.954 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:27 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:27.848+0000 7fbcd08f0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:39:28.673 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:28 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:28.451+0000 7fbcd08f0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:39:28.674 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:28 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:28.673+0000 7fbcd08f0200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:39:29.453 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:29 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:29.105+0000 7fbcd08f0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:39:29.454 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:29 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:29.203+0000 7fbcd08f0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:39:29.892 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:29 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:29.586+0000 7fbcd08f0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:39:30.203 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:29 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:29.947+0000 7fbcd08f0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:39:30.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:30 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:30.042+0000 7fbcd08f0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:39:30.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:30 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:30.288+0000 7fbcd08f0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:39:30.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:30 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:30.421+0000 7fbcd08f0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:39:30.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:30 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:30.523+0000 7fbcd08f0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:39:31.203 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:30 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:30.848+0000 7fbcd08f0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:39:31.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:30 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:30.942+0000 7fbcd08f0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:39:31.819 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:31 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:31.544+0000 7fbcd08f0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:39:31.820 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:39:31 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:39:31.764+0000 7fbcd08f0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:39:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: Active manager daemon x restarted 2024-05-14T16:39:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: Activating manager daemon x 2024-05-14T16:39:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: osdmap e3: 0 total, 0 up, 0 in 2024-05-14T16:39:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: mgrmap e10: x(active, starting, since 0.00402116s) 2024-05-14T16:39:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:39:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:39:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:39:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:39:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:39:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: Manager daemon x is now available 2024-05-14T16:39:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:39:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:39:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:39:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:32 smithi069 ceph-mon[31507]: [14/May/2024:16:39:32] ENGINE Bus STARTING 2024-05-14T16:39:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:32 smithi069 ceph-mon[31507]: [14/May/2024:16:39:32] ENGINE Serving on https://172.21.15.69:7150 2024-05-14T16:39:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:32 smithi069 ceph-mon[31507]: mgrmap e11: x(active, since 1.006s) 2024-05-14T16:39:33.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:39:33.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-14T16:39:33.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-14T16:39:33.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:39:33.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.883+0000 7f4f1f099700 1 Processor -- start 2024-05-14T16:39:33.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.883+0000 7f4f1f099700 1 -- start start 2024-05-14T16:39:33.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.884+0000 7f4f1f099700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1810bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:33.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.884+0000 7f4f1f099700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1807d680 con 0x7f4f18109940 2024-05-14T16:39:33.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.884+0000 7f4f1ce35700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1810bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.884+0000 7f4f1ce35700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1810bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41342/0 (socket says 172.21.15.69:41342) 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.884+0000 7f4f1ce35700 1 -- 172.21.15.69:0/748154768 learned_addr learned my addr 172.21.15.69:0/748154768 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.884+0000 7f4f1ce35700 1 -- 172.21.15.69:0/748154768 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1810c270 con 0x7f4f18109940 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.884+0000 7f4f1ce35700 1 --2- 172.21.15.69:0/748154768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1810bd30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0098a0 tx=0x7f4f0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad02b9c367db7bab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.884+0000 7f4f177fe700 1 -- 172.21.15.69:0/748154768 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f0c004160 con 0x7f4f18109940 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.884+0000 7f4f177fe700 1 -- 172.21.15.69:0/748154768 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c004300 con 0x7f4f18109940 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.884+0000 7f4f177fe700 1 -- 172.21.15.69:0/748154768 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f0c011a20 con 0x7f4f18109940 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.885+0000 7f4f1f099700 1 -- 172.21.15.69:0/748154768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 msgr2=0x7f4f1810bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.885+0000 7f4f1f099700 1 --2- 172.21.15.69:0/748154768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1810bd30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0098a0 tx=0x7f4f0c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.885+0000 7f4f1f099700 1 -- 172.21.15.69:0/748154768 shutdown_connections 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.885+0000 7f4f1f099700 1 --2- 172.21.15.69:0/748154768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1810bd30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.885+0000 7f4f1f099700 1 -- 172.21.15.69:0/748154768 >> 172.21.15.69:0/748154768 conn(0x7f4f18103330 msgr2=0x7f4f18105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.885+0000 7f4f1f099700 1 -- 172.21.15.69:0/748154768 shutdown_connections 2024-05-14T16:39:33.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.885+0000 7f4f1f099700 1 -- 172.21.15.69:0/748154768 wait complete. 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.886+0000 7f4f1f099700 1 Processor -- start 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.886+0000 7f4f1f099700 1 -- start start 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.886+0000 7f4f1f099700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.886+0000 7f4f1f099700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1810d270 con 0x7f4f18109940 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.886+0000 7f4f1ce35700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.886+0000 7f4f1ce35700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41352/0 (socket says 172.21.15.69:41352) 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.886+0000 7f4f1ce35700 1 -- 172.21.15.69:0/2035633734 learned_addr learned my addr 172.21.15.69:0/2035633734 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.886+0000 7f4f1ce35700 1 -- 172.21.15.69:0/2035633734 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1819d2e0 con 0x7f4f18109940 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.886+0000 7f4f1ce35700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1819cda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0116e0 tx=0x7f4f0c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.887+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f0c003b20 con 0x7f4f18109940 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.887+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c01b8f0 con 0x7f4f18109940 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.887+0000 7f4f1f099700 1 -- 172.21.15.69:0/2035633734 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1819d570 con 0x7f4f18109940 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.887+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f0c0238c0 con 0x7f4f18109940 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.887+0000 7f4f1f099700 1 -- 172.21.15.69:0/2035633734 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f181a0260 con 0x7f4f18109940 2024-05-14T16:39:33.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.888+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f4f0c023b70 con 0x7f4f18109940 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.888+0000 7f4f15ffb700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 0x7f4f08040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.888+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 --> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f08041040 con 0x7f4f0803e470 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.888+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4f0c0542d0 con 0x7f4f18109940 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.888+0000 7f4f17fff700 1 -- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 msgr2=0x7f4f08040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/421328414 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:26.888+0000 7f4f17fff700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 0x7f4f08040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:27.089+0000 7f4f17fff700 1 -- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 msgr2=0x7f4f08040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/421328414 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:27.089+0000 7f4f17fff700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 0x7f4f08040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:27.490+0000 7f4f17fff700 1 -- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 msgr2=0x7f4f08040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/421328414 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:27.490+0000 7f4f17fff700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 0x7f4f08040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:28.291+0000 7f4f17fff700 1 -- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 msgr2=0x7f4f08040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/421328414 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:28.291+0000 7f4f17fff700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 0x7f4f08040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:29.892+0000 7f4f17fff700 1 -- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 msgr2=0x7f4f08040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/421328414 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:29.892+0000 7f4f17fff700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 0x7f4f08040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-14T16:39:33.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:31.769+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f4f0c02c3f0 con 0x7f4f18109940 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:31.769+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 msgr2=0x7f4f08040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:31.769+0000 7f4f15ffb700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 0x7f4f08040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.769+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f4f0c056160 con 0x7f4f18109940 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.770+0000 7f4f15ffb700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4f08041ef0 0x7f4f080442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.770+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f08041040 con 0x7f4f08041ef0 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.770+0000 7f4f17fff700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4f08041ef0 0x7f4f080442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.771+0000 7f4f17fff700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4f08041ef0 0x7f4f080442e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4f04003a10 tx=0x7f4f040092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.772+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f4f08041040 con 0x7f4f08041ef0 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.780+0000 7f4f1f099700 1 -- 172.21.15.69:0/2035633734 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4f1806a650 con 0x7f4f08041ef0 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.780+0000 7f4f15ffb700 1 -- 172.21.15.69:0/2035633734 <== mgr.14150 v2:172.21.15.69:6800/4012740750 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f4f1806a650 con 0x7f4f08041ef0 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 -- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4f08041ef0 msgr2=0x7f4f080442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4f08041ef0 0x7f4f080442e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4f04003a10 tx=0x7f4f040092b0 comp rx=0 tx=0).stop 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 -- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 msgr2=0x7f4f1819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1819cda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0116e0 tx=0x7f4f0c011aa0 comp rx=0 tx=0).stop 2024-05-14T16:39:33.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 -- 172.21.15.69:0/2035633734 shutdown_connections 2024-05-14T16:39:33.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4f08041ef0 0x7f4f080442e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:33.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f18109940 0x7f4f1819cda0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:33.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 --2- 172.21.15.69:0/2035633734 >> [v2:172.21.15.69:6800/421328414,v1:172.21.15.69:6801/421328414] conn(0x7f4f0803e470 0x7f4f08040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:33.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 -- 172.21.15.69:0/2035633734 >> 172.21.15.69:0/2035633734 conn(0x7f4f18103330 msgr2=0x7f4f18103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:33.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 -- 172.21.15.69:0/2035633734 shutdown_connections 2024-05-14T16:39:33.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:32.781+0000 7f4f1f099700 1 -- 172.21.15.69:0/2035633734 wait complete. 2024-05-14T16:39:33.644 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 9 is available 2024-05-14T16:39:33.644 INFO:teuthology.orchestra.run.smithi069.stdout:Generating a dashboard self-signed certificate... 2024-05-14T16:39:34.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:34 smithi069 ceph-mon[31507]: [14/May/2024:16:39:32] ENGINE Serving on http://172.21.15.69:8765 2024-05-14T16:39:34.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:34 smithi069 ceph-mon[31507]: [14/May/2024:16:39:32] ENGINE Bus STARTED 2024-05-14T16:39:34.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:34 smithi069 ceph-mon[31507]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-14T16:39:34.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:34 smithi069 ceph-mon[31507]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-14T16:39:34.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:34 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:34.998 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-14T16:39:35.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.112+0000 7f9e31a03700 1 Processor -- start 2024-05-14T16:39:35.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.112+0000 7f9e31a03700 1 -- start start 2024-05-14T16:39:35.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.112+0000 7f9e31a03700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c10bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:35.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.112+0000 7f9e31a03700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c07ce10 con 0x7f9e2c109780 2024-05-14T16:39:35.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.113+0000 7f9e2affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c10bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:35.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.113+0000 7f9e2affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c10bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41452/0 (socket says 172.21.15.69:41452) 2024-05-14T16:39:35.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.113+0000 7f9e2affd700 1 -- 172.21.15.69:0/1882623651 learned_addr learned my addr 172.21.15.69:0/1882623651 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:35.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.113+0000 7f9e2affd700 1 -- 172.21.15.69:0/1882623651 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c10c0b0 con 0x7f9e2c109780 2024-05-14T16:39:35.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.113+0000 7f9e2affd700 1 --2- 172.21.15.69:0/1882623651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c10bb70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9e140098a0 tx=0x7f9e14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6a9772a250b394 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:35.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.113+0000 7f9e29ffb700 1 -- 172.21.15.69:0/1882623651 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e14004160 con 0x7f9e2c109780 2024-05-14T16:39:35.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.113+0000 7f9e29ffb700 1 -- 172.21.15.69:0/1882623651 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e14004300 con 0x7f9e2c109780 2024-05-14T16:39:35.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.113+0000 7f9e29ffb700 1 -- 172.21.15.69:0/1882623651 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e14011a20 con 0x7f9e2c109780 2024-05-14T16:39:35.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.114+0000 7f9e31a03700 1 -- 172.21.15.69:0/1882623651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 msgr2=0x7f9e2c10bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:35.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.114+0000 7f9e31a03700 1 --2- 172.21.15.69:0/1882623651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c10bb70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9e140098a0 tx=0x7f9e14009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:35.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.114+0000 7f9e31a03700 1 -- 172.21.15.69:0/1882623651 shutdown_connections 2024-05-14T16:39:35.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.114+0000 7f9e31a03700 1 --2- 172.21.15.69:0/1882623651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c10bb70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:35.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.114+0000 7f9e31a03700 1 -- 172.21.15.69:0/1882623651 >> 172.21.15.69:0/1882623651 conn(0x7f9e2c1031d0 msgr2=0x7f9e2c1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:35.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.114+0000 7f9e31a03700 1 -- 172.21.15.69:0/1882623651 shutdown_connections 2024-05-14T16:39:35.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.114+0000 7f9e31a03700 1 -- 172.21.15.69:0/1882623651 wait complete. 2024-05-14T16:39:35.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.114+0000 7f9e31a03700 1 Processor -- start 2024-05-14T16:39:35.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.115+0000 7f9e31a03700 1 -- start start 2024-05-14T16:39:35.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.115+0000 7f9e31a03700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c108fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:35.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.115+0000 7f9e31a03700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c10d0b0 con 0x7f9e2c109780 2024-05-14T16:39:35.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.115+0000 7f9e2affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c108fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:35.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.115+0000 7f9e2affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c108fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41466/0 (socket says 172.21.15.69:41466) 2024-05-14T16:39:35.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.115+0000 7f9e2affd700 1 -- 172.21.15.69:0/1830113180 learned_addr learned my addr 172.21.15.69:0/1830113180 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:35.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.115+0000 7f9e2affd700 1 -- 172.21.15.69:0/1830113180 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c1094f0 con 0x7f9e2c109780 2024-05-14T16:39:35.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.115+0000 7f9e2affd700 1 --2- 172.21.15.69:0/1830113180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c108fb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e140116e0 tx=0x7f9e14011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:35.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.116+0000 7f9e30a01700 1 -- 172.21.15.69:0/1830113180 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e14003b20 con 0x7f9e2c109780 2024-05-14T16:39:35.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.116+0000 7f9e30a01700 1 -- 172.21.15.69:0/1830113180 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e1401b8f0 con 0x7f9e2c109780 2024-05-14T16:39:35.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.116+0000 7f9e31a03700 1 -- 172.21.15.69:0/1830113180 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c107730 con 0x7f9e2c109780 2024-05-14T16:39:35.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.116+0000 7f9e30a01700 1 -- 172.21.15.69:0/1830113180 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e140238c0 con 0x7f9e2c109780 2024-05-14T16:39:35.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.116+0000 7f9e31a03700 1 -- 172.21.15.69:0/1830113180 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c107c70 con 0x7f9e2c109780 2024-05-14T16:39:35.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.117+0000 7f9e31a03700 1 -- 172.21.15.69:0/1830113180 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2c06a650 con 0x7f9e2c109780 2024-05-14T16:39:35.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.117+0000 7f9e30a01700 1 -- 172.21.15.69:0/1830113180 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f9e14023a60 con 0x7f9e2c109780 2024-05-14T16:39:35.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.117+0000 7f9e30a01700 1 --2- 172.21.15.69:0/1830113180 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9e1803e3b0 0x7f9e18040870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:35.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.117+0000 7f9e30a01700 1 -- 172.21.15.69:0/1830113180 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9e14053e80 con 0x7f9e2c109780 2024-05-14T16:39:35.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.117+0000 7f9e2a7fc700 1 --2- 172.21.15.69:0/1830113180 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9e1803e3b0 0x7f9e18040870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:35.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.118+0000 7f9e2a7fc700 1 --2- 172.21.15.69:0/1830113180 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9e1803e3b0 0x7f9e18040870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c006fd0 tx=0x7f9e1c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:35.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.122+0000 7f9e30a01700 1 -- 172.21.15.69:0/1830113180 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e1401f030 con 0x7f9e2c109780 2024-05-14T16:39:35.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.333+0000 7f9e31a03700 1 -- 172.21.15.69:0/1830113180 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f9e2c108480 con 0x7f9e1803e3b0 2024-05-14T16:39:35.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.379+0000 7f9e30a01700 1 -- 172.21.15.69:0/1830113180 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f9e1401baa0 con 0x7f9e2c109780 2024-05-14T16:39:35.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.404+0000 7f9e30a01700 1 -- 172.21.15.69:0/1830113180 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f9e2c108480 con 0x7f9e1803e3b0 2024-05-14T16:39:35.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.407+0000 7f9e31a03700 1 -- 172.21.15.69:0/1830113180 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9e1803e3b0 msgr2=0x7f9e18040870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:35.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.407+0000 7f9e31a03700 1 --2- 172.21.15.69:0/1830113180 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9e1803e3b0 0x7f9e18040870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c006fd0 tx=0x7f9e1c006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:35.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.407+0000 7f9e31a03700 1 -- 172.21.15.69:0/1830113180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 msgr2=0x7f9e2c108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:35.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.407+0000 7f9e31a03700 1 --2- 172.21.15.69:0/1830113180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c108fb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e140116e0 tx=0x7f9e14011aa0 comp rx=0 tx=0).stop 2024-05-14T16:39:35.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.407+0000 7f9e31a03700 1 -- 172.21.15.69:0/1830113180 shutdown_connections 2024-05-14T16:39:35.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.407+0000 7f9e31a03700 1 --2- 172.21.15.69:0/1830113180 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9e1803e3b0 0x7f9e18040870 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:35.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.407+0000 7f9e31a03700 1 --2- 172.21.15.69:0/1830113180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e2c109780 0x7f9e2c108fb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:35.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.407+0000 7f9e31a03700 1 -- 172.21.15.69:0/1830113180 >> 172.21.15.69:0/1830113180 conn(0x7f9e2c1031d0 msgr2=0x7f9e2c103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:35.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.407+0000 7f9e31a03700 1 -- 172.21.15.69:0/1830113180 shutdown_connections 2024-05-14T16:39:35.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:34.407+0000 7f9e31a03700 1 -- 172.21.15.69:0/1830113180 wait complete. 2024-05-14T16:39:35.012 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial admin user... 2024-05-14T16:39:35.618 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:35 smithi069 ceph-mon[31507]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:35.619 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:35 smithi069 ceph-mon[31507]: mgrmap e12: x(active, since 2s) 2024-05-14T16:39:35.619 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:35.619 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:35.619 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:36.561 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$08BOu83UNToYitOiJMWlDu0nC4jocwauE4sZ4F9yLem4R51oZoeCO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715704775, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.382+0000 7fa9ae042700 1 Processor -- start 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.382+0000 7fa9ae042700 1 -- start start 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.382+0000 7fa9ae042700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a81075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.382+0000 7fa9ae042700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a8107bb0 con 0x7fa9a81071c0 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.382+0000 7fa9a77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a81075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.382+0000 7fa9a77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a81075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41666/0 (socket says 172.21.15.69:41666) 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.382+0000 7fa9a77fe700 1 -- 172.21.15.69:0/2856792352 learned_addr learned my addr 172.21.15.69:0/2856792352 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.383+0000 7fa9a77fe700 1 -- 172.21.15.69:0/2856792352 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a8107d30 con 0x7fa9a81071c0 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.383+0000 7fa9a77fe700 1 --2- 172.21.15.69:0/2856792352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a81075e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9980098a0 tx=0x7fa998009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e548cd1e8807b0e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.383+0000 7fa9a67fc700 1 -- 172.21.15.69:0/2856792352 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa998004160 con 0x7fa9a81071c0 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.383+0000 7fa9a67fc700 1 -- 172.21.15.69:0/2856792352 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa998004300 con 0x7fa9a81071c0 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.383+0000 7fa9a67fc700 1 -- 172.21.15.69:0/2856792352 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa998011a20 con 0x7fa9a81071c0 2024-05-14T16:39:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.384+0000 7fa9ae042700 1 -- 172.21.15.69:0/2856792352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 msgr2=0x7fa9a81075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.384+0000 7fa9ae042700 1 --2- 172.21.15.69:0/2856792352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a81075e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9980098a0 tx=0x7fa998009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.384+0000 7fa9ae042700 1 -- 172.21.15.69:0/2856792352 shutdown_connections 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.384+0000 7fa9ae042700 1 --2- 172.21.15.69:0/2856792352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a81075e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.384+0000 7fa9ae042700 1 -- 172.21.15.69:0/2856792352 >> 172.21.15.69:0/2856792352 conn(0x7fa9a8102b20 msgr2=0x7fa9a8104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.384+0000 7fa9ae042700 1 -- 172.21.15.69:0/2856792352 shutdown_connections 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.384+0000 7fa9ae042700 1 -- 172.21.15.69:0/2856792352 wait complete. 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.385+0000 7fa9ae042700 1 Processor -- start 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.385+0000 7fa9ae042700 1 -- start start 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.385+0000 7fa9ae042700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.385+0000 7fa9ae042700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a810b0d0 con 0x7fa9a81071c0 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.385+0000 7fa9a77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.385+0000 7fa9a77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41674/0 (socket says 172.21.15.69:41674) 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.385+0000 7fa9a77fe700 1 -- 172.21.15.69:0/2241188773 learned_addr learned my addr 172.21.15.69:0/2241188773 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.385+0000 7fa9a77fe700 1 -- 172.21.15.69:0/2241188773 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a819d350 con 0x7fa9a81071c0 2024-05-14T16:39:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.385+0000 7fa9a77fe700 1 --2- 172.21.15.69:0/2241188773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a819ce10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa998000c00 tx=0x7fa998011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.386+0000 7fa9a4ff9700 1 -- 172.21.15.69:0/2241188773 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa998003aa0 con 0x7fa9a81071c0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.386+0000 7fa9a4ff9700 1 -- 172.21.15.69:0/2241188773 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa99801b8f0 con 0x7fa9a81071c0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.386+0000 7fa9ae042700 1 -- 172.21.15.69:0/2241188773 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a819d640 con 0x7fa9a81071c0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.386+0000 7fa9a4ff9700 1 -- 172.21.15.69:0/2241188773 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa998023890 con 0x7fa9a81071c0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.386+0000 7fa9ae042700 1 -- 172.21.15.69:0/2241188773 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a81a02d0 con 0x7fa9a81071c0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.387+0000 7fa9a4ff9700 1 -- 172.21.15.69:0/2241188773 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fa99802c3f0 con 0x7fa9a81071c0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.387+0000 7fa9ae042700 1 -- 172.21.15.69:0/2241188773 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a8064ba0 con 0x7fa9a81071c0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.387+0000 7fa9a4ff9700 1 --2- 172.21.15.69:0/2241188773 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa99003e520 0x7fa9900409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.387+0000 7fa9a4ff9700 1 -- 172.21.15.69:0/2241188773 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa998053e90 con 0x7fa9a81071c0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.387+0000 7fa9a6ffd700 1 --2- 172.21.15.69:0/2241188773 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa99003e520 0x7fa9900409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.388+0000 7fa9a6ffd700 1 --2- 172.21.15.69:0/2241188773 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa99003e520 0x7fa9900409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006fd0 tx=0x7fa99c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.392+0000 7fa9a4ff9700 1 -- 172.21.15.69:0/2241188773 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa99801f030 con 0x7fa9a81071c0 2024-05-14T16:39:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.617+0000 7fa9ae042700 1 -- 172.21.15.69:0/2241188773 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa9a81a04a0 con 0x7fa99003e520 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.842+0000 7fa9a4ff9700 1 -- 172.21.15.69:0/2241188773 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa9a81a04a0 con 0x7fa99003e520 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.844+0000 7fa9ae042700 1 -- 172.21.15.69:0/2241188773 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa99003e520 msgr2=0x7fa9900409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.844+0000 7fa9ae042700 1 --2- 172.21.15.69:0/2241188773 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa99003e520 0x7fa9900409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006fd0 tx=0x7fa99c006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.844+0000 7fa9ae042700 1 -- 172.21.15.69:0/2241188773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 msgr2=0x7fa9a819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.844+0000 7fa9ae042700 1 --2- 172.21.15.69:0/2241188773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a819ce10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa998000c00 tx=0x7fa998011aa0 comp rx=0 tx=0).stop 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.845+0000 7fa9ae042700 1 -- 172.21.15.69:0/2241188773 shutdown_connections 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.845+0000 7fa9ae042700 1 --2- 172.21.15.69:0/2241188773 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa99003e520 0x7fa9900409e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.845+0000 7fa9ae042700 1 --2- 172.21.15.69:0/2241188773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a81071c0 0x7fa9a819ce10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.845+0000 7fa9ae042700 1 -- 172.21.15.69:0/2241188773 >> 172.21.15.69:0/2241188773 conn(0x7fa9a8102b20 msgr2=0x7fa9a8103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.845+0000 7fa9ae042700 1 -- 172.21.15.69:0/2241188773 shutdown_connections 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:35.845+0000 7fa9ae042700 1 -- 172.21.15.69:0/2241188773 wait complete. 2024-05-14T16:39:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:Fetching dashboard port number... 2024-05-14T16:39:36.682 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:36 smithi069 ceph-mon[31507]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:36.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:37.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:37 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1549954299' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-14T16:39:38.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 8443 2024-05-14T16:39:38.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.428+0000 7f326ac75700 1 Processor -- start 2024-05-14T16:39:38.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.428+0000 7f326ac75700 1 -- start start 2024-05-14T16:39:38.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.428+0000 7f326ac75700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f3264108220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:38.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.428+0000 7f326ac75700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32641087f0 con 0x7f3264107e00 2024-05-14T16:39:38.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.428+0000 7f3268a11700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f3264108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:38.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.429+0000 7f3268a11700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f3264108220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41678/0 (socket says 172.21.15.69:41678) 2024-05-14T16:39:38.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.429+0000 7f3268a11700 1 -- 172.21.15.69:0/690395328 learned_addr learned my addr 172.21.15.69:0/690395328 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:38.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.429+0000 7f3268a11700 1 -- 172.21.15.69:0/690395328 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3264108fb0 con 0x7f3264107e00 2024-05-14T16:39:38.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.429+0000 7f3268a11700 1 --2- 172.21.15.69:0/690395328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f3264108220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f325800bb40 tx=0x7f325800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c9314880621985 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.429+0000 7f32637fe700 1 -- 172.21.15.69:0/690395328 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f325800dbe0 con 0x7f3264107e00 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.429+0000 7f32637fe700 1 -- 172.21.15.69:0/690395328 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32580044e0 con 0x7f3264107e00 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.429+0000 7f32637fe700 1 -- 172.21.15.69:0/690395328 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3258010470 con 0x7f3264107e00 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 -- 172.21.15.69:0/690395328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 msgr2=0x7f3264108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 --2- 172.21.15.69:0/690395328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f3264108220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f325800bb40 tx=0x7f325800be50 comp rx=0 tx=0).stop 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 -- 172.21.15.69:0/690395328 shutdown_connections 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 --2- 172.21.15.69:0/690395328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f3264108220 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 -- 172.21.15.69:0/690395328 >> 172.21.15.69:0/690395328 conn(0x7f32641033e0 msgr2=0x7f3264105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 -- 172.21.15.69:0/690395328 shutdown_connections 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 -- 172.21.15.69:0/690395328 wait complete. 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 Processor -- start 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 -- start start 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f326419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:38.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.430+0000 7f326ac75700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f326407c7e0 con 0x7f3264107e00 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.431+0000 7f3268a11700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f326419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.431+0000 7f3268a11700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f326419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41682/0 (socket says 172.21.15.69:41682) 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.431+0000 7f3268a11700 1 -- 172.21.15.69:0/1549954299 learned_addr learned my addr 172.21.15.69:0/1549954299 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.431+0000 7f3268a11700 1 -- 172.21.15.69:0/1549954299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326419b110 con 0x7f3264107e00 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.431+0000 7f3268a11700 1 --2- 172.21.15.69:0/1549954299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f326419abd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f325800b340 tx=0x7f3258003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.431+0000 7f3261ffb700 1 -- 172.21.15.69:0/1549954299 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3258027070 con 0x7f3264107e00 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.431+0000 7f3261ffb700 1 -- 172.21.15.69:0/1549954299 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3258010c20 con 0x7f3264107e00 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.431+0000 7f326ac75700 1 -- 172.21.15.69:0/1549954299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326419b3a0 con 0x7f3264107e00 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.431+0000 7f3261ffb700 1 -- 172.21.15.69:0/1549954299 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3258022b70 con 0x7f3264107e00 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.431+0000 7f326ac75700 1 -- 172.21.15.69:0/1549954299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326419e090 con 0x7f3264107e00 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.432+0000 7f3261ffb700 1 -- 172.21.15.69:0/1549954299 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f325802a070 con 0x7f3264107e00 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.432+0000 7f326ac75700 1 -- 172.21.15.69:0/1549954299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3264064ba0 con 0x7f3264107e00 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.432+0000 7f3261ffb700 1 --2- 172.21.15.69:0/1549954299 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f325403e470 0x7f3254040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.433+0000 7f3261ffb700 1 -- 172.21.15.69:0/1549954299 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f325801f070 con 0x7f3264107e00 2024-05-14T16:39:38.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.433+0000 7f3263fff700 1 --2- 172.21.15.69:0/1549954299 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f325403e470 0x7f3254040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.433+0000 7f3263fff700 1 --2- 172.21.15.69:0/1549954299 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f325403e470 0x7f3254040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3250006fd0 tx=0x7f3250006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.438+0000 7f3261ffb700 1 -- 172.21.15.69:0/1549954299 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3258015070 con 0x7f3264107e00 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.638+0000 7f326ac75700 1 -- 172.21.15.69:0/1549954299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f3264194130 con 0x7f3264107e00 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.638+0000 7f3261ffb700 1 -- 172.21.15.69:0/1549954299 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f3264194130 con 0x7f3264107e00 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.641+0000 7f326ac75700 1 -- 172.21.15.69:0/1549954299 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f325403e470 msgr2=0x7f3254040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.641+0000 7f326ac75700 1 --2- 172.21.15.69:0/1549954299 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f325403e470 0x7f3254040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3250006fd0 tx=0x7f3250006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.641+0000 7f326ac75700 1 -- 172.21.15.69:0/1549954299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 msgr2=0x7f326419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.641+0000 7f326ac75700 1 --2- 172.21.15.69:0/1549954299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f326419abd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f325800b340 tx=0x7f3258003dc0 comp rx=0 tx=0).stop 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.641+0000 7f326ac75700 1 -- 172.21.15.69:0/1549954299 shutdown_connections 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.641+0000 7f326ac75700 1 --2- 172.21.15.69:0/1549954299 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f325403e470 0x7f3254040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.641+0000 7f326ac75700 1 --2- 172.21.15.69:0/1549954299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264107e00 0x7f326419abd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.641+0000 7f326ac75700 1 -- 172.21.15.69:0/1549954299 >> 172.21.15.69:0/1549954299 conn(0x7f32641033e0 msgr2=0x7f32641911f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.641+0000 7f326ac75700 1 -- 172.21.15.69:0/1549954299 shutdown_connections 2024-05-14T16:39:38.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:37.641+0000 7f326ac75700 1 -- 172.21.15.69:0/1549954299 wait complete. 2024-05-14T16:39:38.251 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:39:38.251 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-05-14T16:39:38.258 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:39:38.258 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-05-14T16:39:38.258 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-05-14T16:39:38.259 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-14T16:39:38.260 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph Dashboard is now available at: 2024-05-14T16:39:38.260 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:38.261 INFO:teuthology.orchestra.run.smithi069.stdout: URL: https://smithi069.front.sepia.ceph.com:8443/ 2024-05-14T16:39:38.261 INFO:teuthology.orchestra.run.smithi069.stdout: User: admin 2024-05-14T16:39:38.261 INFO:teuthology.orchestra.run.smithi069.stdout: Password: rl1mrdotv3 2024-05-14T16:39:38.261 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:38.261 INFO:teuthology.orchestra.run.smithi069.stdout:Saving cluster configuration to /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config directory 2024-05-14T16:39:38.261 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling autotune for osd_memory_target 2024-05-14T16:39:39.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.659+0000 7ffab1806700 1 Processor -- start 2024-05-14T16:39:39.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.659+0000 7ffab1806700 1 -- start start 2024-05-14T16:39:39.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.659+0000 7ffab1806700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:39.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.659+0000 7ffab1806700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1087d0 con 0x7ffaac107de0 2024-05-14T16:39:39.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.660+0000 7ffaaaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.660+0000 7ffaaaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41684/0 (socket says 172.21.15.69:41684) 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.660+0000 7ffaaaffd700 1 -- 172.21.15.69:0/2423074566 learned_addr learned my addr 172.21.15.69:0/2423074566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.660+0000 7ffaaaffd700 1 -- 172.21.15.69:0/2423074566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac108fe0 con 0x7ffaac107de0 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.660+0000 7ffaaaffd700 1 --2- 172.21.15.69:0/2423074566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac108200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0098a0 tx=0x7ffa9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e9a3760e2ff172 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.660+0000 7ffaaa7fc700 1 -- 172.21.15.69:0/2423074566 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c004160 con 0x7ffaac107de0 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.660+0000 7ffaaa7fc700 1 -- 172.21.15.69:0/2423074566 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa9c004300 con 0x7ffaac107de0 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.660+0000 7ffaaa7fc700 1 -- 172.21.15.69:0/2423074566 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c011a20 con 0x7ffaac107de0 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.661+0000 7ffab1806700 1 -- 172.21.15.69:0/2423074566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 msgr2=0x7ffaac108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.661+0000 7ffab1806700 1 --2- 172.21.15.69:0/2423074566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac108200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0098a0 tx=0x7ffa9c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.661+0000 7ffab1806700 1 -- 172.21.15.69:0/2423074566 shutdown_connections 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.661+0000 7ffab1806700 1 --2- 172.21.15.69:0/2423074566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac108200 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.661+0000 7ffab1806700 1 -- 172.21.15.69:0/2423074566 >> 172.21.15.69:0/2423074566 conn(0x7ffaac1033a0 msgr2=0x7ffaac1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:39.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.661+0000 7ffab1806700 1 -- 172.21.15.69:0/2423074566 shutdown_connections 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.661+0000 7ffab1806700 1 -- 172.21.15.69:0/2423074566 wait complete. 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffab1806700 1 Processor -- start 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffab1806700 1 -- start start 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffab1806700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac07c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffab1806700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac10c390 con 0x7ffaac107de0 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffaaaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac07c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffaaaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac07c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41698/0 (socket says 172.21.15.69:41698) 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffaaaffd700 1 -- 172.21.15.69:0/1264797188 learned_addr learned my addr 172.21.15.69:0/1264797188 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffaaaffd700 1 -- 172.21.15.69:0/1264797188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac19fb80 con 0x7ffaac107de0 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffaaaffd700 1 --2- 172.21.15.69:0/1264797188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac07c9d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0119f0 tx=0x7ffa9c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffab0804700 1 -- 172.21.15.69:0/1264797188 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c0039e0 con 0x7ffaac107de0 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffab0804700 1 -- 172.21.15.69:0/1264797188 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa9c01b8f0 con 0x7ffaac107de0 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffab1806700 1 -- 172.21.15.69:0/1264797188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaac19fd50 con 0x7ffaac107de0 2024-05-14T16:39:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.662+0000 7ffab0804700 1 -- 172.21.15.69:0/1264797188 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c0237a0 con 0x7ffaac107de0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.663+0000 7ffab1806700 1 -- 172.21.15.69:0/1264797188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaac1a00f0 con 0x7ffaac107de0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.663+0000 7ffab0804700 1 -- 172.21.15.69:0/1264797188 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7ffa9c023aa0 con 0x7ffaac107de0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.663+0000 7ffab1806700 1 -- 172.21.15.69:0/1264797188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaac064ba0 con 0x7ffaac107de0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.664+0000 7ffab0804700 1 --2- 172.21.15.69:0/1264797188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa9803e470 0x7ffa98040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.664+0000 7ffab0804700 1 -- 172.21.15.69:0/1264797188 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa9c054040 con 0x7ffaac107de0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.664+0000 7ffaa3fff700 1 --2- 172.21.15.69:0/1264797188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa9803e470 0x7ffa98040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.664+0000 7ffaa3fff700 1 --2- 172.21.15.69:0/1264797188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa9803e470 0x7ffa98040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffa94006fd0 tx=0x7ffa94006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.669+0000 7ffab0804700 1 -- 172.21.15.69:0/1264797188 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa9c01f030 con 0x7ffaac107de0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.692+0000 7ffab0804700 1 -- 172.21.15.69:0/1264797188 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ffa9c02bd00 con 0x7ffaac107de0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.875+0000 7ffab1806700 1 -- 172.21.15.69:0/1264797188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ffaac1a0550 con 0x7ffaac107de0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.875+0000 7ffab0804700 1 -- 172.21.15.69:0/1264797188 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ffa9c019e20 con 0x7ffaac107de0 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.878+0000 7ffab1806700 1 -- 172.21.15.69:0/1264797188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa9803e470 msgr2=0x7ffa98040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:39.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.878+0000 7ffab1806700 1 --2- 172.21.15.69:0/1264797188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa9803e470 0x7ffa98040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffa94006fd0 tx=0x7ffa94006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:39.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.878+0000 7ffab1806700 1 -- 172.21.15.69:0/1264797188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 msgr2=0x7ffaac07c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:39.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.878+0000 7ffab1806700 1 --2- 172.21.15.69:0/1264797188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac07c9d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0119f0 tx=0x7ffa9c011ad0 comp rx=0 tx=0).stop 2024-05-14T16:39:39.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.878+0000 7ffab1806700 1 -- 172.21.15.69:0/1264797188 shutdown_connections 2024-05-14T16:39:39.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.878+0000 7ffab1806700 1 --2- 172.21.15.69:0/1264797188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa9803e470 0x7ffa98040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:39.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.878+0000 7ffab1806700 1 --2- 172.21.15.69:0/1264797188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffaac107de0 0x7ffaac07c9d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:39.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.878+0000 7ffab1806700 1 -- 172.21.15.69:0/1264797188 >> 172.21.15.69:0/1264797188 conn(0x7ffaac1033a0 msgr2=0x7ffaac103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:39.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.878+0000 7ffab1806700 1 -- 172.21.15.69:0/1264797188 shutdown_connections 2024-05-14T16:39:39.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:38.878+0000 7ffab1806700 1 -- 172.21.15.69:0/1264797188 wait complete. 2024-05-14T16:39:39.854 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:39 smithi069 ceph-mon[31507]: mgrmap e13: x(active, since 6s) 2024-05-14T16:39:40.699 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.890+0000 7f96b66b2700 1 Processor -- start 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.890+0000 7f96b66b2700 1 -- start start 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.890+0000 7f96b66b2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b01092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.890+0000 7f96b66b2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0109810 con 0x7f96b0108eb0 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.890+0000 7f96affff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b01092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.891+0000 7f96affff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b01092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41714/0 (socket says 172.21.15.69:41714) 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.891+0000 7f96affff700 1 -- 172.21.15.69:0/3030517568 learned_addr learned my addr 172.21.15.69:0/3030517568 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.891+0000 7f96affff700 1 -- 172.21.15.69:0/3030517568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b010a050 con 0x7f96b0108eb0 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.891+0000 7f96affff700 1 --2- 172.21.15.69:0/3030517568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b01092d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96a00098a0 tx=0x7f96a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0229754fd72ad5f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.891+0000 7f96af7fe700 1 -- 172.21.15.69:0/3030517568 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a0004160 con 0x7f96b0108eb0 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.891+0000 7f96af7fe700 1 -- 172.21.15.69:0/3030517568 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96a0004300 con 0x7f96b0108eb0 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.891+0000 7f96af7fe700 1 -- 172.21.15.69:0/3030517568 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a0011a20 con 0x7f96b0108eb0 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.892+0000 7f96b66b2700 1 -- 172.21.15.69:0/3030517568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 msgr2=0x7f96b01092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:40.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.892+0000 7f96b66b2700 1 --2- 172.21.15.69:0/3030517568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b01092d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96a00098a0 tx=0x7f96a0009bb0 comp rx=0 tx=0).stop 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.892+0000 7f96b66b2700 1 -- 172.21.15.69:0/3030517568 shutdown_connections 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.892+0000 7f96b66b2700 1 --2- 172.21.15.69:0/3030517568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b01092d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.892+0000 7f96b66b2700 1 -- 172.21.15.69:0/3030517568 >> 172.21.15.69:0/3030517568 conn(0x7f96b0104500 msgr2=0x7f96b0106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.892+0000 7f96b66b2700 1 -- 172.21.15.69:0/3030517568 shutdown_connections 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.892+0000 7f96b66b2700 1 -- 172.21.15.69:0/3030517568 wait complete. 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.892+0000 7f96b66b2700 1 Processor -- start 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.892+0000 7f96b66b2700 1 -- start start 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96b66b2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96b66b2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f96b010d310 con 0x7f96b0108eb0 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96affff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96affff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41716/0 (socket says 172.21.15.69:41716) 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96affff700 1 -- 172.21.15.69:0/935878230 learned_addr learned my addr 172.21.15.69:0/935878230 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96affff700 1 -- 172.21.15.69:0/935878230 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b019d330 con 0x7f96b0108eb0 2024-05-14T16:39:40.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96affff700 1 --2- 172.21.15.69:0/935878230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b019cdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f96a00119f0 tx=0x7f96a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96adffb700 1 -- 172.21.15.69:0/935878230 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a00039e0 con 0x7f96b0108eb0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96adffb700 1 -- 172.21.15.69:0/935878230 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96a001b8f0 con 0x7f96b0108eb0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96adffb700 1 -- 172.21.15.69:0/935878230 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a00237a0 con 0x7f96b0108eb0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96b66b2700 1 -- 172.21.15.69:0/935878230 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b019d5c0 con 0x7f96b0108eb0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.893+0000 7f96b66b2700 1 -- 172.21.15.69:0/935878230 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b01a02b0 con 0x7f96b0108eb0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.894+0000 7f96b66b2700 1 -- 172.21.15.69:0/935878230 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b0064ba0 con 0x7f96b0108eb0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.894+0000 7f96adffb700 1 -- 172.21.15.69:0/935878230 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f96a001b440 con 0x7f96b0108eb0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.895+0000 7f96adffb700 1 --2- 172.21.15.69:0/935878230 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f969c03e570 0x7f969c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.895+0000 7f96adffb700 1 -- 172.21.15.69:0/935878230 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f96a0054080 con 0x7f96b0108eb0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.895+0000 7f96a7fff700 1 --2- 172.21.15.69:0/935878230 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f969c03e570 0x7f969c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.895+0000 7f96a7fff700 1 --2- 172.21.15.69:0/935878230 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f969c03e570 0x7f969c040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9698006fd0 tx=0x7f9698006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:39.899+0000 7f96adffb700 1 -- 172.21.15.69:0/935878230 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96a001f030 con 0x7f96b0108eb0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.216+0000 7f96b66b2700 1 -- 172.21.15.69:0/935878230 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f96b01a0860 con 0x7f96b0108eb0 2024-05-14T16:39:40.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.219+0000 7f96adffb700 1 -- 172.21.15.69:0/935878230 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f96a001baa0 con 0x7f96b0108eb0 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.221+0000 7f96b66b2700 1 -- 172.21.15.69:0/935878230 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f969c03e570 msgr2=0x7f969c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.221+0000 7f96b66b2700 1 --2- 172.21.15.69:0/935878230 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f969c03e570 0x7f969c040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9698006fd0 tx=0x7f9698006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.221+0000 7f96b66b2700 1 -- 172.21.15.69:0/935878230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 msgr2=0x7f96b019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.221+0000 7f96b66b2700 1 --2- 172.21.15.69:0/935878230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b019cdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f96a00119f0 tx=0x7f96a0011ad0 comp rx=0 tx=0).stop 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.221+0000 7f96b66b2700 1 -- 172.21.15.69:0/935878230 shutdown_connections 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.221+0000 7f96b66b2700 1 --2- 172.21.15.69:0/935878230 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f969c03e570 0x7f969c040a30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.221+0000 7f96b66b2700 1 --2- 172.21.15.69:0/935878230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96b0108eb0 0x7f96b019cdf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.221+0000 7f96b66b2700 1 -- 172.21.15.69:0/935878230 >> 172.21.15.69:0/935878230 conn(0x7f96b0104500 msgr2=0x7f96b0105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.221+0000 7f96b66b2700 1 -- 172.21.15.69:0/935878230 shutdown_connections 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-05-14T16:39:40.221+0000 7f96b66b2700 1 -- 172.21.15.69:0/935878230 wait complete. 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-14T16:39:40.703 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout:Or, if you are only running a single cluster on this host: 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout: ceph telemetry on 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout:For more information see: 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:40.704 INFO:teuthology.orchestra.run.smithi069.stdout:Bootstrap complete. 2024-05-14T16:39:40.735 INFO:tasks.cephadm:Fetching config... 2024-05-14T16:39:40.735 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:39:40.736 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-14T16:39:40.751 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-14T16:39:40.751 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:39:40.751 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-14T16:39:40.808 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-14T16:39:40.808 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:39:40.808 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-14T16:39:40.873 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-14T16:39:40.873 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:39:40.874 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-14T16:39:40.930 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-14T16:39:40.930 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJrE/J1SQekF8P0jH8uWb/HFR6iud/QCSeyVtuN8BsYabSgD5NLA6R5U2vdBIGh/V/mZ1TGT33DXeN7tEIIQwL/iqj5VPQR93fDxCm8tv3TDwfaQoK+4XvHIrWr4OywDiSG/YuouEbbRg0bdSks6ZYJxzW5uCYRy99r2N+K7lP+YtxjBR4JZZv9LO4r1O2rgaxre/m6PgxJTQFMSnoDNP1jEa7k852zvH79cmh0bHYLHcAeZQEoSS/YrWzNFQ0KNpZ36YKP22xxgQdsu3k6I0yb/ksGaJUsG2a+H8B/XY+iIvOqx+oMcGK+sFIZOKq25onFdZh0Eoaq0a/M8p+CgjTO+oX3r7IWhRvPX4g1xbiXz4eDYCvoXXDJCFTtRfNGApUnYwo95YpTQedMTpLm4ORxGWtbzqUhyRI1PYUkovI7f0fhQUqpnxSJTS5E5fCPlS/kHX/o+odeAx1Fh7Nj0JR7migU5S2GMzZQaYfujGHvOteqwrrWwXBBUVfW2oCI40= ceph-356e185e-1210-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T16:39:41.005 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJrE/J1SQekF8P0jH8uWb/HFR6iud/QCSeyVtuN8BsYabSgD5NLA6R5U2vdBIGh/V/mZ1TGT33DXeN7tEIIQwL/iqj5VPQR93fDxCm8tv3TDwfaQoK+4XvHIrWr4OywDiSG/YuouEbbRg0bdSks6ZYJxzW5uCYRy99r2N+K7lP+YtxjBR4JZZv9LO4r1O2rgaxre/m6PgxJTQFMSnoDNP1jEa7k852zvH79cmh0bHYLHcAeZQEoSS/YrWzNFQ0KNpZ36YKP22xxgQdsu3k6I0yb/ksGaJUsG2a+H8B/XY+iIvOqx+oMcGK+sFIZOKq25onFdZh0Eoaq0a/M8p+CgjTO+oX3r7IWhRvPX4g1xbiXz4eDYCvoXXDJCFTtRfNGApUnYwo95YpTQedMTpLm4ORxGWtbzqUhyRI1PYUkovI7f0fhQUqpnxSJTS5E5fCPlS/kHX/o+odeAx1Fh7Nj0JR7migU5S2GMzZQaYfujGHvOteqwrrWwXBBUVfW2oCI40= ceph-356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:39:41.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJrE/J1SQekF8P0jH8uWb/HFR6iud/QCSeyVtuN8BsYabSgD5NLA6R5U2vdBIGh/V/mZ1TGT33DXeN7tEIIQwL/iqj5VPQR93fDxCm8tv3TDwfaQoK+4XvHIrWr4OywDiSG/YuouEbbRg0bdSks6ZYJxzW5uCYRy99r2N+K7lP+YtxjBR4JZZv9LO4r1O2rgaxre/m6PgxJTQFMSnoDNP1jEa7k852zvH79cmh0bHYLHcAeZQEoSS/YrWzNFQ0KNpZ36YKP22xxgQdsu3k6I0yb/ksGaJUsG2a+H8B/XY+iIvOqx+oMcGK+sFIZOKq25onFdZh0Eoaq0a/M8p+CgjTO+oX3r7IWhRvPX4g1xbiXz4eDYCvoXXDJCFTtRfNGApUnYwo95YpTQedMTpLm4ORxGWtbzqUhyRI1PYUkovI7f0fhQUqpnxSJTS5E5fCPlS/kHX/o+odeAx1Fh7Nj0JR7migU5S2GMzZQaYfujGHvOteqwrrWwXBBUVfW2oCI40= ceph-356e185e-1210-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T16:39:41.061 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJrE/J1SQekF8P0jH8uWb/HFR6iud/QCSeyVtuN8BsYabSgD5NLA6R5U2vdBIGh/V/mZ1TGT33DXeN7tEIIQwL/iqj5VPQR93fDxCm8tv3TDwfaQoK+4XvHIrWr4OywDiSG/YuouEbbRg0bdSks6ZYJxzW5uCYRy99r2N+K7lP+YtxjBR4JZZv9LO4r1O2rgaxre/m6PgxJTQFMSnoDNP1jEa7k852zvH79cmh0bHYLHcAeZQEoSS/YrWzNFQ0KNpZ36YKP22xxgQdsu3k6I0yb/ksGaJUsG2a+H8B/XY+iIvOqx+oMcGK+sFIZOKq25onFdZh0Eoaq0a/M8p+CgjTO+oX3r7IWhRvPX4g1xbiXz4eDYCvoXXDJCFTtRfNGApUnYwo95YpTQedMTpLm4ORxGWtbzqUhyRI1PYUkovI7f0fhQUqpnxSJTS5E5fCPlS/kHX/o+odeAx1Fh7Nj0JR7migU5S2GMzZQaYfujGHvOteqwrrWwXBBUVfW2oCI40= ceph-356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:39:41.072 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJrE/J1SQekF8P0jH8uWb/HFR6iud/QCSeyVtuN8BsYabSgD5NLA6R5U2vdBIGh/V/mZ1TGT33DXeN7tEIIQwL/iqj5VPQR93fDxCm8tv3TDwfaQoK+4XvHIrWr4OywDiSG/YuouEbbRg0bdSks6ZYJxzW5uCYRy99r2N+K7lP+YtxjBR4JZZv9LO4r1O2rgaxre/m6PgxJTQFMSnoDNP1jEa7k852zvH79cmh0bHYLHcAeZQEoSS/YrWzNFQ0KNpZ36YKP22xxgQdsu3k6I0yb/ksGaJUsG2a+H8B/XY+iIvOqx+oMcGK+sFIZOKq25onFdZh0Eoaq0a/M8p+CgjTO+oX3r7IWhRvPX4g1xbiXz4eDYCvoXXDJCFTtRfNGApUnYwo95YpTQedMTpLm4ORxGWtbzqUhyRI1PYUkovI7f0fhQUqpnxSJTS5E5fCPlS/kHX/o+odeAx1Fh7Nj0JR7migU5S2GMzZQaYfujGHvOteqwrrWwXBBUVfW2oCI40= ceph-356e185e-1210-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T16:39:41.111 INFO:teuthology.orchestra.run.smithi203.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJrE/J1SQekF8P0jH8uWb/HFR6iud/QCSeyVtuN8BsYabSgD5NLA6R5U2vdBIGh/V/mZ1TGT33DXeN7tEIIQwL/iqj5VPQR93fDxCm8tv3TDwfaQoK+4XvHIrWr4OywDiSG/YuouEbbRg0bdSks6ZYJxzW5uCYRy99r2N+K7lP+YtxjBR4JZZv9LO4r1O2rgaxre/m6PgxJTQFMSnoDNP1jEa7k852zvH79cmh0bHYLHcAeZQEoSS/YrWzNFQ0KNpZ36YKP22xxgQdsu3k6I0yb/ksGaJUsG2a+H8B/XY+iIvOqx+oMcGK+sFIZOKq25onFdZh0Eoaq0a/M8p+CgjTO+oX3r7IWhRvPX4g1xbiXz4eDYCvoXXDJCFTtRfNGApUnYwo95YpTQedMTpLm4ORxGWtbzqUhyRI1PYUkovI7f0fhQUqpnxSJTS5E5fCPlS/kHX/o+odeAx1Fh7Nj0JR7migU5S2GMzZQaYfujGHvOteqwrrWwXBBUVfW2oCI40= ceph-356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:39:41.123 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-14T16:39:41.361 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:39:41.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:41 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/935878230' entity='client.admin' 2024-05-14T16:39:43.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.464+0000 7f8ac08e3700 1 -- 172.21.15.69:0/2563736975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc101e50 msgr2=0x7f8abc102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:43.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.464+0000 7f8ac08e3700 1 --2- 172.21.15.69:0/2563736975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc101e50 0x7f8abc102270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009910 tx=0x7f8aac009c20 comp rx=0 tx=0).stop 2024-05-14T16:39:43.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.465+0000 7f8ac08e3700 1 -- 172.21.15.69:0/2563736975 shutdown_connections 2024-05-14T16:39:43.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.465+0000 7f8ac08e3700 1 --2- 172.21.15.69:0/2563736975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc101e50 0x7f8abc102270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:43.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.465+0000 7f8ac08e3700 1 -- 172.21.15.69:0/2563736975 >> 172.21.15.69:0/2563736975 conn(0x7f8abc0fd440 msgr2=0x7f8abc0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:43.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.465+0000 7f8ac08e3700 1 -- 172.21.15.69:0/2563736975 shutdown_connections 2024-05-14T16:39:43.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.465+0000 7f8ac08e3700 1 -- 172.21.15.69:0/2563736975 wait complete. 2024-05-14T16:39:43.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.465+0000 7f8ac08e3700 1 Processor -- start 2024-05-14T16:39:43.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.465+0000 7f8ac08e3700 1 -- start start 2024-05-14T16:39:43.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.465+0000 7f8ac08e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc101e50 0x7f8abc198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:43.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.465+0000 7f8ac08e3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc107d50 con 0x7f8abc101e50 2024-05-14T16:39:43.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.466+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc101e50 0x7f8abc198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:43.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.466+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc101e50 0x7f8abc198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41726/0 (socket says 172.21.15.69:41726) 2024-05-14T16:39:43.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.466+0000 7f8aba59c700 1 -- 172.21.15.69:0/1088740288 learned_addr learned my addr 172.21.15.69:0/1088740288 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:43.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.466+0000 7f8aba59c700 1 -- 172.21.15.69:0/1088740288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc198b50 con 0x7f8abc101e50 2024-05-14T16:39:43.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.466+0000 7f8aba59c700 1 --2- 172.21.15.69:0/1088740288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc101e50 0x7f8abc198610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8aac005800 tx=0x7f8aac009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:43.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.466+0000 7f8aab7fe700 1 -- 172.21.15.69:0/1088740288 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aac027070 con 0x7f8abc101e50 2024-05-14T16:39:43.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.466+0000 7f8aab7fe700 1 -- 172.21.15.69:0/1088740288 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aac00be10 con 0x7f8abc101e50 2024-05-14T16:39:43.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.466+0000 7f8ac08e3700 1 -- 172.21.15.69:0/1088740288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc198de0 con 0x7f8abc101e50 2024-05-14T16:39:43.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.466+0000 7f8aab7fe700 1 -- 172.21.15.69:0/1088740288 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aac00f460 con 0x7f8abc101e50 2024-05-14T16:39:43.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.466+0000 7f8ac08e3700 1 -- 172.21.15.69:0/1088740288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc19bad0 con 0x7f8abc101e50 2024-05-14T16:39:43.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.467+0000 7f8ac08e3700 1 -- 172.21.15.69:0/1088740288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc1084d0 con 0x7f8abc101e50 2024-05-14T16:39:43.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.467+0000 7f8aab7fe700 1 -- 172.21.15.69:0/1088740288 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8aac02a070 con 0x7f8abc101e50 2024-05-14T16:39:43.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.468+0000 7f8aab7fe700 1 --2- 172.21.15.69:0/1088740288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8aa403e570 0x7f8aa4040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:43.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.468+0000 7f8aab7fe700 1 -- 172.21.15.69:0/1088740288 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8aac01e030 con 0x7f8abc101e50 2024-05-14T16:39:43.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.468+0000 7f8ab9d9b700 1 --2- 172.21.15.69:0/1088740288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8aa403e570 0x7f8aa4040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:43.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.468+0000 7f8ab9d9b700 1 --2- 172.21.15.69:0/1088740288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8aa403e570 0x7f8aa4040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0006fd0 tx=0x7f8ab0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:43.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.473+0000 7f8aab7fe700 1 -- 172.21.15.69:0/1088740288 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8aac013070 con 0x7f8abc101e50 2024-05-14T16:39:43.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.682+0000 7f8ac08e3700 1 -- 172.21.15.69:0/1088740288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f8abc19bed0 con 0x7f8abc101e50 2024-05-14T16:39:43.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:43.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:43.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:39:43.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:43.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:39:43.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:43.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.687+0000 7f8aab7fe700 1 -- 172.21.15.69:0/1088740288 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8aac031070 con 0x7f8abc101e50 2024-05-14T16:39:43.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.690+0000 7f8ac08e3700 1 -- 172.21.15.69:0/1088740288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8aa403e570 msgr2=0x7f8aa4040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:43.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.690+0000 7f8ac08e3700 1 --2- 172.21.15.69:0/1088740288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8aa403e570 0x7f8aa4040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0006fd0 tx=0x7f8ab0006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:43.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.690+0000 7f8ac08e3700 1 -- 172.21.15.69:0/1088740288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc101e50 msgr2=0x7f8abc198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:43.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.690+0000 7f8ac08e3700 1 --2- 172.21.15.69:0/1088740288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc101e50 0x7f8abc198610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8aac005800 tx=0x7f8aac009f60 comp rx=0 tx=0).stop 2024-05-14T16:39:43.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.690+0000 7f8ac08e3700 1 -- 172.21.15.69:0/1088740288 shutdown_connections 2024-05-14T16:39:43.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.690+0000 7f8ac08e3700 1 --2- 172.21.15.69:0/1088740288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8aa403e570 0x7f8aa4040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:43.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.690+0000 7f8ac08e3700 1 --2- 172.21.15.69:0/1088740288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc101e50 0x7f8abc198610 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:43.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.690+0000 7f8ac08e3700 1 -- 172.21.15.69:0/1088740288 >> 172.21.15.69:0/1088740288 conn(0x7f8abc0fd440 msgr2=0x7f8abc06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:43.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.690+0000 7f8ac08e3700 1 -- 172.21.15.69:0/1088740288 shutdown_connections 2024-05-14T16:39:43.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:43.690+0000 7f8ac08e3700 1 -- 172.21.15.69:0/1088740288 wait complete. 2024-05-14T16:39:44.272 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-14T16:39:44.273 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-14T16:39:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:39:44.913 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:44 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1088740288' entity='client.admin' 2024-05-14T16:39:44.913 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:44 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:39:44.914 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:44 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:46.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.036+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/2749922306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d38103ae0 msgr2=0x7f7d38105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:46.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.036+0000 7f7d3e0c9700 1 --2- 172.21.15.69:0/2749922306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d38103ae0 0x7f7d38105ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800b0a0 tx=0x7f7d2800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:39:46.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.036+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/2749922306 shutdown_connections 2024-05-14T16:39:46.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.036+0000 7f7d3e0c9700 1 --2- 172.21.15.69:0/2749922306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d38103ae0 0x7f7d38105ed0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:46.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.036+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/2749922306 >> 172.21.15.69:0/2749922306 conn(0x7f7d380fd450 msgr2=0x7f7d380ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:46.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.036+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/2749922306 shutdown_connections 2024-05-14T16:39:46.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.036+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/2749922306 wait complete. 2024-05-14T16:39:46.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.036+0000 7f7d3e0c9700 1 Processor -- start 2024-05-14T16:39:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d3e0c9700 1 -- start start 2024-05-14T16:39:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d3e0c9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d38103ae0 0x7f7d38198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d3e0c9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7d38108f40 con 0x7f7d38103ae0 2024-05-14T16:39:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d377fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d38103ae0 0x7f7d38198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d377fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d38103ae0 0x7f7d38198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57202/0 (socket says 172.21.15.69:57202) 2024-05-14T16:39:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d377fe700 1 -- 172.21.15.69:0/3275402439 learned_addr learned my addr 172.21.15.69:0/3275402439 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d377fe700 1 -- 172.21.15.69:0/3275402439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d38198a50 con 0x7f7d38103ae0 2024-05-14T16:39:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d377fe700 1 --2- 172.21.15.69:0/3275402439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d38103ae0 0x7f7d38198510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d28015040 tx=0x7f7d28003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d34ff9700 1 -- 172.21.15.69:0/3275402439 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d28021020 con 0x7f7d38103ae0 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d34ff9700 1 -- 172.21.15.69:0/3275402439 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2800bdb0 con 0x7f7d38103ae0 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.037+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/3275402439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d38198ce0 con 0x7f7d38103ae0 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.038+0000 7f7d34ff9700 1 -- 172.21.15.69:0/3275402439 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d28017040 con 0x7f7d38103ae0 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.038+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/3275402439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3819b9d0 con 0x7f7d38103ae0 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.038+0000 7f7d34ff9700 1 -- 172.21.15.69:0/3275402439 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7d28024020 con 0x7f7d38103ae0 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.038+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/3275402439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d381991a0 con 0x7f7d38103ae0 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.039+0000 7f7d34ff9700 1 --2- 172.21.15.69:0/3275402439 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7d2403e510 0x7f7d240409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.039+0000 7f7d34ff9700 1 -- 172.21.15.69:0/3275402439 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7d2800f050 con 0x7f7d38103ae0 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.039+0000 7f7d36ffd700 1 --2- 172.21.15.69:0/3275402439 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7d2403e510 0x7f7d240409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.039+0000 7f7d36ffd700 1 --2- 172.21.15.69:0/3275402439 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7d2403e510 0x7f7d240409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006fd0 tx=0x7f7d20006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:46.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.044+0000 7f7d34ff9700 1 -- 172.21.15.69:0/3275402439 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d2802b020 con 0x7f7d38103ae0 2024-05-14T16:39:46.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.250+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/3275402439 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f7d3802e670 con 0x7f7d2403e510 2024-05-14T16:39:46.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.254+0000 7f7d34ff9700 1 -- 172.21.15.69:0/3275402439 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7d3802e670 con 0x7f7d2403e510 2024-05-14T16:39:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.256+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/3275402439 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7d2403e510 msgr2=0x7f7d240409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.256+0000 7f7d3e0c9700 1 --2- 172.21.15.69:0/3275402439 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7d2403e510 0x7f7d240409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006fd0 tx=0x7f7d20006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.256+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/3275402439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d38103ae0 msgr2=0x7f7d38198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.256+0000 7f7d3e0c9700 1 --2- 172.21.15.69:0/3275402439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d38103ae0 0x7f7d38198510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d28015040 tx=0x7f7d28003ce0 comp rx=0 tx=0).stop 2024-05-14T16:39:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.257+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/3275402439 shutdown_connections 2024-05-14T16:39:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.257+0000 7f7d3e0c9700 1 --2- 172.21.15.69:0/3275402439 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7d2403e510 0x7f7d240409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.257+0000 7f7d3e0c9700 1 --2- 172.21.15.69:0/3275402439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d38103ae0 0x7f7d38198510 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.257+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/3275402439 >> 172.21.15.69:0/3275402439 conn(0x7f7d380fd450 msgr2=0x7f7d380fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.257+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/3275402439 shutdown_connections 2024-05-14T16:39:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:46.257+0000 7f7d3e0c9700 1 -- 172.21.15.69:0/3275402439 wait complete. 2024-05-14T16:39:47.105 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi092 2024-05-14T16:39:47.106 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:39:47.106 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.conf 2024-05-14T16:39:47.123 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:39:47.123 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:39:47.181 INFO:tasks.cephadm:Adding host smithi092 to orchestrator... 2024-05-14T16:39:47.181 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch host add smithi092 2024-05-14T16:39:47.413 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:39:47.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:47 smithi069 ceph-mon[31507]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:47.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:47.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:39:47.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:39:47.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:39:47.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:47 smithi069 ceph-mon[31507]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-14T16:39:47.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:47 smithi069 ceph-mon[31507]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:39:47.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:47.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:47.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:48.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:48 smithi069 ceph-mon[31507]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:39:48.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:48 smithi069 ceph-mon[31507]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:39:48.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.847+0000 7fd7d0e09700 1 -- 172.21.15.69:0/1171759595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7cc101730 msgr2=0x7fd7cc101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:48.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.848+0000 7fd7d0e09700 1 --2- 172.21.15.69:0/1171759595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7cc101730 0x7fd7cc101b10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009910 tx=0x7fd7bc009c20 comp rx=0 tx=0).stop 2024-05-14T16:39:48.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.848+0000 7fd7d0e09700 1 -- 172.21.15.69:0/1171759595 shutdown_connections 2024-05-14T16:39:48.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.848+0000 7fd7d0e09700 1 --2- 172.21.15.69:0/1171759595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7cc101730 0x7fd7cc101b10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:48.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.848+0000 7fd7d0e09700 1 -- 172.21.15.69:0/1171759595 >> 172.21.15.69:0/1171759595 conn(0x7fd7cc0785c0 msgr2=0x7fd7cc0789d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:48.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.848+0000 7fd7d0e09700 1 -- 172.21.15.69:0/1171759595 shutdown_connections 2024-05-14T16:39:48.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.848+0000 7fd7d0e09700 1 -- 172.21.15.69:0/1171759595 wait complete. 2024-05-14T16:39:48.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.848+0000 7fd7d0e09700 1 Processor -- start 2024-05-14T16:39:48.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.848+0000 7fd7d0e09700 1 -- start start 2024-05-14T16:39:48.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7d0e09700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7cc101730 0x7fd7cc1026b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:48.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7d0e09700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc10d470 con 0x7fd7cc101730 2024-05-14T16:39:48.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7ca59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7cc101730 0x7fd7cc1026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:48.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7ca59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7cc101730 0x7fd7cc1026b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57230/0 (socket says 172.21.15.69:57230) 2024-05-14T16:39:48.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7ca59c700 1 -- 172.21.15.69:0/2154866919 learned_addr learned my addr 172.21.15.69:0/2154866919 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7ca59c700 1 -- 172.21.15.69:0/2154866919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc1045a0 con 0x7fd7cc101730 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7ca59c700 1 --2- 172.21.15.69:0/2154866919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7cc101730 0x7fd7cc1026b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0056d0 tx=0x7fd7bc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7c37fe700 1 -- 172.21.15.69:0/2154866919 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc026070 con 0x7fd7cc101730 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7c37fe700 1 -- 172.21.15.69:0/2154866919 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7bc00bc00 con 0x7fd7cc101730 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7c37fe700 1 -- 172.21.15.69:0/2154866919 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc003920 con 0x7fd7cc101730 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7d0e09700 1 -- 172.21.15.69:0/2154866919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7cc102c50 con 0x7fd7cc101730 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.849+0000 7fd7d0e09700 1 -- 172.21.15.69:0/2154866919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7cc103110 con 0x7fd7cc101730 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.850+0000 7fd7c37fe700 1 -- 172.21.15.69:0/2154866919 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd7bc029070 con 0x7fd7cc101730 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.850+0000 7fd7d0e09700 1 -- 172.21.15.69:0/2154866919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7cc064ba0 con 0x7fd7cc101730 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.850+0000 7fd7c37fe700 1 --2- 172.21.15.69:0/2154866919 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd7b803e570 0x7fd7b8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.851+0000 7fd7c37fe700 1 -- 172.21.15.69:0/2154866919 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd7bc01e070 con 0x7fd7cc101730 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.851+0000 7fd7c9d9b700 1 --2- 172.21.15.69:0/2154866919 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd7b803e570 0x7fd7b8040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:48.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.851+0000 7fd7c9d9b700 1 --2- 172.21.15.69:0/2154866919 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd7b803e570 0x7fd7b8040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4006fd0 tx=0x7fd7b4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:48.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:48.856+0000 7fd7c37fe700 1 -- 172.21.15.69:0/2154866919 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd7bc013070 con 0x7fd7cc101730 2024-05-14T16:39:49.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:49.062+0000 7fd7d0e09700 1 -- 172.21.15.69:0/2154866919 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}) -- 0x7fd7cc104810 con 0x7fd7b803e570 2024-05-14T16:39:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:49 smithi069 ceph-mon[31507]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:39:50.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:50 smithi069 ceph-mon[31507]: Deploying cephadm binary to smithi092 2024-05-14T16:39:53.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:52 smithi069 ceph-mon[31507]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:39:53.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.222+0000 7fd7c37fe700 1 -- 172.21.15.69:0/2154866919 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fd7cc104810 con 0x7fd7b803e570 2024-05-14T16:39:53.222 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi092' with addr '172.21.15.92' 2024-05-14T16:39:53.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.225+0000 7fd7d0e09700 1 -- 172.21.15.69:0/2154866919 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd7b803e570 msgr2=0x7fd7b8040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:53.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.225+0000 7fd7d0e09700 1 --2- 172.21.15.69:0/2154866919 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd7b803e570 0x7fd7b8040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4006fd0 tx=0x7fd7b4006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:53.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.225+0000 7fd7d0e09700 1 -- 172.21.15.69:0/2154866919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7cc101730 msgr2=0x7fd7cc1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:53.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.225+0000 7fd7d0e09700 1 --2- 172.21.15.69:0/2154866919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7cc101730 0x7fd7cc1026b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0056d0 tx=0x7fd7bc0057b0 comp rx=0 tx=0).stop 2024-05-14T16:39:53.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.225+0000 7fd7d0e09700 1 -- 172.21.15.69:0/2154866919 shutdown_connections 2024-05-14T16:39:53.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.225+0000 7fd7d0e09700 1 --2- 172.21.15.69:0/2154866919 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd7b803e570 0x7fd7b8040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:53.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.225+0000 7fd7d0e09700 1 --2- 172.21.15.69:0/2154866919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7cc101730 0x7fd7cc1026b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:53.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.225+0000 7fd7d0e09700 1 -- 172.21.15.69:0/2154866919 >> 172.21.15.69:0/2154866919 conn(0x7fd7cc0785c0 msgr2=0x7fd7cc104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:53.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.225+0000 7fd7d0e09700 1 -- 172.21.15.69:0/2154866919 shutdown_connections 2024-05-14T16:39:53.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:53.225+0000 7fd7d0e09700 1 -- 172.21.15.69:0/2154866919 wait complete. 2024-05-14T16:39:53.882 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-14T16:39:54.116 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:39:54.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:54.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:54 smithi069 ceph-mon[31507]: Added host smithi092 2024-05-14T16:39:54.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:39:54.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:55.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:55 smithi069 ceph-mon[31507]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:39:55.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:55.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.549+0000 7f5ccdf73700 1 -- 172.21.15.69:0/3585197963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cc8104110 msgr2=0x7f5cc81044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:55.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.549+0000 7f5ccdf73700 1 --2- 172.21.15.69:0/3585197963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cc8104110 0x7f5cc81044f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800b0a0 tx=0x7f5cb800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:39:55.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.549+0000 7f5ccdf73700 1 -- 172.21.15.69:0/3585197963 shutdown_connections 2024-05-14T16:39:55.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.549+0000 7f5ccdf73700 1 --2- 172.21.15.69:0/3585197963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cc8104110 0x7f5cc81044f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:55.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.549+0000 7f5ccdf73700 1 -- 172.21.15.69:0/3585197963 >> 172.21.15.69:0/3585197963 conn(0x7f5cc80fd940 msgr2=0x7f5cc80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:55.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.549+0000 7f5ccdf73700 1 -- 172.21.15.69:0/3585197963 shutdown_connections 2024-05-14T16:39:55.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.549+0000 7f5ccdf73700 1 -- 172.21.15.69:0/3585197963 wait complete. 2024-05-14T16:39:55.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.549+0000 7f5ccdf73700 1 Processor -- start 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.550+0000 7f5ccdf73700 1 -- start start 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.550+0000 7f5ccdf73700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cc8104110 0x7f5cc819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.550+0000 7f5ccdf73700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc810d4b0 con 0x7f5cc8104110 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.550+0000 7f5cc77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cc8104110 0x7f5cc819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.550+0000 7f5cc77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cc8104110 0x7f5cc819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54172/0 (socket says 172.21.15.69:54172) 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.550+0000 7f5cc77fe700 1 -- 172.21.15.69:0/1715776485 learned_addr learned my addr 172.21.15.69:0/1715776485 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.550+0000 7f5cc77fe700 1 -- 172.21.15.69:0/1715776485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc819eb60 con 0x7f5cc8104110 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.551+0000 7f5cc77fe700 1 --2- 172.21.15.69:0/1715776485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cc8104110 0x7f5cc819e620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8015040 tx=0x7f5cb8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.551+0000 7f5cc4ff9700 1 -- 172.21.15.69:0/1715776485 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cb8021020 con 0x7f5cc8104110 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.551+0000 7f5ccdf73700 1 -- 172.21.15.69:0/1715776485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc819edf0 con 0x7f5cc8104110 2024-05-14T16:39:55.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.551+0000 7f5cc4ff9700 1 -- 172.21.15.69:0/1715776485 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb8004000 con 0x7f5cc8104110 2024-05-14T16:39:55.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.551+0000 7f5cc4ff9700 1 -- 172.21.15.69:0/1715776485 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cb8017040 con 0x7f5cc8104110 2024-05-14T16:39:55.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.551+0000 7f5ccdf73700 1 -- 172.21.15.69:0/1715776485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc81987a0 con 0x7f5cc8104110 2024-05-14T16:39:55.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.552+0000 7f5cc4ff9700 1 -- 172.21.15.69:0/1715776485 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f5cb8024020 con 0x7f5cc8104110 2024-05-14T16:39:55.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.552+0000 7f5ccdf73700 1 -- 172.21.15.69:0/1715776485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc806bc90 con 0x7f5cc8104110 2024-05-14T16:39:55.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.552+0000 7f5cc4ff9700 1 --2- 172.21.15.69:0/1715776485 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5cb003e510 0x7f5cb00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:55.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.552+0000 7f5cc4ff9700 1 -- 172.21.15.69:0/1715776485 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5cb800f050 con 0x7f5cc8104110 2024-05-14T16:39:55.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.552+0000 7f5cc6ffd700 1 --2- 172.21.15.69:0/1715776485 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5cb003e510 0x7f5cb00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:55.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.552+0000 7f5cc6ffd700 1 --2- 172.21.15.69:0/1715776485 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5cb003e510 0x7f5cb00409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc006fd0 tx=0x7f5cbc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:55.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.557+0000 7f5cc4ff9700 1 -- 172.21.15.69:0/1715776485 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5cb802b020 con 0x7f5cc8104110 2024-05-14T16:39:55.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.763+0000 7f5ccdf73700 1 -- 172.21.15.69:0/1715776485 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cc8064ba0 con 0x7f5cb003e510 2024-05-14T16:39:55.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.764+0000 7f5cc4ff9700 1 -- 172.21.15.69:0/1715776485 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f5cc8064ba0 con 0x7f5cb003e510 2024-05-14T16:39:55.765 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:39:55.765 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}] 2024-05-14T16:39:55.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.767+0000 7f5ccdf73700 1 -- 172.21.15.69:0/1715776485 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5cb003e510 msgr2=0x7f5cb00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:55.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.767+0000 7f5ccdf73700 1 --2- 172.21.15.69:0/1715776485 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5cb003e510 0x7f5cb00409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc006fd0 tx=0x7f5cbc006e40 comp rx=0 tx=0).stop 2024-05-14T16:39:55.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.767+0000 7f5ccdf73700 1 -- 172.21.15.69:0/1715776485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cc8104110 msgr2=0x7f5cc819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:55.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.767+0000 7f5ccdf73700 1 --2- 172.21.15.69:0/1715776485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cc8104110 0x7f5cc819e620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8015040 tx=0x7f5cb8009800 comp rx=0 tx=0).stop 2024-05-14T16:39:55.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.767+0000 7f5ccdf73700 1 -- 172.21.15.69:0/1715776485 shutdown_connections 2024-05-14T16:39:55.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.767+0000 7f5ccdf73700 1 --2- 172.21.15.69:0/1715776485 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5cb003e510 0x7f5cb00409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:55.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.767+0000 7f5ccdf73700 1 --2- 172.21.15.69:0/1715776485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cc8104110 0x7f5cc819e620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:55.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.767+0000 7f5ccdf73700 1 -- 172.21.15.69:0/1715776485 >> 172.21.15.69:0/1715776485 conn(0x7f5cc80fd940 msgr2=0x7f5cc8104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:55.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.767+0000 7f5ccdf73700 1 -- 172.21.15.69:0/1715776485 shutdown_connections 2024-05-14T16:39:55.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:55.767+0000 7f5ccdf73700 1 -- 172.21.15.69:0/1715776485 wait complete. 2024-05-14T16:39:56.360 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi203 2024-05-14T16:39:56.361 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:39:56.361 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.conf 2024-05-14T16:39:56.380 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:39:56.380 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:39:56.437 INFO:tasks.cephadm:Adding host smithi203 to orchestrator... 2024-05-14T16:39:56.438 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch host add smithi203 2024-05-14T16:39:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:39:57.054 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:56 smithi069 ceph-mon[31507]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:39:57.054 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:39:57.054 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:56 smithi069 ceph-mon[31507]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:39:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.069+0000 7fb6b67e1700 1 -- 172.21.15.69:0/806124287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6b0104e60 msgr2=0x7fb6b0105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:39:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.069+0000 7fb6b67e1700 1 --2- 172.21.15.69:0/806124287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6b0104e60 0x7fb6b0105240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009910 tx=0x7fb6a0009c20 comp rx=0 tx=0).stop 2024-05-14T16:39:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.069+0000 7fb6b67e1700 1 -- 172.21.15.69:0/806124287 shutdown_connections 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.069+0000 7fb6b67e1700 1 --2- 172.21.15.69:0/806124287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6b0104e60 0x7fb6b0105240 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.069+0000 7fb6b67e1700 1 -- 172.21.15.69:0/806124287 >> 172.21.15.69:0/806124287 conn(0x7fb6b00fd0b0 msgr2=0x7fb6b00ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.069+0000 7fb6b67e1700 1 -- 172.21.15.69:0/806124287 shutdown_connections 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.069+0000 7fb6b67e1700 1 -- 172.21.15.69:0/806124287 wait complete. 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.070+0000 7fb6b67e1700 1 Processor -- start 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.070+0000 7fb6b67e1700 1 -- start start 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6b67e1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6b0104e60 0x7fb6b019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6b67e1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b010d4d0 con 0x7fb6b0104e60 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6affff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6b0104e60 0x7fb6b019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6affff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6b0104e60 0x7fb6b019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54188/0 (socket says 172.21.15.69:54188) 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6affff700 1 -- 172.21.15.69:0/2355071863 learned_addr learned my addr 172.21.15.69:0/2355071863 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6affff700 1 -- 172.21.15.69:0/2355071863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b019eba0 con 0x7fb6b0104e60 2024-05-14T16:39:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6affff700 1 --2- 172.21.15.69:0/2355071863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6b0104e60 0x7fb6b019e660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0006010 tx=0x7fb6a0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6ad7fa700 1 -- 172.21.15.69:0/2355071863 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a0026070 con 0x7fb6b0104e60 2024-05-14T16:39:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6ad7fa700 1 -- 172.21.15.69:0/2355071863 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a000bc00 con 0x7fb6b0104e60 2024-05-14T16:39:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6b67e1700 1 -- 172.21.15.69:0/2355071863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b019ee90 con 0x7fb6b0104e60 2024-05-14T16:39:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6ad7fa700 1 -- 172.21.15.69:0/2355071863 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a0003910 con 0x7fb6b0104e60 2024-05-14T16:39:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.071+0000 7fb6b67e1700 1 -- 172.21.15.69:0/2355071863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b01987e0 con 0x7fb6b0104e60 2024-05-14T16:39:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.072+0000 7fb6ad7fa700 1 -- 172.21.15.69:0/2355071863 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb6a0029070 con 0x7fb6b0104e60 2024-05-14T16:39:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.072+0000 7fb6b67e1700 1 -- 172.21.15.69:0/2355071863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b0104790 con 0x7fb6b0104e60 2024-05-14T16:39:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.072+0000 7fb6ad7fa700 1 --2- 172.21.15.69:0/2355071863 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb69803e570 0x7fb698040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:39:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.073+0000 7fb6ad7fa700 1 -- 172.21.15.69:0/2355071863 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb6a001e070 con 0x7fb6b0104e60 2024-05-14T16:39:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.073+0000 7fb6af7fe700 1 --2- 172.21.15.69:0/2355071863 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb69803e570 0x7fb698040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:39:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.073+0000 7fb6af7fe700 1 --2- 172.21.15.69:0/2355071863 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb69803e570 0x7fb698040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4006fd0 tx=0x7fb6a4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:39:58.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.078+0000 7fb6ad7fa700 1 -- 172.21.15.69:0/2355071863 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb6a0013070 con 0x7fb6b0104e60 2024-05-14T16:39:58.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:39:58.285+0000 7fb6b67e1700 1 -- 172.21.15.69:0/2355071863 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}) -- 0x7fb6b0199d10 con 0x7fb69803e570 2024-05-14T16:39:59.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:58 smithi069 ceph-mon[31507]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:39:59.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:39:58 smithi069 ceph-mon[31507]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:40:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:00 smithi069 ceph-mon[31507]: Deploying cephadm binary to smithi203 2024-05-14T16:40:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:40:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:00 smithi069 ceph-mon[31507]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-14T16:40:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:00 smithi069 ceph-mon[31507]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:01.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:01 smithi069 ceph-mon[31507]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.378+0000 7fb6ad7fa700 1 -- 172.21.15.69:0/2355071863 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb6b0199d10 con 0x7fb69803e570 2024-05-14T16:40:02.379 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi203' with addr '172.21.15.203' 2024-05-14T16:40:02.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.381+0000 7fb6b67e1700 1 -- 172.21.15.69:0/2355071863 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb69803e570 msgr2=0x7fb698040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:02.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.381+0000 7fb6b67e1700 1 --2- 172.21.15.69:0/2355071863 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb69803e570 0x7fb698040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4006fd0 tx=0x7fb6a4006e40 comp rx=0 tx=0).stop 2024-05-14T16:40:02.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.381+0000 7fb6b67e1700 1 -- 172.21.15.69:0/2355071863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6b0104e60 msgr2=0x7fb6b019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:02.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.381+0000 7fb6b67e1700 1 --2- 172.21.15.69:0/2355071863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6b0104e60 0x7fb6b019e660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0006010 tx=0x7fb6a0005780 comp rx=0 tx=0).stop 2024-05-14T16:40:02.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.381+0000 7fb6b67e1700 1 -- 172.21.15.69:0/2355071863 shutdown_connections 2024-05-14T16:40:02.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.381+0000 7fb6b67e1700 1 --2- 172.21.15.69:0/2355071863 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb69803e570 0x7fb698040a30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:02.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.381+0000 7fb6b67e1700 1 --2- 172.21.15.69:0/2355071863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6b0104e60 0x7fb6b019e660 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:02.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.381+0000 7fb6b67e1700 1 -- 172.21.15.69:0/2355071863 >> 172.21.15.69:0/2355071863 conn(0x7fb6b00fd0b0 msgr2=0x7fb6b010abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:02.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.381+0000 7fb6b67e1700 1 -- 172.21.15.69:0/2355071863 shutdown_connections 2024-05-14T16:40:02.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:02.381+0000 7fb6b67e1700 1 -- 172.21.15.69:0/2355071863 wait complete. 2024-05-14T16:40:02.957 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-14T16:40:03.188 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:40:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:03 smithi069 ceph-mon[31507]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:03 smithi069 ceph-mon[31507]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:40:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:03 smithi069 ceph-mon[31507]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:40:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:03 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:03 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:03 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:03 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:03 smithi069 ceph-mon[31507]: Added host smithi203 2024-05-14T16:40:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:03 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:03 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:04.571 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:04 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:04.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.673+0000 7f622db4a700 1 -- 172.21.15.69:0/1326962664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62281031b0 msgr2=0x7f6228103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:04.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.673+0000 7f622db4a700 1 --2- 172.21.15.69:0/1326962664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62281031b0 0x7f6228103590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6218009910 tx=0x7f6218009c20 comp rx=0 tx=0).stop 2024-05-14T16:40:04.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.673+0000 7f622db4a700 1 -- 172.21.15.69:0/1326962664 shutdown_connections 2024-05-14T16:40:04.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.673+0000 7f622db4a700 1 --2- 172.21.15.69:0/1326962664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62281031b0 0x7f6228103590 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:04.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.673+0000 7f622db4a700 1 -- 172.21.15.69:0/1326962664 >> 172.21.15.69:0/1326962664 conn(0x7f62280fea80 msgr2=0x7f6228100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.673+0000 7f622db4a700 1 -- 172.21.15.69:0/1326962664 shutdown_connections 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.673+0000 7f622db4a700 1 -- 172.21.15.69:0/1326962664 wait complete. 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.674+0000 7f622db4a700 1 Processor -- start 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.674+0000 7f622db4a700 1 -- start start 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.674+0000 7f622db4a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62281031b0 0x7f622819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.674+0000 7f622db4a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f622810d430 con 0x7f62281031b0 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.675+0000 7f62277fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62281031b0 0x7f622819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.675+0000 7f62277fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62281031b0 0x7f622819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52968/0 (socket says 172.21.15.69:52968) 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.675+0000 7f62277fe700 1 -- 172.21.15.69:0/3686040572 learned_addr learned my addr 172.21.15.69:0/3686040572 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.675+0000 7f62277fe700 1 -- 172.21.15.69:0/3686040572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622819eb60 con 0x7f62281031b0 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.675+0000 7f62277fe700 1 --2- 172.21.15.69:0/3686040572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62281031b0 0x7f622819e620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f62180056d0 tx=0x7f62180057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.675+0000 7f6224ff9700 1 -- 172.21.15.69:0/3686040572 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6218026070 con 0x7f62281031b0 2024-05-14T16:40:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.675+0000 7f622db4a700 1 -- 172.21.15.69:0/3686040572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622819ee50 con 0x7f62281031b0 2024-05-14T16:40:04.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.675+0000 7f6224ff9700 1 -- 172.21.15.69:0/3686040572 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f621800bba0 con 0x7f62281031b0 2024-05-14T16:40:04.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.675+0000 7f6224ff9700 1 -- 172.21.15.69:0/3686040572 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6218003920 con 0x7f62281031b0 2024-05-14T16:40:04.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.675+0000 7f622db4a700 1 -- 172.21.15.69:0/3686040572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62281987a0 con 0x7f62281031b0 2024-05-14T16:40:04.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.676+0000 7f6224ff9700 1 -- 172.21.15.69:0/3686040572 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6218029070 con 0x7f62281031b0 2024-05-14T16:40:04.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.676+0000 7f622db4a700 1 -- 172.21.15.69:0/3686040572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6228104690 con 0x7f62281031b0 2024-05-14T16:40:04.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.676+0000 7f6224ff9700 1 --2- 172.21.15.69:0/3686040572 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f621003e510 0x7f62100409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:04.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.676+0000 7f6224ff9700 1 -- 172.21.15.69:0/3686040572 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f621801e070 con 0x7f62281031b0 2024-05-14T16:40:04.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.676+0000 7f6226ffd700 1 --2- 172.21.15.69:0/3686040572 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f621003e510 0x7f62100409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:04.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.677+0000 7f6226ffd700 1 --2- 172.21.15.69:0/3686040572 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f621003e510 0x7f62100409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f621c006fd0 tx=0x7f621c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:04.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.681+0000 7f6224ff9700 1 -- 172.21.15.69:0/3686040572 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6218013070 con 0x7f62281031b0 2024-05-14T16:40:04.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.889+0000 7f622db4a700 1 -- 172.21.15.69:0/3686040572 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6228199cd0 con 0x7f621003e510 2024-05-14T16:40:04.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.890+0000 7f6224ff9700 1 -- 172.21.15.69:0/3686040572 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f6228199cd0 con 0x7f621003e510 2024-05-14T16:40:04.890 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:40:04.890 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.203", "hostname": "smithi203", "labels": [], "status": ""}] 2024-05-14T16:40:04.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.893+0000 7f622db4a700 1 -- 172.21.15.69:0/3686040572 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f621003e510 msgr2=0x7f62100409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:04.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.893+0000 7f622db4a700 1 --2- 172.21.15.69:0/3686040572 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f621003e510 0x7f62100409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f621c006fd0 tx=0x7f621c006e40 comp rx=0 tx=0).stop 2024-05-14T16:40:04.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.893+0000 7f622db4a700 1 -- 172.21.15.69:0/3686040572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62281031b0 msgr2=0x7f622819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:04.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.893+0000 7f622db4a700 1 --2- 172.21.15.69:0/3686040572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62281031b0 0x7f622819e620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f62180056d0 tx=0x7f62180057b0 comp rx=0 tx=0).stop 2024-05-14T16:40:04.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.893+0000 7f622db4a700 1 -- 172.21.15.69:0/3686040572 shutdown_connections 2024-05-14T16:40:04.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.893+0000 7f622db4a700 1 --2- 172.21.15.69:0/3686040572 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f621003e510 0x7f62100409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:04.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.893+0000 7f622db4a700 1 --2- 172.21.15.69:0/3686040572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62281031b0 0x7f622819e620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:04.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.893+0000 7f622db4a700 1 -- 172.21.15.69:0/3686040572 >> 172.21.15.69:0/3686040572 conn(0x7f62280fea80 msgr2=0x7f6228104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:04.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.893+0000 7f622db4a700 1 -- 172.21.15.69:0/3686040572 shutdown_connections 2024-05-14T16:40:04.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:04.893+0000 7f622db4a700 1 -- 172.21.15.69:0/3686040572 wait complete. 2024-05-14T16:40:05.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:05 smithi069 ceph-mon[31507]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:05.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:05.595 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-14T16:40:05.595 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-14T16:40:05.830 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:40:06.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:06 smithi069 ceph-mon[31507]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:40:07.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.201+0000 7f970490e700 1 -- 172.21.15.69:0/3819303000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97001020b0 msgr2=0x7f9700102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:07.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.201+0000 7f970490e700 1 --2- 172.21.15.69:0/3819303000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97001020b0 0x7f9700102490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009910 tx=0x7f96f0009c20 comp rx=0 tx=0).stop 2024-05-14T16:40:07.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.202+0000 7f970490e700 1 -- 172.21.15.69:0/3819303000 shutdown_connections 2024-05-14T16:40:07.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.202+0000 7f970490e700 1 --2- 172.21.15.69:0/3819303000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97001020b0 0x7f9700102490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:07.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.202+0000 7f970490e700 1 -- 172.21.15.69:0/3819303000 >> 172.21.15.69:0/3819303000 conn(0x7f97000fd940 msgr2=0x7f97000ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:07.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.202+0000 7f970490e700 1 -- 172.21.15.69:0/3819303000 shutdown_connections 2024-05-14T16:40:07.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.202+0000 7f970490e700 1 -- 172.21.15.69:0/3819303000 wait complete. 2024-05-14T16:40:07.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.202+0000 7f970490e700 1 Processor -- start 2024-05-14T16:40:07.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f970490e700 1 -- start start 2024-05-14T16:40:07.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f970490e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97001020b0 0x7f9700103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:07.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f970490e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f970010c330 con 0x7f97001020b0 2024-05-14T16:40:07.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f96fe59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97001020b0 0x7f9700103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:07.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f96fe59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97001020b0 0x7f9700103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52984/0 (socket says 172.21.15.69:52984) 2024-05-14T16:40:07.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f96fe59c700 1 -- 172.21.15.69:0/3883065166 learned_addr learned my addr 172.21.15.69:0/3883065166 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:40:07.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f96fe59c700 1 -- 172.21.15.69:0/3883065166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9700105a40 con 0x7f97001020b0 2024-05-14T16:40:07.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f96fe59c700 1 --2- 172.21.15.69:0/3883065166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97001020b0 0x7f9700103b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f96f00056d0 tx=0x7f96f00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:07.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f96ef7fe700 1 -- 172.21.15.69:0/3883065166 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96f0026070 con 0x7f97001020b0 2024-05-14T16:40:07.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f96ef7fe700 1 -- 172.21.15.69:0/3883065166 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f000bc00 con 0x7f97001020b0 2024-05-14T16:40:07.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f96ef7fe700 1 -- 172.21.15.69:0/3883065166 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96f0003920 con 0x7f97001020b0 2024-05-14T16:40:07.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f970490e700 1 -- 172.21.15.69:0/3883065166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97001040f0 con 0x7f97001020b0 2024-05-14T16:40:07.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.203+0000 7f970490e700 1 -- 172.21.15.69:0/3883065166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97001045b0 con 0x7f97001020b0 2024-05-14T16:40:07.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.204+0000 7f96ef7fe700 1 -- 172.21.15.69:0/3883065166 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f96f0029070 con 0x7f97001020b0 2024-05-14T16:40:07.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.204+0000 7f970490e700 1 -- 172.21.15.69:0/3883065166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9700103590 con 0x7f97001020b0 2024-05-14T16:40:07.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.205+0000 7f96ef7fe700 1 --2- 172.21.15.69:0/3883065166 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f96e803e510 0x7f96e80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:07.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.205+0000 7f96ef7fe700 1 -- 172.21.15.69:0/3883065166 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f96f001e070 con 0x7f97001020b0 2024-05-14T16:40:07.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.205+0000 7f96fdd9b700 1 --2- 172.21.15.69:0/3883065166 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f96e803e510 0x7f96e80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:07.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.205+0000 7f96fdd9b700 1 --2- 172.21.15.69:0/3883065166 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f96e803e510 0x7f96e80409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96f4006fd0 tx=0x7f96f4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:07.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.210+0000 7f96ef7fe700 1 -- 172.21.15.69:0/3883065166 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96f0013070 con 0x7f97001020b0 2024-05-14T16:40:07.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.416+0000 7f970490e700 1 -- 172.21.15.69:0/3883065166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9700105080 con 0x7f97001020b0 2024-05-14T16:40:07.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.426+0000 7f96ef7fe700 1 -- 172.21.15.69:0/3883065166 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f96f0030090 con 0x7f97001020b0 2024-05-14T16:40:07.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.428+0000 7f970490e700 1 -- 172.21.15.69:0/3883065166 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f96e803e510 msgr2=0x7f96e80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:07.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.428+0000 7f970490e700 1 --2- 172.21.15.69:0/3883065166 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f96e803e510 0x7f96e80409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96f4006fd0 tx=0x7f96f4006e40 comp rx=0 tx=0).stop 2024-05-14T16:40:07.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.428+0000 7f970490e700 1 -- 172.21.15.69:0/3883065166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97001020b0 msgr2=0x7f9700103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:07.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.428+0000 7f970490e700 1 --2- 172.21.15.69:0/3883065166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97001020b0 0x7f9700103b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f96f00056d0 tx=0x7f96f00057b0 comp rx=0 tx=0).stop 2024-05-14T16:40:07.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.428+0000 7f970490e700 1 -- 172.21.15.69:0/3883065166 shutdown_connections 2024-05-14T16:40:07.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.428+0000 7f970490e700 1 --2- 172.21.15.69:0/3883065166 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f96e803e510 0x7f96e80409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:07.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.428+0000 7f970490e700 1 --2- 172.21.15.69:0/3883065166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97001020b0 0x7f9700103b50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:07.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.428+0000 7f970490e700 1 -- 172.21.15.69:0/3883065166 >> 172.21.15.69:0/3883065166 conn(0x7f97000fd940 msgr2=0x7f970006b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:07.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.429+0000 7f970490e700 1 -- 172.21.15.69:0/3883065166 shutdown_connections 2024-05-14T16:40:07.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:07.429+0000 7f970490e700 1 -- 172.21.15.69:0/3883065166 wait complete. 2024-05-14T16:40:07.430 INFO:teuthology.orchestra.run.smithi069.stderr:adjusted tunables profile to default 2024-05-14T16:40:07.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:07 smithi069 ceph-mon[31507]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:07.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:07 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3883065166' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T16:40:07.956 INFO:tasks.cephadm:Adding mon.a on smithi069 2024-05-14T16:40:07.956 INFO:tasks.cephadm:Adding mon.b on smithi092 2024-05-14T16:40:07.956 INFO:tasks.cephadm:Adding mon.c on smithi203 2024-05-14T16:40:07.956 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi069:172.21.15.69=a;smithi092:172.21.15.92=b;smithi203:172.21.15.203=c' 2024-05-14T16:40:08.189 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-14T16:40:08.235 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-14T16:40:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:08 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3883065166' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T16:40:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:08 smithi069 ceph-mon[31507]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:40:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:40:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:08.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:09.698 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.696+0000 7f5973ae5700 1 -- 172.21.15.203:0/464137218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f596c1020f0 msgr2=0x7f596c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:09.698 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.696+0000 7f5973ae5700 1 --2- 172.21.15.203:0/464137218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f596c1020f0 0x7f596c1024d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5968009910 tx=0x7f5968009c20 comp rx=0 tx=0).stop 2024-05-14T16:40:09.698 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.696+0000 7f5973ae5700 1 -- 172.21.15.203:0/464137218 shutdown_connections 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.696+0000 7f5973ae5700 1 --2- 172.21.15.203:0/464137218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f596c1020f0 0x7f596c1024d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.696+0000 7f5973ae5700 1 -- 172.21.15.203:0/464137218 >> 172.21.15.203:0/464137218 conn(0x7f596c0fd9a0 msgr2=0x7f596c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.696+0000 7f5973ae5700 1 -- 172.21.15.203:0/464137218 shutdown_connections 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.696+0000 7f5973ae5700 1 -- 172.21.15.203:0/464137218 wait complete. 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.697+0000 7f5973ae5700 1 Processor -- start 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.697+0000 7f5973ae5700 1 -- start start 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.697+0000 7f5973ae5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f596c1020f0 0x7f596c103b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.697+0000 7f5973ae5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f596c10c370 con 0x7f596c1020f0 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.697+0000 7f5971881700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f596c1020f0 0x7f596c103b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.697+0000 7f5971881700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f596c1020f0 0x7f596c103b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.203:48246/0 (socket says 172.21.15.203:48246) 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.697+0000 7f5971881700 1 -- 172.21.15.203:0/3518406579 learned_addr learned my addr 172.21.15.203:0/3518406579 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.698+0000 7f5971881700 1 -- 172.21.15.203:0/3518406579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c105a80 con 0x7f596c1020f0 2024-05-14T16:40:09.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.698+0000 7f5971881700 1 --2- 172.21.15.203:0/3518406579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f596c1020f0 0x7f596c103b90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f59680056d0 tx=0x7f59680057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:09.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.698+0000 7f5962ffd700 1 -- 172.21.15.203:0/3518406579 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5968026070 con 0x7f596c1020f0 2024-05-14T16:40:09.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.698+0000 7f5962ffd700 1 -- 172.21.15.203:0/3518406579 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596800bc00 con 0x7f596c1020f0 2024-05-14T16:40:09.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.698+0000 7f5973ae5700 1 -- 172.21.15.203:0/3518406579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c104130 con 0x7f596c1020f0 2024-05-14T16:40:09.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.698+0000 7f5962ffd700 1 -- 172.21.15.203:0/3518406579 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5968003920 con 0x7f596c1020f0 2024-05-14T16:40:09.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.698+0000 7f5973ae5700 1 -- 172.21.15.203:0/3518406579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1045f0 con 0x7f596c1020f0 2024-05-14T16:40:09.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.699+0000 7f5973ae5700 1 -- 172.21.15.203:0/3518406579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c1035d0 con 0x7f596c1020f0 2024-05-14T16:40:09.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.699+0000 7f5962ffd700 1 -- 172.21.15.203:0/3518406579 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f5968029070 con 0x7f596c1020f0 2024-05-14T16:40:09.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.699+0000 7f5962ffd700 1 --2- 172.21.15.203:0/3518406579 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f595803e510 0x7f59580409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:09.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.699+0000 7f5962ffd700 1 -- 172.21.15.203:0/3518406579 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f596801e070 con 0x7f596c1020f0 2024-05-14T16:40:09.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.699+0000 7f5971080700 1 --2- 172.21.15.203:0/3518406579 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f595803e510 0x7f59580409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:09.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.700+0000 7f5971080700 1 --2- 172.21.15.203:0/3518406579 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f595803e510 0x7f59580409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f595c006fd0 tx=0x7f595c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:09.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:09 smithi069 ceph-mon[31507]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:09.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:09 smithi069 ceph-mon[31507]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-14T16:40:09.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:09 smithi069 ceph-mon[31507]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:09.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:09 smithi069 ceph-mon[31507]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:40:09.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:09 smithi069 ceph-mon[31507]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:40:09.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:09.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:09.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:09.704 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.704+0000 7f5962ffd700 1 -- 172.21.15.203:0/3518406579 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5968013070 con 0x7f596c1020f0 2024-05-14T16:40:09.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.917+0000 7f5973ae5700 1 -- 172.21.15.203:0/3518406579 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi092:172.21.15.92=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}) -- 0x7f596c1050c0 con 0x7f595803e510 2024-05-14T16:40:09.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.923+0000 7f5962ffd700 1 -- 172.21.15.203:0/3518406579 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f596c1050c0 con 0x7f595803e510 2024-05-14T16:40:09.924 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mon update... 2024-05-14T16:40:09.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.926+0000 7f5973ae5700 1 -- 172.21.15.203:0/3518406579 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f595803e510 msgr2=0x7f59580409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:09.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.926+0000 7f5973ae5700 1 --2- 172.21.15.203:0/3518406579 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f595803e510 0x7f59580409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f595c006fd0 tx=0x7f595c006e40 comp rx=0 tx=0).stop 2024-05-14T16:40:09.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.926+0000 7f5973ae5700 1 -- 172.21.15.203:0/3518406579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f596c1020f0 msgr2=0x7f596c103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:09.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.926+0000 7f5973ae5700 1 --2- 172.21.15.203:0/3518406579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f596c1020f0 0x7f596c103b90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f59680056d0 tx=0x7f59680057b0 comp rx=0 tx=0).stop 2024-05-14T16:40:09.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.926+0000 7f5973ae5700 1 -- 172.21.15.203:0/3518406579 shutdown_connections 2024-05-14T16:40:09.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.926+0000 7f5973ae5700 1 --2- 172.21.15.203:0/3518406579 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f595803e510 0x7f59580409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:09.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.926+0000 7f5973ae5700 1 --2- 172.21.15.203:0/3518406579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f596c1020f0 0x7f596c103b90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:09.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.926+0000 7f5973ae5700 1 -- 172.21.15.203:0/3518406579 >> 172.21.15.203:0/3518406579 conn(0x7f596c0fd9a0 msgr2=0x7f596c06b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:09.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.926+0000 7f5973ae5700 1 -- 172.21.15.203:0/3518406579 shutdown_connections 2024-05-14T16:40:09.927 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:09.926+0000 7f5973ae5700 1 -- 172.21.15.203:0/3518406579 wait complete. 2024-05-14T16:40:10.529 DEBUG:teuthology.orchestra.run.smithi092:mon.b> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.b.service 2024-05-14T16:40:10.532 DEBUG:teuthology.orchestra.run.smithi203:mon.c> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.c.service 2024-05-14T16:40:10.534 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T16:40:10.534 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T16:40:11.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:11.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi092:172.21.15.92=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:40:11.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi092:172.21.15.92=b;smithi203:172.21.15.203=c;count:3 2024-05-14T16:40:11.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:11.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:11.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:11.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:11.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:11.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:11.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:11.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:10 smithi069 ceph-mon[31507]: Deploying daemon mon.c on smithi203 2024-05-14T16:40:12.613 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:40:13.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:12 smithi069 ceph-mon[31507]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:14.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 podman[35740]: 2024-05-14 16:40:14.287972853 +0000 UTC m=+0.126876337 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.633+0000 7fc644b4a700 1 -- 172.21.15.203:0/2054702947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6401021b0 msgr2=0x7fc640102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.633+0000 7fc644b4a700 1 --2- 172.21.15.203:0/2054702947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6401021b0 0x7fc640102590 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc634009910 tx=0x7fc634009c20 comp rx=0 tx=0).stop 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.633+0000 7fc644b4a700 1 -- 172.21.15.203:0/2054702947 shutdown_connections 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.633+0000 7fc644b4a700 1 --2- 172.21.15.203:0/2054702947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6401021b0 0x7fc640102590 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.633+0000 7fc644b4a700 1 -- 172.21.15.203:0/2054702947 >> 172.21.15.203:0/2054702947 conn(0x7fc6400fda80 msgr2=0x7fc6400ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.633+0000 7fc644b4a700 1 -- 172.21.15.203:0/2054702947 shutdown_connections 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.633+0000 7fc644b4a700 1 -- 172.21.15.203:0/2054702947 wait complete. 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.633+0000 7fc644b4a700 1 Processor -- start 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.634+0000 7fc644b4a700 1 -- start start 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.634+0000 7fc644b4a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6401021b0 0x7fc640103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:14.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.634+0000 7fc644b4a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc64010c430 con 0x7fc6401021b0 2024-05-14T16:40:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.634+0000 7fc63e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6401021b0 0x7fc640103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.634+0000 7fc63e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6401021b0 0x7fc640103c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.203:53388/0 (socket says 172.21.15.203:53388) 2024-05-14T16:40:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.634+0000 7fc63e59c700 1 -- 172.21.15.203:0/34796354 learned_addr learned my addr 172.21.15.203:0/34796354 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-14T16:40:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.634+0000 7fc63e59c700 1 -- 172.21.15.203:0/34796354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc640105b40 con 0x7fc6401021b0 2024-05-14T16:40:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.634+0000 7fc63e59c700 1 --2- 172.21.15.203:0/34796354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6401021b0 0x7fc640103c50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc634006010 tx=0x7fc634005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.635+0000 7fc62f7fe700 1 -- 172.21.15.203:0/34796354 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc634026070 con 0x7fc6401021b0 2024-05-14T16:40:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.635+0000 7fc62f7fe700 1 -- 172.21.15.203:0/34796354 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc63400bc00 con 0x7fc6401021b0 2024-05-14T16:40:14.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.635+0000 7fc644b4a700 1 -- 172.21.15.203:0/34796354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6401041f0 con 0x7fc6401021b0 2024-05-14T16:40:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.635+0000 7fc62f7fe700 1 -- 172.21.15.203:0/34796354 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc634003910 con 0x7fc6401021b0 2024-05-14T16:40:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.635+0000 7fc644b4a700 1 -- 172.21.15.203:0/34796354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6401046b0 con 0x7fc6401021b0 2024-05-14T16:40:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.636+0000 7fc644b4a700 1 -- 172.21.15.203:0/34796354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc640103690 con 0x7fc6401021b0 2024-05-14T16:40:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.636+0000 7fc62f7fe700 1 -- 172.21.15.203:0/34796354 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc634029070 con 0x7fc6401021b0 2024-05-14T16:40:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.636+0000 7fc62f7fe700 1 --2- 172.21.15.203:0/34796354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc62803e510 0x7fc6280409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.636+0000 7fc62f7fe700 1 -- 172.21.15.203:0/34796354 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc63401e070 con 0x7fc6401021b0 2024-05-14T16:40:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.636+0000 7fc63dd9b700 1 --2- 172.21.15.203:0/34796354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc62803e510 0x7fc6280409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:14.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.637+0000 7fc63dd9b700 1 --2- 172.21.15.203:0/34796354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc62803e510 0x7fc6280409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc630006fd0 tx=0x7fc630006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:14.641 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.641+0000 7fc62f7fe700 1 -- 172.21.15.203:0/34796354 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc634013070 con 0x7fc6401021b0 2024-05-14T16:40:14.784 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.784+0000 7fc62f7fe700 1 -- 172.21.15.203:0/34796354 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc6340142b0 con 0x7fc6401021b0 2024-05-14T16:40:14.941 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:14.941+0000 7fc644b4a700 1 -- 172.21.15.203:0/34796354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc640105220 con 0x7fc6401021b0 2024-05-14T16:40:15.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 podman[35740]: 2024-05-14 16:40:14.723578085 +0000 UTC m=+0.562481548 container init 92ff3665e8b07f36fa7f0773e265175a56c978e384c0b418373314b926cb55cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-14T16:40:15.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 podman[35740]: 2024-05-14 16:40:14.727286286 +0000 UTC m=+0.566189758 container start 92ff3665e8b07f36fa7f0773e265175a56c978e384c0b418373314b926cb55cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-14T16:40:15.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: set uid:gid to 167:167 (ceph:ceph) 2024-05-14T16:40:15.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev), process ceph-mon, pid 2 2024-05-14T16:40:15.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: pidfile_write: ignore empty --pid-file 2024-05-14T16:40:15.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: load: jerasure load: lrc 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: RocksDB version: 7.9.2 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Git sha 0 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Compile date 2024-05-14 14:11:45 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: DB SUMMARY 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: DB Session ID: Q2524YZTBQ6TNXTVE9N3 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: CURRENT file: CURRENT 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: IDENTITY file: IDENTITY 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.error_if_exists: 0 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.create_if_missing: 0 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.paranoid_checks: 1 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-14T16:40:15.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-14T16:40:15.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-14T16:40:15.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.env: 0x55d0e0373cc0 2024-05-14T16:40:15.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.fs: PosixFileSystem 2024-05-14T16:40:15.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.info_log: 0x55d0e291fda0 2024-05-14T16:40:15.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_file_opening_threads: 16 2024-05-14T16:40:15.213 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.statistics: (nil) 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.use_fsync: 0 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_log_file_size: 0 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.keep_log_file_num: 1000 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.recycle_log_file_num: 0 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.allow_fallocate: 1 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.allow_mmap_reads: 0 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.allow_mmap_writes: 0 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.use_direct_reads: 0 2024-05-14T16:40:15.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.create_missing_column_families: 0 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.db_log_dir: 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.wal_dir: 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.advise_random_on_open: 1 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.db_write_buffer_size: 0 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.write_buffer_manager: 0x55d0e28f10e0 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-14T16:40:15.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.rate_limiter: (nil) 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.wal_recovery_mode: 2 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.enable_thread_tracking: 0 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.enable_pipelined_write: 0 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.unordered_write: 0 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.row_cache: None 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.wal_filter: None 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-14T16:40:15.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.allow_ingest_behind: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.two_write_queues: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.manual_wal_flush: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.wal_compression: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.atomic_flush: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.log_readahead_size: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.best_efforts_recovery: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.allow_data_in_errors: 0 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.db_host_id: __hostname__ 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-14T16:40:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_background_jobs: 2 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_background_compactions: -1 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_subcompactions: 1 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_total_wal_size: 0 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_open_files: -1 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bytes_per_sync: 0 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_readahead_size: 0 2024-05-14T16:40:15.218 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_background_flushes: -1 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Compression algorithms supported: 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: kZSTD supported: 0 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: kXpressCompression supported: 0 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: kLZ4HCCompression supported: 1 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: kZlibCompression supported: 1 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: kSnappyCompression supported: 1 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: kLZ4Compression supported: 1 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: kBZip2Compression supported: 0 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.merge_operator: 2024-05-14T16:40:15.219 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_filter: None 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_filter_factory: None 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.sst_partitioner_factory: None 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d0e291ff60) 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout: cache_index_and_filter_blocks: 1 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout: pin_top_level_index_and_filter: 1 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_type: 0 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout: data_block_index_type: 0 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_shortening: 1 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout: checksum: 4 2024-05-14T16:40:15.220 INFO:journalctl@ceph.mon.c.smithi203.stdout: no_block_cache: 0 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache: 0x55d0e29231f0 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_name: BinnedLRUCache 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_options: 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: capacity : 536870912 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: num_shard_bits : 4 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: strict_capacity_limit : 0 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: high_pri_pool_ratio: 0.000 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_compressed: (nil) 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: persistent_cache: (nil) 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_size: 4096 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_size_deviation: 10 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_restart_interval: 16 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_block_restart_interval: 1 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: metadata_block_size: 4096 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: partition_filters: 0 2024-05-14T16:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout: use_delta_encoding: 1 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: filter_policy: bloomfilter 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: whole_key_filtering: 1 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: verify_compression: 0 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: read_amp_bytes_per_bit: 0 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: format_version: 5 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_index_compression: 1 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_align: 0 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: max_auto_readahead_size: 262144 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: prepopulate_block_cache: 0 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: initial_auto_readahead_size: 8192 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout: num_file_reads_for_auto_readahead: 2 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.write_buffer_size: 33554432 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_write_buffer_number: 2 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compression: NoCompression 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bottommost_compression: Disabled 2024-05-14T16:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.prefix_extractor: nullptr 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.num_levels: 7 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compression_opts.level: 32767 2024-05-14T16:40:15.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compression_opts.strategy: 0 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compression_opts.enabled: false 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.target_file_size_base: 67108864 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-14T16:40:15.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.arena_block_size: 1048576 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.disable_auto_compactions: 0 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-14T16:40:15.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.inplace_update_support: 0 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.bloom_locality: 0 2024-05-14T16:40:15.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.max_successive_merges: 0 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.paranoid_file_checks: 0 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.force_consistency_checks: 1 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.report_bg_io_stats: 0 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.ttl: 2592000 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.enable_blob_files: false 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.min_blob_size: 0 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.blob_file_size: 268435456 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-14T16:40:15.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.blob_file_starting_level: 0 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5452658c-b76e-47f2-b142-9ee923a611a9 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715704814757510, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715704814758090, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715704814, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5452658c-b76e-47f2-b142-9ee923a611a9", "db_session_id": "Q2524YZTBQ6TNXTVE9N3", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715704814758196, "job": 1, "event": "recovery_finished"} 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d0e2a40000 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: DB pointer 0x55d0e2a2c000 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-14T16:40:15.228 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** DB Stats ** 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:40:15.229 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-14T16:40:15.230 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache BinnedLRUCache@0x55d0e29231f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: using public_addr v2:172.21.15.203:0/0 -> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: starting mon.c rank -1 at public addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] at bind addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(???) e0 preinit fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).mds e1 new map 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout: e1 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T16:40:15.231 INFO:journalctl@ceph.mon.c.smithi203.stdout: legacy client fscid: -1 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: No filesystems configured 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:40:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Deploying cephadm binary to smithi203 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Added host smithi203 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3883065166' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3883065166' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:15.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi092:172.21.15.92=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi092:172.21.15.92=b;smithi203:172.21.15.203=c;count:3 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: Deploying daemon mon.c on smithi203 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 ceph-mon[35787]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 bash[35740]: 92ff3665e8b07f36fa7f0773e265175a56c978e384c0b418373314b926cb55cf 2024-05-14T16:40:15.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:14 smithi203 systemd[1]: Started Ceph mon.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:40:19.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.795+0000 7fc62f7fe700 1 -- 172.21.15.203:0/34796354 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fc634030090 con 0x7fc6401021b0 2024-05-14T16:40:19.796 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:40:19.796 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","modified":"2024-05-14T16:40:14.784227Z","created":"2024-05-14T16:38:30.757480Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-14T16:40:19.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.798+0000 7fc644b4a700 1 -- 172.21.15.203:0/34796354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc62803e510 msgr2=0x7fc6280409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:19.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.798+0000 7fc644b4a700 1 --2- 172.21.15.203:0/34796354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc62803e510 0x7fc6280409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc630006fd0 tx=0x7fc630006e40 comp rx=0 tx=0).stop 2024-05-14T16:40:19.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.798+0000 7fc644b4a700 1 -- 172.21.15.203:0/34796354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6401021b0 msgr2=0x7fc640103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:19.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.798+0000 7fc644b4a700 1 --2- 172.21.15.203:0/34796354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6401021b0 0x7fc640103c50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc634006010 tx=0x7fc634005780 comp rx=0 tx=0).stop 2024-05-14T16:40:19.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.798+0000 7fc644b4a700 1 -- 172.21.15.203:0/34796354 shutdown_connections 2024-05-14T16:40:19.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.798+0000 7fc644b4a700 1 --2- 172.21.15.203:0/34796354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc62803e510 0x7fc6280409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:19.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.798+0000 7fc644b4a700 1 --2- 172.21.15.203:0/34796354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6401021b0 0x7fc640103c50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:19.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.798+0000 7fc644b4a700 1 -- 172.21.15.203:0/34796354 >> 172.21.15.203:0/34796354 conn(0x7fc6400fda80 msgr2=0x7fc64006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:19.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.799+0000 7fc644b4a700 1 -- 172.21.15.203:0/34796354 shutdown_connections 2024-05-14T16:40:19.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:19.799+0000 7fc644b4a700 1 -- 172.21.15.203:0/34796354 wait complete. 2024-05-14T16:40:19.800 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-05-14T16:40:20.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:40:20.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: mon.a calling monitor election 2024-05-14T16:40:20.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='client.? 172.21.15.203:0/34796354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:40:20.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:20.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: mon.c calling monitor election 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: monmap epoch 2 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: last_changed 2024-05-14T16:40:14.784227+0000 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: created 2024-05-14T16:38:30.757480+0000 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: min_mon_release 19 (squid) 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: election_strategy: 1 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: fsmap 2024-05-14T16:40:20.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:40:20.120 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: mgrmap e13: x(active, since 48s) 2024-05-14T16:40:20.120 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: overall HEALTH_OK 2024-05-14T16:40:20.120 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:20.120 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:20.120 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:20.120 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:20.120 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:20.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: mon.a calling monitor election 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='client.? 172.21.15.203:0/34796354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: mon.c calling monitor election 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T16:40:20.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: monmap epoch 2 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: last_changed 2024-05-14T16:40:14.784227+0000 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: created 2024-05-14T16:38:30.757480+0000 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: min_mon_release 19 (squid) 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: election_strategy: 1 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: fsmap 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: mgrmap e13: x(active, since 48s) 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: overall HEALTH_OK 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:20.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:20.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:20.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:20.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:21.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:20 smithi069 ceph-mon[31507]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:21.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:20 smithi069 ceph-mon[31507]: Deploying daemon mon.b on smithi092 2024-05-14T16:40:21.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:20 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:21.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:20 smithi203 ceph-mon[35787]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:21.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:20 smithi203 ceph-mon[35787]: Deploying daemon mon.b on smithi092 2024-05-14T16:40:21.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:20 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:21.426 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T16:40:21.429 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T16:40:21.672 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:40:22.938 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:22 smithi203 ceph-mon[35787]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.136+0000 7f8d515f2700 1 -- 172.21.15.203:0/3034747480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c1021d0 msgr2=0x7f8d4c1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.136+0000 7f8d515f2700 1 --2- 172.21.15.203:0/3034747480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c1021d0 0x7f8d4c1025b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009910 tx=0x7f8d3c009c20 comp rx=0 tx=0).stop 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.136+0000 7f8d515f2700 1 -- 172.21.15.203:0/3034747480 shutdown_connections 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.136+0000 7f8d515f2700 1 --2- 172.21.15.203:0/3034747480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c1021d0 0x7f8d4c1025b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.136+0000 7f8d515f2700 1 -- 172.21.15.203:0/3034747480 >> 172.21.15.203:0/3034747480 conn(0x7f8d4c0fda80 msgr2=0x7f8d4c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.136+0000 7f8d515f2700 1 -- 172.21.15.203:0/3034747480 shutdown_connections 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.136+0000 7f8d515f2700 1 -- 172.21.15.203:0/3034747480 wait complete. 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d515f2700 1 Processor -- start 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d515f2700 1 -- start start 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d515f2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d4c1021d0 0x7f8d4c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:23.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d515f2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c19ee20 0x7f8d4c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d515f2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c10c450 con 0x7f8d4c19ee20 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d515f2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c10c2d0 con 0x7f8d4c1021d0 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d4affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d4c1021d0 0x7f8d4c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d4affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d4c1021d0 0x7f8d4c19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:54204/0 (socket says 172.21.15.203:54204) 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d4affd700 1 -- 172.21.15.203:0/3972655636 learned_addr learned my addr 172.21.15.203:0/3972655636 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d4affd700 1 -- 172.21.15.203:0/3972655636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d4c1021d0 msgr2=0x7f8d4c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d4affd700 1 -- 172.21.15.203:0/3972655636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d4c1021d0 msgr2=0x7f8d4c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d4affd700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d4c1021d0 0x7f8d4c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d4a7fc700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c19ee20 0x7f8d4c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.137+0000 7f8d4affd700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d4c1021d0 0x7f8d4c19e8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.138+0000 7f8d4a7fc700 1 -- 172.21.15.203:0/3972655636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d4c1021d0 msgr2=0x7f8d4c19e8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.138+0000 7f8d4a7fc700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d4c1021d0 0x7f8d4c19e8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:23.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.138+0000 7f8d4a7fc700 1 -- 172.21.15.203:0/3972655636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c19f460 con 0x7f8d4c19ee20 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.138+0000 7f8d4a7fc700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c19ee20 0x7f8d4c198aa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000b4d0 tx=0x7f8d4000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.138+0000 7f8d33fff700 1 -- 172.21.15.203:0/3972655636 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8d40011630 con 0x7f8d4c19ee20 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.138+0000 7f8d33fff700 1 -- 172.21.15.203:0/3972655636 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d40011c80 con 0x7f8d4c19ee20 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.138+0000 7f8d33fff700 1 -- 172.21.15.203:0/3972655636 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8d40010da0 con 0x7f8d4c19ee20 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.138+0000 7f8d515f2700 1 -- 172.21.15.203:0/3972655636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4c198fe0 con 0x7f8d4c19ee20 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.138+0000 7f8d515f2700 1 -- 172.21.15.203:0/3972655636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4c199570 con 0x7f8d4c19ee20 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.139+0000 7f8d515f2700 1 -- 172.21.15.203:0/3972655636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4c1036b0 con 0x7f8d4c19ee20 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.139+0000 7f8d33fff700 1 -- 172.21.15.203:0/3972655636 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8d40018020 con 0x7f8d4c19ee20 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.139+0000 7f8d33fff700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8d3403e2b0 0x7f8d34040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.139+0000 7f8d33fff700 1 -- 172.21.15.203:0/3972655636 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8d40052180 con 0x7f8d4c19ee20 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.140+0000 7f8d4affd700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8d3403e2b0 0x7f8d34040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:23.141 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.140+0000 7f8d4affd700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8d3403e2b0 0x7f8d34040770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009750 tx=0x7f8d3c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:23.145 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.145+0000 7f8d33fff700 1 -- 172.21.15.203:0/3972655636 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8d4001f020 con 0x7f8d4c19ee20 2024-05-14T16:40:23.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:22 smithi069 ceph-mon[31507]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:23.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.433+0000 7f8d515f2700 1 -- 172.21.15.203:0/3972655636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8d4c199fd0 con 0x7f8d4c19ee20 2024-05-14T16:40:23.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.433+0000 7f8d33fff700 1 -- 172.21.15.203:0/3972655636 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f8d400151c0 con 0x7f8d4c19ee20 2024-05-14T16:40:23.434 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:40:23.434 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","modified":"2024-05-14T16:40:14.784227Z","created":"2024-05-14T16:38:30.757480Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-14T16:40:23.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 -- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8d3403e2b0 msgr2=0x7f8d34040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:23.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8d3403e2b0 0x7f8d34040770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009750 tx=0x7f8d3c01a040 comp rx=0 tx=0).stop 2024-05-14T16:40:23.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 -- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c19ee20 msgr2=0x7f8d4c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:23.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c19ee20 0x7f8d4c198aa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000b4d0 tx=0x7f8d4000b890 comp rx=0 tx=0).stop 2024-05-14T16:40:23.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 -- 172.21.15.203:0/3972655636 shutdown_connections 2024-05-14T16:40:23.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d4c1021d0 0x7f8d4c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:23.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f8d3403e2b0 0x7f8d34040770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:23.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 --2- 172.21.15.203:0/3972655636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c19ee20 0x7f8d4c198aa0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:23.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 -- 172.21.15.203:0/3972655636 >> 172.21.15.203:0/3972655636 conn(0x7f8d4c0fda80 msgr2=0x7f8d4c109fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:23.437 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 -- 172.21.15.203:0/3972655636 shutdown_connections 2024-05-14T16:40:23.437 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:23.436+0000 7f8d515f2700 1 -- 172.21.15.203:0/3972655636 wait complete. 2024-05-14T16:40:23.437 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-05-14T16:40:24.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:23 smithi092 systemd[1]: Started Ceph mon.b for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:40:24.998 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T16:40:24.999 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T16:40:25.235 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:40:29.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:29.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:40:29.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: mon.a calling monitor election 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: mon.c calling monitor election 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: mon.b calling monitor election 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: monmap epoch 3 2024-05-14T16:40:29.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: last_changed 2024-05-14T16:40:23.791494+0000 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: created 2024-05-14T16:38:30.757480+0000 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: min_mon_release 19 (squid) 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: election_strategy: 1 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: fsmap 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: mgrmap e13: x(active, since 57s) 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: overall HEALTH_OK 2024-05-14T16:40:29.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:29.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: mon.a calling monitor election 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: mon.c calling monitor election 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: mon.b calling monitor election 2024-05-14T16:40:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: monmap epoch 3 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: last_changed 2024-05-14T16:40:23.791494+0000 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: created 2024-05-14T16:38:30.757480+0000 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: min_mon_release 19 (squid) 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: election_strategy: 1 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: fsmap 2024-05-14T16:40:29.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:40:29.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: mgrmap e13: x(active, since 57s) 2024-05-14T16:40:29.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: overall HEALTH_OK 2024-05-14T16:40:29.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:29.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:29.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: mon.a calling monitor election 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: mon.c calling monitor election 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: mon.b calling monitor election 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: monmap epoch 3 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: last_changed 2024-05-14T16:40:23.791494+0000 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: created 2024-05-14T16:38:30.757480+0000 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: min_mon_release 19 (squid) 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: election_strategy: 1 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: fsmap 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: mgrmap e13: x(active, since 57s) 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: overall HEALTH_OK 2024-05-14T16:40:29.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:29.253 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:29.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.614+0000 7fe15e304700 1 -- 172.21.15.203:0/2109920937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1400035c0 msgr2=0x7fe140005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:29.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.614+0000 7fe15e304700 1 --2- 172.21.15.203:0/2109920937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1400035c0 0x7fe140005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe15807a780 tx=0x7fe14800feb0 comp rx=0 tx=0).stop 2024-05-14T16:40:29.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.614+0000 7fe15e304700 1 -- 172.21.15.203:0/2109920937 shutdown_connections 2024-05-14T16:40:29.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.614+0000 7fe15e304700 1 --2- 172.21.15.203:0/2109920937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1400035c0 0x7fe140005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:29.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.614+0000 7fe15e304700 1 -- 172.21.15.203:0/2109920937 >> 172.21.15.203:0/2109920937 conn(0x7fe1580785f0 msgr2=0x7fe158078a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:29.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.614+0000 7fe15e304700 1 -- 172.21.15.203:0/2109920937 shutdown_connections 2024-05-14T16:40:29.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.614+0000 7fe15e304700 1 -- 172.21.15.203:0/2109920937 wait complete. 2024-05-14T16:40:29.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.615+0000 7fe15e304700 1 Processor -- start 2024-05-14T16:40:29.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.615+0000 7fe15e304700 1 -- start start 2024-05-14T16:40:29.618 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.615+0000 7fe15e304700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1400035c0 0x7fe15818f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:29.618 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.615+0000 7fe15e304700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15818fb90 0x7fe158190010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.615+0000 7fe15e304700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1581a4dc0 0x7fe1581a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe15e304700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe158109ee0 con 0x7fe15818fb90 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe15e304700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe158109d60 con 0x7fe1400035c0 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe15e304700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe15810a060 con 0x7fe1581a4dc0 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe157fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1400035c0 0x7fe15818f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe157fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1400035c0 0x7fe15818f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.203:57810/0 (socket says 172.21.15.203:57810) 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe157fff700 1 -- 172.21.15.203:0/1829394475 learned_addr learned my addr 172.21.15.203:0/1829394475 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe15c8a1700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1581a4dc0 0x7fe1581a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe1577fe700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15818fb90 0x7fe158190010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe157fff700 1 -- 172.21.15.203:0/1829394475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1400035c0 msgr2=0x7fe15818f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe157fff700 1 -- 172.21.15.203:0/1829394475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1400035c0 msgr2=0x7fe15818f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe157fff700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1400035c0 0x7fe15818f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe157fff700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1400035c0 0x7fe15818f650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:40:29.619 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe15c8a1700 1 -- 172.21.15.203:0/1829394475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1581a4dc0 msgr2=0x7fe1581a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe15c8a1700 1 -- 172.21.15.203:0/1829394475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1581a4dc0 msgr2=0x7fe1581a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe15c8a1700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1581a4dc0 0x7fe1581a71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe15c8a1700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1581a4dc0 0x7fe1581a71b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe1577fe700 1 -- 172.21.15.203:0/1829394475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1581a4dc0 msgr2=0x7fe1581a71b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe1577fe700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1581a4dc0 0x7fe1581a71b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe1577fe700 1 -- 172.21.15.203:0/1829394475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1400035c0 msgr2=0x7fe15818f650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe1577fe700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1400035c0 0x7fe15818f650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe1577fe700 1 -- 172.21.15.203:0/1829394475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1581a78e0 con 0x7fe15818fb90 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.616+0000 7fe1577fe700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15818fb90 0x7fe158190010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00b550 tx=0x7fe14c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.617+0000 7fe1557fa700 1 -- 172.21.15.203:0/1829394475 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c004230 con 0x7fe15818fb90 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.617+0000 7fe1557fa700 1 -- 172.21.15.203:0/1829394475 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe14c0043d0 con 0x7fe15818fb90 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.617+0000 7fe15e304700 1 -- 172.21.15.203:0/1829394475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1581a7bd0 con 0x7fe15818fb90 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.617+0000 7fe1557fa700 1 -- 172.21.15.203:0/1829394475 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c011b90 con 0x7fe15818fb90 2024-05-14T16:40:29.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.617+0000 7fe15e304700 1 -- 172.21.15.203:0/1829394475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1581a8160 con 0x7fe15818fb90 2024-05-14T16:40:29.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.618+0000 7fe1557fa700 1 -- 172.21.15.203:0/1829394475 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe14c01a400 con 0x7fe15818fb90 2024-05-14T16:40:29.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.618+0000 7fe1557fa700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe14403e9a0 0x7fe144040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:29.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.618+0000 7fe15e304700 1 -- 172.21.15.203:0/1829394475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe158064ba0 con 0x7fe15818fb90 2024-05-14T16:40:29.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.618+0000 7fe1557fa700 1 -- 172.21.15.203:0/1829394475 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe14c0522e0 con 0x7fe15818fb90 2024-05-14T16:40:29.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.618+0000 7fe157fff700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe14403e9a0 0x7fe144040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:29.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.619+0000 7fe157fff700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe14403e9a0 0x7fe144040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe14800fa60 tx=0x7fe148005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:29.624 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.624+0000 7fe1557fa700 1 -- 172.21.15.203:0/1829394475 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe14c020740 con 0x7fe15818fb90 2024-05-14T16:40:29.920 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.919+0000 7fe15e304700 1 -- 172.21.15.203:0/1829394475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe1581a84f0 con 0x7fe15818fb90 2024-05-14T16:40:29.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.920+0000 7fe1557fa700 1 -- 172.21.15.203:0/1829394475 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7fe14c0151c0 con 0x7fe15818fb90 2024-05-14T16:40:29.921 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-14T16:40:29.921 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":3,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","modified":"2024-05-14T16:40:23.791494Z","created":"2024-05-14T16:38:30.757480Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-14T16:40:29.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 -- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe14403e9a0 msgr2=0x7fe144040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:29.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe14403e9a0 0x7fe144040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe14800fa60 tx=0x7fe148005a90 comp rx=0 tx=0).stop 2024-05-14T16:40:29.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 -- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15818fb90 msgr2=0x7fe158190010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:29.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15818fb90 0x7fe158190010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00b550 tx=0x7fe14c00b910 comp rx=0 tx=0).stop 2024-05-14T16:40:29.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 -- 172.21.15.203:0/1829394475 shutdown_connections 2024-05-14T16:40:29.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1581a4dc0 0x7fe1581a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:29.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1400035c0 0x7fe15818f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:29.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe14403e9a0 0x7fe144040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:29.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 --2- 172.21.15.203:0/1829394475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15818fb90 0x7fe158190010 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:29.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 -- 172.21.15.203:0/1829394475 >> 172.21.15.203:0/1829394475 conn(0x7fe1580785f0 msgr2=0x7fe1580ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:29.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 -- 172.21.15.203:0/1829394475 shutdown_connections 2024-05-14T16:40:29.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:29.922+0000 7fe15e304700 1 -- 172.21.15.203:0/1829394475 wait complete. 2024-05-14T16:40:29.925 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 3 2024-05-14T16:40:30.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:29 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:30.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:29 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:30.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:29 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:30.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:29 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:29 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:29 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:30.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:29 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:30.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:29 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:29 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:29 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:29 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:29 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:40:30.946 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-14T16:40:30.947 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-14T16:40:31.188 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: from='client.? 172.21.15.203:0/1829394475' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-14T16:40:31.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:30 smithi069 ceph-mon[31507]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-14T16:40:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: from='client.? 172.21.15.203:0/1829394475' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:40:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-14T16:40:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-14T16:40:31.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:30 smithi203 ceph-mon[35787]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-14T16:40:31.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:31.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: from='client.? 172.21.15.203:0/1829394475' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:40:31.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:31.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:31.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:31.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-14T16:40:31.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-14T16:40:31.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:30 smithi092 ceph-mon[35763]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-14T16:40:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: Reconfiguring daemon mon.a on smithi069 2024-05-14T16:40:32.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:32.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:32.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:32.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: Reconfiguring daemon mon.a on smithi069 2024-05-14T16:40:32.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:32.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T16:40:32.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:32.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:40:32.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:32.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: Reconfiguring daemon mon.a on smithi069 2024-05-14T16:40:32.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.240+0000 7f51b52e7700 1 -- 172.21.15.69:0/4080060796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51b01031b0 msgr2=0x7f51b0103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.240+0000 7f51b52e7700 1 --2- 172.21.15.69:0/4080060796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51b01031b0 0x7f51b0103590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009910 tx=0x7f51a0009c20 comp rx=0 tx=0).stop 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.240+0000 7f51b52e7700 1 -- 172.21.15.69:0/4080060796 shutdown_connections 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.240+0000 7f51b52e7700 1 --2- 172.21.15.69:0/4080060796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51b01031b0 0x7f51b0103590 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.240+0000 7f51b52e7700 1 -- 172.21.15.69:0/4080060796 >> 172.21.15.69:0/4080060796 conn(0x7f51b00fea60 msgr2=0x7f51b0100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.240+0000 7f51b52e7700 1 -- 172.21.15.69:0/4080060796 shutdown_connections 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.240+0000 7f51b52e7700 1 -- 172.21.15.69:0/4080060796 wait complete. 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.241+0000 7f51b52e7700 1 Processor -- start 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.241+0000 7f51b52e7700 1 -- start start 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.241+0000 7f51b52e7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51b01031b0 0x7f51b0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.241+0000 7f51b52e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b0075d80 0x7f51b0076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:33.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.241+0000 7f51b52e7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b01a4dd0 0x7f51b01a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:33.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.241+0000 7f51b52e7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f51b010d430 con 0x7f51b01031b0 2024-05-14T16:40:33.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.241+0000 7f51b52e7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f51b010d2b0 con 0x7f51b0075d80 2024-05-14T16:40:33.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.241+0000 7f51b52e7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f51b010d5b0 con 0x7f51b01a4dd0 2024-05-14T16:40:33.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51aeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51b01031b0 0x7f51b0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:33.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51aeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51b01031b0 0x7f51b0075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58072/0 (socket says 172.21.15.69:58072) 2024-05-14T16:40:33.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51aeffd700 1 -- 172.21.15.69:0/54213643 learned_addr learned my addr 172.21.15.69:0/54213643 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:40:33.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51aeffd700 1 -- 172.21.15.69:0/54213643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b01a4dd0 msgr2=0x7f51b01a71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:33.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51ae7fc700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b0075d80 0x7f51b0076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:33.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51aeffd700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b01a4dd0 0x7f51b01a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:33.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51aeffd700 1 -- 172.21.15.69:0/54213643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b0075d80 msgr2=0x7f51b0076200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:33.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51aeffd700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b0075d80 0x7f51b0076200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:33.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51aeffd700 1 -- 172.21.15.69:0/54213643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b01a78f0 con 0x7f51b01031b0 2024-05-14T16:40:33.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51aeffd700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51b01031b0 0x7f51b0075840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f51a0004f80 tx=0x7f51a0005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:33.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f5197fff700 1 -- 172.21.15.69:0/54213643 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0026070 con 0x7f51b01031b0 2024-05-14T16:40:33.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f5197fff700 1 -- 172.21.15.69:0/54213643 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a000b6f0 con 0x7f51b01031b0 2024-05-14T16:40:33.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51b52e7700 1 -- 172.21.15.69:0/54213643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b01a7be0 con 0x7f51b01031b0 2024-05-14T16:40:33.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f5197fff700 1 -- 172.21.15.69:0/54213643 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0003970 con 0x7f51b01031b0 2024-05-14T16:40:33.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.242+0000 7f51b52e7700 1 -- 172.21.15.69:0/54213643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b01a8170 con 0x7f51b01031b0 2024-05-14T16:40:33.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.243+0000 7f5197fff700 1 -- 172.21.15.69:0/54213643 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f51a0029070 con 0x7f51b01031b0 2024-05-14T16:40:33.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.243+0000 7f51b52e7700 1 -- 172.21.15.69:0/54213643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b0064ba0 con 0x7f51b01031b0 2024-05-14T16:40:33.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.243+0000 7f5197fff700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f519803e9a0 0x7f5198040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:33.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.243+0000 7f5197fff700 1 -- 172.21.15.69:0/54213643 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f51a00535d0 con 0x7f51b01031b0 2024-05-14T16:40:33.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.243+0000 7f51ae7fc700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f519803e9a0 0x7f5198040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:33.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.244+0000 7f51ae7fc700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f519803e9a0 0x7f5198040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51a40078e0 tx=0x7f51a4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:33.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.248+0000 7f5197fff700 1 -- 172.21.15.69:0/54213643 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51a0013070 con 0x7f51b01031b0 2024-05-14T16:40:33.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:33 smithi092 ceph-mon[35763]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:33.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.454+0000 7f51b52e7700 1 -- 172.21.15.69:0/54213643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f51b0076d70 con 0x7f51b01031b0 2024-05-14T16:40:33.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:33 smithi069 ceph-mon[31507]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:33.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.454+0000 7f5197fff700 1 -- 172.21.15.69:0/54213643 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f51a0030020 con 0x7f51b01031b0 2024-05-14T16:40:33.455 INFO:teuthology.orchestra.run.smithi069.stdout:# minimal ceph.conf for 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:40:33.455 INFO:teuthology.orchestra.run.smithi069.stdout:[global] 2024-05-14T16:40:33.455 INFO:teuthology.orchestra.run.smithi069.stdout: fsid = 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:40:33.455 INFO:teuthology.orchestra.run.smithi069.stdout: mon_host = [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-05-14T16:40:33.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.456+0000 7f51b52e7700 1 -- 172.21.15.69:0/54213643 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f519803e9a0 msgr2=0x7f5198040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:33.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f519803e9a0 0x7f5198040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51a40078e0 tx=0x7f51a4008040 comp rx=0 tx=0).stop 2024-05-14T16:40:33.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 -- 172.21.15.69:0/54213643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51b01031b0 msgr2=0x7f51b0075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:33.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51b01031b0 0x7f51b0075840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f51a0004f80 tx=0x7f51a0005060 comp rx=0 tx=0).stop 2024-05-14T16:40:33.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 -- 172.21.15.69:0/54213643 shutdown_connections 2024-05-14T16:40:33.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f519803e9a0 0x7f5198040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:33.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51b01031b0 0x7f51b0075840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:33.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b01a4dd0 0x7f51b01a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:33.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 --2- 172.21.15.69:0/54213643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b0075d80 0x7f51b0076200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:33.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 -- 172.21.15.69:0/54213643 >> 172.21.15.69:0/54213643 conn(0x7f51b00fea60 msgr2=0x7f51b0104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:33.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 -- 172.21.15.69:0/54213643 shutdown_connections 2024-05-14T16:40:33.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:33.457+0000 7f51b52e7700 1 -- 172.21.15.69:0/54213643 wait complete. 2024-05-14T16:40:33.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:33 smithi203 ceph-mon[35787]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:34.325 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:34 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/54213643' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:34.325 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:34 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:34.326 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:34 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:34.326 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:34 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:34.326 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:34 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:40:34.326 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:34 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:34.352 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-14T16:40:34.352 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:40:34.352 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T16:40:34.381 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:40:34.381 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:40:34.446 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:40:34.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T16:40:34.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:34 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/54213643' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:34.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:34 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:34.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:34 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:34.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:34 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:34.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:34 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:40:34.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:34 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:34.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:34 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/54213643' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:34.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:34 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:34.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:34 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:34.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:34 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:34.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:34 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:40:34.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:34 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:34.472 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:40:34.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:40:34.538 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:40:34.538 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T16:40:34.570 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:40:34.570 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:40:34.637 INFO:tasks.cephadm:Adding mgr.x on smithi069 2024-05-14T16:40:34.638 INFO:tasks.cephadm:Adding mgr.y on smithi092 2024-05-14T16:40:34.638 INFO:tasks.cephadm:Adding mgr.z on smithi203 2024-05-14T16:40:34.638 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi069=x;smithi092=y;smithi203=z' 2024-05-14T16:40:34.911 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:40:35.308 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:35 smithi203 ceph-mon[35787]: Reconfiguring mon.b (monmap changed)... 2024-05-14T16:40:35.309 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:35 smithi203 ceph-mon[35787]: Reconfiguring daemon mon.b on smithi092 2024-05-14T16:40:35.309 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:35 smithi203 ceph-mon[35787]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:35.309 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:35.309 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:35.309 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:35.309 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:40:35.309 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:35.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:35 smithi092 ceph-mon[35763]: Reconfiguring mon.b (monmap changed)... 2024-05-14T16:40:35.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:35 smithi092 ceph-mon[35763]: Reconfiguring daemon mon.b on smithi092 2024-05-14T16:40:35.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:35 smithi092 ceph-mon[35763]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:35.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:35.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:35.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:35.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:40:35.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:35.573 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:35 smithi069 ceph-mon[31507]: Reconfiguring mon.b (monmap changed)... 2024-05-14T16:40:35.573 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:35 smithi069 ceph-mon[31507]: Reconfiguring daemon mon.b on smithi092 2024-05-14T16:40:35.574 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:35 smithi069 ceph-mon[31507]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:35.574 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:35.574 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:35.574 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:40:35.574 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:40:35.574 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:36.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:36 smithi092 ceph-mon[35763]: Reconfiguring mon.c (monmap changed)... 2024-05-14T16:40:36.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:36 smithi092 ceph-mon[35763]: Reconfiguring daemon mon.c on smithi203 2024-05-14T16:40:36.682 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:36 smithi203 ceph-mon[35787]: Reconfiguring mon.c (monmap changed)... 2024-05-14T16:40:36.682 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:36 smithi203 ceph-mon[35787]: Reconfiguring daemon mon.c on smithi203 2024-05-14T16:40:36.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:36 smithi069 ceph-mon[31507]: Reconfiguring mon.c (monmap changed)... 2024-05-14T16:40:36.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:36 smithi069 ceph-mon[31507]: Reconfiguring daemon mon.c on smithi203 2024-05-14T16:40:36.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.997+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/957437057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc1030d0 msgr2=0x7ff8bc1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:36.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.997+0000 7ff8c0a6d700 1 --2- 172.21.15.203:0/957437057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc1030d0 0x7ff8bc1034f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009910 tx=0x7ff8b0009c20 comp rx=0 tx=0).stop 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.997+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/957437057 shutdown_connections 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.997+0000 7ff8c0a6d700 1 --2- 172.21.15.203:0/957437057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc1030d0 0x7ff8bc1034f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.997+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/957437057 >> 172.21.15.203:0/957437057 conn(0x7ff8bc0fe670 msgr2=0x7ff8bc100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.997+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/957437057 shutdown_connections 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.997+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/957437057 wait complete. 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8c0a6d700 1 Processor -- start 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8c0a6d700 1 -- start start 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8c0a6d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc1030d0 0x7ff8bc0772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8c0a6d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc075930 0x7ff8bc075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8c0a6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8bc0762f0 0x7ff8bc1a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8c0a6d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc109010 con 0x7ff8bc1030d0 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8c0a6d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc108e90 con 0x7ff8bc0762f0 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8c0a6d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc109190 con 0x7ff8bc075930 2024-05-14T16:40:37.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8b9d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc075930 0x7ff8bc075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8b9d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc075930 0x7ff8bc075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50382/0 (socket says 172.21.15.203:50382) 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8b9d9b700 1 -- 172.21.15.203:0/4280982859 learned_addr learned my addr 172.21.15.203:0/4280982859 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8ba59c700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc1030d0 0x7ff8bc0772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8bad9d700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8bc0762f0 0x7ff8bc1a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8b9d9b700 1 -- 172.21.15.203:0/4280982859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8bc0762f0 msgr2=0x7ff8bc1a2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8b9d9b700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8bc0762f0 0x7ff8bc1a2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8b9d9b700 1 -- 172.21.15.203:0/4280982859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc1030d0 msgr2=0x7ff8bc0772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8b9d9b700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc1030d0 0x7ff8bc0772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8b9d9b700 1 -- 172.21.15.203:0/4280982859 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc1a2e40 con 0x7ff8bc075930 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.998+0000 7ff8b9d9b700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc075930 0x7ff8bc075db0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00b780 tx=0x7ff8ac00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.999+0000 7ff8ab7fe700 1 -- 172.21.15.203:0/4280982859 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac016070 con 0x7ff8bc075930 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.999+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/4280982859 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8bc1a3130 con 0x7ff8bc075930 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.999+0000 7ff8ab7fe700 1 -- 172.21.15.203:0/4280982859 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ac0036a0 con 0x7ff8bc075930 2024-05-14T16:40:37.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.999+0000 7ff8ab7fe700 1 -- 172.21.15.203:0/4280982859 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac011970 con 0x7ff8bc075930 2024-05-14T16:40:37.002 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:36.999+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/4280982859 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8bc1a3740 con 0x7ff8bc075930 2024-05-14T16:40:37.002 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.000+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/4280982859 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8bc064ba0 con 0x7ff8bc075930 2024-05-14T16:40:37.002 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.000+0000 7ff8ab7fe700 1 -- 172.21.15.203:0/4280982859 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff8ac019070 con 0x7ff8bc075930 2024-05-14T16:40:37.002 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.000+0000 7ff8ab7fe700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff8a403e9a0 0x7ff8a4040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:37.002 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.000+0000 7ff8ab7fe700 1 -- 172.21.15.203:0/4280982859 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff8ac0512f0 con 0x7ff8bc075930 2024-05-14T16:40:37.003 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.000+0000 7ff8ba59c700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff8a403e9a0 0x7ff8a4040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:37.003 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.001+0000 7ff8ba59c700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff8a403e9a0 0x7ff8a4040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005190 tx=0x7ff8b001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:37.005 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.005+0000 7ff8ab7fe700 1 -- 172.21.15.203:0/4280982859 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8ac01d030 con 0x7ff8bc075930 2024-05-14T16:40:37.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.226+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/4280982859 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi092=y;smithi203=z", "target": ["mon-mgr", ""]}) -- 0x7ff8bc063a10 con 0x7ff8a403e9a0 2024-05-14T16:40:37.232 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.233+0000 7ff8ab7fe700 1 -- 172.21.15.203:0/4280982859 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff8bc063a10 con 0x7ff8a403e9a0 2024-05-14T16:40:37.233 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mgr update... 2024-05-14T16:40:37.235 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/4280982859 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff8a403e9a0 msgr2=0x7ff8a4040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:37.235 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff8a403e9a0 0x7ff8a4040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005190 tx=0x7ff8b001a040 comp rx=0 tx=0).stop 2024-05-14T16:40:37.235 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/4280982859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc075930 msgr2=0x7ff8bc075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:37.235 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc075930 0x7ff8bc075db0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00b780 tx=0x7ff8ac00ba90 comp rx=0 tx=0).stop 2024-05-14T16:40:37.236 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/4280982859 shutdown_connections 2024-05-14T16:40:37.236 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff8a403e9a0 0x7ff8a4040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:37.236 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc1030d0 0x7ff8bc0772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:37.236 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8bc0762f0 0x7ff8bc1a2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:37.236 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 --2- 172.21.15.203:0/4280982859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8bc075930 0x7ff8bc075db0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:37.236 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/4280982859 >> 172.21.15.203:0/4280982859 conn(0x7ff8bc0fe670 msgr2=0x7ff8bc0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:37.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:37 smithi203 ceph-mon[35787]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:37.236 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/4280982859 shutdown_connections 2024-05-14T16:40:37.236 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:40:37.235+0000 7ff8c0a6d700 1 -- 172.21.15.203:0/4280982859 wait complete. 2024-05-14T16:40:37.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:37 smithi092 ceph-mon[35763]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:37.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:37 smithi069 ceph-mon[31507]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:37.884 DEBUG:teuthology.orchestra.run.smithi092:mgr.y> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.y.service 2024-05-14T16:40:37.887 DEBUG:teuthology.orchestra.run.smithi203:mgr.z> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.z.service 2024-05-14T16:40:37.891 INFO:tasks.cephadm:Deploying OSDs... 2024-05-14T16:40:37.891 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:40:37.892 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T16:40:37.908 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T16:40:37.908 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2024-05-14T16:40:37.964 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T16:40:37.964 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:37.964 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-14T16:40:37.964 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:37.964 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:37.964 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-14 16:39:41.645759717 +0000 2024-05-14T16:40:37.964 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-14 16:33:43.886664287 +0000 2024-05-14T16:40:37.964 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-14 16:33:43.886664287 +0000 2024-05-14T16:40:37.964 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-14 16:33:43.886664287 +0000 2024-05-14T16:40:37.965 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T16:40:38.028 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-14T16:40:38.029 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-14T16:40:38.029 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000216532 s, 2.4 MB/s 2024-05-14T16:40:38.030 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T16:40:38.087 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2024-05-14T16:40:38.142 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T16:40:38.142 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:38.143 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-14T16:40:38.143 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:38.143 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:38.143 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-14 16:39:41.670759161 +0000 2024-05-14T16:40:38.143 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-14 16:33:44.218656913 +0000 2024-05-14T16:40:38.143 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-14 16:33:44.218656913 +0000 2024-05-14T16:40:38.143 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-14 16:33:44.218656913 +0000 2024-05-14T16:40:38.143 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T16:40:38.206 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-14T16:40:38.206 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-14T16:40:38.206 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000198137 s, 2.6 MB/s 2024-05-14T16:40:38.207 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T16:40:38.264 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2024-05-14T16:40:38.320 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T16:40:38.320 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:38.321 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-14T16:40:38.321 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:38.321 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:38.321 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-14 16:39:41.695758605 +0000 2024-05-14T16:40:38.321 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-14 16:33:44.548649583 +0000 2024-05-14T16:40:38.321 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-14 16:33:44.548649583 +0000 2024-05-14T16:40:38.321 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-14 16:33:44.548649583 +0000 2024-05-14T16:40:38.322 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T16:40:38.382 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi092=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:40:38.382 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: Saving service mgr spec with placement smithi069=x;smithi092=y;smithi203=z;count:3 2024-05-14T16:40:38.382 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.382 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.382 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.382 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:38.382 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:38.382 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:38.383 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.383 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:40:38.383 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:40:38.383 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:40:38.383 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:38.384 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-14T16:40:38.384 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-14T16:40:38.384 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000205825 s, 2.5 MB/s 2024-05-14T16:40:38.385 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T16:40:38.442 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2024-05-14T16:40:38.497 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T16:40:38.497 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:38.497 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-14T16:40:38.497 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:38.498 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:38.498 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-14 16:39:41.719758071 +0000 2024-05-14T16:40:38.498 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-14 16:33:44.892641942 +0000 2024-05-14T16:40:38.498 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-14 16:33:44.892641942 +0000 2024-05-14T16:40:38.498 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-14 16:33:44.892641942 +0000 2024-05-14T16:40:38.498 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T16:40:38.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi092=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:40:38.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: Saving service mgr spec with placement smithi069=x;smithi092=y;smithi203=z;count:3 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:40:38.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:38.561 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-14T16:40:38.561 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-14T16:40:38.561 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000204656 s, 2.5 MB/s 2024-05-14T16:40:38.562 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T16:40:38.619 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-14T16:40:38.619 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T16:40:38.637 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T16:40:38.637 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi092=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: Saving service mgr spec with placement smithi069=x;smithi092=y;smithi203=z;count:3 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:40:38.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:40:38.693 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:38.695 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T16:40:38.695 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:38.695 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-14T16:40:38.695 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:38.695 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:38.695 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-14 16:40:23.232873161 +0000 2024-05-14T16:40:38.695 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-14 16:34:42.234209950 +0000 2024-05-14T16:40:38.695 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-14 16:34:42.234209950 +0000 2024-05-14T16:40:38.695 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-14 16:34:42.234209950 +0000 2024-05-14T16:40:38.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T16:40:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-14T16:40:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-14T16:40:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000341776 s, 1.5 MB/s 2024-05-14T16:40:38.763 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T16:40:38.821 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-05-14T16:40:38.878 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T16:40:38.878 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:38.879 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-05-14T16:40:38.879 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:38.879 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:38.879 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-14 16:40:23.232873161 +0000 2024-05-14T16:40:38.879 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-14 16:34:42.561200860 +0000 2024-05-14T16:40:38.879 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-14 16:34:42.561200860 +0000 2024-05-14T16:40:38.879 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-14 16:34:42.561200860 +0000 2024-05-14T16:40:38.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T16:40:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-14T16:40:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-14T16:40:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000220221 s, 2.3 MB/s 2024-05-14T16:40:38.945 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T16:40:39.001 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-05-14T16:40:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T16:40:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:39.058 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-14T16:40:39.058 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:39.058 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:39.058 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-14 16:40:23.233873133 +0000 2024-05-14T16:40:39.058 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-14 16:34:42.891191686 +0000 2024-05-14T16:40:39.058 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-14 16:34:42.891191686 +0000 2024-05-14T16:40:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-14 16:34:42.891191686 +0000 2024-05-14T16:40:39.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T16:40:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-14T16:40:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-14T16:40:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000224937 s, 2.3 MB/s 2024-05-14T16:40:39.124 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T16:40:39.181 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-05-14T16:40:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T16:40:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:39.237 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-14T16:40:39.237 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:39.238 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:39.238 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-14 16:40:23.233873133 +0000 2024-05-14T16:40:39.238 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-14 16:34:43.232182207 +0000 2024-05-14T16:40:39.238 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-14 16:34:43.232182207 +0000 2024-05-14T16:40:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-14 16:34:43.232182207 +0000 2024-05-14T16:40:39.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T16:40:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-14T16:40:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-14T16:40:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000240113 s, 2.1 MB/s 2024-05-14T16:40:39.303 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T16:40:39.359 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-14T16:40:39.359 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T16:40:39.375 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T16:40:39.375 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-05-14T16:40:39.431 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T16:40:39.431 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:39.431 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-14T16:40:39.431 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:39.432 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:39.432 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-14 16:40:13.702339478 +0000 2024-05-14T16:40:39.432 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-14 16:34:01.725644030 +0000 2024-05-14T16:40:39.432 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-14 16:34:01.725644030 +0000 2024-05-14T16:40:39.432 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-14 16:34:01.725644030 +0000 2024-05-14T16:40:39.432 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T16:40:39.495 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-14T16:40:39.495 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-14T16:40:39.496 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000254735 s, 2.0 MB/s 2024-05-14T16:40:39.497 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T16:40:39.553 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-05-14T16:40:39.609 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T16:40:39.609 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:39.609 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-14T16:40:39.609 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:39.609 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:39.609 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-14 16:40:13.702339478 +0000 2024-05-14T16:40:39.609 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-14 16:34:02.086635318 +0000 2024-05-14T16:40:39.609 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-14 16:34:02.086635318 +0000 2024-05-14T16:40:39.609 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-14 16:34:02.086635318 +0000 2024-05-14T16:40:39.610 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T16:40:39.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:39 smithi069 ceph-mon[31507]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:39.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:39 smithi069 ceph-mon[31507]: Deploying daemon mgr.z on smithi203 2024-05-14T16:40:39.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:39 smithi203 ceph-mon[35787]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:39.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:39 smithi203 ceph-mon[35787]: Deploying daemon mgr.z on smithi203 2024-05-14T16:40:39.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:39 smithi092 ceph-mon[35763]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:39.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:39 smithi092 ceph-mon[35763]: Deploying daemon mgr.z on smithi203 2024-05-14T16:40:39.781 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-14T16:40:39.781 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-14T16:40:39.781 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000201034 s, 2.5 MB/s 2024-05-14T16:40:39.783 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T16:40:39.798 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-05-14T16:40:39.857 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T16:40:39.857 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:39.857 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-14T16:40:39.857 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:39.857 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:39.857 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-14 16:40:39.737726784 +0000 2024-05-14T16:40:39.857 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-14 16:34:02.414627403 +0000 2024-05-14T16:40:39.857 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-14 16:34:02.414627403 +0000 2024-05-14T16:40:39.857 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-14 16:34:02.414627403 +0000 2024-05-14T16:40:39.858 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T16:40:39.957 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-14T16:40:39.957 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-14T16:40:39.957 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000177071 s, 2.9 MB/s 2024-05-14T16:40:39.958 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T16:40:39.973 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-05-14T16:40:40.031 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T16:40:40.031 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:40:40.031 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-14T16:40:40.031 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:40:40.031 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:40:40.031 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-14 16:40:39.914722619 +0000 2024-05-14T16:40:40.031 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-14 16:34:02.745619414 +0000 2024-05-14T16:40:40.031 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-14 16:34:02.745619414 +0000 2024-05-14T16:40:40.031 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-14 16:34:02.745619414 +0000 2024-05-14T16:40:40.031 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T16:40:40.092 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:39 smithi203 systemd[1]: Starting Ceph mgr.z for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:40:40.095 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-14T16:40:40.095 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-14T16:40:40.095 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000205734 s, 2.5 MB/s 2024-05-14T16:40:40.096 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T16:40:40.152 INFO:tasks.cephadm:Deploying osd.0 on smithi069 with /dev/vg_nvme/lv_4... 2024-05-14T16:40:40.153 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T16:40:40.393 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:40:40.401 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:40 smithi203 podman[37583]: 2024-05-14 16:40:40.191052667 +0000 UTC m=+0.112315789 container create de0a6ecb180747f755d6d6abc7d6fc3f159e53d22a993d78008eb2739f67ea9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-14T16:40:40.401 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:40 smithi203 podman[37583]: 2024-05-14 16:40:40.097453742 +0000 UTC m=+0.018716893 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:40:40.401 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:40 smithi203 podman[37583]: 2024-05-14 16:40:40.401815459 +0000 UTC m=+0.323078584 container init de0a6ecb180747f755d6d6abc7d6fc3f159e53d22a993d78008eb2739f67ea9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-14T16:40:40.710 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:40 smithi203 podman[37583]: 2024-05-14 16:40:40.404385429 +0000 UTC m=+0.325648566 container start de0a6ecb180747f755d6d6abc7d6fc3f159e53d22a993d78008eb2739f67ea9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_CLEAN=True) 2024-05-14T16:40:40.710 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:40 smithi203 ceph-mgr[37640]: -- 172.21.15.203:0/3064501793 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c7857a5a0 con 0x555c7854e800 2024-05-14T16:40:40.710 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:40 smithi203 bash[37583]: de0a6ecb180747f755d6d6abc7d6fc3f159e53d22a993d78008eb2739f67ea9b 2024-05-14T16:40:40.710 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:40 smithi203 systemd[1]: Started Ceph mgr.z for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:40:40.710 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:40 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:40.568+0000 7fee47c7b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:40:41.161 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:40 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:40.753+0000 7fee47c7b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:40:41.161 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:40 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:40.854+0000 7fee47c7b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:40:41.161 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:41 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:41.161+0000 7fee47c7b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:40:41.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:41 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:41.247+0000 7fee47c7b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:40:41.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:41 smithi069 ceph-mon[31507]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:41.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:41.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:41.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:41.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:40:41.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:40:41.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:40:41.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:41.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:41 smithi069 ceph-mon[31507]: Deploying daemon mgr.y on smithi092 2024-05-14T16:40:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:41 smithi203 ceph-mon[35787]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:40:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:40:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:40:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:41 smithi203 ceph-mon[35787]: Deploying daemon mgr.y on smithi092 2024-05-14T16:40:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:41 smithi092 ceph-mon[35763]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:40:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:40:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:40:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:41 smithi092 ceph-mon[35763]: Deploying daemon mgr.y on smithi092 2024-05-14T16:40:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T16:40:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T16:40:42.144 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-14T16:40:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-14T16:40:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111633 s, 939 MB/s 2024-05-14T16:40:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-14T16:40:42.710 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:42 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:42.450+0000 7fee47c7b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:40:42.710 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:42 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:42.630+0000 7fee47c7b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:40:42.750 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:42 smithi092 systemd[1]: Starting Ceph mgr.y for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:40:42.772 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2024-05-14T16:40:43.006 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:40:43.080 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:42 smithi092 podman[37115]: 2024-05-14 16:40:42.724925036 +0000 UTC m=+0.017305640 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:40:43.081 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:42 smithi092 podman[37115]: 2024-05-14 16:40:42.83472569 +0000 UTC m=+0.127110229 container create d1f975a7b58b84ce254fd7122fb4c3366b58e162f73b6110c7265d0a874e4425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:40:43.081 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:43 smithi092 podman[37115]: 2024-05-14 16:40:43.053929194 +0000 UTC m=+0.346309777 container init d1f975a7b58b84ce254fd7122fb4c3366b58e162f73b6110c7265d0a874e4425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-14T16:40:43.081 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:43 smithi092 podman[37115]: 2024-05-14 16:40:43.056397838 +0000 UTC m=+0.348778429 container start d1f975a7b58b84ce254fd7122fb4c3366b58e162f73b6110c7265d0a874e4425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T16:40:43.081 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:43 smithi092 ceph-mgr[37134]: -- 172.21.15.92:0/2076992111 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6da38e5a0 con 0x55b6da362800 2024-05-14T16:40:43.105 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:42 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:42.807+0000 7fee47c7b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:40:43.105 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:43 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:43.105+0000 7fee47c7b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:40:43.404 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:43 smithi092 bash[37115]: d1f975a7b58b84ce254fd7122fb4c3366b58e162f73b6110c7265d0a874e4425 2024-05-14T16:40:43.404 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:43 smithi092 systemd[1]: Started Ceph mgr.y for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:40:43.404 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:43 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:43.207+0000 7f8648efb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:40:43.404 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:43 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:43.404+0000 7f8648efb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:40:43.412 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:43 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:43.199+0000 7fee47c7b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:40:43.412 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:43 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:43.412+0000 7fee47c7b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:40:43.709 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:43 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:43.620+0000 7fee47c7b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:40:43.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:43 smithi203 ceph-mon[35787]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:43.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:43.750 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:43 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:43.518+0000 7f8648efb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:40:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:43 smithi092 ceph-mon[35763]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:43.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:43 smithi069 ceph-mon[31507]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:43.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:43.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:44.199 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:43 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:43.851+0000 7f8648efb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:40:44.199 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:43 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:43.948+0000 7f8648efb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:40:44.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.383+0000 7f4b18524700 1 -- 172.21.15.69:0/2324267804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b101036e0 msgr2=0x7f4b10103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:44.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.383+0000 7f4b18524700 1 --2- 172.21.15.69:0/2324267804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b101036e0 0x7f4b10103b40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b000099c0 tx=0x7f4b00009cd0 comp rx=0 tx=0).stop 2024-05-14T16:40:44.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.383+0000 7f4b18524700 1 -- 172.21.15.69:0/2324267804 shutdown_connections 2024-05-14T16:40:44.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.383+0000 7f4b18524700 1 --2- 172.21.15.69:0/2324267804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b10104080 0x7f4b1010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:44.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.383+0000 7f4b18524700 1 --2- 172.21.15.69:0/2324267804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b101024e0 0x7f4b10102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:44.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.383+0000 7f4b18524700 1 --2- 172.21.15.69:0/2324267804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b101036e0 0x7f4b10103b40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:44.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.383+0000 7f4b18524700 1 -- 172.21.15.69:0/2324267804 >> 172.21.15.69:0/2324267804 conn(0x7f4b100fdaa0 msgr2=0x7f4b100ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:40:44.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.383+0000 7f4b18524700 1 -- 172.21.15.69:0/2324267804 shutdown_connections 2024-05-14T16:40:44.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.383+0000 7f4b18524700 1 -- 172.21.15.69:0/2324267804 wait complete. 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b18524700 1 Processor -- start 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b18524700 1 -- start start 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b18524700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b101024e0 0x7f4b10199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b18524700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b101036e0 0x7f4b10199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b18524700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10104080 0x7f4b101a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b18524700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1010f0f0 con 0x7f4b10104080 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b18524700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1010ef70 con 0x7f4b101036e0 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b18524700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1010f270 con 0x7f4b101024e0 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b16ac1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10104080 0x7f4b101a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b16ac1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10104080 0x7f4b101a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48606/0 (socket says 172.21.15.69:48606) 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b16ac1700 1 -- 172.21.15.69:0/3478333140 learned_addr learned my addr 172.21.15.69:0/3478333140 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b15abf700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b101036e0 0x7f4b10199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.384+0000 7f4b162c0700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b101024e0 0x7f4b10199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b16ac1700 1 -- 172.21.15.69:0/3478333140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b101024e0 msgr2=0x7f4b10199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b16ac1700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b101024e0 0x7f4b10199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b16ac1700 1 -- 172.21.15.69:0/3478333140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b101036e0 msgr2=0x7f4b10199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b16ac1700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b101036e0 0x7f4b10199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b16ac1700 1 -- 172.21.15.69:0/3478333140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b101a13c0 con 0x7f4b10104080 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b16ac1700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10104080 0x7f4b101a0d20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4b08007f10 tx=0x7f4b0800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b077fe700 1 -- 172.21.15.69:0/3478333140 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800dd20 con 0x7f4b10104080 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b18524700 1 -- 172.21.15.69:0/3478333140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b101a16b0 con 0x7f4b10104080 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b077fe700 1 -- 172.21.15.69:0/3478333140 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b08004020 con 0x7f4b10104080 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b077fe700 1 -- 172.21.15.69:0/3478333140 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08015660 con 0x7f4b10104080 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.385+0000 7f4b18524700 1 -- 172.21.15.69:0/3478333140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b101a1c40 con 0x7f4b10104080 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.386+0000 7f4b077fe700 1 -- 172.21.15.69:0/3478333140 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4b08015800 con 0x7f4b10104080 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.386+0000 7f4b18524700 1 -- 172.21.15.69:0/3478333140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1006a230 con 0x7f4b10104080 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.386+0000 7f4b077fe700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4afc03e750 0x7f4afc040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.386+0000 7f4b077fe700 1 -- 172.21.15.69:0/3478333140 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4b08052290 con 0x7f4b10104080 2024-05-14T16:40:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.386+0000 7f4b162c0700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4afc03e750 0x7f4afc040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:40:44.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.387+0000 7f4b162c0700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4afc03e750 0x7f4afc040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b10103540 tx=0x7f4b0c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:40:44.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.391+0000 7f4b077fe700 1 -- 172.21.15.69:0/3478333140 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b08010070 con 0x7f4b10104080 2024-05-14T16:40:44.460 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:44.260+0000 7fee47c7b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:40:44.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:44.615+0000 7f4b18524700 1 -- 172.21.15.69:0/3478333140 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4b10063980 con 0x7f4afc03e750 2024-05-14T16:40:44.923 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:44.488+0000 7fee47c7b200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:40:45.209 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:44.922+0000 7fee47c7b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:40:45.210 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:45.018+0000 7fee47c7b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:40:45.500 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:45 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:45.182+0000 7f8648efb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:40:45.500 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:45 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:45.350+0000 7f8648efb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:40:45.709 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:45.389+0000 7fee47c7b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:45.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.711 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:40:45.711 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:40:45.711 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:45.803 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:45 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:45.517+0000 7f8648efb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:45.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:40:45.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:40:45.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:45.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:45.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:40:45.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:40:45.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:46.082 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:45.748+0000 7fee47c7b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:40:46.082 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:45.840+0000 7fee47c7b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:40:46.092 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:45 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:45.803+0000 7f8648efb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:40:46.092 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:45 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:45.894+0000 7f8648efb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:40:46.092 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:46 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:46.092+0000 7f8648efb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:40:46.460 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:46.081+0000 7fee47c7b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:40:46.460 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:46.216+0000 7fee47c7b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:40:46.460 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:46.311+0000 7fee47c7b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:40:46.500 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:46 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:46.281+0000 7f8648efb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:40:46.873 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:46 smithi092 ceph-mon[35763]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T16:40:46.873 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:46 smithi092 ceph-mon[35763]: Reconfiguring daemon mgr.x on smithi069 2024-05-14T16:40:46.897 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:46 smithi069 ceph-mon[31507]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T16:40:46.897 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:46 smithi069 ceph-mon[31507]: Reconfiguring daemon mgr.x on smithi069 2024-05-14T16:40:46.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:46 smithi203 ceph-mon[35787]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T16:40:46.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:46 smithi203 ceph-mon[35787]: Reconfiguring daemon mgr.x on smithi069 2024-05-14T16:40:46.960 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:46.639+0000 7fee47c7b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:40:46.960 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:46.731+0000 7fee47c7b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:40:47.250 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:46 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:46.873+0000 7f8648efb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:40:47.251 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:47.091+0000 7f8648efb200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:40:47.604 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:47.331+0000 7fee47c7b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:40:47.604 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:40:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:40:47.547+0000 7fee47c7b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:40:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: Standby manager daemon z started 2024-05-14T16:40:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:40:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:40:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:40:47.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:47 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:40:47.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:47.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: Standby manager daemon z started 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:40:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:47 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:40:47.969 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:47.510+0000 7f8648efb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:40:47.970 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:47.605+0000 7f8648efb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:40:47.970 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:47.970+0000 7f8648efb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:40:47.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:47.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:40:47.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:47.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:40:47.971 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:40:47.971 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: Standby manager daemon z started 2024-05-14T16:40:47.971 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:40:47.971 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:40:47.971 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:40:47.971 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:47 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.203:0/714987916' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:40:48.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:48.281+0000 7f4b077fe700 1 -- 172.21.15.69:0/3478333140 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f4b0801cba0 con 0x7f4b10104080 2024-05-14T16:40:48.654 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:48.322+0000 7f8648efb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:40:48.654 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:48.414+0000 7f8648efb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:40:49.000 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:48.654+0000 7f8648efb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:40:49.000 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:48.785+0000 7f8648efb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:40:49.001 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:48.879+0000 7f8648efb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:40:49.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:49 smithi092 ceph-mon[35763]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:49.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:49 smithi092 ceph-mon[35763]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-14T16:40:49.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:40:49.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:49 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/673408652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e3c095-398a-4772-b0ef-655789199a67"}]: dispatch 2024-05-14T16:40:49.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:49 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/673408652' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e3c095-398a-4772-b0ef-655789199a67"}]': finished 2024-05-14T16:40:49.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:49 smithi092 ceph-mon[35763]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T16:40:49.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:40:49.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:49 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2159663765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:40:49.501 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:49.202+0000 7f8648efb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:40:49.501 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:49.293+0000 7f8648efb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:40:49.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:49 smithi069 ceph-mon[31507]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:49.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:49 smithi069 ceph-mon[31507]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-14T16:40:49.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:40:49.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:49 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/673408652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e3c095-398a-4772-b0ef-655789199a67"}]: dispatch 2024-05-14T16:40:49.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:49 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/673408652' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e3c095-398a-4772-b0ef-655789199a67"}]': finished 2024-05-14T16:40:49.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:49 smithi069 ceph-mon[31507]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T16:40:49.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:40:49.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:49 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2159663765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:40:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:49 smithi203 ceph-mon[35787]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:49 smithi203 ceph-mon[35787]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-14T16:40:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:40:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:49 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/673408652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e3c095-398a-4772-b0ef-655789199a67"}]: dispatch 2024-05-14T16:40:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:49 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/673408652' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e3c095-398a-4772-b0ef-655789199a67"}]': finished 2024-05-14T16:40:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:49 smithi203 ceph-mon[35787]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T16:40:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:40:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:49 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2159663765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:40:50.250 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:49.886+0000 7f8648efb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:40:50.251 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:40:50 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:40:50.098+0000 7f8648efb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:40:50.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:40:50.292+0000 7f4b077fe700 1 -- 172.21.15.69:0/3478333140 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f4b080044f0 con 0x7f4b10104080 2024-05-14T16:40:50.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:50 smithi069 ceph-mon[31507]: Standby manager daemon y started 2024-05-14T16:40:50.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:50 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:40:50.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:50 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:40:50.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:50 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:40:50.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:50 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:40:50.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:50 smithi203 ceph-mon[35787]: Standby manager daemon y started 2024-05-14T16:40:50.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:50 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:40:50.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:50 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:40:50.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:50 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:40:50.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:50 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:40:50.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:50 smithi092 ceph-mon[35763]: Standby manager daemon y started 2024-05-14T16:40:50.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:50 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:40:50.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:50 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:40:50.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:50 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:40:50.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:50 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.92:0/1396342630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:40:51.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:51 smithi069 ceph-mon[31507]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:51.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:51 smithi069 ceph-mon[31507]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-05-14T16:40:51.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:51 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:40:51.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:51 smithi203 ceph-mon[35787]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:51.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:51 smithi203 ceph-mon[35787]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-05-14T16:40:51.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:51 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:40:51.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:51 smithi092 ceph-mon[35763]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:51.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:51 smithi092 ceph-mon[35763]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-05-14T16:40:51.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:51 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:40:53.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:53 smithi069 ceph-mon[31507]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:53.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:53 smithi203 ceph-mon[35787]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:53.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:53 smithi092 ceph-mon[35763]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:55.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:55 smithi069 ceph-mon[31507]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T16:40:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:55.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:55 smithi203 ceph-mon[35787]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:55.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T16:40:55.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:55.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:55 smithi092 ceph-mon[35763]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:55.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T16:40:55.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:40:56.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:56 smithi069 ceph-mon[31507]: Deploying daemon osd.0 on smithi069 2024-05-14T16:40:56.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:56 smithi203 ceph-mon[35787]: Deploying daemon osd.0 on smithi069 2024-05-14T16:40:56.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:56 smithi092 ceph-mon[35763]: Deploying daemon osd.0 on smithi069 2024-05-14T16:40:57.587 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:57 smithi069 ceph-mon[31507]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:57.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:57 smithi203 ceph-mon[35787]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:57.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:57 smithi092 ceph-mon[35763]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:59.577 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:40:59 smithi069 ceph-mon[31507]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:59.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:40:59 smithi203 ceph-mon[35787]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:40:59.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:40:59 smithi092 ceph-mon[35763]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:01.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:01 smithi069 ceph-mon[31507]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:01.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:01 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:01.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:01 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:01.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:01 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:01.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:01 smithi203 ceph-mon[35787]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:01.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:01 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:01.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:01 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:01.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:01 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:01 smithi092 ceph-mon[35763]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:01 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:01.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:01 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:01.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:01 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:03.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.467+0000 7f4b077fe700 1 -- 172.21.15.69:0/3478333140 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4b10063980 con 0x7f4afc03e750 2024-05-14T16:41:03.468 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 0 on host 'smithi069' 2024-05-14T16:41:03.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 -- 172.21.15.69:0/3478333140 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4afc03e750 msgr2=0x7f4afc040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:03.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4afc03e750 0x7f4afc040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b10103540 tx=0x7f4b0c00b410 comp rx=0 tx=0).stop 2024-05-14T16:41:03.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 -- 172.21.15.69:0/3478333140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10104080 msgr2=0x7f4b101a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:03.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10104080 0x7f4b101a0d20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4b08007f10 tx=0x7f4b0800d3b0 comp rx=0 tx=0).stop 2024-05-14T16:41:03.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 -- 172.21.15.69:0/3478333140 shutdown_connections 2024-05-14T16:41:03.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f4afc03e750 0x7f4afc040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:03.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10104080 0x7f4b101a0d20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:03.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b101024e0 0x7f4b10199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:03.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 --2- 172.21.15.69:0/3478333140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b101036e0 0x7f4b10199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:03.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 -- 172.21.15.69:0/3478333140 >> 172.21.15.69:0/3478333140 conn(0x7f4b100fdaa0 msgr2=0x7f4b10106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:41:03.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 -- 172.21.15.69:0/3478333140 shutdown_connections 2024-05-14T16:41:03.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:03.469+0000 7f4b18524700 1 -- 172.21.15.69:0/3478333140 wait complete. 2024-05-14T16:41:03.661 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:03 smithi069 ceph-mon[31507]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:03.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:03 smithi203 ceph-mon[35787]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:03.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:03 smithi092 ceph-mon[35763]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:04.318 DEBUG:teuthology.orchestra.run.smithi069:osd.0> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.0.service 2024-05-14T16:41:04.320 INFO:tasks.cephadm:Deploying osd.1 on smithi092 with /dev/vg_nvme/lv_4... 2024-05-14T16:41:04.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T16:41:04.566 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:41:04.672 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:04 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:04.672 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:04 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:04.672 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:04 smithi092 ceph-mon[35763]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T16:41:04.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:04 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:04.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:04 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:04.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:04 smithi069 ceph-mon[31507]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T16:41:04.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:04 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:04.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:04 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:04.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:04 smithi203 ceph-mon[35787]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T16:41:05.675 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:05 smithi092 ceph-mon[35763]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:05.675 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:05 smithi092 ceph-mon[35763]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T16:41:05.675 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:05 smithi092 ceph-mon[35763]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T16:41:05.675 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:05 smithi092 ceph-mon[35763]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:41:05.675 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:05.675 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:05.675 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:05.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:05 smithi069 ceph-mon[31507]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:05.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:05 smithi069 ceph-mon[31507]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T16:41:05.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:05 smithi069 ceph-mon[31507]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T16:41:05.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:05 smithi069 ceph-mon[31507]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:41:05.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:05.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:05.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:05.705 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 16:41:05 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0[43503]: 2024-05-14T16:41:05.441+0000 7f810a24b700 -1 osd.0 0 waiting for initial osdmap 2024-05-14T16:41:05.705 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 16:41:05 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0[43503]: 2024-05-14T16:41:05.445+0000 7f810603e700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:41:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:05 smithi203 ceph-mon[35787]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:05 smithi203 ceph-mon[35787]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T16:41:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:05 smithi203 ceph-mon[35787]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T16:41:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:05 smithi203 ceph-mon[35787]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:41:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T16:41:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T16:41:06.274 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-14T16:41:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-14T16:41:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125586 s, 835 MB/s 2024-05-14T16:41:06.275 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-14T16:41:06.674 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:06 smithi092 ceph-mon[35763]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:41:06.675 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:06 smithi092 ceph-mon[35763]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T16:41:06.675 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:06 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:06.675 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:06 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:06.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:06 smithi069 ceph-mon[31507]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:41:06.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:06 smithi069 ceph-mon[31507]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T16:41:06.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:06 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:06.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:06 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:06.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:06 smithi203 ceph-mon[35787]: from='osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:41:06.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:06 smithi203 ceph-mon[35787]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T16:41:06.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:06 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:06.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:06 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:06.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-05-14T16:41:07.153 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:41:07.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:07 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:41:07.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:07 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:41:07.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:07 smithi092 ceph-mon[35763]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:07.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:07.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:07 smithi092 ceph-mon[35763]: osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] boot 2024-05-14T16:41:07.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:07 smithi092 ceph-mon[35763]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T16:41:07.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:07.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:07 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:41:07.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:07 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:41:07.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:07 smithi069 ceph-mon[31507]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:07.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:07.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:07 smithi069 ceph-mon[31507]: osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] boot 2024-05-14T16:41:07.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:07 smithi069 ceph-mon[31507]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T16:41:07.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:07.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:07 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:41:07.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:07 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:41:07.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:07 smithi203 ceph-mon[35787]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:41:07.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:07.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:07 smithi203 ceph-mon[35787]: osd.0 [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] boot 2024-05-14T16:41:07.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:07 smithi203 ceph-mon[35787]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T16:41:07.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:41:08.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:08 smithi092 ceph-mon[35763]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T16:41:08.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:08 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:08 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:08 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:41:08.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:08 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:08 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:08.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:08 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:08.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:08 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:08 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:08.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.694+0000 7f09d3c4c700 1 -- 172.21.15.92:0/68098855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc1047d0 msgr2=0x7f09cc104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:08.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.694+0000 7f09d3c4c700 1 --2- 172.21.15.92:0/68098855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc1047d0 0x7f09cc104c30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f09c80099c0 tx=0x7f09c8009cd0 comp rx=0 tx=0).stop 2024-05-14T16:41:08.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.694+0000 7f09d3c4c700 1 -- 172.21.15.92:0/68098855 shutdown_connections 2024-05-14T16:41:08.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.694+0000 7f09d3c4c700 1 --2- 172.21.15.92:0/68098855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc105170 0x7f09cc10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:08.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.694+0000 7f09d3c4c700 1 --2- 172.21.15.92:0/68098855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09cc1035d0 0x7f09cc1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:08.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.694+0000 7f09d3c4c700 1 --2- 172.21.15.92:0/68098855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc1047d0 0x7f09cc104c30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:08.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.694+0000 7f09d3c4c700 1 -- 172.21.15.92:0/68098855 >> 172.21.15.92:0/68098855 conn(0x7f09cc0feb50 msgr2=0x7f09cc100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.694+0000 7f09d3c4c700 1 -- 172.21.15.92:0/68098855 shutdown_connections 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.694+0000 7f09d3c4c700 1 -- 172.21.15.92:0/68098855 wait complete. 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d3c4c700 1 Processor -- start 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d3c4c700 1 -- start start 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d3c4c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc1035d0 0x7f09cc197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d3c4c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc1047d0 0x7f09cc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d3c4c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09cc105170 0x7f09cc19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d3c4c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc10cf20 con 0x7f09cc1035d0 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d3c4c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc10cda0 con 0x7f09cc1047d0 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d3c4c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc10d0a0 con 0x7f09cc105170 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d11e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc1047d0 0x7f09cc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:08.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d11e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc1047d0 0x7f09cc197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49402/0 (socket says 172.21.15.92:49402) 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d11e7700 1 -- 172.21.15.92:0/1588613747 learned_addr learned my addr 172.21.15.92:0/1588613747 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d19e8700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc1035d0 0x7f09cc197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d21e9700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09cc105170 0x7f09cc19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d11e7700 1 -- 172.21.15.92:0/1588613747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09cc105170 msgr2=0x7f09cc19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d11e7700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09cc105170 0x7f09cc19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d11e7700 1 -- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc1035d0 msgr2=0x7f09cc197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d11e7700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc1035d0 0x7f09cc197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d11e7700 1 -- 172.21.15.92:0/1588613747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09cc19f160 con 0x7f09cc1047d0 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d19e8700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc1035d0 0x7f09cc197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.695+0000 7f09d11e7700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc1047d0 0x7f09cc197a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009990 tx=0x7f09c800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.696+0000 7f09c2ffd700 1 -- 172.21.15.92:0/1588613747 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8009e60 con 0x7f09cc1047d0 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.696+0000 7f09d3c4c700 1 -- 172.21.15.92:0/1588613747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09cc19f3f0 con 0x7f09cc1047d0 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.696+0000 7f09c2ffd700 1 -- 172.21.15.92:0/1588613747 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09c8003d70 con 0x7f09cc1047d0 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.696+0000 7f09c2ffd700 1 -- 172.21.15.92:0/1588613747 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8005670 con 0x7f09cc1047d0 2024-05-14T16:41:08.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.696+0000 7f09d3c4c700 1 -- 172.21.15.92:0/1588613747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09cc19f8a0 con 0x7f09cc1047d0 2024-05-14T16:41:08.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.697+0000 7f09d3c4c700 1 -- 172.21.15.92:0/1588613747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09cc06a230 con 0x7f09cc1047d0 2024-05-14T16:41:08.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.699+0000 7f09c2ffd700 1 -- 172.21.15.92:0/1588613747 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f09c800fcd0 con 0x7f09cc1047d0 2024-05-14T16:41:08.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.699+0000 7f09c2ffd700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09b80b27f0 0x7f09b80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:08.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.700+0000 7f09c2ffd700 1 -- 172.21.15.92:0/1588613747 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f09c80e1710 con 0x7f09cc1047d0 2024-05-14T16:41:08.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.700+0000 7f09d19e8700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09b80b27f0 0x7f09b80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:08.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.700+0000 7f09d19e8700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09b80b27f0 0x7f09b80b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09cc104630 tx=0x7f09bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:41:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:08 smithi069 ceph-mon[31507]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T16:41:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:41:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:08.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:08.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:08.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.703+0000 7f09c2ffd700 1 -- 172.21.15.92:0/1588613747 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09c8013070 con 0x7f09cc1047d0 2024-05-14T16:41:08.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:08 smithi203 ceph-mon[35787]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T16:41:08.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:08 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:08 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:08 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:41:08.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:08 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:08 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:08.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:08 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:08.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:08 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:08.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:08 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:08.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:08.919+0000 7f09d3c4c700 1 -- 172.21.15.92:0/1588613747 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f09cc063a40 con 0x7f09b80b27f0 2024-05-14T16:41:09.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:09 smithi092 ceph-mon[35763]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:09.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:09 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi069 2024-05-14T16:41:09.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:09 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi069 to 17088M 2024-05-14T16:41:09.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:09 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:09.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:09 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:09.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:09 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:09.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:09 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:41:09.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:09 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:41:09.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:09 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:09.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:09 smithi069 ceph-mon[31507]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:09.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:09 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi069 2024-05-14T16:41:09.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:09 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi069 to 17088M 2024-05-14T16:41:09.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:09.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:09.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:09.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:41:09.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:41:09.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:09.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:09 smithi203 ceph-mon[35787]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:09 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi069 2024-05-14T16:41:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:09 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi069 to 17088M 2024-05-14T16:41:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:09 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:09 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:09 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:09 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:41:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:09 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:41:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:09 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:10.727 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:10 smithi092 ceph-mon[35763]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:41:10.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:10 smithi069 ceph-mon[31507]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:41:10.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:10 smithi203 ceph-mon[35787]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:41:11.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:11 smithi092 ceph-mon[35763]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:11.861 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:11 smithi203 ceph-mon[35787]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:11.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:11 smithi069 ceph-mon[31507]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:12.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:12 smithi092 ceph-mon[35763]: from='client.? 172.21.15.92:0/1650592648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0678a843-a395-41d6-892f-7d0d24b30e40"}]: dispatch 2024-05-14T16:41:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:12 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0678a843-a395-41d6-892f-7d0d24b30e40"}]: dispatch 2024-05-14T16:41:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:12 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0678a843-a395-41d6-892f-7d0d24b30e40"}]': finished 2024-05-14T16:41:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:12 smithi092 ceph-mon[35763]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T16:41:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:12 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:12 smithi092 ceph-mon[35763]: from='client.? 172.21.15.92:0/1886672789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:41:12.877 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:12 smithi203 ceph-mon[35787]: from='client.? 172.21.15.92:0/1650592648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0678a843-a395-41d6-892f-7d0d24b30e40"}]: dispatch 2024-05-14T16:41:12.877 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:12 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0678a843-a395-41d6-892f-7d0d24b30e40"}]: dispatch 2024-05-14T16:41:12.877 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:12 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0678a843-a395-41d6-892f-7d0d24b30e40"}]': finished 2024-05-14T16:41:12.878 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:12 smithi203 ceph-mon[35787]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T16:41:12.878 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:12 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:12.878 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:12 smithi203 ceph-mon[35787]: from='client.? 172.21.15.92:0/1886672789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:41:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:12 smithi069 ceph-mon[31507]: from='client.? 172.21.15.92:0/1650592648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0678a843-a395-41d6-892f-7d0d24b30e40"}]: dispatch 2024-05-14T16:41:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:12 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0678a843-a395-41d6-892f-7d0d24b30e40"}]: dispatch 2024-05-14T16:41:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:12 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0678a843-a395-41d6-892f-7d0d24b30e40"}]': finished 2024-05-14T16:41:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:12 smithi069 ceph-mon[31507]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T16:41:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:12 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:12 smithi069 ceph-mon[31507]: from='client.? 172.21.15.92:0/1886672789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:41:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:13 smithi069 ceph-mon[31507]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:13.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:13 smithi203 ceph-mon[35787]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:14.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:13 smithi092 ceph-mon[35763]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:15.806 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:15 smithi092 ceph-mon[35763]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:15 smithi069 ceph-mon[31507]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:15.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:15 smithi203 ceph-mon[35787]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:17.935 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:17 smithi069 ceph-mon[31507]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:17 smithi203 ceph-mon[35787]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:17.978 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:17 smithi092 ceph-mon[35763]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:19.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:19 smithi069 ceph-mon[31507]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:19.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T16:41:19.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:19.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:19 smithi069 ceph-mon[31507]: Deploying daemon osd.1 on smithi092 2024-05-14T16:41:19.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:19 smithi203 ceph-mon[35787]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:19.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T16:41:19.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:19.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:19 smithi203 ceph-mon[35787]: Deploying daemon osd.1 on smithi092 2024-05-14T16:41:19.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:19 smithi092 ceph-mon[35763]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:19.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T16:41:19.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:19.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:19 smithi092 ceph-mon[35763]: Deploying daemon osd.1 on smithi092 2024-05-14T16:41:21.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:21 smithi092 ceph-mon[35763]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:21.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:21 smithi069 ceph-mon[31507]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:21.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:21 smithi203 ceph-mon[35787]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:23.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:23 smithi092 ceph-mon[35763]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:23.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:23 smithi069 ceph-mon[31507]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:23.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:23 smithi203 ceph-mon[35787]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:25.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:24 smithi069 ceph-mon[31507]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:25.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:25.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:25.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:25.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:24 smithi203 ceph-mon[35787]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:25.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:25.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:25.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:25.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:24 smithi092 ceph-mon[35763]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:25.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:25.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:25.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:27.300 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:27 smithi092 ceph-mon[35763]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:27.300 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:27 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:27.300 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:27 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:27.301 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:27 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:27.301 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:27 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:27.301 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:27 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:27.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.383+0000 7f09c2ffd700 1 -- 172.21.15.92:0/1588613747 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f09cc063a40 con 0x7f09b80b27f0 2024-05-14T16:41:27.383 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 1 on host 'smithi092' 2024-05-14T16:41:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.385+0000 7f09d3c4c700 1 -- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09b80b27f0 msgr2=0x7f09b80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.385+0000 7f09d3c4c700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09b80b27f0 0x7f09b80b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09cc104630 tx=0x7f09bc009260 comp rx=0 tx=0).stop 2024-05-14T16:41:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.385+0000 7f09d3c4c700 1 -- 172.21.15.92:0/1588613747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc1047d0 msgr2=0x7f09cc197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.385+0000 7f09d3c4c700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc1047d0 0x7f09cc197a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009990 tx=0x7f09c800f690 comp rx=0 tx=0).stop 2024-05-14T16:41:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.385+0000 7f09d3c4c700 1 -- 172.21.15.92:0/1588613747 shutdown_connections 2024-05-14T16:41:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.386+0000 7f09d3c4c700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09b80b27f0 0x7f09b80b4cb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.386+0000 7f09d3c4c700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc1035d0 0x7f09cc197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.386+0000 7f09d3c4c700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09cc105170 0x7f09cc19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.386+0000 7f09d3c4c700 1 --2- 172.21.15.92:0/1588613747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc1047d0 0x7f09cc197a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.386+0000 7f09d3c4c700 1 -- 172.21.15.92:0/1588613747 >> 172.21.15.92:0/1588613747 conn(0x7f09cc0feb50 msgr2=0x7f09cc107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:41:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.386+0000 7f09d3c4c700 1 -- 172.21.15.92:0/1588613747 shutdown_connections 2024-05-14T16:41:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:41:27.386+0000 7f09d3c4c700 1 -- 172.21.15.92:0/1588613747 wait complete. 2024-05-14T16:41:27.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:27 smithi069 ceph-mon[31507]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:27.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:27 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:27.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:27 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:27.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:27 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:27.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:27 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:27.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:27 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:27.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:27 smithi203 ceph-mon[35787]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:27.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:27 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:27.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:27 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:27.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:27 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:27.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:27 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:27.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:27 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:28.064 DEBUG:teuthology.orchestra.run.smithi092:osd.1> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.1.service 2024-05-14T16:41:28.066 INFO:tasks.cephadm:Deploying osd.2 on smithi203 with /dev/vg_nvme/lv_4... 2024-05-14T16:41:28.066 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T16:41:28.309 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:41:28.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:28.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:28.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:28.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:28 smithi203 ceph-mon[35787]: from='osd.1 [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:41:28.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:28 smithi203 ceph-mon[35787]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:41:28.574 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:28.575 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:28.575 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:28.575 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:28 smithi092 ceph-mon[35763]: from='osd.1 [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:41:28.575 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:28 smithi092 ceph-mon[35763]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:41:28.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:28.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:28.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:28.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:28 smithi069 ceph-mon[31507]: from='osd.1 [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:41:28.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:28 smithi069 ceph-mon[31507]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:41:29.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:29 smithi092 ceph-mon[35763]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:29.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:29 smithi092 ceph-mon[35763]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T16:41:29.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:29 smithi092 ceph-mon[35763]: from='osd.1 [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:41:29.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:29 smithi092 ceph-mon[35763]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T16:41:29.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:29 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:29.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:29 smithi092 ceph-mon[35763]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:41:29.466 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 16:41:29 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1[40326]: 2024-05-14T16:41:29.348+0000 7f3ab3a3e700 -1 osd.1 0 waiting for initial osdmap 2024-05-14T16:41:29.466 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 16:41:29 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1[40326]: 2024-05-14T16:41:29.352+0000 7f3aaf831700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:41:29.549 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:29 smithi203 ceph-mon[35787]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:29.549 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:29 smithi203 ceph-mon[35787]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T16:41:29.550 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:29 smithi203 ceph-mon[35787]: from='osd.1 [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:41:29.550 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:29 smithi203 ceph-mon[35787]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T16:41:29.550 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:29 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:29.550 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:29 smithi203 ceph-mon[35787]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:41:29.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:29 smithi069 ceph-mon[31507]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:29.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:29 smithi069 ceph-mon[31507]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T16:41:29.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:29 smithi069 ceph-mon[31507]: from='osd.1 [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:41:29.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:29 smithi069 ceph-mon[31507]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T16:41:29.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:29 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:29.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:29 smithi069 ceph-mon[31507]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:41:30.144 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T16:41:30.144 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T16:41:30.144 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-05-14T16:41:30.144 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-05-14T16:41:30.144 INFO:teuthology.orchestra.run.smithi203.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113961 s, 920 MB/s 2024-05-14T16:41:30.144 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 2024-05-14T16:41:30.144 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-05-14T16:41:30.624 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:30 smithi203 ceph-mon[35787]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:41:30.624 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:30 smithi203 ceph-mon[35787]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T16:41:30.624 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:30.624 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:30.692 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_4 2024-05-14T16:41:30.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:30 smithi069 ceph-mon[31507]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:41:30.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:30 smithi069 ceph-mon[31507]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T16:41:30.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:30.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:30.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:30 smithi092 ceph-mon[35763]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:41:30.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:30 smithi092 ceph-mon[35763]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T16:41:30.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:30.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:30.929 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:41:31.693 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:41:31.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:41:31.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:31.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: osd.1 [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] boot 2024-05-14T16:41:31.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T16:41:31.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:31.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:41:31.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:31.695 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:31.695 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: osd.1 [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] boot 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:31.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:31.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:41:31.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: osd.1 [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] boot 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:32.478 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.475+0000 7faa7a8cb700 1 -- 172.21.15.203:0/660302184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74105140 msgr2=0x7faa7410b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.475+0000 7faa7a8cb700 1 --2- 172.21.15.203:0/660302184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74105140 0x7faa7410b9f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00b0a0 tx=0x7faa6c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 -- 172.21.15.203:0/660302184 shutdown_connections 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 --2- 172.21.15.203:0/660302184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa741035a0 0x7faa741039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 --2- 172.21.15.203:0/660302184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74105140 0x7faa7410b9f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 --2- 172.21.15.203:0/660302184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa741047a0 0x7faa74104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 -- 172.21.15.203:0/660302184 >> 172.21.15.203:0/660302184 conn(0x7faa740feb40 msgr2=0x7faa74100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 -- 172.21.15.203:0/660302184 shutdown_connections 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 -- 172.21.15.203:0/660302184 wait complete. 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 Processor -- start 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 -- start start 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa741035a0 0x7faa741995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa741047a0 0x7faa74199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.476+0000 7faa7a8cb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa74105140 0x7faa741a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:32.479 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa7a8cb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faa7410f020 con 0x7faa74105140 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa7a8cb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa7410eea0 con 0x7faa741047a0 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa7a8cb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa7410f1a0 con 0x7faa741035a0 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa73fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa741035a0 0x7faa741995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa73fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa741035a0 0x7faa741995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:40888/0 (socket says 172.21.15.203:40888) 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa73fff700 1 -- 172.21.15.203:0/4243843163 learned_addr learned my addr 172.21.15.203:0/4243843163 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa78e68700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa74105140 0x7faa741a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa737fe700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa741047a0 0x7faa74199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa73fff700 1 -- 172.21.15.203:0/4243843163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa741047a0 msgr2=0x7faa74199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa73fff700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa741047a0 0x7faa74199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa73fff700 1 -- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa74105140 msgr2=0x7faa741a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa73fff700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa74105140 0x7faa741a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa73fff700 1 -- 172.21.15.203:0/4243843163 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa741a1230 con 0x7faa741035a0 2024-05-14T16:41:32.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa78e68700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa74105140 0x7faa741a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:41:32.481 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa73fff700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa741035a0 0x7faa741995d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00c9c0 tx=0x7faa5c00cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:41:32.481 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa717fa700 1 -- 172.21.15.203:0/4243843163 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c007ac0 con 0x7faa741035a0 2024-05-14T16:41:32.481 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa717fa700 1 -- 172.21.15.203:0/4243843163 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa5c007c60 con 0x7faa741035a0 2024-05-14T16:41:32.481 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa7a8cb700 1 -- 172.21.15.203:0/4243843163 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa741a1520 con 0x7faa741035a0 2024-05-14T16:41:32.481 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa717fa700 1 -- 172.21.15.203:0/4243843163 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c018660 con 0x7faa741035a0 2024-05-14T16:41:32.481 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.477+0000 7faa7a8cb700 1 -- 172.21.15.203:0/4243843163 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa741a1ab0 con 0x7faa741035a0 2024-05-14T16:41:32.481 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.478+0000 7faa7a8cb700 1 -- 172.21.15.203:0/4243843163 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7406a230 con 0x7faa741035a0 2024-05-14T16:41:32.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.481+0000 7faa717fa700 1 -- 172.21.15.203:0/4243843163 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7faa5c004020 con 0x7faa741035a0 2024-05-14T16:41:32.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.481+0000 7faa717fa700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7faa600b2870 0x7faa600b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:32.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.481+0000 7faa717fa700 1 -- 172.21.15.203:0/4243843163 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7faa5c0e0b70 con 0x7faa741035a0 2024-05-14T16:41:32.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.481+0000 7faa737fe700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7faa600b2870 0x7faa600b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:32.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.482+0000 7faa737fe700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7faa600b2870 0x7faa600b4d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faa64005fd0 tx=0x7faa6400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:41:32.485 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.485+0000 7faa717fa700 1 -- 172.21.15.203:0/4243843163 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa5c014030 con 0x7faa741035a0 2024-05-14T16:41:32.696 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:32.695+0000 7faa7a8cb700 1 -- 172.21.15.203:0/4243843163 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7faa74063980 con 0x7faa600b2870 2024-05-14T16:41:32.696 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:32 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi092 2024-05-14T16:41:32.696 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:32 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi092 to 17088M 2024-05-14T16:41:32.696 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:32 smithi203 ceph-mon[35787]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T16:41:32.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:32 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi092 2024-05-14T16:41:32.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:32 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi092 to 17088M 2024-05-14T16:41:32.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:32 smithi069 ceph-mon[31507]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T16:41:32.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:32 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi092 2024-05-14T16:41:32.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:32 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi092 to 17088M 2024-05-14T16:41:32.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:32 smithi092 ceph-mon[35763]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T16:41:33.542 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:33 smithi203 ceph-mon[35787]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:33.542 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:41:33.542 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:41:33.542 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:33.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:33 smithi069 ceph-mon[31507]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:33.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:41:33.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:41:33.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:33.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:33 smithi092 ceph-mon[35763]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:33.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:41:33.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:41:33.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:34.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:34 smithi069 ceph-mon[31507]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:41:34.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:34 smithi203 ceph-mon[35787]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:41:34.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:34 smithi092 ceph-mon[35763]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:41:35.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:35 smithi069 ceph-mon[31507]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:35.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:35 smithi069 ceph-mon[31507]: from='client.? 172.21.15.203:0/822988450' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4e05d2-6132-476c-9c8e-64526c9bfa08"}]: dispatch 2024-05-14T16:41:35.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:35 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4e05d2-6132-476c-9c8e-64526c9bfa08"}]: dispatch 2024-05-14T16:41:35.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:35 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f4e05d2-6132-476c-9c8e-64526c9bfa08"}]': finished 2024-05-14T16:41:35.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:35 smithi069 ceph-mon[31507]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T16:41:35.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:35.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:35 smithi203 ceph-mon[35787]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:35.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:35 smithi203 ceph-mon[35787]: from='client.? 172.21.15.203:0/822988450' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4e05d2-6132-476c-9c8e-64526c9bfa08"}]: dispatch 2024-05-14T16:41:35.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:35 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4e05d2-6132-476c-9c8e-64526c9bfa08"}]: dispatch 2024-05-14T16:41:35.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:35 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f4e05d2-6132-476c-9c8e-64526c9bfa08"}]': finished 2024-05-14T16:41:35.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:35 smithi203 ceph-mon[35787]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T16:41:35.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:35.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:35 smithi092 ceph-mon[35763]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:35.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:35 smithi092 ceph-mon[35763]: from='client.? 172.21.15.203:0/822988450' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4e05d2-6132-476c-9c8e-64526c9bfa08"}]: dispatch 2024-05-14T16:41:35.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:35 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4e05d2-6132-476c-9c8e-64526c9bfa08"}]: dispatch 2024-05-14T16:41:35.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:35 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f4e05d2-6132-476c-9c8e-64526c9bfa08"}]': finished 2024-05-14T16:41:35.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:35 smithi092 ceph-mon[35763]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T16:41:35.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:36.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:36 smithi069 ceph-mon[31507]: from='client.? 172.21.15.203:0/1567546305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:41:36.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:36 smithi203 ceph-mon[35787]: from='client.? 172.21.15.203:0/1567546305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:41:36.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:36 smithi092 ceph-mon[35763]: from='client.? 172.21.15.203:0/1567546305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:41:37.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:37 smithi069 ceph-mon[31507]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:37.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:37 smithi203 ceph-mon[35787]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:37.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:37 smithi092 ceph-mon[35763]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:39.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:39 smithi203 ceph-mon[35787]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:39.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:39 smithi069 ceph-mon[31507]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:39.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:39 smithi092 ceph-mon[35763]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:41.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:41 smithi069 ceph-mon[31507]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:41.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:41 smithi203 ceph-mon[35787]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:41.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:41 smithi092 ceph-mon[35763]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:42.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:42 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T16:41:42.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:42 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:42.705 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:42 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T16:41:42.705 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:42 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:42.749 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:42 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T16:41:42.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:42 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:43 smithi069 ceph-mon[31507]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:43 smithi069 ceph-mon[31507]: Deploying daemon osd.2 on smithi203 2024-05-14T16:41:43.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:43 smithi203 ceph-mon[35787]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:43.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:43 smithi203 ceph-mon[35787]: Deploying daemon osd.2 on smithi203 2024-05-14T16:41:43.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:43 smithi092 ceph-mon[35763]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:43.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:43 smithi092 ceph-mon[35763]: Deploying daemon osd.2 on smithi203 2024-05-14T16:41:45.582 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:45 smithi203 ceph-mon[35787]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:45.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:45 smithi069 ceph-mon[31507]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:45.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:45 smithi092 ceph-mon[35763]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:47.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:47 smithi203 ceph-mon[35787]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:47.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:47.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:47.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:47.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:47 smithi069 ceph-mon[31507]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:47.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:47.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:47.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:47.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:47 smithi092 ceph-mon[35763]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:47.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:47.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:47.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:49.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:49 smithi069 ceph-mon[31507]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:49.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:49 smithi203 ceph-mon[35787]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:49.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:49 smithi092 ceph-mon[35763]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:50.610 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.610+0000 7faa717fa700 1 -- 172.21.15.203:0/4243843163 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faa74063980 con 0x7faa600b2870 2024-05-14T16:41:50.611 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 2 on host 'smithi203' 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.612+0000 7faa7a8cb700 1 -- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7faa600b2870 msgr2=0x7faa600b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.612+0000 7faa7a8cb700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7faa600b2870 0x7faa600b4d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faa64005fd0 tx=0x7faa6400a560 comp rx=0 tx=0).stop 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.612+0000 7faa7a8cb700 1 -- 172.21.15.203:0/4243843163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa741035a0 msgr2=0x7faa741995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.612+0000 7faa7a8cb700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa741035a0 0x7faa741995d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00c9c0 tx=0x7faa5c00cd80 comp rx=0 tx=0).stop 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.612+0000 7faa7a8cb700 1 -- 172.21.15.203:0/4243843163 shutdown_connections 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.612+0000 7faa7a8cb700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7faa600b2870 0x7faa600b4d30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.612+0000 7faa7a8cb700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa74105140 0x7faa741a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.612+0000 7faa7a8cb700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa741035a0 0x7faa741995d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.612+0000 7faa7a8cb700 1 --2- 172.21.15.203:0/4243843163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa741047a0 0x7faa74199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.613+0000 7faa7a8cb700 1 -- 172.21.15.203:0/4243843163 >> 172.21.15.203:0/4243843163 conn(0x7faa740feb40 msgr2=0x7faa74107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.613+0000 7faa7a8cb700 1 -- 172.21.15.203:0/4243843163 shutdown_connections 2024-05-14T16:41:50.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:41:50.613+0000 7faa7a8cb700 1 -- 172.21.15.203:0/4243843163 wait complete. 2024-05-14T16:41:50.906 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:50 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:50.906 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:50 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:50.906 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:50 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:50.906 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:50 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:50.906 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:50 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:50.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:50 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:50.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:50 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:50.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:50 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:50.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:50 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:50.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:50 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:51.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:50 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:51.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:50 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:51.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:50 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:51.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:50 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:51.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:50 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:51.208 DEBUG:teuthology.orchestra.run.smithi203:osd.2> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.2.service 2024-05-14T16:41:51.210 INFO:tasks.cephadm:Deploying osd.3 on smithi069 with /dev/vg_nvme/lv_3... 2024-05-14T16:41:51.210 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T16:41:51.455 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:41:51.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:51 smithi069 ceph-mon[31507]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:51.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:51 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:51.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:51 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:51.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:51 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:51.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:51 smithi069 ceph-mon[31507]: from='osd.2 [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:41:51.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:51 smithi069 ceph-mon[31507]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:41:51.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:51 smithi203 ceph-mon[35787]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:51.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:51 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:51.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:51 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:51.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:51 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:51.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:51 smithi203 ceph-mon[35787]: from='osd.2 [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:41:51.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:51 smithi203 ceph-mon[35787]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:41:52.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:51 smithi092 ceph-mon[35763]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:52.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:51 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:52.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:51 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:52.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:51 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:41:52.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:51 smithi092 ceph-mon[35763]: from='osd.2 [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:41:52.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:51 smithi092 ceph-mon[35763]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:41:52.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:52 smithi069 ceph-mon[31507]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T16:41:52.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:52 smithi069 ceph-mon[31507]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T16:41:52.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:52 smithi069 ceph-mon[31507]: from='osd.2 [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:41:52.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:52 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:52.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:52 smithi069 ceph-mon[31507]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:41:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:52 smithi203 ceph-mon[35787]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T16:41:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:52 smithi203 ceph-mon[35787]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T16:41:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:52 smithi203 ceph-mon[35787]: from='osd.2 [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:41:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:52 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:52 smithi203 ceph-mon[35787]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:41:52.960 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 16:41:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2[40916]: 2024-05-14T16:41:52.616+0000 7fc9d7ed5700 -1 osd.2 0 waiting for initial osdmap 2024-05-14T16:41:52.960 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 16:41:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2[40916]: 2024-05-14T16:41:52.620+0000 7fc9d3cc8700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:41:53.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:52 smithi092 ceph-mon[35763]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T16:41:53.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:52 smithi092 ceph-mon[35763]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T16:41:53.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:52 smithi092 ceph-mon[35763]: from='osd.2 [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:41:53.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:52 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:53.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:52 smithi092 ceph-mon[35763]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:41:53.465 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T16:41:53.465 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T16:41:53.465 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-14T16:41:53.465 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-14T16:41:53.465 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011092 s, 945 MB/s 2024-05-14T16:41:53.465 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-14T16:41:53.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:53 smithi203 ceph-mon[35787]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:53.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:53 smithi203 ceph-mon[35787]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:41:53.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:53 smithi203 ceph-mon[35787]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T16:41:53.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:53 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:53.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:53 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:53.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:53 smithi069 ceph-mon[31507]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:53.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:53 smithi069 ceph-mon[31507]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:41:53.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:53 smithi069 ceph-mon[31507]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T16:41:53.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:53 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:53.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:53 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:54.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:53 smithi092 ceph-mon[35763]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:41:54.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:53 smithi092 ceph-mon[35763]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:41:54.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:53 smithi092 ceph-mon[35763]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T16:41:54.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:53 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:54.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:53 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:54.095 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2024-05-14T16:41:54.339 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: osd.2 [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] boot 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: osd.2 [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] boot 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:55.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:41:55.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: osd.2 [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] boot 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:41:55.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:55.704 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 16:41:55 smithi069 sudo[47988]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:41:55.704 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 16:41:55 smithi069 sudo[47988]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:55.704 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 16:41:55 smithi069 sudo[47988]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 ceph-mon[31507]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:41:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi203 2024-05-14T16:41:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi203 to 17088M 2024-05-14T16:41:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T16:41:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 ceph-mon[31507]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T16:41:55.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:41:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.805+0000 7f044bdcc700 1 -- 172.21.15.69:0/3574827132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0444101e10 msgr2=0x7f0444104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.805+0000 7f044bdcc700 1 --2- 172.21.15.69:0/3574827132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0444101e10 0x7f0444104240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f044000b0a0 tx=0x7f044000b3b0 comp rx=0 tx=0).stop 2024-05-14T16:41:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.805+0000 7f044bdcc700 1 -- 172.21.15.69:0/3574827132 shutdown_connections 2024-05-14T16:41:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.805+0000 7f044bdcc700 1 --2- 172.21.15.69:0/3574827132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0444101e10 0x7f0444104240 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.805+0000 7f044bdcc700 1 --2- 172.21.15.69:0/3574827132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0444107150 0x7f04441075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.805+0000 7f044bdcc700 1 --2- 172.21.15.69:0/3574827132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0444104780 0x7f0444106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.805+0000 7f044bdcc700 1 -- 172.21.15.69:0/3574827132 >> 172.21.15.69:0/3574827132 conn(0x7f04440fda30 msgr2=0x7f04440ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:41:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.805+0000 7f044bdcc700 1 -- 172.21.15.69:0/3574827132 shutdown_connections 2024-05-14T16:41:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.805+0000 7f044bdcc700 1 -- 172.21.15.69:0/3574827132 wait complete. 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f044bdcc700 1 Processor -- start 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f044bdcc700 1 -- start start 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f044bdcc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0444101e10 0x7f044419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f044bdcc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0444104780 0x7f04441a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f044bdcc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0444107150 0x7f04441a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f044bdcc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0444115580 con 0x7f0444104780 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f044bdcc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0444115400 con 0x7f0444101e10 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f044bdcc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0444115700 con 0x7f0444107150 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f0449367700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0444104780 0x7f04441a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f0449b68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0444101e10 0x7f044419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f0449367700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0444104780 0x7f04441a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48996/0 (socket says 172.21.15.69:48996) 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.806+0000 7f0449367700 1 -- 172.21.15.69:0/1287611040 learned_addr learned my addr 172.21.15.69:0/1287611040 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:41:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.807+0000 7f044a369700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0444107150 0x7f04441a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.807+0000 7f0449367700 1 -- 172.21.15.69:0/1287611040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0444107150 msgr2=0x7f04441a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.807+0000 7f0449367700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0444107150 0x7f04441a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.807+0000 7f0449367700 1 -- 172.21.15.69:0/1287611040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0444101e10 msgr2=0x7f044419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:41:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.807+0000 7f0449367700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0444101e10 0x7f044419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:41:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.807+0000 7f0449367700 1 -- 172.21.15.69:0/1287611040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04441a78c0 con 0x7f0444104780 2024-05-14T16:41:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.807+0000 7f0449b68700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0444101e10 0x7f044419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:41:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.807+0000 7f0449367700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0444104780 0x7f04441a01b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f043400d980 tx=0x7f043400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:41:55.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.809+0000 7f043affd700 1 -- 172.21.15.69:0/1287611040 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434016070 con 0x7f0444104780 2024-05-14T16:41:55.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.809+0000 7f044bdcc700 1 -- 172.21.15.69:0/1287611040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04441a7bb0 con 0x7f0444104780 2024-05-14T16:41:55.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.809+0000 7f044bdcc700 1 -- 172.21.15.69:0/1287611040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04441a8140 con 0x7f0444104780 2024-05-14T16:41:55.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.809+0000 7f043affd700 1 -- 172.21.15.69:0/1287611040 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0434004830 con 0x7f0444104780 2024-05-14T16:41:55.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.809+0000 7f043affd700 1 -- 172.21.15.69:0/1287611040 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434005540 con 0x7f0444104780 2024-05-14T16:41:55.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.810+0000 7f044bdcc700 1 -- 172.21.15.69:0/1287611040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f044406a230 con 0x7f0444104780 2024-05-14T16:41:55.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.811+0000 7f043affd700 1 -- 172.21.15.69:0/1287611040 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f0434004220 con 0x7f0444104780 2024-05-14T16:41:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.812+0000 7f043affd700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f04300b27f0 0x7f04300b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:41:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.812+0000 7f0449b68700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f04300b27f0 0x7f04300b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:41:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.812+0000 7f043affd700 1 -- 172.21.15.69:0/1287611040 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f04340e10a0 con 0x7f0444104780 2024-05-14T16:41:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.812+0000 7f0449b68700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f04300b27f0 0x7f04300b4cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04441a1150 tx=0x7f0440009d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:41:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:55.816+0000 7f043affd700 1 -- 172.21.15.69:0/1287611040 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f043401f020 con 0x7f0444104780 2024-05-14T16:41:55.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 ceph-mon[35787]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:41:55.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi203 2024-05-14T16:41:55.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi203 to 17088M 2024-05-14T16:41:55.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T16:41:55.920 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 ceph-mon[35787]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T16:41:55.920 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:41:55.920 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44969]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:41:55.920 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44969]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:55.920 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44969]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:55.920 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44969]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:55.920 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44972]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:41:55.920 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44972]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:55.920 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44972]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:55.920 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44972]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:55.957 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 ceph-mon[35763]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:41:55.957 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi203 2024-05-14T16:41:55.958 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi203 to 17088M 2024-05-14T16:41:55.958 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T16:41:55.958 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 ceph-mon[35763]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T16:41:55.958 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:41:55.958 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44447]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:41:55.958 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44447]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:55.958 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44447]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:55.958 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44447]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:55.958 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44450]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:41:55.958 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44450]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:55.958 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44450]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:55.958 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44450]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:56.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:56.042+0000 7f044bdcc700 1 -- 172.21.15.69:0/1287611040 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0444063980 con 0x7f04300b27f0 2024-05-14T16:41:56.042 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 sudo[48016]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:41:56.042 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 sudo[48016]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:56.042 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 sudo[48016]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:56.042 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 sudo[48016]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:56.042 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 sudo[48019]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:41:56.042 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 sudo[48019]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:56.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 sudo[48019]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:56.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:55 smithi069 sudo[48019]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:56.043 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 16:41:55 smithi069 sudo[47988]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:56.043 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 16:41:55 smithi069 sudo[47993]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:41:56.043 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 16:41:55 smithi069 sudo[47993]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:56.043 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 16:41:55 smithi069 sudo[47993]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:56.043 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 16:41:55 smithi069 sudo[47993]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:56.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44975]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:41:56.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44975]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:56.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44975]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:56.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44975]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:56.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44978]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:41:56.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44978]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:56.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44978]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:56.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:55 smithi203 sudo[44978]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44453]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:41:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44453]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44453]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44453]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44456]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:41:56.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44456]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:41:56.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44456]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:41:56.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:55 smithi092 sudo[44456]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:41:56.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:41:56.817+0000 7f043affd700 1 -- 172.21.15.69:0/1287611040 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f04340a5e00 con 0x7f0444104780 2024-05-14T16:41:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T16:41:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T16:41:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:41:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:41:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:41:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:41:56.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:56.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T16:41:56.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T16:41:56.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:56.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:56.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:56.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:41:56.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:41:56.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:41:56.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:57.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:41:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:41:57.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:41:57.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:41:57.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:41:57.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:41:57.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:41:57.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:41:57.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:41:57.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:57 smithi069 ceph-mon[31507]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:41:57.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:57 smithi069 ceph-mon[31507]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:41:57.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:57 smithi069 ceph-mon[31507]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T16:41:57.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:57 smithi069 ceph-mon[31507]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T16:41:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:57 smithi203 ceph-mon[35787]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:41:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:57 smithi203 ceph-mon[35787]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:41:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:57 smithi203 ceph-mon[35787]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T16:41:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:57 smithi203 ceph-mon[35787]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T16:41:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:57 smithi092 ceph-mon[35763]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:41:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:57 smithi092 ceph-mon[35763]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:41:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:57 smithi092 ceph-mon[35763]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T16:41:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:57 smithi092 ceph-mon[35763]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T16:41:59.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:59 smithi069 ceph-mon[31507]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:41:59.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:59 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2050428794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097ec507-bd81-441b-9b33-61e9c55c9f7e"}]: dispatch 2024-05-14T16:41:59.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:59 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2050428794' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "097ec507-bd81-441b-9b33-61e9c55c9f7e"}]': finished 2024-05-14T16:41:59.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:59 smithi069 ceph-mon[31507]: osdmap e22: 4 total, 3 up, 4 in 2024-05-14T16:41:59.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:59 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:41:59.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:41:59 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/207562786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:41:59.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:59 smithi203 ceph-mon[35787]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:41:59.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:59 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2050428794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097ec507-bd81-441b-9b33-61e9c55c9f7e"}]: dispatch 2024-05-14T16:41:59.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:59 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2050428794' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "097ec507-bd81-441b-9b33-61e9c55c9f7e"}]': finished 2024-05-14T16:41:59.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:59 smithi203 ceph-mon[35787]: osdmap e22: 4 total, 3 up, 4 in 2024-05-14T16:41:59.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:59 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:41:59.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:41:59 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/207562786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:42:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:59 smithi092 ceph-mon[35763]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:59 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2050428794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097ec507-bd81-441b-9b33-61e9c55c9f7e"}]: dispatch 2024-05-14T16:42:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:59 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2050428794' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "097ec507-bd81-441b-9b33-61e9c55c9f7e"}]': finished 2024-05-14T16:42:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:59 smithi092 ceph-mon[35763]: osdmap e22: 4 total, 3 up, 4 in 2024-05-14T16:42:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:59 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:41:59 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/207562786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:42:01.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:01 smithi069 ceph-mon[31507]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:01 smithi203 ceph-mon[35787]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:01 smithi092 ceph-mon[35763]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:03.846 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:03 smithi069 ceph-mon[31507]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:03.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:03 smithi203 ceph-mon[35787]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:03 smithi092 ceph-mon[35763]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:05.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:05 smithi069 ceph-mon[31507]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:05 smithi203 ceph-mon[35787]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:05 smithi092 ceph-mon[35763]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:06.819 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:06 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T16:42:06.819 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:06 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:06.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:06 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T16:42:06.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:06 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:07.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:06 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T16:42:07.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:06 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:07.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:07 smithi069 ceph-mon[31507]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:07.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:07 smithi069 ceph-mon[31507]: Deploying daemon osd.3 on smithi069 2024-05-14T16:42:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:07 smithi203 ceph-mon[35787]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:07 smithi203 ceph-mon[35787]: Deploying daemon osd.3 on smithi069 2024-05-14T16:42:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:07 smithi092 ceph-mon[35763]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:07 smithi092 ceph-mon[35763]: Deploying daemon osd.3 on smithi069 2024-05-14T16:42:08.822 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:08 smithi069 ceph-mon[31507]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:09.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:08 smithi092 ceph-mon[35763]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:09.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:08 smithi203 ceph-mon[35787]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:10.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:10 smithi203 ceph-mon[35787]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:11.118 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:10 smithi069 ceph-mon[31507]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:11.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:10 smithi092 ceph-mon[35763]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:12.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:12 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:12.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:12 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:12.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:12 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:12.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:12 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:12.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:12 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:12.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:12 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:12.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:12 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:12.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:12 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:12.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:12 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:13.665 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:13 smithi069 ceph-mon[31507]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:13.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:13 smithi203 ceph-mon[35787]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:13.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:13 smithi092 ceph-mon[35763]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:14.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:14.998+0000 7f043affd700 1 -- 172.21.15.69:0/1287611040 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0444063980 con 0x7f04300b27f0 2024-05-14T16:42:14.999 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 3 on host 'smithi069' 2024-05-14T16:42:15.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.001+0000 7f044bdcc700 1 -- 172.21.15.69:0/1287611040 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f04300b27f0 msgr2=0x7f04300b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:42:15.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.001+0000 7f044bdcc700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f04300b27f0 0x7f04300b4cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04441a1150 tx=0x7f0440009d80 comp rx=0 tx=0).stop 2024-05-14T16:42:15.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.001+0000 7f044bdcc700 1 -- 172.21.15.69:0/1287611040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0444104780 msgr2=0x7f04441a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:42:15.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.001+0000 7f044bdcc700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0444104780 0x7f04441a01b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f043400d980 tx=0x7f043400dc90 comp rx=0 tx=0).stop 2024-05-14T16:42:15.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.001+0000 7f044bdcc700 1 -- 172.21.15.69:0/1287611040 shutdown_connections 2024-05-14T16:42:15.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.001+0000 7f044bdcc700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0444107150 0x7f04441a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:15.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.001+0000 7f044bdcc700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0444101e10 0x7f044419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:15.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.001+0000 7f044bdcc700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f04300b27f0 0x7f04300b4cb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:15.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.001+0000 7f044bdcc700 1 --2- 172.21.15.69:0/1287611040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0444104780 0x7f04441a01b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:15.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.001+0000 7f044bdcc700 1 -- 172.21.15.69:0/1287611040 >> 172.21.15.69:0/1287611040 conn(0x7f04440fda30 msgr2=0x7f04441055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:42:15.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.002+0000 7f044bdcc700 1 -- 172.21.15.69:0/1287611040 shutdown_connections 2024-05-14T16:42:15.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:42:15.002+0000 7f044bdcc700 1 -- 172.21.15.69:0/1287611040 wait complete. 2024-05-14T16:42:15.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:15 smithi069 ceph-mon[31507]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:15.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:15.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:15.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:15.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:15 smithi203 ceph-mon[35787]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:15.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:15.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:15.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:15.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:15 smithi092 ceph-mon[35763]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:15.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:15.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:15.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:15.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:15.711 DEBUG:teuthology.orchestra.run.smithi069:osd.3> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.3.service 2024-05-14T16:42:15.713 INFO:tasks.cephadm:Deploying osd.4 on smithi092 with /dev/vg_nvme/lv_3... 2024-05-14T16:42:15.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T16:42:15.950 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:42:16.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:16 smithi069 ceph-mon[31507]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T16:42:16.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:16 smithi092 ceph-mon[35763]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T16:42:16.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:16 smithi203 ceph-mon[35787]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T16:42:17.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:17 smithi069 ceph-mon[31507]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:17.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:17 smithi069 ceph-mon[31507]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T16:42:17.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:17 smithi069 ceph-mon[31507]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T16:42:17.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:17 smithi069 ceph-mon[31507]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:42:17.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:17 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:17.204 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 16:42:17 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3[50250]: 2024-05-14T16:42:17.145+0000 7f8c64001700 -1 osd.3 0 waiting for initial osdmap 2024-05-14T16:42:17.204 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 16:42:17 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3[50250]: 2024-05-14T16:42:17.150+0000 7f8c5fdf4700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:42:17.344 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:17 smithi092 ceph-mon[35763]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:17.344 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:17 smithi092 ceph-mon[35763]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T16:42:17.344 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:17 smithi092 ceph-mon[35763]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T16:42:17.344 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:17 smithi092 ceph-mon[35763]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:42:17.344 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:17 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:17 smithi203 ceph-mon[35787]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:17 smithi203 ceph-mon[35787]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T16:42:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:17 smithi203 ceph-mon[35787]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T16:42:17.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:17 smithi203 ceph-mon[35787]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:42:17.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:17 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T16:42:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T16:42:17.927 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-14T16:42:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-14T16:42:17.927 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114594 s, 915 MB/s 2024-05-14T16:42:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-14T16:42:18.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:18 smithi069 ceph-mon[31507]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:42:18.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:18 smithi069 ceph-mon[31507]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T16:42:18.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:18 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:18.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:18 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:18.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:18 smithi203 ceph-mon[35787]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:42:18.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:18 smithi203 ceph-mon[35787]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T16:42:18.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:18 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:18.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:18 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:18.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:18 smithi092 ceph-mon[35763]: from='osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:42:18.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:18 smithi092 ceph-mon[35763]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T16:42:18.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:18 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:18.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:18 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:18.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-05-14T16:42:18.846 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] boot 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: osdmap e25: 4 total, 4 up, 4 in 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:19.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:19.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:19.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:19.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:42:19.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:42:19.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] boot 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: osdmap e25: 4 total, 4 up, 4 in 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:19.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: osd.3 [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] boot 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: osdmap e25: 4 total, 4 up, 4 in 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:19.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:19.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:19.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:19.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:19.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:20.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 -- 172.21.15.92:0/370714497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf01035a0 msgr2=0x7fcaf01039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:42:20.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 --2- 172.21.15.92:0/370714497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf01035a0 0x7fcaf01039c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0099c0 tx=0x7fcaec009cd0 comp rx=0 tx=0).stop 2024-05-14T16:42:20.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 -- 172.21.15.92:0/370714497 shutdown_connections 2024-05-14T16:42:20.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 --2- 172.21.15.92:0/370714497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf0105140 0x7fcaf010b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:20.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 --2- 172.21.15.92:0/370714497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf01035a0 0x7fcaf01039c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:20.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 --2- 172.21.15.92:0/370714497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf01047a0 0x7fcaf0104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 -- 172.21.15.92:0/370714497 >> 172.21.15.92:0/370714497 conn(0x7fcaf00feb40 msgr2=0x7fcaf0100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 -- 172.21.15.92:0/370714497 shutdown_connections 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 -- 172.21.15.92:0/370714497 wait complete. 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 Processor -- start 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 -- start start 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf01035a0 0x7fcaf019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf01047a0 0x7fcaf019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf0105140 0x7fcaf01a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf010f090 con 0x7fcaf01035a0 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf010ef10 con 0x7fcaf01047a0 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.340+0000 7fcaf7e90700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf010f210 con 0x7fcaf0105140 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf542b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf01047a0 0x7fcaf019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:42:20.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf542b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf01047a0 0x7fcaf019bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48266/0 (socket says 172.21.15.92:48266) 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf542b700 1 -- 172.21.15.92:0/4244695935 learned_addr learned my addr 172.21.15.92:0/4244695935 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf5c2c700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf01035a0 0x7fcaf019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf642d700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf0105140 0x7fcaf01a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf542b700 1 -- 172.21.15.92:0/4244695935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf0105140 msgr2=0x7fcaf01a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf542b700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf0105140 0x7fcaf01a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf542b700 1 -- 172.21.15.92:0/4244695935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf01035a0 msgr2=0x7fcaf019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf542b700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf01035a0 0x7fcaf019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf542b700 1 -- 172.21.15.92:0/4244695935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf01a3600 con 0x7fcaf01047a0 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf542b700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf01047a0 0x7fcaf019bee0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcae000b530 tx=0x7fcae000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcae6ffd700 1 -- 172.21.15.92:0/4244695935 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0015020 con 0x7fcaf01047a0 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcae6ffd700 1 -- 172.21.15.92:0/4244695935 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae0011ce0 con 0x7fcaf01047a0 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcae6ffd700 1 -- 172.21.15.92:0/4244695935 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae000f2e0 con 0x7fcaf01047a0 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf7e90700 1 -- 172.21.15.92:0/4244695935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf01a38f0 con 0x7fcaf01047a0 2024-05-14T16:42:20.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.341+0000 7fcaf7e90700 1 -- 172.21.15.92:0/4244695935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf01a3de0 con 0x7fcaf01047a0 2024-05-14T16:42:20.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.342+0000 7fcaf7e90700 1 -- 172.21.15.92:0/4244695935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf006a260 con 0x7fcaf01047a0 2024-05-14T16:42:20.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.343+0000 7fcae6ffd700 1 -- 172.21.15.92:0/4244695935 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcae0018020 con 0x7fcaf01047a0 2024-05-14T16:42:20.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.344+0000 7fcae6ffd700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fcadc0b2900 0x7fcadc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:42:20.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.345+0000 7fcaf5c2c700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fcadc0b2900 0x7fcadc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:42:20.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.345+0000 7fcae6ffd700 1 -- 172.21.15.92:0/4244695935 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7fcae00e0d40 con 0x7fcaf01047a0 2024-05-14T16:42:20.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.345+0000 7fcaf5c2c700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fcadc0b2900 0x7fcadc0b4dc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcaec006010 tx=0x7fcaec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:42:20.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.348+0000 7fcae6ffd700 1 -- 172.21.15.92:0/4244695935 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcaf006a260 con 0x7fcaf01047a0 2024-05-14T16:42:20.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:20.562+0000 7fcaf7e90700 1 -- 172.21.15.92:0/4244695935 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcaf0076e90 con 0x7fcadc0b2900 2024-05-14T16:42:20.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:20 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi069 2024-05-14T16:42:20.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:20 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi069 to 8544M 2024-05-14T16:42:20.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:20 smithi092 ceph-mon[35763]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T16:42:20.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:20 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi069 2024-05-14T16:42:20.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:20 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi069 to 8544M 2024-05-14T16:42:20.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:20 smithi069 ceph-mon[31507]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T16:42:20.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:20 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi069 2024-05-14T16:42:20.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:20 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi069 to 8544M 2024-05-14T16:42:20.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:20 smithi203 ceph-mon[35787]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T16:42:21.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:21 smithi092 ceph-mon[35763]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:21.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:21 smithi092 ceph-mon[35763]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:42:21.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:21 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:42:21.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:21 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:42:21.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:21 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:21.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:21 smithi069 ceph-mon[31507]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:21.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:21 smithi069 ceph-mon[31507]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:42:21.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:21 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:42:21.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:21 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:42:21.856 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:21 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:21.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:21 smithi203 ceph-mon[35787]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:21.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:21 smithi203 ceph-mon[35787]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:42:21.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:21 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:42:21.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:21 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:42:21.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:21 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:23.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:23 smithi203 ceph-mon[35787]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-14T16:42:23.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:23 smithi203 ceph-mon[35787]: from='client.? 172.21.15.92:0/410969813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76556e0f-a08a-4c01-a662-f84af601cf5c"}]: dispatch 2024-05-14T16:42:23.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:23 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76556e0f-a08a-4c01-a662-f84af601cf5c"}]: dispatch 2024-05-14T16:42:23.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:23 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76556e0f-a08a-4c01-a662-f84af601cf5c"}]': finished 2024-05-14T16:42:23.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:23 smithi203 ceph-mon[35787]: osdmap e28: 5 total, 4 up, 5 in 2024-05-14T16:42:23.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:23 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:23.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:23 smithi092 ceph-mon[35763]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-14T16:42:23.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:23 smithi092 ceph-mon[35763]: from='client.? 172.21.15.92:0/410969813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76556e0f-a08a-4c01-a662-f84af601cf5c"}]: dispatch 2024-05-14T16:42:23.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:23 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76556e0f-a08a-4c01-a662-f84af601cf5c"}]: dispatch 2024-05-14T16:42:23.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:23 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76556e0f-a08a-4c01-a662-f84af601cf5c"}]': finished 2024-05-14T16:42:23.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:23 smithi092 ceph-mon[35763]: osdmap e28: 5 total, 4 up, 5 in 2024-05-14T16:42:23.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:23 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:23.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:23 smithi069 ceph-mon[31507]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-14T16:42:23.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:23 smithi069 ceph-mon[31507]: from='client.? 172.21.15.92:0/410969813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76556e0f-a08a-4c01-a662-f84af601cf5c"}]: dispatch 2024-05-14T16:42:23.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:23 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76556e0f-a08a-4c01-a662-f84af601cf5c"}]: dispatch 2024-05-14T16:42:23.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:23 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76556e0f-a08a-4c01-a662-f84af601cf5c"}]': finished 2024-05-14T16:42:23.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:23 smithi069 ceph-mon[31507]: osdmap e28: 5 total, 4 up, 5 in 2024-05-14T16:42:23.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:23 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:24.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:24 smithi092 ceph-mon[35763]: from='client.? 172.21.15.92:0/4201049087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:42:24.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:24 smithi069 ceph-mon[31507]: from='client.? 172.21.15.92:0/4201049087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:42:24.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:24 smithi203 ceph-mon[35787]: from='client.? 172.21.15.92:0/4201049087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:42:25.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:25 smithi092 ceph-mon[35763]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-14T16:42:25.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:25 smithi069 ceph-mon[31507]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-14T16:42:25.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:25 smithi203 ceph-mon[35787]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-14T16:42:27.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:27 smithi092 ceph-mon[35763]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-14T16:42:27.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:27 smithi069 ceph-mon[31507]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-14T16:42:27.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:27 smithi203 ceph-mon[35787]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-14T16:42:29.669 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:29 smithi092 ceph-mon[35763]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-14T16:42:29.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:29 smithi069 ceph-mon[31507]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-14T16:42:29.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:29 smithi203 ceph-mon[35787]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-14T16:42:30.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T16:42:30.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:30.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T16:42:30.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:30.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T16:42:30.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:31.664 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:31 smithi092 ceph-mon[35763]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:42:31.665 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:31 smithi092 ceph-mon[35763]: Deploying daemon osd.4 on smithi092 2024-05-14T16:42:31.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:31 smithi069 ceph-mon[31507]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:42:31.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:31 smithi069 ceph-mon[31507]: Deploying daemon osd.4 on smithi092 2024-05-14T16:42:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:31 smithi203 ceph-mon[35787]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:42:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:31 smithi203 ceph-mon[35787]: Deploying daemon osd.4 on smithi092 2024-05-14T16:42:33.744 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:33 smithi092 ceph-mon[35763]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:33.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:33 smithi069 ceph-mon[31507]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:33.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:33 smithi203 ceph-mon[35787]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:35.695 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:35 smithi092 ceph-mon[35763]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:35.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:35 smithi069 ceph-mon[31507]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:35.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:35 smithi203 ceph-mon[35787]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:37.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:36 smithi092 ceph-mon[35763]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:37.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:37.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:37.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:37.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:36 smithi069 ceph-mon[31507]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:37.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:37.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:37.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:37.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:36 smithi203 ceph-mon[35787]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:37.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:37.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:37.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:39.185 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:38 smithi092 ceph-mon[35763]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:39.228 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 4 on host 'smithi092' 2024-05-14T16:42:39.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.228+0000 7fcae6ffd700 1 -- 172.21.15.92:0/4244695935 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcaf0076e90 con 0x7fcadc0b2900 2024-05-14T16:42:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 -- 172.21.15.92:0/4244695935 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fcadc0b2900 msgr2=0x7fcadc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:42:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fcadc0b2900 0x7fcadc0b4dc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcaec006010 tx=0x7fcaec000bc0 comp rx=0 tx=0).stop 2024-05-14T16:42:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 -- 172.21.15.92:0/4244695935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf01047a0 msgr2=0x7fcaf019bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:42:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf01047a0 0x7fcaf019bee0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcae000b530 tx=0x7fcae000b840 comp rx=0 tx=0).stop 2024-05-14T16:42:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 -- 172.21.15.92:0/4244695935 shutdown_connections 2024-05-14T16:42:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fcadc0b2900 0x7fcadc0b4dc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf01035a0 0x7fcaf019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf0105140 0x7fcaf01a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 --2- 172.21.15.92:0/4244695935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaf01047a0 0x7fcaf019bee0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 -- 172.21.15.92:0/4244695935 >> 172.21.15.92:0/4244695935 conn(0x7fcaf00feb40 msgr2=0x7fcaf0075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:42:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 -- 172.21.15.92:0/4244695935 shutdown_connections 2024-05-14T16:42:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:42:39.230+0000 7fcaf7e90700 1 -- 172.21.15.92:0/4244695935 wait complete. 2024-05-14T16:42:39.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:38 smithi069 ceph-mon[31507]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:39.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:38 smithi203 ceph-mon[35787]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:39.878 DEBUG:teuthology.orchestra.run.smithi092:osd.4> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.4.service 2024-05-14T16:42:39.880 INFO:tasks.cephadm:Deploying osd.5 on smithi203 with /dev/vg_nvme/lv_3... 2024-05-14T16:42:39.880 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T16:42:40.127 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:42:40.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:40 smithi092 ceph-mon[35763]: from='osd.4 [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:42:40.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:40 smithi092 ceph-mon[35763]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:42:40.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:40 smithi203 ceph-mon[35787]: from='osd.4 [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:42:40.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:40 smithi203 ceph-mon[35787]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:42:40.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:40.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:40 smithi069 ceph-mon[31507]: from='osd.4 [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:42:40.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:40 smithi069 ceph-mon[31507]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:42:41.502 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:41 smithi203 ceph-mon[35787]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:41.503 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:41 smithi203 ceph-mon[35787]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T16:42:41.503 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:41 smithi203 ceph-mon[35787]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T16:42:41.503 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:41.503 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:41 smithi203 ceph-mon[35787]: from='osd.4 [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:42:41.503 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:41 smithi203 ceph-mon[35787]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:42:41.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:41 smithi069 ceph-mon[31507]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:41.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:41 smithi069 ceph-mon[31507]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T16:42:41.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:41 smithi069 ceph-mon[31507]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T16:42:41.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:41.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:41 smithi069 ceph-mon[31507]: from='osd.4 [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:42:41.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:41 smithi069 ceph-mon[31507]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:42:41.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:41 smithi092 ceph-mon[35763]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:41.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:41 smithi092 ceph-mon[35763]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T16:42:41.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:41 smithi092 ceph-mon[35763]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T16:42:41.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:41.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:41 smithi092 ceph-mon[35763]: from='osd.4 [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:42:41.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:41 smithi092 ceph-mon[35763]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:42:42.058 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T16:42:42.058 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T16:42:42.058 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-05-14T16:42:42.058 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-05-14T16:42:42.058 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113579 s, 923 MB/s 2024-05-14T16:42:42.058 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-05-14T16:42:42.210 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 16:42:41 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T16:42:41.750+0000 7fa25fa80700 -1 osd.4 0 waiting for initial osdmap 2024-05-14T16:42:42.210 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 16:42:41 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T16:42:41.756+0000 7fa25786b700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:42:42.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:42 smithi092 ceph-mon[35763]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:42:42.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:42 smithi092 ceph-mon[35763]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T16:42:42.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:42 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:42.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:42 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:42.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:42 smithi092 ceph-mon[35763]: from='osd.4 ' entity='osd.4' 2024-05-14T16:42:42.672 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:42 smithi203 ceph-mon[35787]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:42:42.672 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:42 smithi203 ceph-mon[35787]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T16:42:42.672 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:42 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:42.672 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:42 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:42.672 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:42 smithi203 ceph-mon[35787]: from='osd.4 ' entity='osd.4' 2024-05-14T16:42:42.701 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_3 2024-05-14T16:42:42.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:42 smithi069 ceph-mon[31507]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:42:42.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:42 smithi069 ceph-mon[31507]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T16:42:42.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:42 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:42.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:42 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:42.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:42 smithi069 ceph-mon[31507]: from='osd.4 ' entity='osd.4' 2024-05-14T16:42:42.946 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:42:43.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:42:43.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:42:43.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:43.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:43.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: osd.4 [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] boot 2024-05-14T16:42:43.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: osdmap e31: 5 total, 5 up, 5 in 2024-05-14T16:42:43.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:43.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:43.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:43.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:43.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:43.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:43.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:43.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:43.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: osd.4 [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] boot 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: osdmap e31: 5 total, 5 up, 5 in 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:43.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:43.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:43.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:43.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:43.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:42:43.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:42:43.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-14T16:42:43.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:43.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: osd.4 [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] boot 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: osdmap e31: 5 total, 5 up, 5 in 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:42:43.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:42:44.504 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.504+0000 7feb85322700 1 -- 172.21.15.203:0/1761554250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb80104030 msgr2=0x7feb8010a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:42:44.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.504+0000 7feb85322700 1 --2- 172.21.15.203:0/1761554250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb80104030 0x7feb8010a8e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb7800b0a0 tx=0x7feb7800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:42:44.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.504+0000 7feb85322700 1 -- 172.21.15.203:0/1761554250 shutdown_connections 2024-05-14T16:42:44.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.504+0000 7feb85322700 1 --2- 172.21.15.203:0/1761554250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb80104030 0x7feb8010a8e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:44.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.504+0000 7feb85322700 1 --2- 172.21.15.203:0/1761554250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb80102490 0x7feb801028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:44.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.504+0000 7feb85322700 1 --2- 172.21.15.203:0/1761554250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb80103690 0x7feb80103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:44.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.504+0000 7feb85322700 1 -- 172.21.15.203:0/1761554250 >> 172.21.15.203:0/1761554250 conn(0x7feb800fda30 msgr2=0x7feb800ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:42:44.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.504+0000 7feb85322700 1 -- 172.21.15.203:0/1761554250 shutdown_connections 2024-05-14T16:42:44.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.504+0000 7feb85322700 1 -- 172.21.15.203:0/1761554250 wait complete. 2024-05-14T16:42:44.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb85322700 1 Processor -- start 2024-05-14T16:42:44.507 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb85322700 1 -- start start 2024-05-14T16:42:44.507 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb85322700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb80102490 0x7feb8006c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:42:44.507 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb85322700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb80103690 0x7feb8019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:42:44.507 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb85322700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb80104030 0x7feb801a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:42:44.507 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb85322700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb8010df10 con 0x7feb80104030 2024-05-14T16:42:44.507 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb85322700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feb8010dd90 con 0x7feb80103690 2024-05-14T16:42:44.507 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb85322700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb8010e090 con 0x7feb80102490 2024-05-14T16:42:44.507 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb7effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb80102490 0x7feb8006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb7effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb80102490 0x7feb8006c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55766/0 (socket says 172.21.15.203:55766) 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb7effd700 1 -- 172.21.15.203:0/121989109 learned_addr learned my addr 172.21.15.203:0/121989109 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb7e7fc700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb80103690 0x7feb8019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.505+0000 7feb7f7fe700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb80104030 0x7feb801a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb7effd700 1 -- 172.21.15.203:0/121989109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb80103690 msgr2=0x7feb8019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb7effd700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb80103690 0x7feb8019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb7effd700 1 -- 172.21.15.203:0/121989109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb80104030 msgr2=0x7feb801a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb7effd700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb80104030 0x7feb801a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb7effd700 1 -- 172.21.15.203:0/121989109 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb801a12c0 con 0x7feb80102490 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb7effd700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb80102490 0x7feb8006c610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb7400d980 tx=0x7feb7400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb67fff700 1 -- 172.21.15.203:0/121989109 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74017030 con 0x7feb80102490 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb85322700 1 -- 172.21.15.203:0/121989109 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb801a15b0 con 0x7feb80102490 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb67fff700 1 -- 172.21.15.203:0/121989109 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb740044d0 con 0x7feb80102490 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb67fff700 1 -- 172.21.15.203:0/121989109 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74003970 con 0x7feb80102490 2024-05-14T16:42:44.508 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.506+0000 7feb85322700 1 -- 172.21.15.203:0/121989109 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb801a1af0 con 0x7feb80102490 2024-05-14T16:42:44.509 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.507+0000 7feb85322700 1 -- 172.21.15.203:0/121989109 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb8006a230 con 0x7feb80102490 2024-05-14T16:42:44.509 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.508+0000 7feb67fff700 1 -- 172.21.15.203:0/121989109 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7feb7401a070 con 0x7feb80102490 2024-05-14T16:42:44.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.509+0000 7feb67fff700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7feb680b2880 0x7feb680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:42:44.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.509+0000 7feb67fff700 1 -- 172.21.15.203:0/121989109 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7feb740e0a50 con 0x7feb80102490 2024-05-14T16:42:44.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.509+0000 7feb7e7fc700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7feb680b2880 0x7feb680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:42:44.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.510+0000 7feb7e7fc700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7feb680b2880 0x7feb680b4d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feb8006cb90 tx=0x7feb7000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:42:44.513 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.514+0000 7feb67fff700 1 -- 172.21.15.203:0/121989109 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb740a97e0 con 0x7feb80102490 2024-05-14T16:42:44.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:44 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi092 2024-05-14T16:42:44.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:44 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi092 to 8544M 2024-05-14T16:42:44.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:44 smithi069 ceph-mon[31507]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T16:42:44.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:44 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi092 2024-05-14T16:42:44.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:44 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi092 to 8544M 2024-05-14T16:42:44.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:44 smithi203 ceph-mon[35787]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T16:42:44.725 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:42:44.725+0000 7feb85322700 1 -- 172.21.15.203:0/121989109 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7feb80063980 con 0x7feb680b2880 2024-05-14T16:42:44.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:44 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi092 2024-05-14T16:42:44.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:44 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi092 to 8544M 2024-05-14T16:42:44.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:44 smithi092 ceph-mon[35763]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T16:42:45.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:45 smithi203 ceph-mon[35787]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:45.602 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:42:45.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:42:45.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:45.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:45 smithi069 ceph-mon[31507]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:45.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:42:45.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:42:45.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:45.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:45 smithi092 ceph-mon[35763]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:45.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:42:45.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:42:45.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:46.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:46 smithi203 ceph-mon[35787]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:42:46.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:46 smithi069 ceph-mon[31507]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:42:46.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:46 smithi092 ceph-mon[35763]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:42:47.700 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:47 smithi203 ceph-mon[35787]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:47.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:47 smithi069 ceph-mon[31507]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:47.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:47 smithi092 ceph-mon[35763]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:48.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:48 smithi069 ceph-mon[31507]: from='client.? 172.21.15.203:0/3758475079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ebc58a-f720-4000-a6aa-a2cfa0223481"}]: dispatch 2024-05-14T16:42:48.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:48 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ebc58a-f720-4000-a6aa-a2cfa0223481"}]: dispatch 2024-05-14T16:42:48.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:48 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23ebc58a-f720-4000-a6aa-a2cfa0223481"}]': finished 2024-05-14T16:42:48.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:48 smithi069 ceph-mon[31507]: osdmap e33: 6 total, 5 up, 6 in 2024-05-14T16:42:48.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:48 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:42:48.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:48 smithi069 ceph-mon[31507]: from='client.? 172.21.15.203:0/3896925015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:42:48.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:48 smithi203 ceph-mon[35787]: from='client.? 172.21.15.203:0/3758475079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ebc58a-f720-4000-a6aa-a2cfa0223481"}]: dispatch 2024-05-14T16:42:48.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:48 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ebc58a-f720-4000-a6aa-a2cfa0223481"}]: dispatch 2024-05-14T16:42:48.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:48 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23ebc58a-f720-4000-a6aa-a2cfa0223481"}]': finished 2024-05-14T16:42:48.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:48 smithi203 ceph-mon[35787]: osdmap e33: 6 total, 5 up, 6 in 2024-05-14T16:42:48.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:48 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:42:48.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:48 smithi203 ceph-mon[35787]: from='client.? 172.21.15.203:0/3896925015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:42:48.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:48 smithi092 ceph-mon[35763]: from='client.? 172.21.15.203:0/3758475079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ebc58a-f720-4000-a6aa-a2cfa0223481"}]: dispatch 2024-05-14T16:42:48.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:48 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ebc58a-f720-4000-a6aa-a2cfa0223481"}]: dispatch 2024-05-14T16:42:48.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:48 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23ebc58a-f720-4000-a6aa-a2cfa0223481"}]': finished 2024-05-14T16:42:48.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:48 smithi092 ceph-mon[35763]: osdmap e33: 6 total, 5 up, 6 in 2024-05-14T16:42:48.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:48 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:42:48.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:48 smithi092 ceph-mon[35763]: from='client.? 172.21.15.203:0/3896925015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:42:49.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:49 smithi069 ceph-mon[31507]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:49.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:49 smithi203 ceph-mon[35787]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:49.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:49 smithi092 ceph-mon[35763]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:51.633 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:51 smithi203 ceph-mon[35787]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:51.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:51 smithi069 ceph-mon[31507]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:51.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:51 smithi092 ceph-mon[35763]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:53.607 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:53 smithi203 ceph-mon[35787]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:53.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:53 smithi069 ceph-mon[31507]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:53.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:53 smithi092 ceph-mon[35763]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:55.501 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:55 smithi203 ceph-mon[35787]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:55.501 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T16:42:55.501 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:55.501 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:55 smithi203 ceph-mon[35787]: Deploying daemon osd.5 on smithi203 2024-05-14T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:55 smithi069 ceph-mon[31507]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:55 smithi069 ceph-mon[31507]: Deploying daemon osd.5 on smithi203 2024-05-14T16:42:55.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:55 smithi092 ceph-mon[35763]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:55.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T16:42:55.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:42:55.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:55 smithi092 ceph-mon[35763]: Deploying daemon osd.5 on smithi203 2024-05-14T16:42:57.630 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:57 smithi203 ceph-mon[35787]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:57.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:57 smithi069 ceph-mon[31507]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:57.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:57 smithi092 ceph-mon[35763]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:59.557 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:42:59 smithi203 ceph-mon[35787]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:59.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:42:59 smithi069 ceph-mon[31507]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:42:59.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:42:59 smithi092 ceph-mon[35763]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:01.408 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:01 smithi203 ceph-mon[35787]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:01.408 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:01 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:01.408 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:01 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:01.409 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:01 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:01.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:01 smithi069 ceph-mon[31507]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:01.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:01 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:01.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:01 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:01.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:01 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:01.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:01 smithi092 ceph-mon[35763]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:01.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:01 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:01.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:01 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:01.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:01 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:03.204 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:03 smithi203 ceph-mon[35787]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:03.296 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 5 on host 'smithi203' 2024-05-14T16:43:03.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.295+0000 7feb67fff700 1 -- 172.21.15.203:0/121989109 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7feb80063980 con 0x7feb680b2880 2024-05-14T16:43:03.299 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 -- 172.21.15.203:0/121989109 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7feb680b2880 msgr2=0x7feb680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:03.299 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7feb680b2880 0x7feb680b4d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feb8006cb90 tx=0x7feb7000b410 comp rx=0 tx=0).stop 2024-05-14T16:43:03.299 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 -- 172.21.15.203:0/121989109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb80102490 msgr2=0x7feb8006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:03.300 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb80102490 0x7feb8006c610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb7400d980 tx=0x7feb7400dc90 comp rx=0 tx=0).stop 2024-05-14T16:43:03.300 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 -- 172.21.15.203:0/121989109 shutdown_connections 2024-05-14T16:43:03.300 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7feb680b2880 0x7feb680b4d40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:03.300 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb80104030 0x7feb801a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:03.300 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb80102490 0x7feb8006c610 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:03.300 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 --2- 172.21.15.203:0/121989109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb80103690 0x7feb8019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:03.300 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 -- 172.21.15.203:0/121989109 >> 172.21.15.203:0/121989109 conn(0x7feb800fda30 msgr2=0x7feb80106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:03.300 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 -- 172.21.15.203:0/121989109 shutdown_connections 2024-05-14T16:43:03.300 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:43:03.299+0000 7feb85322700 1 -- 172.21.15.203:0/121989109 wait complete. 2024-05-14T16:43:03.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:03 smithi069 ceph-mon[31507]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:03.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:03 smithi092 ceph-mon[35763]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:03.903 DEBUG:teuthology.orchestra.run.smithi203:osd.5> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.5.service 2024-05-14T16:43:03.905 INFO:tasks.cephadm:Deploying osd.6 on smithi069 with /dev/vg_nvme/lv_2... 2024-05-14T16:43:03.905 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T16:43:04.153 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:43:04.202 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 16:43:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T16:43:04.011+0000 7f7a614187c0 -1 osd.5 0 log_to_monitors true 2024-05-14T16:43:04.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:04 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:04 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:04 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:04 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:04 smithi203 ceph-mon[35787]: from='osd.5 [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:43:04.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:04 smithi203 ceph-mon[35787]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:43:04.492 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:04 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.492 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:04 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.492 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:04 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.493 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:04 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.493 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:04 smithi069 ceph-mon[31507]: from='osd.5 [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:43:04.493 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:04 smithi069 ceph-mon[31507]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:43:04.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:04 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:04 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:04 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:04 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:04.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:04 smithi092 ceph-mon[35763]: from='osd.5 [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:43:04.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:04 smithi092 ceph-mon[35763]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:43:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:05 smithi203 ceph-mon[35787]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:05 smithi203 ceph-mon[35787]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T16:43:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:05 smithi203 ceph-mon[35787]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T16:43:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:05 smithi203 ceph-mon[35787]: from='osd.5 [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:43:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:05.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:05 smithi203 ceph-mon[35787]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:43:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:05 smithi069 ceph-mon[31507]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:05 smithi069 ceph-mon[31507]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T16:43:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:05 smithi069 ceph-mon[31507]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T16:43:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:05 smithi069 ceph-mon[31507]: from='osd.5 [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:43:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:05 smithi069 ceph-mon[31507]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:43:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:05 smithi092 ceph-mon[35763]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:05 smithi092 ceph-mon[35763]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T16:43:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:05 smithi092 ceph-mon[35763]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T16:43:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:05 smithi092 ceph-mon[35763]: from='osd.5 [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:43:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:05 smithi092 ceph-mon[35763]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:43:06.142 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T16:43:06.143 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T16:43:06.143 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-14T16:43:06.143 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-14T16:43:06.143 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113791 s, 921 MB/s 2024-05-14T16:43:06.143 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-14T16:43:06.460 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 16:43:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T16:43:06.041+0000 7f7a56a7c700 -1 osd.5 0 waiting for initial osdmap 2024-05-14T16:43:06.460 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 16:43:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T16:43:06.048+0000 7f7a4e867700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:43:06.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:06 smithi069 ceph-mon[31507]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:43:06.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:06 smithi069 ceph-mon[31507]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T16:43:06.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:06 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:06.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:06 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:06.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:06 smithi069 ceph-mon[31507]: from='osd.5 ' entity='osd.5' 2024-05-14T16:43:06.817 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2024-05-14T16:43:06.870 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:06 smithi203 ceph-mon[35787]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:43:06.871 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:06 smithi203 ceph-mon[35787]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T16:43:06.871 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:06 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:06.871 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:06 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:06.871 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:06 smithi203 ceph-mon[35787]: from='osd.5 ' entity='osd.5' 2024-05-14T16:43:07.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:06 smithi092 ceph-mon[35763]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:43:07.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:06 smithi092 ceph-mon[35763]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T16:43:07.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:06 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:07.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:06 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:07.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:06 smithi092 ceph-mon[35763]: from='osd.5 ' entity='osd.5' 2024-05-14T16:43:07.073 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:43:07.885 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: osd.5 [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] boot 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: osdmap e36: 6 total, 6 up, 6 in 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:07.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:07.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:07.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:07.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:07.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:07.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: osd.5 [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] boot 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: osdmap e36: 6 total, 6 up, 6 in 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:07.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:07.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:07.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:07.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:07.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: osd.5 [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] boot 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: osdmap e36: 6 total, 6 up, 6 in 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:08.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:08.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:08.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:08.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:08.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:08.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:08.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:08.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:08.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.703+0000 7fc418b9e700 1 -- 172.21.15.69:0/2729753501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4141047a0 msgr2=0x7fc414104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.704+0000 7fc418b9e700 1 --2- 172.21.15.69:0/2729753501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4141047a0 0x7fc414104c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc4040099c0 tx=0x7fc404009cd0 comp rx=0 tx=0).stop 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.704+0000 7fc418b9e700 1 -- 172.21.15.69:0/2729753501 shutdown_connections 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.704+0000 7fc418b9e700 1 --2- 172.21.15.69:0/2729753501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414105140 0x7fc41410b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.704+0000 7fc418b9e700 1 --2- 172.21.15.69:0/2729753501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4141035a0 0x7fc4141039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.704+0000 7fc418b9e700 1 --2- 172.21.15.69:0/2729753501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4141047a0 0x7fc414104c00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.704+0000 7fc418b9e700 1 -- 172.21.15.69:0/2729753501 >> 172.21.15.69:0/2729753501 conn(0x7fc4140feb40 msgr2=0x7fc414100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.704+0000 7fc418b9e700 1 -- 172.21.15.69:0/2729753501 shutdown_connections 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.704+0000 7fc418b9e700 1 -- 172.21.15.69:0/2729753501 wait complete. 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.704+0000 7fc418b9e700 1 Processor -- start 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.704+0000 7fc418b9e700 1 -- start start 2024-05-14T16:43:08.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc418b9e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4141035a0 0x7fc4141996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc418b9e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4141047a0 0x7fc414199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc418b9e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414105140 0x7fc4141a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc418b9e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410f090 con 0x7fc4141035a0 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc418b9e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410ef10 con 0x7fc414105140 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc418b9e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410f210 con 0x7fc4141047a0 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc41259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4141035a0 0x7fc4141996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc41259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4141035a0 0x7fc4141996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54368/0 (socket says 172.21.15.69:54368) 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc41259c700 1 -- 172.21.15.69:0/2844269977 learned_addr learned my addr 172.21.15.69:0/2844269977 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc411d9b700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4141047a0 0x7fc414199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc412d9d700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414105140 0x7fc4141a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc41259c700 1 -- 172.21.15.69:0/2844269977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4141047a0 msgr2=0x7fc414199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc41259c700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4141047a0 0x7fc414199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc41259c700 1 -- 172.21.15.69:0/2844269977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414105140 msgr2=0x7fc4141a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc41259c700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414105140 0x7fc4141a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc41259c700 1 -- 172.21.15.69:0/2844269977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4141a1310 con 0x7fc4141035a0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc411d9b700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4141047a0 0x7fc414199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc412d9d700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414105140 0x7fc4141a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc41259c700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4141035a0 0x7fc4141996b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c9c0 tx=0x7fc40800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc4037fe700 1 -- 172.21.15.69:0/2844269977 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408007ac0 con 0x7fc4141035a0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc418b9e700 1 -- 172.21.15.69:0/2844269977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4141a1600 con 0x7fc4141035a0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc4037fe700 1 -- 172.21.15.69:0/2844269977 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc408007c60 con 0x7fc4141035a0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc4037fe700 1 -- 172.21.15.69:0/2844269977 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408018660 con 0x7fc4141035a0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.705+0000 7fc418b9e700 1 -- 172.21.15.69:0/2844269977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4141a1b90 con 0x7fc4141035a0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.706+0000 7fc418b9e700 1 -- 172.21.15.69:0/2844269977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41406a230 con 0x7fc4141035a0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.708+0000 7fc4037fe700 1 -- 172.21.15.69:0/2844269977 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc408004020 con 0x7fc4141035a0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.708+0000 7fc4037fe700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3fc0b2880 0x7fc3fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.708+0000 7fc411d9b700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3fc0b2880 0x7fc3fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.709+0000 7fc4037fe700 1 -- 172.21.15.69:0/2844269977 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fc4080e14f0 con 0x7fc4141035a0 2024-05-14T16:43:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.709+0000 7fc411d9b700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3fc0b2880 0x7fc3fc0b4d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc404000c00 tx=0x7fc404005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:08.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.712+0000 7fc4037fe700 1 -- 172.21.15.69:0/2844269977 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc408053df0 con 0x7fc4141035a0 2024-05-14T16:43:08.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:08.926+0000 7fc418b9e700 1 -- 172.21.15.69:0/2844269977 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc414063980 con 0x7fc3fc0b2880 2024-05-14T16:43:08.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:08 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi203 2024-05-14T16:43:08.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:08 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi203 to 8544M 2024-05-14T16:43:08.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:08 smithi069 ceph-mon[31507]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T16:43:08.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:08 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi203 2024-05-14T16:43:08.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:08 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi203 to 8544M 2024-05-14T16:43:08.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:08 smithi203 ceph-mon[35787]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T16:43:09.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:08 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi203 2024-05-14T16:43:09.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:08 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi203 to 8544M 2024-05-14T16:43:09.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:08 smithi092 ceph-mon[35763]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T16:43:09.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:09 smithi069 ceph-mon[31507]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:09.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:09 smithi069 ceph-mon[31507]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T16:43:09.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:43:09.860 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:43:09.860 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:09 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:09.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:09 smithi203 ceph-mon[35787]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:09 smithi203 ceph-mon[35787]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T16:43:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:09 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:43:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:09 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:43:09.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:09 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:09 smithi092 ceph-mon[35763]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:09 smithi092 ceph-mon[35763]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T16:43:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:09 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:43:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:09 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:43:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:09 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:10.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:10 smithi069 ceph-mon[31507]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:10.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:10 smithi203 ceph-mon[35787]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:11.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:10 smithi092 ceph-mon[35763]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:11.901 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:11 smithi092 ceph-mon[35763]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:11.902 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:11 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/541820667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2"}]: dispatch 2024-05-14T16:43:11.902 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:11 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/541820667' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2"}]': finished 2024-05-14T16:43:11.902 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:11 smithi092 ceph-mon[35763]: osdmap e39: 7 total, 6 up, 7 in 2024-05-14T16:43:11.902 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:11 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:11.912 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:11 smithi069 ceph-mon[31507]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:11.912 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:11 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/541820667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2"}]: dispatch 2024-05-14T16:43:11.912 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:11 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/541820667' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2"}]': finished 2024-05-14T16:43:11.912 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:11 smithi069 ceph-mon[31507]: osdmap e39: 7 total, 6 up, 7 in 2024-05-14T16:43:11.912 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:11 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:11.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:11 smithi203 ceph-mon[35787]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:11.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:11 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/541820667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2"}]: dispatch 2024-05-14T16:43:11.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:11 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/541820667' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2"}]': finished 2024-05-14T16:43:11.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:11 smithi203 ceph-mon[35787]: osdmap e39: 7 total, 6 up, 7 in 2024-05-14T16:43:11.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:11 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:12.917 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:12 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1326065234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:43:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:12 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1326065234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:43:12.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:12 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1326065234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:43:13.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:13 smithi069 ceph-mon[31507]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-14T16:43:13.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:13 smithi203 ceph-mon[35787]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-14T16:43:14.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:13 smithi092 ceph-mon[35763]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-14T16:43:15.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:15 smithi069 ceph-mon[31507]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-14T16:43:15.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:15 smithi203 ceph-mon[35787]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-14T16:43:16.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:15 smithi092 ceph-mon[35763]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-14T16:43:17.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:17 smithi069 ceph-mon[31507]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:43:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:17 smithi203 ceph-mon[35787]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:43:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:17 smithi092 ceph-mon[35763]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:43:19.780 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:19 smithi069 ceph-mon[31507]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T16:43:19.780 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T16:43:19.780 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:19.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:19 smithi203 ceph-mon[35787]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T16:43:19.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T16:43:19.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:19 smithi092 ceph-mon[35763]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T16:43:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T16:43:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:20.854 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:20 smithi069 ceph-mon[31507]: Deploying daemon osd.6 on smithi069 2024-05-14T16:43:20.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:20 smithi203 ceph-mon[35787]: Deploying daemon osd.6 on smithi069 2024-05-14T16:43:21.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:20 smithi092 ceph-mon[35763]: Deploying daemon osd.6 on smithi069 2024-05-14T16:43:21.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:21 smithi069 ceph-mon[31507]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:43:21.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:21 smithi203 ceph-mon[35787]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:43:22.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:21 smithi092 ceph-mon[35763]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:43:23.777 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:23 smithi069 ceph-mon[31507]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:23.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:23 smithi203 ceph-mon[35787]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:24.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:23 smithi092 ceph-mon[35763]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:25.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:25 smithi069 ceph-mon[31507]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:25.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:25 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:25.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:25 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:25.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:25 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:25.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:25 smithi203 ceph-mon[35787]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:25.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:25 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:25.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:25 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:25.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:25 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:26.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:25 smithi092 ceph-mon[35763]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:26.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:25 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:26.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:25 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:26.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:25 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:27.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:27 smithi069 ceph-mon[31507]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:27.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:27 smithi203 ceph-mon[35787]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:28.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:27 smithi092 ceph-mon[35763]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:28.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.731+0000 7fc4037fe700 1 -- 172.21.15.69:0/2844269977 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc414063980 con 0x7fc3fc0b2880 2024-05-14T16:43:28.732 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 6 on host 'smithi069' 2024-05-14T16:43:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 -- 172.21.15.69:0/2844269977 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3fc0b2880 msgr2=0x7fc3fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3fc0b2880 0x7fc3fc0b4d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc404000c00 tx=0x7fc404005fb0 comp rx=0 tx=0).stop 2024-05-14T16:43:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 -- 172.21.15.69:0/2844269977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4141035a0 msgr2=0x7fc4141996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4141035a0 0x7fc4141996b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c9c0 tx=0x7fc40800cd80 comp rx=0 tx=0).stop 2024-05-14T16:43:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 -- 172.21.15.69:0/2844269977 shutdown_connections 2024-05-14T16:43:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3fc0b2880 0x7fc3fc0b4d40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4141035a0 0x7fc4141996b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc414105140 0x7fc4141a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 --2- 172.21.15.69:0/2844269977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4141047a0 0x7fc414199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 -- 172.21.15.69:0/2844269977 >> 172.21.15.69:0/2844269977 conn(0x7fc4140feb40 msgr2=0x7fc414107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 -- 172.21.15.69:0/2844269977 shutdown_connections 2024-05-14T16:43:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:43:28.734+0000 7fc418b9e700 1 -- 172.21.15.69:0/2844269977 wait complete. 2024-05-14T16:43:28.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:28.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:28.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:28.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:28.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:28.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:28 smithi069 ceph-mon[31507]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:28.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:28 smithi069 ceph-mon[31507]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T16:43:28.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:28.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:28.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:29.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:29.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:28 smithi092 ceph-mon[35763]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:29.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:28 smithi092 ceph-mon[35763]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T16:43:29.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:29.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:28 smithi203 ceph-mon[35787]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:28 smithi203 ceph-mon[35787]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T16:43:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:29.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:29.498 DEBUG:teuthology.orchestra.run.smithi069:osd.6> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.6.service 2024-05-14T16:43:29.500 INFO:tasks.cephadm:Deploying osd.7 on smithi092 with /dev/vg_nvme/lv_2... 2024-05-14T16:43:29.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T16:43:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:43:29.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:29 smithi092 ceph-mon[35763]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T16:43:29.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:29 smithi092 ceph-mon[35763]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T16:43:29.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:29 smithi092 ceph-mon[35763]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:43:29.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:29 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:30.079 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:29 smithi069 ceph-mon[31507]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T16:43:30.079 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:29 smithi069 ceph-mon[31507]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T16:43:30.079 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:29 smithi069 ceph-mon[31507]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:43:30.079 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:29 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:30.079 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 16:43:29 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T16:43:29.822+0000 7fa401d13700 -1 osd.6 0 waiting for initial osdmap 2024-05-14T16:43:30.079 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 16:43:29 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T16:43:29.830+0000 7fa3fdb06700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:43:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:29 smithi203 ceph-mon[35787]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T16:43:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:29 smithi203 ceph-mon[35787]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T16:43:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:29 smithi203 ceph-mon[35787]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:43:30.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:29 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:30.881 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:30 smithi092 ceph-mon[35763]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:43:30.881 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:30 smithi092 ceph-mon[35763]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T16:43:30.881 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:30.881 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:30 smithi092 ceph-mon[35763]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:30.881 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:30 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:30.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:30 smithi069 ceph-mon[31507]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:43:30.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:30 smithi069 ceph-mon[31507]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T16:43:30.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:30.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:30 smithi069 ceph-mon[31507]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:30.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:30 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:31.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:30 smithi203 ceph-mon[35787]: from='osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:43:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:30 smithi203 ceph-mon[35787]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T16:43:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:30 smithi203 ceph-mon[35787]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:43:31.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:30 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:31.749 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T16:43:31.749 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T16:43:31.749 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-14T16:43:31.749 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-14T16:43:31.749 INFO:teuthology.orchestra.run.smithi092.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114502 s, 916 MB/s 2024-05-14T16:43:31.749 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 2024-05-14T16:43:31.750 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-14T16:43:32.062 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:31 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:43:32.062 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:31 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:43:32.062 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:32.063 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:31 smithi092 ceph-mon[35763]: osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] boot 2024-05-14T16:43:32.063 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:31 smithi092 ceph-mon[35763]: osdmap e42: 7 total, 7 up, 7 in 2024-05-14T16:43:32.063 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:32.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:31 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:43:32.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:31 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:43:32.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:32.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:31 smithi069 ceph-mon[31507]: osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] boot 2024-05-14T16:43:32.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:31 smithi069 ceph-mon[31507]: osdmap e42: 7 total, 7 up, 7 in 2024-05-14T16:43:32.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:32.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:31 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:43:32.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:31 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:43:32.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:32.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:31 smithi203 ceph-mon[35787]: osd.6 [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] boot 2024-05-14T16:43:32.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:31 smithi203 ceph-mon[35787]: osdmap e42: 7 total, 7 up, 7 in 2024-05-14T16:43:32.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:43:32.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-05-14T16:43:32.633 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:43:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T16:43:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:32.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:32.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:32.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:32.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:33.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T16:43:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:33.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T16:43:33.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:33.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:33.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:33.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:33.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:33.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:33.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:33.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:33.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:33.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:33.983 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:33 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi069 2024-05-14T16:43:33.983 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:33 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi069 to 5696M 2024-05-14T16:43:34.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:33 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi069 2024-05-14T16:43:34.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:33 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi069 to 5696M 2024-05-14T16:43:34.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:33 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi069 2024-05-14T16:43:34.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:33 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi069 to 5696M 2024-05-14T16:43:34.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.328+0000 7f74fde0b700 1 -- 172.21.15.92:0/3620818188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f81035a0 msgr2=0x7f74f81039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:34.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.328+0000 7f74fde0b700 1 --2- 172.21.15.92:0/3620818188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f81035a0 0x7f74f81039c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f74e80099c0 tx=0x7f74e8009cd0 comp rx=0 tx=0).stop 2024-05-14T16:43:34.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.328+0000 7f74fde0b700 1 -- 172.21.15.92:0/3620818188 shutdown_connections 2024-05-14T16:43:34.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.328+0000 7f74fde0b700 1 --2- 172.21.15.92:0/3620818188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f8105140 0x7f74f810b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:34.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.328+0000 7f74fde0b700 1 --2- 172.21.15.92:0/3620818188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f81035a0 0x7f74f81039c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:34.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.328+0000 7f74fde0b700 1 --2- 172.21.15.92:0/3620818188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f81047a0 0x7f74f8104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:34.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.328+0000 7f74fde0b700 1 -- 172.21.15.92:0/3620818188 >> 172.21.15.92:0/3620818188 conn(0x7f74f80feb40 msgr2=0x7f74f8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:34.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.328+0000 7f74fde0b700 1 -- 172.21.15.92:0/3620818188 shutdown_connections 2024-05-14T16:43:34.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.328+0000 7f74fde0b700 1 -- 172.21.15.92:0/3620818188 wait complete. 2024-05-14T16:43:34.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74fde0b700 1 Processor -- start 2024-05-14T16:43:34.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74fde0b700 1 -- start start 2024-05-14T16:43:34.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74fde0b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f81035a0 0x7f74f8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:34.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74fde0b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f81047a0 0x7f74f8199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:34.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74fde0b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f8105140 0x7f74f81a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:34.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74fde0b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74f810f020 con 0x7f74f81047a0 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74fde0b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74f810eea0 con 0x7f74f81035a0 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74fde0b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f74f810f1a0 con 0x7f74f8105140 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74f77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f81035a0 0x7f74f8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74f77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f81035a0 0x7f74f8199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35042/0 (socket says 172.21.15.92:35042) 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74f77fe700 1 -- 172.21.15.92:0/1043918702 learned_addr learned my addr 172.21.15.92:0/1043918702 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74f7fff700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f8105140 0x7f74f81a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74effff700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f81047a0 0x7f74f8199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74f77fe700 1 -- 172.21.15.92:0/1043918702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f8105140 msgr2=0x7f74f81a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74f77fe700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f8105140 0x7f74f81a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74f77fe700 1 -- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f81047a0 msgr2=0x7f74f8199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74f77fe700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f81047a0 0x7f74f8199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74f77fe700 1 -- 172.21.15.92:0/1043918702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f81a13c0 con 0x7f74f81035a0 2024-05-14T16:43:34.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.329+0000 7f74effff700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f81047a0 0x7f74f8199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:43:34.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.330+0000 7f74f77fe700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f81035a0 0x7f74f8199760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009f50 tx=0x7f74e8009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:34.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.330+0000 7f74f57fa700 1 -- 172.21.15.92:0/1043918702 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8027070 con 0x7f74f81035a0 2024-05-14T16:43:34.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.330+0000 7f74fde0b700 1 -- 172.21.15.92:0/1043918702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f81a1650 con 0x7f74f81035a0 2024-05-14T16:43:34.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.330+0000 7f74f57fa700 1 -- 172.21.15.92:0/1043918702 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e8022470 con 0x7f74f81035a0 2024-05-14T16:43:34.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.330+0000 7f74f57fa700 1 -- 172.21.15.92:0/1043918702 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e800f6c0 con 0x7f74f81035a0 2024-05-14T16:43:34.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.330+0000 7f74fde0b700 1 -- 172.21.15.92:0/1043918702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f81a1b30 con 0x7f74f81035a0 2024-05-14T16:43:34.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.331+0000 7f74fde0b700 1 -- 172.21.15.92:0/1043918702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f806a230 con 0x7f74f81035a0 2024-05-14T16:43:34.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.332+0000 7f74f57fa700 1 -- 172.21.15.92:0/1043918702 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f74e802a070 con 0x7f74f81035a0 2024-05-14T16:43:34.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.333+0000 7f74f57fa700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74e40b2880 0x7f74e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:34.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.333+0000 7f74f57fa700 1 -- 172.21.15.92:0/1043918702 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f74e80e2840 con 0x7f74f81035a0 2024-05-14T16:43:34.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.333+0000 7f74effff700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74e40b2880 0x7f74e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:34.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.333+0000 7f74effff700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74e40b2880 0x7f74e40b4d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f74e0005950 tx=0x7f74e00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:34.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.336+0000 7f74f57fa700 1 -- 172.21.15.92:0/1043918702 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74e8031080 con 0x7f74f81035a0 2024-05-14T16:43:34.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:34.547+0000 7f74fde0b700 1 -- 172.21.15.92:0/1043918702 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f74f8063980 con 0x7f74e40b2880 2024-05-14T16:43:35.160 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:34 smithi092 ceph-mon[35763]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:35.161 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:34 smithi092 ceph-mon[35763]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:35.161 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:34 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:43:35.161 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:34 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:43:35.161 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:34 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:35.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:34 smithi069 ceph-mon[31507]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:35.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:34 smithi069 ceph-mon[31507]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:35.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:34 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:43:35.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:34 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:43:35.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:34 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:35.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:34 smithi203 ceph-mon[35787]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:35.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:34 smithi203 ceph-mon[35787]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:35.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:34 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:43:35.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:34 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:43:35.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:34 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:37.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:36 smithi069 ceph-mon[31507]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:37.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:36 smithi203 ceph-mon[35787]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:37.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:36 smithi092 ceph-mon[35763]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:38.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:37 smithi092 ceph-mon[35763]: from='client.? 172.21.15.92:0/3748185487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b87d086b-bce8-441b-83bb-9d21051d300c"}]: dispatch 2024-05-14T16:43:38.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:37 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b87d086b-bce8-441b-83bb-9d21051d300c"}]: dispatch 2024-05-14T16:43:38.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:37 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b87d086b-bce8-441b-83bb-9d21051d300c"}]': finished 2024-05-14T16:43:38.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:37 smithi092 ceph-mon[35763]: osdmap e44: 8 total, 7 up, 8 in 2024-05-14T16:43:38.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:37 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:38.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:37 smithi069 ceph-mon[31507]: from='client.? 172.21.15.92:0/3748185487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b87d086b-bce8-441b-83bb-9d21051d300c"}]: dispatch 2024-05-14T16:43:38.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:37 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b87d086b-bce8-441b-83bb-9d21051d300c"}]: dispatch 2024-05-14T16:43:38.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:37 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b87d086b-bce8-441b-83bb-9d21051d300c"}]': finished 2024-05-14T16:43:38.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:37 smithi069 ceph-mon[31507]: osdmap e44: 8 total, 7 up, 8 in 2024-05-14T16:43:38.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:37 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:37 smithi203 ceph-mon[35787]: from='client.? 172.21.15.92:0/3748185487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b87d086b-bce8-441b-83bb-9d21051d300c"}]: dispatch 2024-05-14T16:43:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:37 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b87d086b-bce8-441b-83bb-9d21051d300c"}]: dispatch 2024-05-14T16:43:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:37 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b87d086b-bce8-441b-83bb-9d21051d300c"}]': finished 2024-05-14T16:43:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:37 smithi203 ceph-mon[35787]: osdmap e44: 8 total, 7 up, 8 in 2024-05-14T16:43:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:37 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:39.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:38 smithi069 ceph-mon[31507]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:39.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:38 smithi069 ceph-mon[31507]: from='client.? 172.21.15.92:0/559111288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:43:39.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:38 smithi203 ceph-mon[35787]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:39.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:38 smithi203 ceph-mon[35787]: from='client.? 172.21.15.92:0/559111288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:43:39.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:38 smithi092 ceph-mon[35763]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:39.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:38 smithi092 ceph-mon[35763]: from='client.? 172.21.15.92:0/559111288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:43:41.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:40 smithi069 ceph-mon[31507]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:41.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:40 smithi203 ceph-mon[35787]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:41.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:40 smithi092 ceph-mon[35763]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:43.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:42 smithi092 ceph-mon[35763]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:42 smithi069 ceph-mon[31507]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:43.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:42 smithi203 ceph-mon[35787]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:45.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:44 smithi092 ceph-mon[35763]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:45.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:44 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T16:43:45.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:44 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:44 smithi069 ceph-mon[31507]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:44 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T16:43:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:44 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:45.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:44 smithi203 ceph-mon[35787]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:45.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:44 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T16:43:45.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:44 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:46.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:45 smithi069 ceph-mon[31507]: Deploying daemon osd.7 on smithi092 2024-05-14T16:43:46.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:45 smithi203 ceph-mon[35787]: Deploying daemon osd.7 on smithi092 2024-05-14T16:43:46.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:45 smithi092 ceph-mon[35763]: Deploying daemon osd.7 on smithi092 2024-05-14T16:43:47.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:46 smithi092 ceph-mon[35763]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:47.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:46 smithi069 ceph-mon[31507]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:47.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:46 smithi203 ceph-mon[35787]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:49.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:48 smithi069 ceph-mon[31507]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:49.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:48 smithi203 ceph-mon[35787]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:49.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:48 smithi092 ceph-mon[35763]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:51.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:51 smithi092 ceph-mon[35763]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:51.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:51 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:51.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:51 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:51.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:51 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:51.907 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:51 smithi069 ceph-mon[31507]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:51.907 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:51 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:51.907 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:51 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:51.907 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:51 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:51.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:51 smithi203 ceph-mon[35787]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:51.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:51 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:51.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:51 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:51.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:51 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:53.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:53 smithi069 ceph-mon[31507]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:53.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:53 smithi203 ceph-mon[35787]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:54.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:53 smithi092 ceph-mon[35763]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:54.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.464+0000 7f74f57fa700 1 -- 172.21.15.92:0/1043918702 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f74f8063980 con 0x7f74e40b2880 2024-05-14T16:43:54.464 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 7 on host 'smithi092' 2024-05-14T16:43:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 -- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74e40b2880 msgr2=0x7f74e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74e40b2880 0x7f74e40b4d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f74e0005950 tx=0x7f74e00058e0 comp rx=0 tx=0).stop 2024-05-14T16:43:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 -- 172.21.15.92:0/1043918702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f81035a0 msgr2=0x7f74f8199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f81035a0 0x7f74f8199760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009f50 tx=0x7f74e8009510 comp rx=0 tx=0).stop 2024-05-14T16:43:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 -- 172.21.15.92:0/1043918702 shutdown_connections 2024-05-14T16:43:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f8105140 0x7f74f81a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74f81035a0 0x7f74f8199760 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74e40b2880 0x7f74e40b4d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 --2- 172.21.15.92:0/1043918702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74f81047a0 0x7f74f8199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:54.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 -- 172.21.15.92:0/1043918702 >> 172.21.15.92:0/1043918702 conn(0x7f74f80feb40 msgr2=0x7f74f8107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:54.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 -- 172.21.15.92:0/1043918702 shutdown_connections 2024-05-14T16:43:54.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:43:54.466+0000 7f74fde0b700 1 -- 172.21.15.92:0/1043918702 wait complete. 2024-05-14T16:43:54.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:54.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:54.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:54 smithi092 ceph-mon[35763]: from='osd.7 [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:43:54.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:54 smithi092 ceph-mon[35763]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:43:54.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:54 smithi069 ceph-mon[31507]: from='osd.7 [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:43:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:54 smithi069 ceph-mon[31507]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:43:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:54 smithi203 ceph-mon[35787]: from='osd.7 [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:43:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:54 smithi203 ceph-mon[35787]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:43:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:54.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:55.176 DEBUG:teuthology.orchestra.run.smithi092:osd.7> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.7.service 2024-05-14T16:43:55.178 INFO:tasks.cephadm:Deploying osd.8 on smithi203 with /dev/vg_nvme/lv_2... 2024-05-14T16:43:55.179 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T16:43:55.427 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:43:55.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:55 smithi092 ceph-mon[35763]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:55.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:55 smithi092 ceph-mon[35763]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T16:43:55.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:55 smithi092 ceph-mon[35763]: from='osd.7 [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:43:55.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:55 smithi092 ceph-mon[35763]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T16:43:55.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:55.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:55 smithi092 ceph-mon[35763]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:43:55.755 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 16:43:55 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T16:43:55.623+0000 7fd8a0fab700 -1 osd.7 0 waiting for initial osdmap 2024-05-14T16:43:55.755 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 16:43:55 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T16:43:55.631+0000 7fd89cd9e700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:43:55.804 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:55 smithi203 ceph-mon[35787]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:55.805 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:55 smithi203 ceph-mon[35787]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T16:43:55.805 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:55 smithi203 ceph-mon[35787]: from='osd.7 [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:43:55.805 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:55 smithi203 ceph-mon[35787]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T16:43:55.805 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:55.805 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:55 smithi203 ceph-mon[35787]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:43:55.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:55 smithi069 ceph-mon[31507]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:55.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:55 smithi069 ceph-mon[31507]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T16:43:55.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:55 smithi069 ceph-mon[31507]: from='osd.7 [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:43:55.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:55 smithi069 ceph-mon[31507]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T16:43:55.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:55.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:55 smithi069 ceph-mon[31507]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:43:56.629 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:56 smithi092 ceph-mon[35763]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:43:56.629 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:56 smithi092 ceph-mon[35763]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T16:43:56.629 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:56.629 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:56.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:56 smithi069 ceph-mon[31507]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:43:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:56 smithi069 ceph-mon[31507]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T16:43:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:56.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:56 smithi203 ceph-mon[35787]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:43:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:56 smithi203 ceph-mon[35787]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T16:43:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:57.554 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T16:43:57.555 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T16:43:57.555 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-05-14T16:43:57.555 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-05-14T16:43:57.555 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127996 s, 819 MB/s 2024-05-14T16:43:57.555 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-05-14T16:43:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:57 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:43:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:57 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:43:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:57 smithi203 ceph-mon[35787]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:57 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:57 smithi203 ceph-mon[35787]: osd.7 [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] boot 2024-05-14T16:43:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:57 smithi203 ceph-mon[35787]: osdmap e47: 8 total, 8 up, 8 in 2024-05-14T16:43:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:57 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:57.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:57 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:43:57.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:57 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:43:57.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:57 smithi092 ceph-mon[35763]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:57.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:57 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:57.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:57 smithi092 ceph-mon[35763]: osd.7 [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] boot 2024-05-14T16:43:57.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:57 smithi092 ceph-mon[35763]: osdmap e47: 8 total, 8 up, 8 in 2024-05-14T16:43:57.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:57 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:57 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:43:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:57 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:43:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:57 smithi069 ceph-mon[31507]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:43:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:57 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:57 smithi069 ceph-mon[31507]: osd.7 [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] boot 2024-05-14T16:43:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:57 smithi069 ceph-mon[31507]: osdmap e47: 8 total, 8 up, 8 in 2024-05-14T16:43:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:57 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:43:58.362 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_2 2024-05-14T16:43:58.604 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:43:58.893 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:58 smithi203 ceph-mon[35787]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T16:43:58.893 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:58 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:58.893 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:58 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:58.893 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:58 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:58.893 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:58 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:58.894 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:58 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:58.894 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:58 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:58.894 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:58 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:58.894 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:58 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:58.894 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:58 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:58.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:58 smithi069 ceph-mon[31507]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T16:43:58.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:58 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:58.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:58 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:58.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:58 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:58.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:58 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:58.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:58 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:58.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:58 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:58.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:58 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:58.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:58 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:58.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:58 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:58 smithi092 ceph-mon[35763]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T16:43:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:58 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:58 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:58 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:58 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:58 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:43:59.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:58 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:59.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:58 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:59.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:58 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:43:59.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:58 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:43:59.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:59 smithi203 ceph-mon[35787]: pgmap v171: 1 pgs: 1 remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:43:59.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:59 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi092 2024-05-14T16:43:59.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:59 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-14T16:43:59.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:43:59 smithi203 ceph-mon[35787]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T16:43:59.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:59 smithi069 ceph-mon[31507]: pgmap v171: 1 pgs: 1 remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:43:59.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:59 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi092 2024-05-14T16:43:59.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:59 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-14T16:43:59.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:43:59 smithi069 ceph-mon[31507]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T16:44:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:59 smithi092 ceph-mon[35763]: pgmap v171: 1 pgs: 1 remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:59 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi092 2024-05-14T16:44:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:59 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-14T16:44:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:43:59 smithi092 ceph-mon[35763]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T16:44:00.204 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.202+0000 7f2d2538b700 1 -- 172.21.15.203:0/3586694885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d20104030 msgr2=0x7f2d2010a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.202+0000 7f2d2538b700 1 --2- 172.21.15.203:0/3586694885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d20104030 0x7f2d2010a8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009f90 tx=0x7f2d1800b330 comp rx=0 tx=0).stop 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.203+0000 7f2d2538b700 1 -- 172.21.15.203:0/3586694885 shutdown_connections 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.203+0000 7f2d2538b700 1 --2- 172.21.15.203:0/3586694885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d20104030 0x7f2d2010a8e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.203+0000 7f2d2538b700 1 --2- 172.21.15.203:0/3586694885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d20102490 0x7f2d201028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.203+0000 7f2d2538b700 1 --2- 172.21.15.203:0/3586694885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d20103690 0x7f2d20103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.203+0000 7f2d2538b700 1 -- 172.21.15.203:0/3586694885 >> 172.21.15.203:0/3586694885 conn(0x7f2d200fda30 msgr2=0x7f2d200ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.203+0000 7f2d2538b700 1 -- 172.21.15.203:0/3586694885 shutdown_connections 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.203+0000 7f2d2538b700 1 -- 172.21.15.203:0/3586694885 wait complete. 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.203+0000 7f2d2538b700 1 Processor -- start 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.203+0000 7f2d2538b700 1 -- start start 2024-05-14T16:44:00.205 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d2538b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d20102490 0x7f2d2006c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d2538b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d20103690 0x7f2d2019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d2538b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d20104030 0x7f2d201a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d2538b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2010df10 con 0x7f2d20102490 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d2538b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2010dd90 con 0x7f2d20103690 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d2538b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2010e090 con 0x7f2d20104030 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d20104030 0x7f2d201a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d20102490 0x7f2d2006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d20104030 0x7f2d201a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:46960/0 (socket says 172.21.15.203:46960) 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d20102490 0x7f2d2006c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.203:56416/0 (socket says 172.21.15.203:56416) 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1f7fe700 1 -- 172.21.15.203:0/1994768966 learned_addr learned my addr 172.21.15.203:0/1994768966 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1e7fc700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d20103690 0x7f2d2019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1f7fe700 1 -- 172.21.15.203:0/1994768966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d20103690 msgr2=0x7f2d2019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1f7fe700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d20103690 0x7f2d2019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:00.206 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1f7fe700 1 -- 172.21.15.203:0/1994768966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d20102490 msgr2=0x7f2d2006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:00.207 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1f7fe700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d20102490 0x7f2d2006c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:00.207 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1f7fe700 1 -- 172.21.15.203:0/1994768966 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d201a12c0 con 0x7f2d20104030 2024-05-14T16:44:00.207 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d1f7fe700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d20104030 0x7f2d201a09f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800b6c0 tx=0x7f2d180079b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:00.207 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d17fff700 1 -- 172.21.15.203:0/1994768966 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18021070 con 0x7f2d20104030 2024-05-14T16:44:00.207 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d17fff700 1 -- 172.21.15.203:0/1994768966 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d1801db00 con 0x7f2d20104030 2024-05-14T16:44:00.207 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d2538b700 1 -- 172.21.15.203:0/1994768966 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d201a1550 con 0x7f2d20104030 2024-05-14T16:44:00.207 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d17fff700 1 -- 172.21.15.203:0/1994768966 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1801bb20 con 0x7f2d20104030 2024-05-14T16:44:00.207 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.204+0000 7f2d2538b700 1 -- 172.21.15.203:0/1994768966 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d201a1a00 con 0x7f2d20104030 2024-05-14T16:44:00.208 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.205+0000 7f2d2538b700 1 -- 172.21.15.203:0/1994768966 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2006a230 con 0x7f2d20104030 2024-05-14T16:44:00.208 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.207+0000 7f2d17fff700 1 -- 172.21.15.203:0/1994768966 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2d18024070 con 0x7f2d20104030 2024-05-14T16:44:00.208 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.208+0000 7f2d17fff700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2d0c0b2880 0x7f2d0c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:00.208 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.208+0000 7f2d17fff700 1 -- 172.21.15.203:0/1994768966 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f2d1800f020 con 0x7f2d20104030 2024-05-14T16:44:00.208 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.208+0000 7f2d1effd700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2d0c0b2880 0x7f2d0c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:00.208 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.208+0000 7f2d1effd700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2d0c0b2880 0x7f2d0c0b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d201034f0 tx=0x7f2d1000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:00.211 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.212+0000 7f2d17fff700 1 -- 172.21.15.203:0/1994768966 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d1802b020 con 0x7f2d20104030 2024-05-14T16:44:00.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:00.421+0000 7f2d2538b700 1 -- 172.21.15.203:0/1994768966 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2d20063980 con 0x7f2d0c0b2880 2024-05-14T16:44:00.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:00 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:44:00.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:00 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:44:00.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:00 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:00.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:44:00.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:44:00.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:00 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:44:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:00 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:44:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:00 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:01.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:01 smithi069 ceph-mon[31507]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:01.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:01 smithi069 ceph-mon[31507]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:01 smithi203 ceph-mon[35787]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:01 smithi203 ceph-mon[35787]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:01 smithi092 ceph-mon[35763]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:01 smithi092 ceph-mon[35763]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:03.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:03 smithi069 ceph-mon[31507]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:03.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:03 smithi069 ceph-mon[31507]: from='client.? 172.21.15.203:0/1710406382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59490349-0db6-4e24-bc5b-b1cf52663838"}]: dispatch 2024-05-14T16:44:03.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:03 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59490349-0db6-4e24-bc5b-b1cf52663838"}]: dispatch 2024-05-14T16:44:03.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:03 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59490349-0db6-4e24-bc5b-b1cf52663838"}]': finished 2024-05-14T16:44:03.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:03 smithi069 ceph-mon[31507]: osdmap e50: 9 total, 8 up, 9 in 2024-05-14T16:44:03.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:03 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:03.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:03 smithi203 ceph-mon[35787]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:03.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:03 smithi203 ceph-mon[35787]: from='client.? 172.21.15.203:0/1710406382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59490349-0db6-4e24-bc5b-b1cf52663838"}]: dispatch 2024-05-14T16:44:03.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:03 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59490349-0db6-4e24-bc5b-b1cf52663838"}]: dispatch 2024-05-14T16:44:03.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:03 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59490349-0db6-4e24-bc5b-b1cf52663838"}]': finished 2024-05-14T16:44:03.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:03 smithi203 ceph-mon[35787]: osdmap e50: 9 total, 8 up, 9 in 2024-05-14T16:44:03.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:03 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:03 smithi092 ceph-mon[35763]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:03 smithi092 ceph-mon[35763]: from='client.? 172.21.15.203:0/1710406382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59490349-0db6-4e24-bc5b-b1cf52663838"}]: dispatch 2024-05-14T16:44:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:03 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59490349-0db6-4e24-bc5b-b1cf52663838"}]: dispatch 2024-05-14T16:44:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:03 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59490349-0db6-4e24-bc5b-b1cf52663838"}]': finished 2024-05-14T16:44:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:03 smithi092 ceph-mon[35763]: osdmap e50: 9 total, 8 up, 9 in 2024-05-14T16:44:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:03 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:04.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:04 smithi069 ceph-mon[31507]: from='client.? 172.21.15.203:0/4028110510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:44:04.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:04 smithi203 ceph-mon[35787]: from='client.? 172.21.15.203:0/4028110510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:44:05.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:04 smithi092 ceph-mon[35763]: from='client.? 172.21.15.203:0/4028110510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:44:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:05 smithi069 ceph-mon[31507]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 186 KiB/s, 0 objects/s recovering 2024-05-14T16:44:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:05 smithi203 ceph-mon[35787]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 186 KiB/s, 0 objects/s recovering 2024-05-14T16:44:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:05 smithi092 ceph-mon[35763]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 186 KiB/s, 0 objects/s recovering 2024-05-14T16:44:07.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:07 smithi069 ceph-mon[31507]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-14T16:44:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:07 smithi203 ceph-mon[35787]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-14T16:44:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:07 smithi092 ceph-mon[35763]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-14T16:44:09.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:09 smithi203 ceph-mon[35787]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-14T16:44:09.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:09 smithi069 ceph-mon[31507]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-14T16:44:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:09 smithi092 ceph-mon[35763]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-14T16:44:10.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:10 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T16:44:10.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:10 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:10.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:10 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T16:44:10.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:10 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:11.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:10 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T16:44:11.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:10 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:11.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:11 smithi203 ceph-mon[35787]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-14T16:44:11.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:11 smithi203 ceph-mon[35787]: Deploying daemon osd.8 on smithi203 2024-05-14T16:44:11.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:11 smithi069 ceph-mon[31507]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-14T16:44:11.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:11 smithi069 ceph-mon[31507]: Deploying daemon osd.8 on smithi203 2024-05-14T16:44:12.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:11 smithi092 ceph-mon[35763]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-14T16:44:12.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:11 smithi092 ceph-mon[35763]: Deploying daemon osd.8 on smithi203 2024-05-14T16:44:13.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:13 smithi203 ceph-mon[35787]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-14T16:44:13.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:13 smithi069 ceph-mon[31507]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-14T16:44:14.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:13 smithi092 ceph-mon[35763]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-14T16:44:15.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:15 smithi069 ceph-mon[31507]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:15.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:15 smithi203 ceph-mon[35787]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:16.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:15 smithi092 ceph-mon[35763]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:16.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:16 smithi203 ceph-mon[35787]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:16.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:16 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:16.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:16 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:16.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:16 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:17.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:16 smithi069 ceph-mon[31507]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:17.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:16 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:17.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:16 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:17.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:16 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:17.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:16 smithi092 ceph-mon[35763]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:17.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:16 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:17.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:16 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:17.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:16 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:19.067 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:18 smithi203 ceph-mon[35787]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:19.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:18 smithi069 ceph-mon[31507]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:19.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:18 smithi092 ceph-mon[35763]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:19.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.440+0000 7f2d17fff700 1 -- 172.21.15.203:0/1994768966 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2d20063980 con 0x7f2d0c0b2880 2024-05-14T16:44:19.441 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 8 on host 'smithi203' 2024-05-14T16:44:19.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 -- 172.21.15.203:0/1994768966 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2d0c0b2880 msgr2=0x7f2d0c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:19.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2d0c0b2880 0x7f2d0c0b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d201034f0 tx=0x7f2d1000b410 comp rx=0 tx=0).stop 2024-05-14T16:44:19.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 -- 172.21.15.203:0/1994768966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d20104030 msgr2=0x7f2d201a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:19.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d20104030 0x7f2d201a09f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800b6c0 tx=0x7f2d180079b0 comp rx=0 tx=0).stop 2024-05-14T16:44:19.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 -- 172.21.15.203:0/1994768966 shutdown_connections 2024-05-14T16:44:19.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2d0c0b2880 0x7f2d0c0b4d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:19.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d20102490 0x7f2d2006c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:19.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d20104030 0x7f2d201a09f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:19.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 --2- 172.21.15.203:0/1994768966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d20103690 0x7f2d2019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:19.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 -- 172.21.15.203:0/1994768966 >> 172.21.15.203:0/1994768966 conn(0x7f2d200fda30 msgr2=0x7f2d20106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:19.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 -- 172.21.15.203:0/1994768966 shutdown_connections 2024-05-14T16:44:19.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:44:19.443+0000 7f2d2538b700 1 -- 172.21.15.203:0/1994768966 wait complete. 2024-05-14T16:44:20.091 DEBUG:teuthology.orchestra.run.smithi203:osd.8> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.8.service 2024-05-14T16:44:20.093 INFO:tasks.cephadm:Deploying osd.9 on smithi069 with /dev/vg_nvme/lv_1... 2024-05-14T16:44:20.093 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T16:44:20.354 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:44:20.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:20 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:20 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:20 smithi069 ceph-mon[31507]: from='osd.8 [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:44:20.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:20 smithi069 ceph-mon[31507]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:44:20.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:20 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:20 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:20 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:20 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:20 smithi203 ceph-mon[35787]: from='osd.8 [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:44:20.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:20 smithi203 ceph-mon[35787]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:44:20.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:20 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:20 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:20 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:20 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:20 smithi092 ceph-mon[35763]: from='osd.8 [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:44:20.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:20 smithi092 ceph-mon[35763]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:44:20.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:20 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:20.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:20 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:21.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:21 smithi069 ceph-mon[31507]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:21.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:21 smithi069 ceph-mon[31507]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T16:44:21.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:21 smithi069 ceph-mon[31507]: from='osd.8 [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:44:21.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:21 smithi069 ceph-mon[31507]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T16:44:21.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:21 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:21.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:21 smithi069 ceph-mon[31507]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:44:21.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:21 smithi203 ceph-mon[35787]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:21.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:21 smithi203 ceph-mon[35787]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T16:44:21.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:21 smithi203 ceph-mon[35787]: from='osd.8 [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:44:21.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:21 smithi203 ceph-mon[35787]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T16:44:21.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:21 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:21.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:21 smithi203 ceph-mon[35787]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:44:21.710 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 16:44:21 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T16:44:21.448+0000 7f481cd19700 -1 osd.8 0 waiting for initial osdmap 2024-05-14T16:44:21.710 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 16:44:21 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T16:44:21.461+0000 7f4818b0c700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:44:21.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:21 smithi092 ceph-mon[35763]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:21.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:21 smithi092 ceph-mon[35763]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T16:44:21.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:21 smithi092 ceph-mon[35763]: from='osd.8 [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:44:21.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:21 smithi092 ceph-mon[35763]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T16:44:21.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:21 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:21.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:21 smithi092 ceph-mon[35763]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:44:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T16:44:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T16:44:22.435 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-14T16:44:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-14T16:44:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125333 s, 837 MB/s 2024-05-14T16:44:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-14T16:44:22.453 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:22 smithi203 ceph-mon[35787]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:44:22.453 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:22 smithi203 ceph-mon[35787]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T16:44:22.453 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:22 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:22.453 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:22 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:22.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:22 smithi069 ceph-mon[31507]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:44:22.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:22 smithi069 ceph-mon[31507]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T16:44:22.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:22 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:22.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:22 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:22.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:22 smithi092 ceph-mon[35763]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:44:22.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:22 smithi092 ceph-mon[35763]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T16:44:22.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:22 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:22.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:22 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:23.042 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_1 2024-05-14T16:44:23.290 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:44:23.557 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:23 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:44:23.557 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:23 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:44:23.557 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:23 smithi203 ceph-mon[35787]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:23.557 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:23 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:23.558 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:23 smithi203 ceph-mon[35787]: osd.8 [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] boot 2024-05-14T16:44:23.558 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:23 smithi203 ceph-mon[35787]: osdmap e53: 9 total, 9 up, 9 in 2024-05-14T16:44:23.558 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:23 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:23.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:23 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:44:23.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:23 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:44:23.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:23 smithi069 ceph-mon[31507]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:23.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:23 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:23.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:23 smithi069 ceph-mon[31507]: osd.8 [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] boot 2024-05-14T16:44:23.668 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:23 smithi069 ceph-mon[31507]: osdmap e53: 9 total, 9 up, 9 in 2024-05-14T16:44:23.668 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:23 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:23.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:23 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:44:23.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:23 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:44:23.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:23 smithi092 ceph-mon[35763]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:44:23.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:23 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:23.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:23 smithi092 ceph-mon[35763]: osd.8 [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] boot 2024-05-14T16:44:23.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:23 smithi092 ceph-mon[35763]: osdmap e53: 9 total, 9 up, 9 in 2024-05-14T16:44:23.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:23 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:44:24.750 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi203 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi203 to 5696M 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:24.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi203 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi203 to 5696M 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:24.752 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:24.753 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:24.753 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.816 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T16:44:24.816 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi203 2024-05-14T16:44:24.816 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.816 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.816 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi203 to 5696M 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:24.817 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:24.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 -- 172.21.15.69:0/3573043104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36e8105140 msgr2=0x7f36e810b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:24.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 --2- 172.21.15.69:0/3573043104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36e8105140 0x7f36e810b9f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f36e000b0d0 tx=0x7f36e000b3e0 comp rx=0 tx=0).stop 2024-05-14T16:44:24.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 -- 172.21.15.69:0/3573043104 shutdown_connections 2024-05-14T16:44:24.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 --2- 172.21.15.69:0/3573043104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36e8105140 0x7f36e810b9f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:24.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 --2- 172.21.15.69:0/3573043104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81035a0 0x7f36e81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:24.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 --2- 172.21.15.69:0/3573043104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e81047a0 0x7f36e8104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 -- 172.21.15.69:0/3573043104 >> 172.21.15.69:0/3573043104 conn(0x7f36e80feb40 msgr2=0x7f36e8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 -- 172.21.15.69:0/3573043104 shutdown_connections 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 -- 172.21.15.69:0/3573043104 wait complete. 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 Processor -- start 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.991+0000 7f36ee687700 1 -- start start 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ee687700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e81035a0 0x7f36e80772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ee687700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81047a0 0x7f36e8075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ee687700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36e8105140 0x7f36e8075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ee687700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f36e810f040 con 0x7f36e8105140 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ee687700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f36e810eec0 con 0x7f36e81047a0 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ee687700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f36e810f1c0 con 0x7f36e81035a0 2024-05-14T16:44:24.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ecc24700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36e8105140 0x7f36e8075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ecc24700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36e8105140 0x7f36e8075e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56944/0 (socket says 172.21.15.69:56944) 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ecc24700 1 -- 172.21.15.69:0/1605024454 learned_addr learned my addr 172.21.15.69:0/1605024454 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36e7fff700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e81035a0 0x7f36e80772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ecc24700 1 -- 172.21.15.69:0/1605024454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e81035a0 msgr2=0x7f36e80772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ecc24700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e81035a0 0x7f36e80772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36e77fe700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81047a0 0x7f36e8075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ecc24700 1 -- 172.21.15.69:0/1605024454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81047a0 msgr2=0x7f36e8075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ecc24700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81047a0 0x7f36e8075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ecc24700 1 -- 172.21.15.69:0/1605024454 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e8076720 con 0x7f36e8105140 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36e77fe700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81047a0 0x7f36e8075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ecc24700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36e8105140 0x7f36e8075e70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009cd0 tx=0x7f36e0004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36e57fa700 1 -- 172.21.15.69:0/1605024454 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e001f030 con 0x7f36e8105140 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36ee687700 1 -- 172.21.15.69:0/1605024454 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e81a3780 con 0x7f36e8105140 2024-05-14T16:44:24.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36e57fa700 1 -- 172.21.15.69:0/1605024454 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e0007a80 con 0x7f36e8105140 2024-05-14T16:44:24.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.992+0000 7f36e57fa700 1 -- 172.21.15.69:0/1605024454 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e001b690 con 0x7f36e8105140 2024-05-14T16:44:24.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.993+0000 7f36ee687700 1 -- 172.21.15.69:0/1605024454 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e81a3c30 con 0x7f36e8105140 2024-05-14T16:44:24.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.993+0000 7f36ee687700 1 -- 172.21.15.69:0/1605024454 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e806a230 con 0x7f36e8105140 2024-05-14T16:44:24.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.995+0000 7f36e57fa700 1 -- 172.21.15.69:0/1605024454 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f36e0022030 con 0x7f36e8105140 2024-05-14T16:44:24.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.995+0000 7f36e57fa700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f36d00b2880 0x7f36d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:24.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.995+0000 7f36e7fff700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f36d00b2880 0x7f36d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:24.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.996+0000 7f36e57fa700 1 -- 172.21.15.69:0/1605024454 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f36e000f020 con 0x7f36e8105140 2024-05-14T16:44:24.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.996+0000 7f36e7fff700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f36d00b2880 0x7f36d00b4d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f36e8104600 tx=0x7f36d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:24.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:24.999+0000 7f36e57fa700 1 -- 172.21.15.69:0/1605024454 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36e0029020 con 0x7f36e8105140 2024-05-14T16:44:25.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:25.209+0000 7f36ee687700 1 -- 172.21.15.69:0/1605024454 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f36e8063980 con 0x7f36d00b2880 2024-05-14T16:44:25.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:25 smithi203 ceph-mon[35787]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:25.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:25 smithi203 ceph-mon[35787]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T16:44:25.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:25 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:44:25.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:25 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:44:25.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:25 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:25.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:25 smithi092 ceph-mon[35763]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:25.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:25 smithi092 ceph-mon[35763]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T16:44:25.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:25 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:44:25.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:25 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:44:25.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:25 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:25.820 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:25 smithi069 ceph-mon[31507]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:25.820 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:25 smithi069 ceph-mon[31507]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T16:44:25.820 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:25 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:44:25.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:25 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:44:25.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:25 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:26.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:26 smithi092 ceph-mon[35763]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:26.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:26 smithi069 ceph-mon[31507]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:26.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:26 smithi203 ceph-mon[35787]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:27.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:27 smithi092 ceph-mon[35763]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:27.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:27 smithi069 ceph-mon[31507]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:27.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:27 smithi203 ceph-mon[35787]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:28.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:28 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/426211854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b029d15-d987-448e-8ea7-513aff17fc95"}]: dispatch 2024-05-14T16:44:28.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:28 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/426211854' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b029d15-d987-448e-8ea7-513aff17fc95"}]': finished 2024-05-14T16:44:28.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:28 smithi069 ceph-mon[31507]: osdmap e56: 10 total, 9 up, 10 in 2024-05-14T16:44:28.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:28.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:28 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/426211854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b029d15-d987-448e-8ea7-513aff17fc95"}]: dispatch 2024-05-14T16:44:28.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:28 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/426211854' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b029d15-d987-448e-8ea7-513aff17fc95"}]': finished 2024-05-14T16:44:28.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:28 smithi092 ceph-mon[35763]: osdmap e56: 10 total, 9 up, 10 in 2024-05-14T16:44:28.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:28.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:28 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/426211854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b029d15-d987-448e-8ea7-513aff17fc95"}]: dispatch 2024-05-14T16:44:28.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:28 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/426211854' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b029d15-d987-448e-8ea7-513aff17fc95"}]': finished 2024-05-14T16:44:28.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:28 smithi203 ceph-mon[35787]: osdmap e56: 10 total, 9 up, 10 in 2024-05-14T16:44:28.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:29.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:29 smithi092 ceph-mon[35763]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:44:29.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:29 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1615776769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:44:29.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:29 smithi069 ceph-mon[31507]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:44:29.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:29 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1615776769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:44:29.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:29 smithi203 ceph-mon[35787]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:44:29.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:29 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1615776769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:44:31.748 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:31 smithi069 ceph-mon[31507]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-14T16:44:31.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:31 smithi092 ceph-mon[35763]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-14T16:44:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:31 smithi203 ceph-mon[35787]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-14T16:44:33.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:33 smithi069 ceph-mon[31507]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:44:33.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:33 smithi092 ceph-mon[35763]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:44:33.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:33 smithi203 ceph-mon[35787]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:44:35.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:35 smithi092 ceph-mon[35763]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-14T16:44:35.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:35 smithi069 ceph-mon[31507]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-14T16:44:35.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:35 smithi203 ceph-mon[35787]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-14T16:44:36.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T16:44:36.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:36.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:36 smithi092 ceph-mon[35763]: Deploying daemon osd.9 on smithi069 2024-05-14T16:44:36.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T16:44:36.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:36.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:36 smithi069 ceph-mon[31507]: Deploying daemon osd.9 on smithi069 2024-05-14T16:44:36.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T16:44:36.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:36.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:36 smithi203 ceph-mon[35787]: Deploying daemon osd.9 on smithi069 2024-05-14T16:44:37.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:37 smithi092 ceph-mon[35763]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:44:37.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:37 smithi069 ceph-mon[31507]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:44:37.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:37 smithi203 ceph-mon[35787]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:44:39.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:39 smithi069 ceph-mon[31507]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:39.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:39 smithi092 ceph-mon[35763]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:39.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:39 smithi203 ceph-mon[35787]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:41.852 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:41 smithi092 ceph-mon[35763]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:41.852 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:41.852 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:41.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:41 smithi069 ceph-mon[31507]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:41.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:41.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:41 smithi203 ceph-mon[35787]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:42.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:42 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:42.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:42 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:42.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:42 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:43.861 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:43 smithi203 ceph-mon[35787]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:43.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:43 smithi069 ceph-mon[31507]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:44.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:43 smithi092 ceph-mon[35763]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:45.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.430+0000 7f36e57fa700 1 -- 172.21.15.69:0/1605024454 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f36e8063980 con 0x7f36d00b2880 2024-05-14T16:44:45.431 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 9 on host 'smithi069' 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 -- 172.21.15.69:0/1605024454 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f36d00b2880 msgr2=0x7f36d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f36d00b2880 0x7f36d00b4d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f36e8104600 tx=0x7f36d8009260 comp rx=0 tx=0).stop 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 -- 172.21.15.69:0/1605024454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36e8105140 msgr2=0x7f36e8075e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36e8105140 0x7f36e8075e70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009cd0 tx=0x7f36e0004e20 comp rx=0 tx=0).stop 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 -- 172.21.15.69:0/1605024454 shutdown_connections 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f36d00b2880 0x7f36d00b4d40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36e8105140 0x7f36e8075e70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e81035a0 0x7f36e80772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 --2- 172.21.15.69:0/1605024454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36e81047a0 0x7f36e8075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 -- 172.21.15.69:0/1605024454 >> 172.21.15.69:0/1605024454 conn(0x7f36e80feb40 msgr2=0x7f36e8107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:45.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 -- 172.21.15.69:0/1605024454 shutdown_connections 2024-05-14T16:44:45.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:44:45.433+0000 7f36ee687700 1 -- 172.21.15.69:0/1605024454 wait complete. 2024-05-14T16:44:45.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:45 smithi069 ceph-mon[31507]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:45.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:45.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:45.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:45.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:45.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:45.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:45 smithi069 ceph-mon[31507]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T16:44:45.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:45.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:45.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:45.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:45 smithi203 ceph-mon[35787]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:45 smithi203 ceph-mon[35787]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T16:44:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:45 smithi092 ceph-mon[35763]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:46.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:46.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:46.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:45 smithi092 ceph-mon[35763]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T16:44:46.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:46.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:46.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:46.126 DEBUG:teuthology.orchestra.run.smithi069:osd.9> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.9.service 2024-05-14T16:44:46.128 INFO:tasks.cephadm:Deploying osd.10 on smithi092 with /dev/vg_nvme/lv_1... 2024-05-14T16:44:46.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T16:44:46.370 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:44:46.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:46 smithi069 ceph-mon[31507]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T16:44:46.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:46 smithi069 ceph-mon[31507]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T16:44:46.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:46 smithi069 ceph-mon[31507]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:44:46.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:46 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:46.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:46 smithi092 ceph-mon[35763]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T16:44:46.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:46 smithi092 ceph-mon[35763]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T16:44:46.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:46 smithi092 ceph-mon[35763]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:44:46.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:46 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:46.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:46 smithi203 ceph-mon[35787]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T16:44:46.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:46 smithi203 ceph-mon[35787]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T16:44:46.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:46 smithi203 ceph-mon[35787]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-14T16:44:46.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:46 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:47.395 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 16:44:46 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T16:44:46.967+0000 7f9b632b5700 -1 osd.9 0 waiting for initial osdmap 2024-05-14T16:44:47.395 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 16:44:46 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T16:44:46.978+0000 7f9b5a89f700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:44:47.557 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:47 smithi069 ceph-mon[31507]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:47.557 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:47 smithi069 ceph-mon[31507]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:44:47.558 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:47 smithi069 ceph-mon[31507]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T16:44:47.558 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:47.558 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:47 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:47.558 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:47 smithi069 ceph-mon[31507]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' 2024-05-14T16:44:47.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:47 smithi092 ceph-mon[35763]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:47.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:47 smithi092 ceph-mon[35763]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:44:47.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:47 smithi092 ceph-mon[35763]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T16:44:47.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:47.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:47 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:47.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:47 smithi092 ceph-mon[35763]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' 2024-05-14T16:44:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:47 smithi203 ceph-mon[35787]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:47 smithi203 ceph-mon[35787]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-14T16:44:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:47 smithi203 ceph-mon[35787]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T16:44:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:47 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:47 smithi203 ceph-mon[35787]: from='osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425]' entity='osd.9' 2024-05-14T16:44:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T16:44:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T16:44:48.383 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-14T16:44:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-14T16:44:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0117344 s, 894 MB/s 2024-05-14T16:44:48.383 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 2024-05-14T16:44:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-14T16:44:48.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:48 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:44:48.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:48 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:44:48.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:48 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:48.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:48 smithi069 ceph-mon[31507]: osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] boot 2024-05-14T16:44:48.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:48 smithi069 ceph-mon[31507]: osdmap e59: 10 total, 10 up, 10 in 2024-05-14T16:44:48.850 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:48 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:48.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:48 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:44:48.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:48 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:44:48.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:48 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:48.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:48 smithi203 ceph-mon[35787]: osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] boot 2024-05-14T16:44:48.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:48 smithi203 ceph-mon[35787]: osdmap e59: 10 total, 10 up, 10 in 2024-05-14T16:44:48.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:48 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:48.966 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:48 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:44:48.966 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:48 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:44:48.966 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:48 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:48.966 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:48 smithi092 ceph-mon[35763]: osd.9 [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] boot 2024-05-14T16:44:48.966 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:48 smithi092 ceph-mon[35763]: osdmap e59: 10 total, 10 up, 10 in 2024-05-14T16:44:48.966 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:48 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:44:48.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-05-14T16:44:49.231 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:44:49.611 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:49.611 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T16:44:49.611 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.611 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.611 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.612 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.612 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.612 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.612 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.612 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:49.612 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:49.612 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:49.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:44:49.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T16:44:49.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:49.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:49.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:49.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:49.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.805+0000 7f2e1429d700 1 -- 172.21.15.92:0/3779499318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e0c1051d0 msgr2=0x7f2e0c10ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.805+0000 7f2e1429d700 1 --2- 172.21.15.92:0/3779499318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e0c1051d0 0x7f2e0c10ba80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009990 tx=0x7f2e04009ca0 comp rx=0 tx=0).stop 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.805+0000 7f2e1429d700 1 -- 172.21.15.92:0/3779499318 shutdown_connections 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.805+0000 7f2e1429d700 1 --2- 172.21.15.92:0/3779499318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0c101850 0x7f2e0c101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.805+0000 7f2e1429d700 1 --2- 172.21.15.92:0/3779499318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e0c1051d0 0x7f2e0c10ba80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.805+0000 7f2e1429d700 1 --2- 172.21.15.92:0/3779499318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e0c104830 0x7f2e0c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.805+0000 7f2e1429d700 1 -- 172.21.15.92:0/3779499318 >> 172.21.15.92:0/3779499318 conn(0x7f2e0c0fd1d0 msgr2=0x7f2e0c0ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.805+0000 7f2e1429d700 1 -- 172.21.15.92:0/3779499318 shutdown_connections 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.805+0000 7f2e1429d700 1 -- 172.21.15.92:0/3779499318 wait complete. 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e1429d700 1 Processor -- start 2024-05-14T16:44:50.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e1429d700 1 -- start start 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e1429d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e0c101850 0x7f2e0c104190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e1429d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0c104830 0x7f2e0c1027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e1429d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e0c1051d0 0x7f2e0c102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e1429d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c10f020 con 0x7f2e0c104830 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e1429d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c10eea0 con 0x7f2e0c101850 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e1429d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c10f1a0 con 0x7f2e0c1051d0 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e12039700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e0c101850 0x7f2e0c104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e12039700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e0c101850 0x7f2e0c104190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42540/0 (socket says 172.21.15.92:42540) 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e12039700 1 -- 172.21.15.92:0/3535793990 learned_addr learned my addr 172.21.15.92:0/3535793990 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e11838700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0c104830 0x7f2e0c1027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e1283a700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e0c1051d0 0x7f2e0c102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e12039700 1 -- 172.21.15.92:0/3535793990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e0c1051d0 msgr2=0x7f2e0c102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e12039700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e0c1051d0 0x7f2e0c102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:50.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e12039700 1 -- 172.21.15.92:0/3535793990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0c104830 msgr2=0x7f2e0c1027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:50.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e12039700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0c104830 0x7f2e0c1027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:50.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e12039700 1 -- 172.21.15.92:0/3535793990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c1035d0 con 0x7f2e0c101850 2024-05-14T16:44:50.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.806+0000 7f2e12039700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e0c101850 0x7f2e0c104190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc007e00 tx=0x7f2dfc00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:50.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.807+0000 7f2e0b7fe700 1 -- 172.21.15.92:0/3535793990 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc016020 con 0x7f2e0c101850 2024-05-14T16:44:50.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.807+0000 7f2e0b7fe700 1 -- 172.21.15.92:0/3535793990 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dfc00ad40 con 0x7f2e0c101850 2024-05-14T16:44:50.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.807+0000 7f2e1429d700 1 -- 172.21.15.92:0/3535793990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c1a3740 con 0x7f2e0c101850 2024-05-14T16:44:50.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.807+0000 7f2e0b7fe700 1 -- 172.21.15.92:0/3535793990 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc011670 con 0x7f2e0c101850 2024-05-14T16:44:50.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.807+0000 7f2e1429d700 1 -- 172.21.15.92:0/3535793990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c1a3c80 con 0x7f2e0c101850 2024-05-14T16:44:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.808+0000 7f2e1429d700 1 -- 172.21.15.92:0/3535793990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c06a230 con 0x7f2e0c101850 2024-05-14T16:44:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.809+0000 7f2e0b7fe700 1 -- 172.21.15.92:0/3535793990 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2dfc019020 con 0x7f2e0c101850 2024-05-14T16:44:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.809+0000 7f2e0b7fe700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2df40b2900 0x7f2df40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.810+0000 7f2e0b7fe700 1 -- 172.21.15.92:0/3535793990 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7f2dfc0e1640 con 0x7f2e0c101850 2024-05-14T16:44:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.810+0000 7f2e11838700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2df40b2900 0x7f2df40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:50.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.810+0000 7f2e11838700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2df40b2900 0x7f2df40b4dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2e00004630 tx=0x7f2e0000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:50.814+0000 7f2e0b7fe700 1 -- 172.21.15.92:0/3535793990 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2dfc020030 con 0x7f2e0c101850 2024-05-14T16:44:50.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:50 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi069 2024-05-14T16:44:50.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:50 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi069 to 4272M 2024-05-14T16:44:50.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:50 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi069 2024-05-14T16:44:50.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:50 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi069 to 4272M 2024-05-14T16:44:50.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:50 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi069 2024-05-14T16:44:50.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:50 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi069 to 4272M 2024-05-14T16:44:51.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:44:51.025+0000 7f2e1429d700 1 -- 172.21.15.92:0/3535793990 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2e0c063980 con 0x7f2df40b2900 2024-05-14T16:44:51.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:51 smithi069 ceph-mon[31507]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:51.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:51 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:44:51.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:51 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:44:51.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:51 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:51.896 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:51 smithi092 ceph-mon[35763]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:51.896 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:51 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:44:51.896 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:51 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:44:51.896 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:51 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:51.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:51 smithi203 ceph-mon[35787]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:51.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:51 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:44:51.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:51 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:44:51.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:51 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:52.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:52 smithi092 ceph-mon[35763]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:52.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:52 smithi069 ceph-mon[31507]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:52.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:52 smithi203 ceph-mon[35787]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:53.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:53 smithi069 ceph-mon[31507]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:53.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:53 smithi203 ceph-mon[35787]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:54.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:53 smithi092 ceph-mon[35763]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:54.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:54 smithi069 ceph-mon[31507]: from='client.? 172.21.15.92:0/3835916572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bce9e619-6203-4ae1-b02c-322eee6f809a"}]: dispatch 2024-05-14T16:44:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:54 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bce9e619-6203-4ae1-b02c-322eee6f809a"}]: dispatch 2024-05-14T16:44:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:54 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bce9e619-6203-4ae1-b02c-322eee6f809a"}]': finished 2024-05-14T16:44:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:54 smithi069 ceph-mon[31507]: osdmap e61: 11 total, 10 up, 11 in 2024-05-14T16:44:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:54 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:44:54.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:54 smithi069 ceph-mon[31507]: from='client.? 172.21.15.92:0/3036693540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:44:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:54 smithi203 ceph-mon[35787]: from='client.? 172.21.15.92:0/3835916572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bce9e619-6203-4ae1-b02c-322eee6f809a"}]: dispatch 2024-05-14T16:44:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:54 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bce9e619-6203-4ae1-b02c-322eee6f809a"}]: dispatch 2024-05-14T16:44:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:54 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bce9e619-6203-4ae1-b02c-322eee6f809a"}]': finished 2024-05-14T16:44:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:54 smithi203 ceph-mon[35787]: osdmap e61: 11 total, 10 up, 11 in 2024-05-14T16:44:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:54 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:44:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:54 smithi203 ceph-mon[35787]: from='client.? 172.21.15.92:0/3036693540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:44:55.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:54 smithi092 ceph-mon[35763]: from='client.? 172.21.15.92:0/3835916572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bce9e619-6203-4ae1-b02c-322eee6f809a"}]: dispatch 2024-05-14T16:44:55.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:54 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bce9e619-6203-4ae1-b02c-322eee6f809a"}]: dispatch 2024-05-14T16:44:55.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:54 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bce9e619-6203-4ae1-b02c-322eee6f809a"}]': finished 2024-05-14T16:44:55.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:54 smithi092 ceph-mon[35763]: osdmap e61: 11 total, 10 up, 11 in 2024-05-14T16:44:55.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:54 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:44:55.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:54 smithi092 ceph-mon[35763]: from='client.? 172.21.15.92:0/3036693540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:44:55.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:55 smithi069 ceph-mon[31507]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:55.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:55 smithi203 ceph-mon[35787]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:55 smithi092 ceph-mon[35763]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:57.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:57 smithi069 ceph-mon[31507]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:57 smithi203 ceph-mon[35787]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:57.968 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:57 smithi092 ceph-mon[35763]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:59.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:44:59 smithi092 ceph-mon[35763]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:59.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:44:59 smithi069 ceph-mon[31507]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:44:59.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:44:59 smithi203 ceph-mon[35787]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:01 smithi092 ceph-mon[35763]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:01 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T16:45:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:01 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:01.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:01 smithi069 ceph-mon[31507]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:01.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:01 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T16:45:01.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:01 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:01 smithi203 ceph-mon[35787]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:01 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T16:45:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:01 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:02.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:02 smithi092 ceph-mon[35763]: Deploying daemon osd.10 on smithi092 2024-05-14T16:45:02.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:02 smithi069 ceph-mon[31507]: Deploying daemon osd.10 on smithi092 2024-05-14T16:45:02.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:02 smithi203 ceph-mon[35787]: Deploying daemon osd.10 on smithi092 2024-05-14T16:45:03.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:03 smithi069 ceph-mon[31507]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:03.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:03 smithi203 ceph-mon[35787]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:03 smithi092 ceph-mon[35763]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:05.861 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:05 smithi092 ceph-mon[35763]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:05.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:05 smithi069 ceph-mon[31507]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:05 smithi203 ceph-mon[35787]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:07.780 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:07 smithi092 ceph-mon[35763]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:07.780 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:07.780 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:07.780 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:07.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:07 smithi069 ceph-mon[31507]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:07.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:07.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:07.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:07 smithi203 ceph-mon[35787]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:09.896 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:09 smithi092 ceph-mon[35763]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:09.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:09 smithi069 ceph-mon[31507]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:09.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:09 smithi203 ceph-mon[35787]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.291+0000 7f2e0b7fe700 1 -- 172.21.15.92:0/3535793990 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2e0c063980 con 0x7f2df40b2900 2024-05-14T16:45:10.292 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 10 on host 'smithi092' 2024-05-14T16:45:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 -- 172.21.15.92:0/3535793990 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2df40b2900 msgr2=0x7f2df40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2df40b2900 0x7f2df40b4dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2e00004630 tx=0x7f2e0000b3c0 comp rx=0 tx=0).stop 2024-05-14T16:45:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 -- 172.21.15.92:0/3535793990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e0c101850 msgr2=0x7f2e0c104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e0c101850 0x7f2e0c104190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc007e00 tx=0x7f2dfc00a340 comp rx=0 tx=0).stop 2024-05-14T16:45:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 -- 172.21.15.92:0/3535793990 shutdown_connections 2024-05-14T16:45:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e0c1051d0 0x7f2e0c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e0c101850 0x7f2e0c104190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2df40b2900 0x7f2df40b4dc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 --2- 172.21.15.92:0/3535793990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0c104830 0x7f2e0c1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 -- 172.21.15.92:0/3535793990 >> 172.21.15.92:0/3535793990 conn(0x7f2e0c0fd1d0 msgr2=0x7f2e0c107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 -- 172.21.15.92:0/3535793990 shutdown_connections 2024-05-14T16:45:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-14T16:45:10.293+0000 7f2e1429d700 1 -- 172.21.15.92:0/3535793990 wait complete. 2024-05-14T16:45:11.319 DEBUG:teuthology.orchestra.run.smithi092:osd.10> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.10.service 2024-05-14T16:45:11.322 INFO:tasks.cephadm:Deploying osd.11 on smithi203 with /dev/vg_nvme/lv_1... 2024-05-14T16:45:11.322 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T16:45:11.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:11 smithi092 ceph-mon[35763]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:11.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:11 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:11.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:11 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:11.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:11 smithi092 ceph-mon[35763]: from='osd.10 [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:45:11.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:11 smithi092 ceph-mon[35763]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:45:11.571 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:45:11.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:11 smithi203 ceph-mon[35787]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:11.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:11 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:11.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:11 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:11.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:11 smithi203 ceph-mon[35787]: from='osd.10 [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:45:11.694 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:11 smithi203 ceph-mon[35787]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:45:11.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:11 smithi069 ceph-mon[31507]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:11.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:11 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:11.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:11 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:11.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:11 smithi069 ceph-mon[31507]: from='osd.10 [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:45:11.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:11 smithi069 ceph-mon[31507]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:45:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:12 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:12 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:12 smithi092 ceph-mon[35763]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T16:45:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:12 smithi092 ceph-mon[35763]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T16:45:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:12 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:12 smithi092 ceph-mon[35763]: from='osd.10 [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:45:12.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:12 smithi092 ceph-mon[35763]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:45:12.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:12 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:12.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:12 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:12.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:12 smithi203 ceph-mon[35787]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T16:45:12.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:12 smithi203 ceph-mon[35787]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T16:45:12.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:12 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:12.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:12 smithi203 ceph-mon[35787]: from='osd.10 [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:45:12.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:12 smithi203 ceph-mon[35787]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:45:12.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:12 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:12.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:12 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:12 smithi069 ceph-mon[31507]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T16:45:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:12 smithi069 ceph-mon[31507]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T16:45:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:12 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:12 smithi069 ceph-mon[31507]: from='osd.10 [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:45:12.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:12 smithi069 ceph-mon[31507]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-14T16:45:13.194 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 16:45:13 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T16:45:13.087+0000 7f86a128c700 -1 osd.10 0 waiting for initial osdmap 2024-05-14T16:45:13.194 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 16:45:13 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T16:45:13.098+0000 7f8699077700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:45:13.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:13 smithi092 ceph-mon[35763]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:13.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:13 smithi092 ceph-mon[35763]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:45:13.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:13 smithi092 ceph-mon[35763]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T16:45:13.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:13 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:13.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:13 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:13.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:13 smithi092 ceph-mon[35763]: from='osd.10 ' entity='osd.10' 2024-05-14T16:45:13.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:13 smithi092 ceph-mon[35763]: osd.10 [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] boot 2024-05-14T16:45:13.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:13 smithi092 ceph-mon[35763]: osdmap e64: 11 total, 11 up, 11 in 2024-05-14T16:45:13.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:13 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:13.786 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T16:45:13.786 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T16:45:13.786 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-05-14T16:45:13.786 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-05-14T16:45:13.786 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011481 s, 913 MB/s 2024-05-14T16:45:13.786 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-05-14T16:45:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:13 smithi069 ceph-mon[31507]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:13 smithi069 ceph-mon[31507]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:45:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:13 smithi069 ceph-mon[31507]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T16:45:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:13 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:13 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:13 smithi069 ceph-mon[31507]: from='osd.10 ' entity='osd.10' 2024-05-14T16:45:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:13 smithi069 ceph-mon[31507]: osd.10 [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] boot 2024-05-14T16:45:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:13 smithi069 ceph-mon[31507]: osdmap e64: 11 total, 11 up, 11 in 2024-05-14T16:45:13.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:13 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:13.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:13 smithi203 ceph-mon[35787]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:45:13.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:13 smithi203 ceph-mon[35787]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-14T16:45:13.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:13 smithi203 ceph-mon[35787]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T16:45:13.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:13 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:13.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:13 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:13.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:13 smithi203 ceph-mon[35787]: from='osd.10 ' entity='osd.10' 2024-05-14T16:45:13.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:13 smithi203 ceph-mon[35787]: osd.10 [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] boot 2024-05-14T16:45:13.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:13 smithi203 ceph-mon[35787]: osdmap e64: 11 total, 11 up, 11 in 2024-05-14T16:45:13.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:13 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:45:14.526 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_1 2024-05-14T16:45:14.774 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:45:14.803 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:14 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:45:14.803 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:14 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:45:14.803 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:14 smithi203 ceph-mon[35787]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T16:45:14.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:14 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:45:14.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:14 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:45:14.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:14 smithi069 ceph-mon[31507]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T16:45:14.971 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:14 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:45:14.971 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:14 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:45:14.971 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:14 smithi092 ceph-mon[35763]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T16:45:15.918 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:15.919 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:15.920 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:15.920 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:15 smithi203 ceph-mon[35787]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T16:45:15.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:15.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:15.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:15.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:15.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:15 smithi069 ceph-mon[31507]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T16:45:16.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:16.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:16.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:16.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:16.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:15 smithi092 ceph-mon[35763]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T16:45:16.431 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 -- 172.21.15.203:0/3365653563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f416c105140 msgr2=0x7f416c10b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:16.431 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 --2- 172.21.15.203:0/3365653563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f416c105140 0x7f416c10b9f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f416400b0a0 tx=0x7f416400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:45:16.431 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 -- 172.21.15.203:0/3365653563 shutdown_connections 2024-05-14T16:45:16.431 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 --2- 172.21.15.203:0/3365653563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f416c105140 0x7f416c10b9f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:16.431 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 --2- 172.21.15.203:0/3365653563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f416c1035a0 0x7f416c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:16.431 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 --2- 172.21.15.203:0/3365653563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416c1047a0 0x7f416c104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:16.431 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 -- 172.21.15.203:0/3365653563 >> 172.21.15.203:0/3365653563 conn(0x7f416c0feb40 msgr2=0x7f416c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:16.431 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 -- 172.21.15.203:0/3365653563 shutdown_connections 2024-05-14T16:45:16.431 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 -- 172.21.15.203:0/3365653563 wait complete. 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 Processor -- start 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.429+0000 7f4171e43700 1 -- start start 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4171e43700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f416c1035a0 0x7f416c0772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4171e43700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f416c1047a0 0x7f416c075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4171e43700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416c105140 0x7f416c075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4171e43700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10f020 con 0x7f416c105140 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4171e43700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10eea0 con 0x7f416c1035a0 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4171e43700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10f1a0 con 0x7f416c1047a0 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f416c1047a0 0x7f416c075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f416c1047a0 0x7f416c075930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:46504/0 (socket says 172.21.15.203:46504) 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416affd700 1 -- 172.21.15.203:0/1766593036 learned_addr learned my addr 172.21.15.203:0/1766593036 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416bfff700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416c105140 0x7f416c075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416b7fe700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f416c1035a0 0x7f416c0772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416affd700 1 -- 172.21.15.203:0/1766593036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f416c1035a0 msgr2=0x7f416c0772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:16.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416affd700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f416c1035a0 0x7f416c0772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:16.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416affd700 1 -- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416c105140 msgr2=0x7f416c075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:16.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416affd700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416c105140 0x7f416c075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:16.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416affd700 1 -- 172.21.15.203:0/1766593036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c076720 con 0x7f416c1047a0 2024-05-14T16:45:16.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416bfff700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416c105140 0x7f416c075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:45:16.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f416affd700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f416c1047a0 0x7f416c075930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f416000c9b0 tx=0x7f416000cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:16.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4168ff9700 1 -- 172.21.15.203:0/1766593036 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160007b00 con 0x7f416c1047a0 2024-05-14T16:45:16.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4168ff9700 1 -- 172.21.15.203:0/1766593036 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4160007ca0 con 0x7f416c1047a0 2024-05-14T16:45:16.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4171e43700 1 -- 172.21.15.203:0/1766593036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1a36d0 con 0x7f416c1047a0 2024-05-14T16:45:16.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4168ff9700 1 -- 172.21.15.203:0/1766593036 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160018730 con 0x7f416c1047a0 2024-05-14T16:45:16.433 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.430+0000 7f4171e43700 1 -- 172.21.15.203:0/1766593036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1a3be0 con 0x7f416c1047a0 2024-05-14T16:45:16.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.432+0000 7f4171e43700 1 -- 172.21.15.203:0/1766593036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c06a230 con 0x7f416c1047a0 2024-05-14T16:45:16.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.433+0000 7f4168ff9700 1 -- 172.21.15.203:0/1766593036 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f416000f450 con 0x7f416c1047a0 2024-05-14T16:45:16.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.433+0000 7f4168ff9700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f41540b2880 0x7f41540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:16.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.433+0000 7f4168ff9700 1 -- 172.21.15.203:0/1766593036 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7f41600e1c40 con 0x7f416c1047a0 2024-05-14T16:45:16.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.433+0000 7f416b7fe700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f41540b2880 0x7f41540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:16.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.434+0000 7f416b7fe700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f41540b2880 0x7f41540b4d40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f415c005fd0 tx=0x7f415c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:16.437 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.438+0000 7f4168ff9700 1 -- 172.21.15.203:0/1766593036 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4160014030 con 0x7f416c1047a0 2024-05-14T16:45:16.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:16.650+0000 7f4171e43700 1 -- 172.21.15.203:0/1766593036 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f416c02e990 con 0x7f41540b2880 2024-05-14T16:45:16.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:16 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi092 2024-05-14T16:45:16.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:16 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi092 to 4272M 2024-05-14T16:45:16.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:16 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi092 2024-05-14T16:45:16.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:16 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi092 to 4272M 2024-05-14T16:45:17.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:16 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi092 2024-05-14T16:45:17.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:16 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi092 to 4272M 2024-05-14T16:45:17.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:17 smithi069 ceph-mon[31507]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:17.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:17 smithi069 ceph-mon[31507]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:17.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:17 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:45:17.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:17 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:45:17.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:17 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:17 smithi203 ceph-mon[35787]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:17 smithi203 ceph-mon[35787]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:17 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:45:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:17 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:45:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:17 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:17 smithi092 ceph-mon[35763]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:17 smithi092 ceph-mon[35763]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:17 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:45:18.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:17 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:45:18.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:17 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:19.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:19 smithi069 ceph-mon[31507]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-14T16:45:19.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:19 smithi069 ceph-mon[31507]: from='client.? 172.21.15.203:0/3818379440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9acab8e8-82ae-44b8-aba6-87b3a99ebe62"}]: dispatch 2024-05-14T16:45:19.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:19 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9acab8e8-82ae-44b8-aba6-87b3a99ebe62"}]: dispatch 2024-05-14T16:45:19.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:19 smithi069 ceph-mon[31507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9acab8e8-82ae-44b8-aba6-87b3a99ebe62"}]': finished 2024-05-14T16:45:19.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:19 smithi069 ceph-mon[31507]: osdmap e67: 12 total, 11 up, 12 in 2024-05-14T16:45:19.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:19.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:19 smithi203 ceph-mon[35787]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-14T16:45:19.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:19 smithi203 ceph-mon[35787]: from='client.? 172.21.15.203:0/3818379440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9acab8e8-82ae-44b8-aba6-87b3a99ebe62"}]: dispatch 2024-05-14T16:45:19.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:19 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9acab8e8-82ae-44b8-aba6-87b3a99ebe62"}]: dispatch 2024-05-14T16:45:19.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:19 smithi203 ceph-mon[35787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9acab8e8-82ae-44b8-aba6-87b3a99ebe62"}]': finished 2024-05-14T16:45:19.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:19 smithi203 ceph-mon[35787]: osdmap e67: 12 total, 11 up, 12 in 2024-05-14T16:45:19.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:19 smithi092 ceph-mon[35763]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-14T16:45:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:19 smithi092 ceph-mon[35763]: from='client.? 172.21.15.203:0/3818379440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9acab8e8-82ae-44b8-aba6-87b3a99ebe62"}]: dispatch 2024-05-14T16:45:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:19 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9acab8e8-82ae-44b8-aba6-87b3a99ebe62"}]: dispatch 2024-05-14T16:45:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:19 smithi092 ceph-mon[35763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9acab8e8-82ae-44b8-aba6-87b3a99ebe62"}]': finished 2024-05-14T16:45:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:19 smithi092 ceph-mon[35763]: osdmap e67: 12 total, 11 up, 12 in 2024-05-14T16:45:20.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:20.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:20 smithi069 ceph-mon[31507]: from='client.? 172.21.15.203:0/3048614227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:45:20.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:20 smithi203 ceph-mon[35787]: from='client.? 172.21.15.203:0/3048614227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:45:21.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:20 smithi092 ceph-mon[35763]: from='client.? 172.21.15.203:0/3048614227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:45:21.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:21 smithi069 ceph-mon[31507]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:45:21.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:21 smithi203 ceph-mon[35787]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:45:22.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:21 smithi092 ceph-mon[35763]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:45:23.931 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:23 smithi203 ceph-mon[35787]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-14T16:45:23.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:23 smithi069 ceph-mon[31507]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-14T16:45:24.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:23 smithi092 ceph-mon[35763]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-14T16:45:25.877 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:25 smithi203 ceph-mon[35787]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-14T16:45:25.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:25 smithi069 ceph-mon[31507]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-14T16:45:26.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:25 smithi092 ceph-mon[35763]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-14T16:45:27.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:27 smithi069 ceph-mon[31507]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:45:27.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:27 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T16:45:27.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:27 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:27.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:27 smithi203 ceph-mon[35787]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:45:27.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:27 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T16:45:27.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:27 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:28.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:27 smithi092 ceph-mon[35763]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:45:28.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:27 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T16:45:28.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:27 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:28.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:28 smithi069 ceph-mon[31507]: Deploying daemon osd.11 on smithi203 2024-05-14T16:45:28.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:28 smithi203 ceph-mon[35787]: Deploying daemon osd.11 on smithi203 2024-05-14T16:45:29.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:28 smithi092 ceph-mon[35763]: Deploying daemon osd.11 on smithi203 2024-05-14T16:45:29.953 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:29 smithi203 ceph-mon[35787]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:29.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:29 smithi069 ceph-mon[31507]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:30.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:29 smithi092 ceph-mon[35763]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:31.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:31 smithi069 ceph-mon[31507]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:31 smithi203 ceph-mon[35787]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:32.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:31 smithi092 ceph-mon[35763]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:33.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:33 smithi069 ceph-mon[31507]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:33.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:33.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:33.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:33.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:33 smithi203 ceph-mon[35787]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:33.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:33.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:33.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:33 smithi092 ceph-mon[35763]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:35.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:35 smithi069 ceph-mon[31507]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:35.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:35 smithi203 ceph-mon[35787]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:36.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:35 smithi092 ceph-mon[35763]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:36.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.867+0000 7f4168ff9700 1 -- 172.21.15.203:0/1766593036 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f416c02e990 con 0x7f41540b2880 2024-05-14T16:45:36.867 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 11 on host 'smithi203' 2024-05-14T16:45:36.869 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 -- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f41540b2880 msgr2=0x7f41540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f41540b2880 0x7f41540b4d40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f415c005fd0 tx=0x7f415c005e50 comp rx=0 tx=0).stop 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 -- 172.21.15.203:0/1766593036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f416c1047a0 msgr2=0x7f416c075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f416c1047a0 0x7f416c075930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f416000c9b0 tx=0x7f416000cd70 comp rx=0 tx=0).stop 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 -- 172.21.15.203:0/1766593036 shutdown_connections 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f41540b2880 0x7f41540b4d40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416c105140 0x7f416c075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f416c1035a0 0x7f416c0772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 --2- 172.21.15.203:0/1766593036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f416c1047a0 0x7f416c075930 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 -- 172.21.15.203:0/1766593036 >> 172.21.15.203:0/1766593036 conn(0x7f416c0feb40 msgr2=0x7f416c107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 -- 172.21.15.203:0/1766593036 shutdown_connections 2024-05-14T16:45:36.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:36.870+0000 7f4171e43700 1 -- 172.21.15.203:0/1766593036 wait complete. 2024-05-14T16:45:37.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:37.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:37.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:37.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: from='osd.11 [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:45:37.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:45:37.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:36 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:37.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:37.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:37.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:37.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: from='osd.11 [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:45:37.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:45:37.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:36 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:37.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:37.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:37.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:37.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: from='osd.11 [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:45:37.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:45:37.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:37.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:36 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:37.603 DEBUG:teuthology.orchestra.run.smithi203:osd.11> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.11.service 2024-05-14T16:45:37.606 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-14T16:45:37.606 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-14T16:45:37.856 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:45:38.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:38 smithi069 ceph-mon[31507]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T16:45:38.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:38 smithi069 ceph-mon[31507]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T16:45:38.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:38.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:38 smithi069 ceph-mon[31507]: from='osd.11 [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:45:38.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:38 smithi069 ceph-mon[31507]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:45:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:38 smithi203 ceph-mon[35787]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T16:45:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:38 smithi203 ceph-mon[35787]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T16:45:38.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:38.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:38 smithi203 ceph-mon[35787]: from='osd.11 [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:45:38.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:38 smithi203 ceph-mon[35787]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:45:38.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 16:45:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T16:45:38.685+0000 7efdb210e700 -1 osd.11 0 waiting for initial osdmap 2024-05-14T16:45:38.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 16:45:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T16:45:38.696+0000 7efdadf01700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:45:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:38 smithi092 ceph-mon[35763]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T16:45:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:38 smithi092 ceph-mon[35763]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T16:45:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:38 smithi092 ceph-mon[35763]: from='osd.11 [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:45:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:38 smithi092 ceph-mon[35763]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-05-14T16:45:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.701+0000 7fc8e4289700 1 -- 172.21.15.69:0/1409596530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8dc101610 msgr2=0x7fc8dc101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.701+0000 7fc8e4289700 1 --2- 172.21.15.69:0/1409596530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8dc101610 0x7fc8dc101a90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0099c0 tx=0x7fc8cc009cd0 comp rx=0 tx=0).stop 2024-05-14T16:45:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.701+0000 7fc8e4289700 1 -- 172.21.15.69:0/1409596530 shutdown_connections 2024-05-14T16:45:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.701+0000 7fc8e4289700 1 --2- 172.21.15.69:0/1409596530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc101fd0 0x7fc8dc10bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.701+0000 7fc8e4289700 1 --2- 172.21.15.69:0/1409596530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8dc104b90 0x7fc8dc104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.701+0000 7fc8e4289700 1 --2- 172.21.15.69:0/1409596530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8dc101610 0x7fc8dc101a90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.701+0000 7fc8e4289700 1 -- 172.21.15.69:0/1409596530 >> 172.21.15.69:0/1409596530 conn(0x7fc8dc0fd1d0 msgr2=0x7fc8dc0ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.701+0000 7fc8e4289700 1 -- 172.21.15.69:0/1409596530 shutdown_connections 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.701+0000 7fc8e4289700 1 -- 172.21.15.69:0/1409596530 wait complete. 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e4289700 1 Processor -- start 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e4289700 1 -- start start 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e4289700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8dc101610 0x7fc8dc1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e4289700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc101fd0 0x7fc8dc199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e4289700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8dc104b90 0x7fc8dc1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e4289700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc10f020 con 0x7fc8dc104b90 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e4289700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc10eea0 con 0x7fc8dc101fd0 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e4289700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc10f1a0 con 0x7fc8dc101610 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e2025700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8dc101610 0x7fc8dc1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e2826700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8dc104b90 0x7fc8dc1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e2025700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8dc101610 0x7fc8dc1996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:49942/0 (socket says 172.21.15.69:49942) 2024-05-14T16:45:39.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e2025700 1 -- 172.21.15.69:0/1368325718 learned_addr learned my addr 172.21.15.69:0/1368325718 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.702+0000 7fc8e1824700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc101fd0 0x7fc8dc199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8e2826700 1 -- 172.21.15.69:0/1368325718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8dc101610 msgr2=0x7fc8dc1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8e2826700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8dc101610 0x7fc8dc1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8e2826700 1 -- 172.21.15.69:0/1368325718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc101fd0 msgr2=0x7fc8dc199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8e2826700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc101fd0 0x7fc8dc199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8e2826700 1 -- 172.21.15.69:0/1368325718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc1a1320 con 0x7fc8dc104b90 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8e2025700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8dc101610 0x7fc8dc1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8e2826700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8dc104b90 0x7fc8dc1a0c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009f60 tx=0x7fc8d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8d37fe700 1 -- 172.21.15.69:0/1368325718 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40092e0 con 0x7fc8dc104b90 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8d37fe700 1 -- 172.21.15.69:0/1368325718 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d4004bb0 con 0x7fc8dc104b90 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8e4289700 1 -- 172.21.15.69:0/1368325718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8dc1a15b0 con 0x7fc8dc104b90 2024-05-14T16:45:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8d37fe700 1 -- 172.21.15.69:0/1368325718 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4004280 con 0x7fc8dc104b90 2024-05-14T16:45:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.703+0000 7fc8e4289700 1 -- 172.21.15.69:0/1368325718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8dc1a1aa0 con 0x7fc8dc104b90 2024-05-14T16:45:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.704+0000 7fc8e4289700 1 -- 172.21.15.69:0/1368325718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8dc06a230 con 0x7fc8dc104b90 2024-05-14T16:45:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.705+0000 7fc8d37fe700 1 -- 172.21.15.69:0/1368325718 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc8d401b030 con 0x7fc8dc104b90 2024-05-14T16:45:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.706+0000 7fc8d37fe700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc8c80b2880 0x7fc8c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.706+0000 7fc8e2025700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc8c80b2880 0x7fc8c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.706+0000 7fc8d37fe700 1 -- 172.21.15.69:0/1368325718 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fc8d40e1880 con 0x7fc8dc104b90 2024-05-14T16:45:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.706+0000 7fc8e2025700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc8c80b2880 0x7fc8c80b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8005950 tx=0x7fc8d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.710+0000 7fc8d37fe700 1 -- 172.21.15.69:0/1368325718 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc8d4022020 con 0x7fc8dc104b90 2024-05-14T16:45:39.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.912+0000 7fc8e4289700 1 -- 172.21.15.69:0/1368325718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc8dc064150 con 0x7fc8dc104b90 2024-05-14T16:45:39.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.912+0000 7fc8d37fe700 1 -- 172.21.15.69:0/1368325718 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7fc8d4010040 con 0x7fc8dc104b90 2024-05-14T16:45:39.913 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:45:39.913 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:39 smithi069 ceph-mon[31507]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:39.913 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:39 smithi069 ceph-mon[31507]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:45:39.913 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:39 smithi069 ceph-mon[31507]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T16:45:39.913 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:39 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:39.913 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:39 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:39.913 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:39 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:39.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 -- 172.21.15.69:0/1368325718 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc8c80b2880 msgr2=0x7fc8c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:39.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc8c80b2880 0x7fc8c80b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8005950 tx=0x7fc8d8009260 comp rx=0 tx=0).stop 2024-05-14T16:45:39.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 -- 172.21.15.69:0/1368325718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8dc104b90 msgr2=0x7fc8dc1a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:39.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8dc104b90 0x7fc8dc1a0c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009f60 tx=0x7fc8d4009f90 comp rx=0 tx=0).stop 2024-05-14T16:45:39.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 -- 172.21.15.69:0/1368325718 shutdown_connections 2024-05-14T16:45:39.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc8c80b2880 0x7fc8c80b4d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:39.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8dc104b90 0x7fc8dc1a0c80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:39.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8dc101610 0x7fc8dc1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:39.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 --2- 172.21.15.69:0/1368325718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc101fd0 0x7fc8dc199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:39.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 -- 172.21.15.69:0/1368325718 >> 172.21.15.69:0/1368325718 conn(0x7fc8dc0fd1d0 msgr2=0x7fc8dc107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:39.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 -- 172.21.15.69:0/1368325718 shutdown_connections 2024-05-14T16:45:39.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:39.915+0000 7fc8e4289700 1 -- 172.21.15.69:0/1368325718 wait complete. 2024-05-14T16:45:39.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:39 smithi203 ceph-mon[35787]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:39.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:39 smithi203 ceph-mon[35787]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:45:39.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:39 smithi203 ceph-mon[35787]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T16:45:39.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:39 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:39.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:39 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:39.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:39 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:39 smithi092 ceph-mon[35763]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:39 smithi092 ceph-mon[35763]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-05-14T16:45:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:39 smithi092 ceph-mon[35763]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T16:45:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:39 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:39 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:40.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:39 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:40.677 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1715705139,"num_in_osds":12,"osd_in_since":1715705119,"num_remapped_pgs":0} 2024-05-14T16:45:40.677 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:45:40.938 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:45:40.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: purged_snaps scrub starts 2024-05-14T16:45:40.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: purged_snaps scrub ok 2024-05-14T16:45:40.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: osd.11 [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] boot 2024-05-14T16:45:40.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: osdmap e70: 12 total, 12 up, 12 in 2024-05-14T16:45:40.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:40.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1368325718' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:45:40.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:40.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:40.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:40.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:40.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:40.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:40 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:40.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: purged_snaps scrub starts 2024-05-14T16:45:40.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: purged_snaps scrub ok 2024-05-14T16:45:40.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: osd.11 [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] boot 2024-05-14T16:45:40.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: osdmap e70: 12 total, 12 up, 12 in 2024-05-14T16:45:40.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:40.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1368325718' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:45:40.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:40.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:40.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:40.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:40.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:40.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:40 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:41.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: purged_snaps scrub starts 2024-05-14T16:45:41.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: purged_snaps scrub ok 2024-05-14T16:45:41.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: osd.11 [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] boot 2024-05-14T16:45:41.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: osdmap e70: 12 total, 12 up, 12 in 2024-05-14T16:45:41.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:45:41.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1368325718' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:45:41.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:41.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:41.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:41.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:41.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:41.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:40 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:41.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:41 smithi069 ceph-mon[31507]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:41.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:41 smithi069 ceph-mon[31507]: Detected new or changed devices on smithi203 2024-05-14T16:45:41.952 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:41 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi203 to 4272M 2024-05-14T16:45:41.952 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:41.952 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:41.952 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:41.952 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:41 smithi069 ceph-mon[31507]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T16:45:41.952 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:41 smithi203 ceph-mon[35787]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:41 smithi203 ceph-mon[35787]: Detected new or changed devices on smithi203 2024-05-14T16:45:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:41 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi203 to 4272M 2024-05-14T16:45:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:41 smithi203 ceph-mon[35787]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T16:45:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:41 smithi092 ceph-mon[35763]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:45:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:41 smithi092 ceph-mon[35763]: Detected new or changed devices on smithi203 2024-05-14T16:45:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:41 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi203 to 4272M 2024-05-14T16:45:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:41 smithi092 ceph-mon[35763]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T16:45:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:42.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.596+0000 7f327b668700 1 -- 172.21.15.69:0/1194388330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3274108710 msgr2=0x7f3274108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:42.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.596+0000 7f327b668700 1 --2- 172.21.15.69:0/1194388330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3274108710 0x7f3274108af0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3264009920 tx=0x7f3264009c30 comp rx=0 tx=0).stop 2024-05-14T16:45:42.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.596+0000 7f327b668700 1 -- 172.21.15.69:0/1194388330 shutdown_connections 2024-05-14T16:45:42.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.596+0000 7f327b668700 1 --2- 172.21.15.69:0/1194388330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3274108710 0x7f3274108af0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:42.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.596+0000 7f327b668700 1 --2- 172.21.15.69:0/1194388330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32741039b0 0x7f327410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:42.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.596+0000 7f327b668700 1 --2- 172.21.15.69:0/1194388330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3274102ff0 0x7f3274103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:42.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.596+0000 7f327b668700 1 -- 172.21.15.69:0/1194388330 >> 172.21.15.69:0/1194388330 conn(0x7f3274077760 msgr2=0x7f3274077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:42.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.596+0000 7f327b668700 1 -- 172.21.15.69:0/1194388330 shutdown_connections 2024-05-14T16:45:42.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.596+0000 7f327b668700 1 -- 172.21.15.69:0/1194388330 wait complete. 2024-05-14T16:45:42.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f327b668700 1 Processor -- start 2024-05-14T16:45:42.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f327b668700 1 -- start start 2024-05-14T16:45:42.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f327b668700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3274102ff0 0x7f327419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:42.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f327b668700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32741039b0 0x7f327419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:42.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f327b668700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3274108710 0x7f3274198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:42.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f327b668700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3274112730 con 0x7f32741039b0 2024-05-14T16:45:42.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f327b668700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f32741125b0 con 0x7f3274108710 2024-05-14T16:45:42.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f327b668700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32741128b0 con 0x7f3274102ff0 2024-05-14T16:45:42.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f3278c03700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32741039b0 0x7f327419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:42.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f3278c03700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32741039b0 0x7f327419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33036/0 (socket says 172.21.15.69:33036) 2024-05-14T16:45:42.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f3278c03700 1 -- 172.21.15.69:0/2967834729 learned_addr learned my addr 172.21.15.69:0/2967834729 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f3279c05700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3274108710 0x7f3274198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f3279404700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3274102ff0 0x7f327419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f3278c03700 1 -- 172.21.15.69:0/2967834729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3274102ff0 msgr2=0x7f327419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f3278c03700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3274102ff0 0x7f327419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f3278c03700 1 -- 172.21.15.69:0/2967834729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3274108710 msgr2=0x7f3274198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f3278c03700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3274108710 0x7f3274198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.597+0000 7f3278c03700 1 -- 172.21.15.69:0/2967834729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3274199210 con 0x7f32741039b0 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.598+0000 7f3278c03700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32741039b0 0x7f327419ee30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f327000d980 tx=0x7f327000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.598+0000 7f326a7fc700 1 -- 172.21.15.69:0/2967834729 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270016070 con 0x7f32741039b0 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.598+0000 7f327b668700 1 -- 172.21.15.69:0/2967834729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3274199500 con 0x7f32741039b0 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.598+0000 7f326a7fc700 1 -- 172.21.15.69:0/2967834729 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3270004830 con 0x7f32741039b0 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.598+0000 7f326a7fc700 1 -- 172.21.15.69:0/2967834729 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32700054c0 con 0x7f32741039b0 2024-05-14T16:45:42.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.598+0000 7f327b668700 1 -- 172.21.15.69:0/2967834729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32741a57c0 con 0x7f32741039b0 2024-05-14T16:45:42.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.599+0000 7f327b668700 1 -- 172.21.15.69:0/2967834729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327406a230 con 0x7f32741039b0 2024-05-14T16:45:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.600+0000 7f326a7fc700 1 -- 172.21.15.69:0/2967834729 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3270004220 con 0x7f32741039b0 2024-05-14T16:45:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.601+0000 7f326a7fc700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f32600b2880 0x7f32600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.601+0000 7f3279404700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f32600b2880 0x7f32600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.601+0000 7f326a7fc700 1 -- 172.21.15.69:0/2967834729 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f32700e2130 con 0x7f32741039b0 2024-05-14T16:45:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.601+0000 7f3279404700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f32600b2880 0x7f32600b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3264005e90 tx=0x7f3264019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.604+0000 7f326a7fc700 1 -- 172.21.15.69:0/2967834729 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f327001f020 con 0x7f32741039b0 2024-05-14T16:45:42.815 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:42 smithi069 ceph-mon[31507]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:42.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.815+0000 7f327b668700 1 -- 172.21.15.69:0/2967834729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f327402e560 con 0x7f32741039b0 2024-05-14T16:45:42.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.817+0000 7f326a7fc700 1 -- 172.21.15.69:0/2967834729 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18420 (secure 0 0 0) 0x7f32700aa2f0 con 0x7f32741039b0 2024-05-14T16:45:42.817 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:45:42.817 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":71,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T16:45:40.695187+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:45:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 -- 172.21.15.69:0/2967834729 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f32600b2880 msgr2=0x7f32600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f32600b2880 0x7f32600b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3264005e90 tx=0x7f3264019040 comp rx=0 tx=0).stop 2024-05-14T16:45:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 -- 172.21.15.69:0/2967834729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32741039b0 msgr2=0x7f327419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32741039b0 0x7f327419ee30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f327000d980 tx=0x7f327000dc90 comp rx=0 tx=0).stop 2024-05-14T16:45:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 -- 172.21.15.69:0/2967834729 shutdown_connections 2024-05-14T16:45:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3274108710 0x7f3274198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3274102ff0 0x7f327419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f32600b2880 0x7f32600b4d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:42.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 --2- 172.21.15.69:0/2967834729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32741039b0 0x7f327419ee30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:42.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 -- 172.21.15.69:0/2967834729 >> 172.21.15.69:0/2967834729 conn(0x7f3274077760 msgr2=0x7f32740fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:42.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 -- 172.21.15.69:0/2967834729 shutdown_connections 2024-05-14T16:45:42.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:42.819+0000 7f327b668700 1 -- 172.21.15.69:0/2967834729 wait complete. 2024-05-14T16:45:42.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:42 smithi203 ceph-mon[35787]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:43.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:42 smithi092 ceph-mon[35763]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:43.543 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T16:41:53.859789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T16:45:43.544 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-14T16:45:43.781 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:45:43.903 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:43 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2967834729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:45:43.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:43 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2967834729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:45:44.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:43 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2967834729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:45:44.882 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:44 smithi069 ceph-mon[31507]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:45.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:44 smithi092 ceph-mon[35763]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:45.138 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:44 smithi203 ceph-mon[35787]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:45.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.383+0000 7f7b62bf1700 1 -- 172.21.15.69:0/979164284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c06bc10 msgr2=0x7f7b5c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:45.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.383+0000 7f7b62bf1700 1 --2- 172.21.15.69:0/979164284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c06bc10 0x7f7b5c06c090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009990 tx=0x7f7b50009ca0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.383+0000 7f7b62bf1700 1 -- 172.21.15.69:0/979164284 shutdown_connections 2024-05-14T16:45:45.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.383+0000 7f7b62bf1700 1 --2- 172.21.15.69:0/979164284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c06c5d0 0x7f7b5c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.383+0000 7f7b62bf1700 1 --2- 172.21.15.69:0/979164284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b5c1095e0 0x7f7b5c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.383+0000 7f7b62bf1700 1 --2- 172.21.15.69:0/979164284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c06bc10 0x7f7b5c06c090 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.383+0000 7f7b62bf1700 1 -- 172.21.15.69:0/979164284 >> 172.21.15.69:0/979164284 conn(0x7f7b5c079480 msgr2=0x7f7b5c079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:45.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.383+0000 7f7b62bf1700 1 -- 172.21.15.69:0/979164284 shutdown_connections 2024-05-14T16:45:45.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.383+0000 7f7b62bf1700 1 -- 172.21.15.69:0/979164284 wait complete. 2024-05-14T16:45:45.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b62bf1700 1 Processor -- start 2024-05-14T16:45:45.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b62bf1700 1 -- start start 2024-05-14T16:45:45.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b62bf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b5c06bc10 0x7f7b5c19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:45.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b62bf1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c06c5d0 0x7f7b5c19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:45.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b62bf1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c1095e0 0x7f7b5c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:45.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b62bf1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c113710 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b62bf1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c113590 con 0x7f7b5c06bc10 2024-05-14T16:45:45.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b62bf1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c113890 con 0x7f7b5c1095e0 2024-05-14T16:45:45.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c06c5d0 0x7f7b5c19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:45.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c06c5d0 0x7f7b5c19fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48780/0 (socket says 172.21.15.69:48780) 2024-05-14T16:45:45.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b5bfff700 1 -- 172.21.15.69:0/3789105094 learned_addr learned my addr 172.21.15.69:0/3789105094 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:45:45.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b6118e700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c1095e0 0x7f7b5c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:45.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b6098d700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b5c06bc10 0x7f7b5c19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:45.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b5bfff700 1 -- 172.21.15.69:0/3789105094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c1095e0 msgr2=0x7f7b5c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:45.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b5bfff700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c1095e0 0x7f7b5c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b5bfff700 1 -- 172.21.15.69:0/3789105094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b5c06bc10 msgr2=0x7f7b5c19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:45.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b5bfff700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b5c06bc10 0x7f7b5c19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.384+0000 7f7b5bfff700 1 -- 172.21.15.69:0/3789105094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c19a210 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.385+0000 7f7b5bfff700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c06c5d0 0x7f7b5c19fe30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b50000c00 tx=0x7f7b500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:45.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.385+0000 7f7b59ffb700 1 -- 172.21.15.69:0/3789105094 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50027070 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.385+0000 7f7b59ffb700 1 -- 172.21.15.69:0/3789105094 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b50003f20 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.385+0000 7f7b62bf1700 1 -- 172.21.15.69:0/3789105094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5c19a4a0 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.385+0000 7f7b59ffb700 1 -- 172.21.15.69:0/3789105094 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50022cf0 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.385+0000 7f7b62bf1700 1 -- 172.21.15.69:0/3789105094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5c1a67c0 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.386+0000 7f7b62bf1700 1 -- 172.21.15.69:0/3789105094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5c104ca0 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.387+0000 7f7b59ffb700 1 -- 172.21.15.69:0/3789105094 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7b5002a070 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.388+0000 7f7b59ffb700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7b440b2900 0x7f7b440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:45.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.388+0000 7f7b6098d700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7b440b2900 0x7f7b440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:45.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.388+0000 7f7b59ffb700 1 -- 172.21.15.69:0/3789105094 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7b500e2e80 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.388+0000 7f7b6098d700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7b440b2900 0x7f7b440b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005fd0 tx=0x7f7b4c005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:45.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.391+0000 7f7b59ffb700 1 -- 172.21.15.69:0/3789105094 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7b50013070 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.595+0000 7f7b62bf1700 1 -- 172.21.15.69:0/3789105094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b5c02e560 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.595+0000 7f7b59ffb700 1 -- 172.21.15.69:0/3789105094 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f7b500ab0d0 con 0x7f7b5c06c5d0 2024-05-14T16:45:45.596 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-05-14T16:45:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 -- 172.21.15.69:0/3789105094 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7b440b2900 msgr2=0x7f7b440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7b440b2900 0x7f7b440b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005fd0 tx=0x7f7b4c005db0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 -- 172.21.15.69:0/3789105094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c06c5d0 msgr2=0x7f7b5c19fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c06c5d0 0x7f7b5c19fe30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b50000c00 tx=0x7f7b500047b0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 -- 172.21.15.69:0/3789105094 shutdown_connections 2024-05-14T16:45:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c1095e0 0x7f7b5c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b5c06bc10 0x7f7b5c19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7b440b2900 0x7f7b440b4dc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 --2- 172.21.15.69:0/3789105094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c06c5d0 0x7f7b5c19fe30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 -- 172.21.15.69:0/3789105094 >> 172.21.15.69:0/3789105094 conn(0x7f7b5c079480 msgr2=0x7f7b5c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:45.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 -- 172.21.15.69:0/3789105094 shutdown_connections 2024-05-14T16:45:45.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:45.598+0000 7f7b62bf1700 1 -- 172.21.15.69:0/3789105094 wait complete. 2024-05-14T16:45:45.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:45 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3789105094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T16:45:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:45 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3789105094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T16:45:46.068 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:45 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3789105094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T16:45:46.387 INFO:tasks.cephadm:Adding mds.a on smithi069 2024-05-14T16:45:46.387 INFO:tasks.cephadm:Adding mds.d on smithi069 2024-05-14T16:45:46.387 INFO:tasks.cephadm:Adding mds.g on smithi069 2024-05-14T16:45:46.387 INFO:tasks.cephadm:Adding mds.j on smithi069 2024-05-14T16:45:46.387 INFO:tasks.cephadm:Adding mds.b on smithi092 2024-05-14T16:45:46.387 INFO:tasks.cephadm:Adding mds.e on smithi092 2024-05-14T16:45:46.387 INFO:tasks.cephadm:Adding mds.h on smithi092 2024-05-14T16:45:46.388 INFO:tasks.cephadm:Adding mds.k on smithi092 2024-05-14T16:45:46.388 INFO:tasks.cephadm:Adding mds.c on smithi203 2024-05-14T16:45:46.388 INFO:tasks.cephadm:Adding mds.f on smithi203 2024-05-14T16:45:46.388 INFO:tasks.cephadm:Adding mds.i on smithi203 2024-05-14T16:45:46.388 INFO:tasks.cephadm:Adding mds.l on smithi203 2024-05-14T16:45:46.388 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l' 2024-05-14T16:45:46.627 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:45:47.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:46 smithi092 ceph-mon[35763]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:47.022 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:46 smithi203 ceph-mon[35787]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:47.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:46 smithi069 ceph-mon[31507]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:48.292 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.292+0000 7f0a40fe7700 1 -- 172.21.15.203:0/3755641934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c102490 msgr2=0x7f0a3c1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:48.292 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.292+0000 7f0a40fe7700 1 --2- 172.21.15.203:0/3755641934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c102490 0x7f0a3c1028b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009990 tx=0x7f0a30009ca0 comp rx=0 tx=0).stop 2024-05-14T16:45:48.293 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.292+0000 7f0a40fe7700 1 -- 172.21.15.203:0/3755641934 shutdown_connections 2024-05-14T16:45:48.293 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.292+0000 7f0a40fe7700 1 --2- 172.21.15.203:0/3755641934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a3c104030 0x7f0a3c10a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:48.293 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.292+0000 7f0a40fe7700 1 --2- 172.21.15.203:0/3755641934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c102490 0x7f0a3c1028b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:48.293 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.292+0000 7f0a40fe7700 1 --2- 172.21.15.203:0/3755641934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c103690 0x7f0a3c103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:48.293 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.292+0000 7f0a40fe7700 1 -- 172.21.15.203:0/3755641934 >> 172.21.15.203:0/3755641934 conn(0x7f0a3c0fda30 msgr2=0x7f0a3c0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:48.293 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.292+0000 7f0a40fe7700 1 -- 172.21.15.203:0/3755641934 shutdown_connections 2024-05-14T16:45:48.293 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.292+0000 7f0a40fe7700 1 -- 172.21.15.203:0/3755641934 wait complete. 2024-05-14T16:45:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a40fe7700 1 Processor -- start 2024-05-14T16:45:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a40fe7700 1 -- start start 2024-05-14T16:45:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a40fe7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c102490 0x7f0a3c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a40fe7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a3c103690 0x7f0a3c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a40fe7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c104030 0x7f0a3c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a40fe7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c10df30 con 0x7f0a3c104030 2024-05-14T16:45:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a40fe7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c10ddb0 con 0x7f0a3c103690 2024-05-14T16:45:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a40fe7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c10e0b0 con 0x7f0a3c102490 2024-05-14T16:45:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a3a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c102490 0x7f0a3c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:48.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a3a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c102490 0x7f0a3c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:41256/0 (socket says 172.21.15.203:41256) 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a3a59c700 1 -- 172.21.15.203:0/2743601862 learned_addr learned my addr 172.21.15.203:0/2743601862 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.293+0000 7f0a39d9b700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a3c103690 0x7f0a3c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a3ad9d700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c104030 0x7f0a3c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a3a59c700 1 -- 172.21.15.203:0/2743601862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a3c103690 msgr2=0x7f0a3c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a3a59c700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a3c103690 0x7f0a3c199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a3a59c700 1 -- 172.21.15.203:0/2743601862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c104030 msgr2=0x7f0a3c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a3a59c700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c104030 0x7f0a3c1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a3a59c700 1 -- 172.21.15.203:0/2743601862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c1a1350 con 0x7f0a3c102490 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a3a59c700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c102490 0x7f0a3c199740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a300047b0 tx=0x7f0a30009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a2b7fe700 1 -- 172.21.15.203:0/2743601862 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30026070 con 0x7f0a3c102490 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a2b7fe700 1 -- 172.21.15.203:0/2743601862 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a30004140 con 0x7f0a3c102490 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a2b7fe700 1 -- 172.21.15.203:0/2743601862 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30005290 con 0x7f0a3c102490 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a40fe7700 1 -- 172.21.15.203:0/2743601862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c1a15e0 con 0x7f0a3c102490 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.294+0000 7f0a40fe7700 1 -- 172.21.15.203:0/2743601862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c1a1a30 con 0x7f0a3c102490 2024-05-14T16:45:48.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.295+0000 7f0a40fe7700 1 -- 172.21.15.203:0/2743601862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3c06a260 con 0x7f0a3c102490 2024-05-14T16:45:48.297 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.296+0000 7f0a2b7fe700 1 -- 172.21.15.203:0/2743601862 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0a30029070 con 0x7f0a3c102490 2024-05-14T16:45:48.297 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.297+0000 7f0a2b7fe700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0a240b2880 0x7f0a240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:48.297 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.297+0000 7f0a2b7fe700 1 -- 172.21.15.203:0/2743601862 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f0a300e2c00 con 0x7f0a3c102490 2024-05-14T16:45:48.297 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.297+0000 7f0a39d9b700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0a240b2880 0x7f0a240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:48.298 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.298+0000 7f0a39d9b700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0a240b2880 0x7f0a240b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c005950 tx=0x7f0a2c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:48.300 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.301+0000 7f0a2b7fe700 1 -- 172.21.15.203:0/2743601862 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a3c06a260 con 0x7f0a3c102490 2024-05-14T16:45:48.513 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.514+0000 7f0a40fe7700 1 -- 172.21.15.203:0/2743601862 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}) -- 0x7f0a3c06c1a0 con 0x7f0a240b2880 2024-05-14T16:45:48.522 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.522+0000 7f0a2b7fe700 1 -- 172.21.15.203:0/2743601862 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0a3c06c1a0 con 0x7f0a240b2880 2024-05-14T16:45:48.522 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mds.all update... 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 -- 172.21.15.203:0/2743601862 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0a240b2880 msgr2=0x7f0a240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0a240b2880 0x7f0a240b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c005950 tx=0x7f0a2c00b410 comp rx=0 tx=0).stop 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 -- 172.21.15.203:0/2743601862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c102490 msgr2=0x7f0a3c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c102490 0x7f0a3c199740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a300047b0 tx=0x7f0a30009580 comp rx=0 tx=0).stop 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 -- 172.21.15.203:0/2743601862 shutdown_connections 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0a240b2880 0x7f0a240b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c104030 0x7f0a3c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3c102490 0x7f0a3c199740 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 --2- 172.21.15.203:0/2743601862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a3c103690 0x7f0a3c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 -- 172.21.15.203:0/2743601862 >> 172.21.15.203:0/2743601862 conn(0x7f0a3c0fda30 msgr2=0x7f0a3c06ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.524+0000 7f0a40fe7700 1 -- 172.21.15.203:0/2743601862 shutdown_connections 2024-05-14T16:45:48.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-14T16:45:48.525+0000 7f0a40fe7700 1 -- 172.21.15.203:0/2743601862 wait complete. 2024-05-14T16:45:49.128 DEBUG:teuthology.orchestra.run.smithi069:mds.a> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.a.service 2024-05-14T16:45:49.131 DEBUG:teuthology.orchestra.run.smithi069:mds.d> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.d.service 2024-05-14T16:45:49.173 DEBUG:teuthology.orchestra.run.smithi069:mds.g> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.g.service 2024-05-14T16:45:49.216 DEBUG:teuthology.orchestra.run.smithi069:mds.j> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.j.service 2024-05-14T16:45:49.259 DEBUG:teuthology.orchestra.run.smithi092:mds.b> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.b.service 2024-05-14T16:45:49.262 DEBUG:teuthology.orchestra.run.smithi092:mds.e> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.e.service 2024-05-14T16:45:49.305 DEBUG:teuthology.orchestra.run.smithi092:mds.h> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.h.service 2024-05-14T16:45:49.348 DEBUG:teuthology.orchestra.run.smithi092:mds.k> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.k.service 2024-05-14T16:45:49.391 DEBUG:teuthology.orchestra.run.smithi203:mds.c> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service 2024-05-14T16:45:49.393 DEBUG:teuthology.orchestra.run.smithi203:mds.f> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.f.service 2024-05-14T16:45:49.436 DEBUG:teuthology.orchestra.run.smithi203:mds.i> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.i.service 2024-05-14T16:45:49.479 DEBUG:teuthology.orchestra.run.smithi203:mds.l> sudo journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.l.service 2024-05-14T16:45:49.522 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-14T16:45:49.522 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:45:49.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-05-14T16:45:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:45:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:45:49.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:49 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:49.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:49.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:49.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-05-14T16:45:49.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:49.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:49.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:49.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:49.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:49.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:45:49.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:45:49.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:49 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:49.757 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:45:49.878 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:49.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:49.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-05-14T16:45:49.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:49.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:49.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:49.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:49.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:49.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:45:49.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:45:49.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:49 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:50.693 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:50 smithi203 ceph-mon[35787]: Deploying daemon mds.l on smithi203 2024-05-14T16:45:50.903 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:50 smithi069 ceph-mon[31507]: Deploying daemon mds.l on smithi203 2024-05-14T16:45:51.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:50 smithi092 ceph-mon[35763]: Deploying daemon mds.l on smithi203 2024-05-14T16:45:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.530+0000 7f21b5f32700 1 -- 172.21.15.69:0/2196513249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21b0102e00 msgr2=0x7f21b010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.530+0000 7f21b5f32700 1 --2- 172.21.15.69:0/2196513249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21b0102e00 0x7f21b010f2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009990 tx=0x7f21a8009ca0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.530+0000 7f21b5f32700 1 -- 172.21.15.69:0/2196513249 shutdown_connections 2024-05-14T16:45:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.530+0000 7f21b5f32700 1 --2- 172.21.15.69:0/2196513249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21b0102e00 0x7f21b010f2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.530+0000 7f21b5f32700 1 --2- 172.21.15.69:0/2196513249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b0108520 0x7f21b0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.530+0000 7f21b5f32700 1 --2- 172.21.15.69:0/2196513249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21b0102440 0x7f21b01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.530+0000 7f21b5f32700 1 -- 172.21.15.69:0/2196513249 >> 172.21.15.69:0/2196513249 conn(0x7f21b00793f0 msgr2=0x7f21b0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.530+0000 7f21b5f32700 1 -- 172.21.15.69:0/2196513249 shutdown_connections 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.530+0000 7f21b5f32700 1 -- 172.21.15.69:0/2196513249 wait complete. 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.531+0000 7f21b5f32700 1 Processor -- start 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.531+0000 7f21b5f32700 1 -- start start 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.531+0000 7f21b5f32700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b0102440 0x7f21b0104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.531+0000 7f21b5f32700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21b0102e00 0x7f21b01045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.531+0000 7f21b5f32700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21b0108520 0x7f21b0104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.531+0000 7f21b5f32700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01125f0 con 0x7f21b0108520 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.531+0000 7f21b5f32700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f21b0112470 con 0x7f21b0102e00 2024-05-14T16:45:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.531+0000 7f21b5f32700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f21b0112770 con 0x7f21b0102440 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21affff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21b0108520 0x7f21b0104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21affff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21b0108520 0x7f21b0104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48802/0 (socket says 172.21.15.69:48802) 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21affff700 1 -- 172.21.15.69:0/3613511288 learned_addr learned my addr 172.21.15.69:0/3613511288 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21af7fe700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b0102440 0x7f21b0104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21aeffd700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21b0102e00 0x7f21b01045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21affff700 1 -- 172.21.15.69:0/3613511288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b0102440 msgr2=0x7f21b0104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21affff700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b0102440 0x7f21b0104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21affff700 1 -- 172.21.15.69:0/3613511288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21b0102e00 msgr2=0x7f21b01045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21affff700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21b0102e00 0x7f21b01045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21affff700 1 -- 172.21.15.69:0/3613511288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b01a8200 con 0x7f21b0108520 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21aeffd700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21b0102e00 0x7f21b01045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21affff700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21b0108520 0x7f21b0104b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f21a800c010 tx=0x7f21a80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21acff9700 1 -- 172.21.15.69:0/3613511288 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8027070 con 0x7f21b0108520 2024-05-14T16:45:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21acff9700 1 -- 172.21.15.69:0/3613511288 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a8022470 con 0x7f21b0108520 2024-05-14T16:45:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21b5f32700 1 -- 172.21.15.69:0/3613511288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b01a8490 con 0x7f21b0108520 2024-05-14T16:45:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21acff9700 1 -- 172.21.15.69:0/3613511288 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8017650 con 0x7f21b0108520 2024-05-14T16:45:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.532+0000 7f21b5f32700 1 -- 172.21.15.69:0/3613511288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b01a8940 con 0x7f21b0108520 2024-05-14T16:45:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.533+0000 7f21b5f32700 1 -- 172.21.15.69:0/3613511288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b006a230 con 0x7f21b0108520 2024-05-14T16:45:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.534+0000 7f21acff9700 1 -- 172.21.15.69:0/3613511288 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f21a802a070 con 0x7f21b0108520 2024-05-14T16:45:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.535+0000 7f21acff9700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f219c0b2900 0x7f219c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.536+0000 7f21af7fe700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f219c0b2900 0x7f219c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.536+0000 7f21acff9700 1 -- 172.21.15.69:0/3613511288 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f21a80e2fe0 con 0x7f21b0108520 2024-05-14T16:45:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.536+0000 7f21af7fe700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f219c0b2900 0x7f219c0b4dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f21a000f540 tx=0x7f21a000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:51.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.539+0000 7f21acff9700 1 -- 172.21.15.69:0/3613511288 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21a8013070 con 0x7f21b0108520 2024-05-14T16:45:51.671 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:51 smithi203 ceph-mon[35787]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:51.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:51 smithi069 ceph-mon[31507]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:51.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.799+0000 7f21b5f32700 1 -- 172.21.15.69:0/3613511288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21b002e560 con 0x7f21b0108520 2024-05-14T16:45:51.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.800+0000 7f21acff9700 1 -- 172.21.15.69:0/3613511288 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f21a80ab1b0 con 0x7f21b0108520 2024-05-14T16:45:51.799 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:45:51.800 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-14T16:45:51.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 -- 172.21.15.69:0/3613511288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f219c0b2900 msgr2=0x7f219c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:51.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f219c0b2900 0x7f219c0b4dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f21a000f540 tx=0x7f21a000b040 comp rx=0 tx=0).stop 2024-05-14T16:45:51.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 -- 172.21.15.69:0/3613511288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21b0108520 msgr2=0x7f21b0104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:51.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21b0108520 0x7f21b0104b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f21a800c010 tx=0x7f21a80047b0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 -- 172.21.15.69:0/3613511288 shutdown_connections 2024-05-14T16:45:51.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f219c0b2900 0x7f219c0b4dc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21b0108520 0x7f21b0104b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b0102440 0x7f21b0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 --2- 172.21.15.69:0/3613511288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21b0102e00 0x7f21b01045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:51.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 -- 172.21.15.69:0/3613511288 >> 172.21.15.69:0/3613511288 conn(0x7f21b00793f0 msgr2=0x7f21b006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:51.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 -- 172.21.15.69:0/3613511288 shutdown_connections 2024-05-14T16:45:51.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:51.802+0000 7f21b5f32700 1 -- 172.21.15.69:0/3613511288 wait complete. 2024-05-14T16:45:51.803 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 1 2024-05-14T16:45:52.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:51 smithi092 ceph-mon[35763]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:52.459 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 16:45:52 smithi203 systemd[1]: Started Ceph mds.l for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:45:52.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:52 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3613511288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:45:52.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:52 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6832/85237132,v1:172.21.15.203:6833/85237132] up:boot 2024-05-14T16:45:52.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:52 smithi092 ceph-mon[35763]: fsmap 1 up:standby 2024-05-14T16:45:52.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:52 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:45:52.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:52 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:52.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:52 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:52.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:52 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:52.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:52 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:45:52.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:52 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:45:52.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:52 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:52.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:52 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3613511288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:45:52.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:52 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6832/85237132,v1:172.21.15.203:6833/85237132] up:boot 2024-05-14T16:45:52.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:52 smithi069 ceph-mon[31507]: fsmap 1 up:standby 2024-05-14T16:45:52.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:52 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:45:52.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:52 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:52.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:52 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:52.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:52 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:52.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:52 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:45:52.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:52 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:45:52.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:52 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:52.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:52 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3613511288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:45:52.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:52 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6832/85237132,v1:172.21.15.203:6833/85237132] up:boot 2024-05-14T16:45:52.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:52 smithi203 ceph-mon[35787]: fsmap 1 up:standby 2024-05-14T16:45:52.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:52 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:45:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:52 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:52 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:52 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:52 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:45:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:52 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:45:52.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:52 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:53.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:53 smithi092 ceph-mon[35763]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:53.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:53 smithi092 ceph-mon[35763]: Deploying daemon mds.k on smithi092 2024-05-14T16:45:53.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:53 smithi069 ceph-mon[31507]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:53.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:53 smithi069 ceph-mon[31507]: Deploying daemon mds.k on smithi092 2024-05-14T16:45:53.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:53 smithi203 ceph-mon[35787]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:53.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:53 smithi203 ceph-mon[35787]: Deploying daemon mds.k on smithi092 2024-05-14T16:45:54.500 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 16:45:54 smithi092 systemd[1]: Starting Ceph mds.k for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:45:54.621 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:45:54.874 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:45:54.878 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 16:45:54 smithi092 podman[65099]: 2024-05-14 16:45:54.508010503 +0000 UTC m=+0.017345023 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:45:54.878 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 16:45:54 smithi092 podman[65099]: 2024-05-14 16:45:54.605979427 +0000 UTC m=+0.115313926 container create 5ae9d798f7632a29ab96bf55e30d52af2c5242ca5dd1177ce3be26fa72cf358d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-14T16:45:54.878 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 16:45:54 smithi092 podman[65099]: 2024-05-14 16:45:54.85312945 +0000 UTC m=+0.362463957 container init 5ae9d798f7632a29ab96bf55e30d52af2c5242ca5dd1177ce3be26fa72cf358d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-14T16:45:54.878 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 16:45:54 smithi092 podman[65099]: 2024-05-14 16:45:54.855641721 +0000 UTC m=+0.364976228 container start 5ae9d798f7632a29ab96bf55e30d52af2c5242ca5dd1177ce3be26fa72cf358d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-k, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:45:55.250 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 16:45:54 smithi092 ceph-mds[65118]: -- 172.21.15.92:0/145648876 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562117d47a40 con 0x562117d1b000 2024-05-14T16:45:55.250 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 16:45:54 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-k[65114]: starting mds.k at 2024-05-14T16:45:55.250 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 16:45:54 smithi092 bash[65099]: 5ae9d798f7632a29ab96bf55e30d52af2c5242ca5dd1177ce3be26fa72cf358d 2024-05-14T16:45:55.250 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 16:45:54 smithi092 systemd[1]: Started Ceph mds.k for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:45:55.587 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:55 smithi069 ceph-mon[31507]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:55.588 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:55.588 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:55.588 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:55.588 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:45:55.588 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:45:55.588 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:55 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:55.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:55 smithi203 ceph-mon[35787]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:55.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:55.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:55.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:55.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:45:55.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:45:55.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:55 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:55 smithi092 ceph-mon[35763]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:45:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:45:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:45:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:55 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:56.847 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:56 smithi069 ceph-mon[31507]: Deploying daemon mds.j on smithi069 2024-05-14T16:45:56.847 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:56 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.92:6832/786223395,v1:172.21.15.92:6833/786223395] up:boot 2024-05-14T16:45:56.847 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:56 smithi069 ceph-mon[31507]: fsmap 2 up:standby 2024-05-14T16:45:56.847 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:56 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:45:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:56 smithi203 ceph-mon[35787]: Deploying daemon mds.j on smithi069 2024-05-14T16:45:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:56 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.92:6832/786223395,v1:172.21.15.92:6833/786223395] up:boot 2024-05-14T16:45:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:56 smithi203 ceph-mon[35787]: fsmap 2 up:standby 2024-05-14T16:45:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:56 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:45:57.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:56 smithi092 ceph-mon[35763]: Deploying daemon mds.j on smithi069 2024-05-14T16:45:57.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:56 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.92:6832/786223395,v1:172.21.15.92:6833/786223395] up:boot 2024-05-14T16:45:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:56 smithi092 ceph-mon[35763]: fsmap 2 up:standby 2024-05-14T16:45:57.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:56 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:45:57.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.027+0000 7f31022b4700 1 -- 172.21.15.69:0/3259298935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc108930 msgr2=0x7f30fc1104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:57.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.027+0000 7f31022b4700 1 --2- 172.21.15.69:0/3259298935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc108930 0x7f30fc1104b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30f400b0d0 tx=0x7f30f400b3e0 comp rx=0 tx=0).stop 2024-05-14T16:45:57.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.027+0000 7f31022b4700 1 -- 172.21.15.69:0/3259298935 shutdown_connections 2024-05-14T16:45:57.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.027+0000 7f31022b4700 1 --2- 172.21.15.69:0/3259298935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc108930 0x7f30fc1104b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:57.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.027+0000 7f31022b4700 1 --2- 172.21.15.69:0/3259298935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30fc101cf0 0x7f30fc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:57.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.027+0000 7f31022b4700 1 --2- 172.21.15.69:0/3259298935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc1026a0 0x7f30fc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:57.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.027+0000 7f31022b4700 1 -- 172.21.15.69:0/3259298935 >> 172.21.15.69:0/3259298935 conn(0x7f30fc078a40 msgr2=0x7f30fc0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:57.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.028+0000 7f31022b4700 1 -- 172.21.15.69:0/3259298935 shutdown_connections 2024-05-14T16:45:57.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.028+0000 7f31022b4700 1 -- 172.21.15.69:0/3259298935 wait complete. 2024-05-14T16:45:57.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.028+0000 7f31022b4700 1 Processor -- start 2024-05-14T16:45:57.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.028+0000 7f31022b4700 1 -- start start 2024-05-14T16:45:57.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.028+0000 7f31022b4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc101cf0 0x7f30fc19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:57.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.028+0000 7f31022b4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30fc1026a0 0x7f30fc19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:57.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.028+0000 7f31022b4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc108930 0x7f30fc199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:57.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f31022b4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc113700 con 0x7f30fc101cf0 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f31022b4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc113580 con 0x7f30fc108930 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f31022b4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc113880 con 0x7f30fc1026a0 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc101cf0 0x7f30fc19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc101cf0 0x7f30fc19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45516/0 (socket says 172.21.15.69:45516) 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fbfff700 1 -- 172.21.15.69:0/3433708498 learned_addr learned my addr 172.21.15.69:0/3433708498 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fb7fe700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30fc1026a0 0x7f30fc19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f3100851700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc108930 0x7f30fc199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fbfff700 1 -- 172.21.15.69:0/3433708498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30fc1026a0 msgr2=0x7f30fc19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fbfff700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30fc1026a0 0x7f30fc19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fbfff700 1 -- 172.21.15.69:0/3433708498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc108930 msgr2=0x7f30fc199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fbfff700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc108930 0x7f30fc199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fbfff700 1 -- 172.21.15.69:0/3433708498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc199fe0 con 0x7f30fc101cf0 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fb7fe700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30fc1026a0 0x7f30fc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.029+0000 7f30fbfff700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc101cf0 0x7f30fc19f6c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00c920 tx=0x7f30ec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:57.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.030+0000 7f30f97fa700 1 -- 172.21.15.69:0/3433708498 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec007b00 con 0x7f30fc101cf0 2024-05-14T16:45:57.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.030+0000 7f31022b4700 1 -- 172.21.15.69:0/3433708498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc19a2d0 con 0x7f30fc101cf0 2024-05-14T16:45:57.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.030+0000 7f30f97fa700 1 -- 172.21.15.69:0/3433708498 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30ec004ce0 con 0x7f30fc101cf0 2024-05-14T16:45:57.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.030+0000 7f31022b4700 1 -- 172.21.15.69:0/3433708498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1a6590 con 0x7f30fc101cf0 2024-05-14T16:45:57.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.030+0000 7f30f97fa700 1 -- 172.21.15.69:0/3433708498 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec005500 con 0x7f30fc101cf0 2024-05-14T16:45:57.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.031+0000 7f31022b4700 1 -- 172.21.15.69:0/3433708498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e8005330 con 0x7f30fc101cf0 2024-05-14T16:45:57.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.032+0000 7f30f97fa700 1 -- 172.21.15.69:0/3433708498 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f30ec007ca0 con 0x7f30fc101cf0 2024-05-14T16:45:57.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.033+0000 7f30f97fa700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f30e40b2900 0x7f30e40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:57.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.033+0000 7f30fb7fe700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f30e40b2900 0x7f30e40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:57.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.033+0000 7f30f97fa700 1 -- 172.21.15.69:0/3433708498 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f30ec0e1bb0 con 0x7f30fc101cf0 2024-05-14T16:45:57.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.034+0000 7f30fb7fe700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f30e40b2900 0x7f30e40b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30f0005fd0 tx=0x7f30f0005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:57.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.037+0000 7f30f97fa700 1 -- 172.21.15.69:0/3433708498 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30ec014030 con 0x7f30fc101cf0 2024-05-14T16:45:57.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.294+0000 7f31022b4700 1 -- 172.21.15.69:0/3433708498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30e8005620 con 0x7f30fc101cf0 2024-05-14T16:45:57.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.295+0000 7f30f97fa700 1 -- 172.21.15.69:0/3433708498 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2212 (secure 0 0 0) 0x7f30ec0b0e20 con 0x7f30fc101cf0 2024-05-14T16:45:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:45:57.295 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-14T16:45:57.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 -- 172.21.15.69:0/3433708498 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f30e40b2900 msgr2=0x7f30e40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:57.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f30e40b2900 0x7f30e40b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30f0005fd0 tx=0x7f30f0005e30 comp rx=0 tx=0).stop 2024-05-14T16:45:57.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 -- 172.21.15.69:0/3433708498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc101cf0 msgr2=0x7f30fc19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:57.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc101cf0 0x7f30fc19f6c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00c920 tx=0x7f30ec00cc30 comp rx=0 tx=0).stop 2024-05-14T16:45:57.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 -- 172.21.15.69:0/3433708498 shutdown_connections 2024-05-14T16:45:57.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f30e40b2900 0x7f30e40b4dc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:57.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc101cf0 0x7f30fc19f6c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:57.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc108930 0x7f30fc199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:57.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 --2- 172.21.15.69:0/3433708498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30fc1026a0 0x7f30fc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:57.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 -- 172.21.15.69:0/3433708498 >> 172.21.15.69:0/3433708498 conn(0x7f30fc078a40 msgr2=0x7f30fc108030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:57.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 -- 172.21.15.69:0/3433708498 shutdown_connections 2024-05-14T16:45:57.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:45:57.297+0000 7f31022b4700 1 -- 172.21.15.69:0/3433708498 wait complete. 2024-05-14T16:45:57.298 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 3 2024-05-14T16:45:57.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:57 smithi069 ceph-mon[31507]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:57.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:57 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3433708498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:45:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:57 smithi203 ceph-mon[35787]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:57 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3433708498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:45:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:57 smithi092 ceph-mon[35763]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:57 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3433708498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:45:59.203 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 16:45:59 smithi069 systemd[1]: Starting Ceph mds.j for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:45:59.552 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 16:45:59 smithi069 podman[68418]: 2024-05-14 16:45:59.202890335 +0000 UTC m=+0.017512069 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:45:59.552 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 16:45:59 smithi069 podman[68418]: 2024-05-14 16:45:59.306859226 +0000 UTC m=+0.121480970 container create 3d50ce5a9d6e5b4f0b1d2be68ebd98984d8c41fab2b70902bfc992bd9c703d15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-j, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-05-14T16:45:59.552 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 16:45:59 smithi069 podman[68418]: 2024-05-14 16:45:59.534592608 +0000 UTC m=+0.349214322 container init 3d50ce5a9d6e5b4f0b1d2be68ebd98984d8c41fab2b70902bfc992bd9c703d15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-14T16:45:59.552 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 16:45:59 smithi069 podman[68418]: 2024-05-14 16:45:59.53731295 +0000 UTC m=+0.351934682 container start 3d50ce5a9d6e5b4f0b1d2be68ebd98984d8c41fab2b70902bfc992bd9c703d15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-14T16:45:59.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:45:59 smithi069 ceph-mon[31507]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:45:59.953 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 16:45:59 smithi069 ceph-mds[68438]: -- 172.21.15.69:0/2524399128 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d5388da40 con 0x562d53860c00 2024-05-14T16:45:59.954 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 16:45:59 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-j[68434]: starting mds.j at 2024-05-14T16:45:59.954 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 16:45:59 smithi069 bash[68418]: 3d50ce5a9d6e5b4f0b1d2be68ebd98984d8c41fab2b70902bfc992bd9c703d15 2024-05-14T16:45:59.954 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 16:45:59 smithi069 systemd[1]: Started Ceph mds.j for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:45:59.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:45:59 smithi203 ceph-mon[35787]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:45:59 smithi092 ceph-mon[35763]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:00.171 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:46:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:00.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:00.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:00.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:00.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:00.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:00.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:00.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:00 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.69:6834/894337492,v1:172.21.15.69:6835/894337492] up:boot 2024-05-14T16:46:00.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:00 smithi069 ceph-mon[31507]: fsmap 3 up:standby 2024-05-14T16:46:00.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:00 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:46:00.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:00 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:00.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:00 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:00.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:00 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:00.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:00 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:00.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:00 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:00.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:00 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:00.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:00 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.69:6834/894337492,v1:172.21.15.69:6835/894337492] up:boot 2024-05-14T16:46:00.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:00 smithi203 ceph-mon[35787]: fsmap 3 up:standby 2024-05-14T16:46:00.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:00 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:46:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:00 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:00 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:00 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:00 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:00 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:00 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:00 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.69:6834/894337492,v1:172.21.15.69:6835/894337492] up:boot 2024-05-14T16:46:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:00 smithi092 ceph-mon[35763]: fsmap 3 up:standby 2024-05-14T16:46:01.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:00 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:46:01.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:01 smithi069 ceph-mon[31507]: Deploying daemon mds.d on smithi069 2024-05-14T16:46:01.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:01 smithi069 ceph-mon[31507]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:01 smithi203 ceph-mon[35787]: Deploying daemon mds.d on smithi069 2024-05-14T16:46:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:01 smithi203 ceph-mon[35787]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:01 smithi092 ceph-mon[35763]: Deploying daemon mds.d on smithi069 2024-05-14T16:46:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:01 smithi092 ceph-mon[35763]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:03.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.189+0000 7f7ce4057700 1 -- 172.21.15.69:0/1397830815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc101b30 msgr2=0x7f7cdc101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:03.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.189+0000 7f7ce4057700 1 --2- 172.21.15.69:0/1397830815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc101b30 0x7f7cdc101fb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0099c0 tx=0x7f7ccc009cd0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.189+0000 7f7ce4057700 1 -- 172.21.15.69:0/1397830815 shutdown_connections 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.189+0000 7f7ce4057700 1 --2- 172.21.15.69:0/1397830815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cdc1024f0 0x7f7cdc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.189+0000 7f7ce4057700 1 --2- 172.21.15.69:0/1397830815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cdc109520 0x7f7cdc109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.189+0000 7f7ce4057700 1 --2- 172.21.15.69:0/1397830815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc101b30 0x7f7cdc101fb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.189+0000 7f7ce4057700 1 -- 172.21.15.69:0/1397830815 >> 172.21.15.69:0/1397830815 conn(0x7f7cdc078ab0 msgr2=0x7f7cdc0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.189+0000 7f7ce4057700 1 -- 172.21.15.69:0/1397830815 shutdown_connections 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.189+0000 7f7ce4057700 1 -- 172.21.15.69:0/1397830815 wait complete. 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce4057700 1 Processor -- start 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce4057700 1 -- start start 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce4057700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cdc101b30 0x7f7cdc19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce4057700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1024f0 0x7f7cdc19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce4057700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cdc109520 0x7f7cdc199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:03.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce4057700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc113700 con 0x7f7cdc1024f0 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce4057700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc113580 con 0x7f7cdc101b30 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce4057700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc113880 con 0x7f7cdc109520 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce15f2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1024f0 0x7f7cdc19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce15f2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1024f0 0x7f7cdc19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45550/0 (socket says 172.21.15.69:45550) 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.190+0000 7f7ce15f2700 1 -- 172.21.15.69:0/1259747338 learned_addr learned my addr 172.21.15.69:0/1259747338 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7ce1df3700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cdc101b30 0x7f7cdc19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7ce15f2700 1 -- 172.21.15.69:0/1259747338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cdc109520 msgr2=0x7f7cdc199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7ce15f2700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cdc109520 0x7f7cdc199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7ce15f2700 1 -- 172.21.15.69:0/1259747338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cdc101b30 msgr2=0x7f7cdc19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7ce15f2700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cdc101b30 0x7f7cdc19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7ce15f2700 1 -- 172.21.15.69:0/1259747338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cdc19a0a0 con 0x7f7cdc1024f0 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7ce15f2700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1024f0 0x7f7cdc19fcc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0098e0 tx=0x7f7ccc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7cd2ffd700 1 -- 172.21.15.69:0/1259747338 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc027070 con 0x7f7cdc1024f0 2024-05-14T16:46:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7cd2ffd700 1 -- 172.21.15.69:0/1259747338 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ccc004020 con 0x7f7cdc1024f0 2024-05-14T16:46:03.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7ce4057700 1 -- 172.21.15.69:0/1259747338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cdc19a330 con 0x7f7cdc1024f0 2024-05-14T16:46:03.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7cd2ffd700 1 -- 172.21.15.69:0/1259747338 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc022e10 con 0x7f7cdc1024f0 2024-05-14T16:46:03.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.191+0000 7f7ce4057700 1 -- 172.21.15.69:0/1259747338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cdc1a6650 con 0x7f7cdc1024f0 2024-05-14T16:46:03.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.192+0000 7f7ce4057700 1 -- 172.21.15.69:0/1259747338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cdc104e50 con 0x7f7cdc1024f0 2024-05-14T16:46:03.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.194+0000 7f7cd2ffd700 1 -- 172.21.15.69:0/1259747338 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7ccc02a070 con 0x7f7cdc1024f0 2024-05-14T16:46:03.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.195+0000 7f7cd2ffd700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7cc80b2880 0x7f7cc80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:03.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.195+0000 7f7ce1df3700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7cc80b2880 0x7f7cc80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:03.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.195+0000 7f7cd2ffd700 1 -- 172.21.15.69:0/1259747338 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7ccc005030 con 0x7f7cdc1024f0 2024-05-14T16:46:03.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.195+0000 7f7ce1df3700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7cc80b2880 0x7f7cc80b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc19aa80 tx=0x7f7cd80079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:03.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.198+0000 7f7cd2ffd700 1 -- 172.21.15.69:0/1259747338 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ccc031020 con 0x7f7cdc1024f0 2024-05-14T16:46:03.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.492+0000 7f7ce4057700 1 -- 172.21.15.69:0/1259747338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cdc19ae70 con 0x7f7cdc1024f0 2024-05-14T16:46:03.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.492+0000 7f7cd2ffd700 1 -- 172.21.15.69:0/1259747338 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3034 (secure 0 0 0) 0x7f7ccc013070 con 0x7f7cdc1024f0 2024-05-14T16:46:03.492 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:46:03.492 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 -- 172.21.15.69:0/1259747338 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7cc80b2880 msgr2=0x7f7cc80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7cc80b2880 0x7f7cc80b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc19aa80 tx=0x7f7cd80079e0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 -- 172.21.15.69:0/1259747338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1024f0 msgr2=0x7f7cdc19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1024f0 0x7f7cdc19fcc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0098e0 tx=0x7f7ccc0057c0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 -- 172.21.15.69:0/1259747338 shutdown_connections 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cdc109520 0x7f7cdc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cdc101b30 0x7f7cdc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7cc80b2880 0x7f7cc80b4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 --2- 172.21.15.69:0/1259747338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1024f0 0x7f7cdc19fcc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 -- 172.21.15.69:0/1259747338 >> 172.21.15.69:0/1259747338 conn(0x7f7cdc078ab0 msgr2=0x7f7cdc1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 -- 172.21.15.69:0/1259747338 shutdown_connections 2024-05-14T16:46:03.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:03.495+0000 7f7ce4057700 1 -- 172.21.15.69:0/1259747338 wait complete. 2024-05-14T16:46:03.496 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 4 2024-05-14T16:46:03.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:03 smithi069 ceph-mon[31507]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:03.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:03 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1259747338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:03.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:03 smithi203 ceph-mon[35787]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:03.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:03 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1259747338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:03 smithi092 ceph-mon[35763]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:03 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1259747338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:04.508 INFO:journalctl@ceph.mds.d.smithi069.stdout:May 14 16:46:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-d[68883]: starting mds.d at 2024-05-14T16:46:04.508 INFO:journalctl@ceph.mds.d.smithi069.stdout:May 14 16:46:04 smithi069 bash[68856]: e8b2847e04895ebc5cfb1ab6d0b23f116ccc2542cd26c2ee670cf9719a4e53e7 2024-05-14T16:46:04.953 INFO:journalctl@ceph.mds.d.smithi069.stdout:May 14 16:46:04 smithi069 systemd[1]: Started Ceph mds.d for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:46:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: Deploying daemon mds.e on smithi092 2024-05-14T16:46:05.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.69:6836/1744189392,v1:172.21.15.69:6837/1744189392] up:boot 2024-05-14T16:46:05.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: fsmap 4 up:standby 2024-05-14T16:46:05.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:05 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:46:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:05.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:05.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:05.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: Deploying daemon mds.e on smithi092 2024-05-14T16:46:05.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.69:6836/1744189392,v1:172.21.15.69:6837/1744189392] up:boot 2024-05-14T16:46:05.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: fsmap 4 up:standby 2024-05-14T16:46:05.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:05 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:46:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: Deploying daemon mds.e on smithi092 2024-05-14T16:46:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.69:6836/1744189392,v1:172.21.15.69:6837/1744189392] up:boot 2024-05-14T16:46:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: fsmap 4 up:standby 2024-05-14T16:46:06.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:05 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:46:06.277 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:46:06.530 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:06.928 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 16:46:06 smithi092 systemd[1]: Starting Ceph mds.e for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:46:07.199 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 16:46:06 smithi092 podman[65415]: 2024-05-14 16:46:06.83172338 +0000 UTC m=+0.017302252 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:46:07.200 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 16:46:06 smithi092 podman[65415]: 2024-05-14 16:46:06.946621869 +0000 UTC m=+0.132200739 container create a61898b8a3a9ee2d96f214dc23c5677c8f83df716d4d84bce35278b79c1a4e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-05-14T16:46:07.200 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 16:46:07 smithi092 podman[65415]: 2024-05-14 16:46:07.174623477 +0000 UTC m=+0.360202333 container init a61898b8a3a9ee2d96f214dc23c5677c8f83df716d4d84bce35278b79c1a4e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-14T16:46:07.200 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 16:46:07 smithi092 podman[65415]: 2024-05-14 16:46:07.177042853 +0000 UTC m=+0.362621723 container start a61898b8a3a9ee2d96f214dc23c5677c8f83df716d4d84bce35278b79c1a4e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-14T16:46:07.500 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 16:46:07 smithi092 ceph-mds[65435]: -- 172.21.15.92:0/969294763 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565519b9ba40 con 0x565519b6e800 2024-05-14T16:46:07.500 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 16:46:07 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e[65431]: starting mds.e at 2024-05-14T16:46:07.500 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 16:46:07 smithi092 bash[65415]: a61898b8a3a9ee2d96f214dc23c5677c8f83df716d4d84bce35278b79c1a4e4d 2024-05-14T16:46:07.500 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 16:46:07 smithi092 systemd[1]: Started Ceph mds.e for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:46:07.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:07 smithi069 ceph-mon[31507]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:07.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:07.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:07.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:07.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:07.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:07.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:07 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:07 smithi203 ceph-mon[35787]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:07.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:07 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:07 smithi092 ceph-mon[35763]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:07 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.191+0000 7f432f24a700 1 -- 172.21.15.69:0/3767040260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4328108510 msgr2=0x7f43281088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.191+0000 7f432f24a700 1 --2- 172.21.15.69:0/3767040260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4328108510 0x7f43281088f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f431c009990 tx=0x7f431c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.191+0000 7f432f24a700 1 -- 172.21.15.69:0/3767040260 shutdown_connections 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.191+0000 7f432f24a700 1 --2- 172.21.15.69:0/3767040260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4328108510 0x7f43281088f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.191+0000 7f432f24a700 1 --2- 172.21.15.69:0/3767040260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4328102df0 0x7f432810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.191+0000 7f432f24a700 1 --2- 172.21.15.69:0/3767040260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4328102430 0x7f43281028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.191+0000 7f432f24a700 1 -- 172.21.15.69:0/3767040260 >> 172.21.15.69:0/3767040260 conn(0x7f43280793b0 msgr2=0x7f43280797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.191+0000 7f432f24a700 1 -- 172.21.15.69:0/3767040260 shutdown_connections 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.191+0000 7f432f24a700 1 -- 172.21.15.69:0/3767040260 wait complete. 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f432f24a700 1 Processor -- start 2024-05-14T16:46:08.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f432f24a700 1 -- start start 2024-05-14T16:46:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f432f24a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4328102430 0x7f432819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f432f24a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4328102df0 0x7f432819afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f432f24a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4328108510 0x7f43281a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f432f24a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f43281125e0 con 0x7f4328102df0 2024-05-14T16:46:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f432f24a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4328112460 con 0x7f4328102430 2024-05-14T16:46:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f432f24a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4328112760 con 0x7f4328108510 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f4327fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4328102df0 0x7f432819afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f432cfe6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4328102430 0x7f432819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f4327fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4328102df0 0x7f432819afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47140/0 (socket says 172.21.15.69:47140) 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f4327fff700 1 -- 172.21.15.69:0/1179207136 learned_addr learned my addr 172.21.15.69:0/1179207136 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.192+0000 7f432d7e7700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4328108510 0x7f43281a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f432cfe6700 1 -- 172.21.15.69:0/1179207136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4328108510 msgr2=0x7f43281a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f432cfe6700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4328108510 0x7f43281a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f432cfe6700 1 -- 172.21.15.69:0/1179207136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4328102df0 msgr2=0x7f432819afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f432cfe6700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4328102df0 0x7f432819afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f432cfe6700 1 -- 172.21.15.69:0/1179207136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43281a6060 con 0x7f4328102430 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f432cfe6700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4328102430 0x7f432819aaa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f431c0059d0 tx=0x7f431c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f4325ffb700 1 -- 172.21.15.69:0/1179207136 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c027070 con 0x7f4328102430 2024-05-14T16:46:08.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f432f24a700 1 -- 172.21.15.69:0/1179207136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43281a6230 con 0x7f4328102430 2024-05-14T16:46:08.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f4325ffb700 1 -- 172.21.15.69:0/1179207136 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f431c022470 con 0x7f4328102430 2024-05-14T16:46:08.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f4325ffb700 1 -- 172.21.15.69:0/1179207136 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c017650 con 0x7f4328102430 2024-05-14T16:46:08.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.193+0000 7f432f24a700 1 -- 172.21.15.69:0/1179207136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43281a66e0 con 0x7f4328102430 2024-05-14T16:46:08.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.194+0000 7f432f24a700 1 -- 172.21.15.69:0/1179207136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432810a9a0 con 0x7f4328102430 2024-05-14T16:46:08.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.196+0000 7f4325ffb700 1 -- 172.21.15.69:0/1179207136 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f431c02a070 con 0x7f4328102430 2024-05-14T16:46:08.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.196+0000 7f4325ffb700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f43180b2900 0x7f43180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:08.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.196+0000 7f4327fff700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f43180b2900 0x7f43180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:08.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.196+0000 7f4325ffb700 1 -- 172.21.15.69:0/1179207136 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f431c0e31b0 con 0x7f4328102430 2024-05-14T16:46:08.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.197+0000 7f4327fff700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f43180b2900 0x7f43180b4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4328079af0 tx=0x7f4314009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:08.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.200+0000 7f4325ffb700 1 -- 172.21.15.69:0/1179207136 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f431c031070 con 0x7f4328102430 2024-05-14T16:46:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.450+0000 7f432f24a700 1 -- 172.21.15.69:0/1179207136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f432806a230 con 0x7f4328102430 2024-05-14T16:46:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.451+0000 7f4325ffb700 1 -- 172.21.15.69:0/1179207136 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4684 (secure 0 0 0) 0x7f431c013070 con 0x7f4328102430 2024-05-14T16:46:08.451 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:46:08.451 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-14T16:46:08.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 -- 172.21.15.69:0/1179207136 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f43180b2900 msgr2=0x7f43180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:08.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f43180b2900 0x7f43180b4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4328079af0 tx=0x7f4314009260 comp rx=0 tx=0).stop 2024-05-14T16:46:08.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 -- 172.21.15.69:0/1179207136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4328102430 msgr2=0x7f432819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:08.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4328102430 0x7f432819aaa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f431c0059d0 tx=0x7f431c0047b0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 -- 172.21.15.69:0/1179207136 shutdown_connections 2024-05-14T16:46:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4328108510 0x7f43281a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4328102430 0x7f432819aaa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f43180b2900 0x7f43180b4dc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 --2- 172.21.15.69:0/1179207136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4328102df0 0x7f432819afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 -- 172.21.15.69:0/1179207136 >> 172.21.15.69:0/1179207136 conn(0x7f43280793b0 msgr2=0x7f4328101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 -- 172.21.15.69:0/1179207136 shutdown_connections 2024-05-14T16:46:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:08.453+0000 7f432f24a700 1 -- 172.21.15.69:0/1179207136 wait complete. 2024-05-14T16:46:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-05-14T16:46:08.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:08 smithi069 ceph-mon[31507]: Deploying daemon mds.f on smithi203 2024-05-14T16:46:08.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:08 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.92:6834/2453543352,v1:172.21.15.92:6835/2453543352] up:boot 2024-05-14T16:46:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:08 smithi069 ceph-mon[31507]: fsmap 5 up:standby 2024-05-14T16:46:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:08 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:46:08.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:08 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1179207136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:08.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:08 smithi203 ceph-mon[35787]: Deploying daemon mds.f on smithi203 2024-05-14T16:46:08.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:08 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.92:6834/2453543352,v1:172.21.15.92:6835/2453543352] up:boot 2024-05-14T16:46:08.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:08 smithi203 ceph-mon[35787]: fsmap 5 up:standby 2024-05-14T16:46:08.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:08 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:46:08.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:08 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1179207136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:09.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:08 smithi092 ceph-mon[35763]: Deploying daemon mds.f on smithi203 2024-05-14T16:46:09.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:08 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.92:6834/2453543352,v1:172.21.15.92:6835/2453543352] up:boot 2024-05-14T16:46:09.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:08 smithi092 ceph-mon[35763]: fsmap 5 up:standby 2024-05-14T16:46:09.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:08 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:46:09.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:08 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1179207136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:09.582 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 16:46:09 smithi203 systemd[1]: Starting Ceph mds.f for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:46:09.929 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:09 smithi203 ceph-mon[35787]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:09.929 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 16:46:09 smithi203 podman[65957]: 2024-05-14 16:46:09.578293184 +0000 UTC m=+0.017229156 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:46:09.929 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 16:46:09 smithi203 podman[65957]: 2024-05-14 16:46:09.685686824 +0000 UTC m=+0.124622787 container create 46634c2ae77ddef549b3f4a2ef519e9d187fb6dcb350741ad9653f387cd838a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:46:09.929 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 16:46:09 smithi203 podman[65957]: 2024-05-14 16:46:09.930055681 +0000 UTC m=+0.368991645 container init 46634c2ae77ddef549b3f4a2ef519e9d187fb6dcb350741ad9653f387cd838a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T16:46:09.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:09 smithi069 ceph-mon[31507]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:09 smithi092 ceph-mon[35763]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:10.209 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 16:46:09 smithi203 podman[65957]: 2024-05-14 16:46:09.933042267 +0000 UTC m=+0.371978238 container start 46634c2ae77ddef549b3f4a2ef519e9d187fb6dcb350741ad9653f387cd838a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-05-14T16:46:10.209 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 16:46:09 smithi203 ceph-mds[65976]: -- 172.21.15.203:0/887667317 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d2fa47a40 con 0x558d2fa1a800 2024-05-14T16:46:10.209 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 16:46:09 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f[65972]: starting mds.f at 2024-05-14T16:46:10.209 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 16:46:10 smithi203 bash[65957]: 46634c2ae77ddef549b3f4a2ef519e9d187fb6dcb350741ad9653f387cd838a0 2024-05-14T16:46:10.210 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 16:46:10 smithi203 systemd[1]: Started Ceph mds.f for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:46:11.113 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:46:11.365 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:11.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:11.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:11.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:11.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:11.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:11.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:11.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:11.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: Deploying daemon mds.c on smithi203 2024-05-14T16:46:11.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6834/3334129544,v1:172.21.15.203:6835/3334129544] up:boot 2024-05-14T16:46:11.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: fsmap 6 up:standby 2024-05-14T16:46:11.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:11 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:46:11.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:11.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:11.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:11.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:11.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:11.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:11.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:11.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: Deploying daemon mds.c on smithi203 2024-05-14T16:46:11.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6834/3334129544,v1:172.21.15.203:6835/3334129544] up:boot 2024-05-14T16:46:11.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: fsmap 6 up:standby 2024-05-14T16:46:11.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:11 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:46:11.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:11.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:11.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:11.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:11.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:11.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:11.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:11.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: Deploying daemon mds.c on smithi203 2024-05-14T16:46:11.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6834/3334129544,v1:172.21.15.203:6835/3334129544] up:boot 2024-05-14T16:46:11.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: fsmap 6 up:standby 2024-05-14T16:46:11.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:11 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:46:12.459 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:46:12 smithi203 systemd[1]: Starting Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:46:12.751 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:46:12 smithi203 podman[66258]: 2024-05-14 16:46:12.404296962 +0000 UTC m=+0.017180232 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:46:12.751 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:46:12 smithi203 podman[66258]: 2024-05-14 16:46:12.52079983 +0000 UTC m=+0.133683079 container create 37afd030daf7762b90a58fb8f92d12f60d0d9b0992490c8e2f2ba980e9423d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T16:46:12.751 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:46:12 smithi203 podman[66258]: 2024-05-14 16:46:12.749355425 +0000 UTC m=+0.362238681 container init 37afd030daf7762b90a58fb8f92d12f60d0d9b0992490c8e2f2ba980e9423d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-14T16:46:12.752 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:46:12 smithi203 podman[66258]: 2024-05-14 16:46:12.752044613 +0000 UTC m=+0.364927878 container start 37afd030daf7762b90a58fb8f92d12f60d0d9b0992490c8e2f2ba980e9423d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-14T16:46:12.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.926+0000 7f05e550a700 1 -- 172.21.15.69:0/4130074843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05e01088f0 msgr2=0x7f05e0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:12.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.926+0000 7f05e550a700 1 --2- 172.21.15.69:0/4130074843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05e01088f0 0x7f05e0110470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f05d800b0d0 tx=0x7f05d800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:46:12.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.926+0000 7f05e550a700 1 -- 172.21.15.69:0/4130074843 shutdown_connections 2024-05-14T16:46:12.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.926+0000 7f05e550a700 1 --2- 172.21.15.69:0/4130074843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05e01088f0 0x7f05e0110470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:12.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.926+0000 7f05e550a700 1 --2- 172.21.15.69:0/4130074843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e01037f0 0x7f05e0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:12.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.926+0000 7f05e550a700 1 --2- 172.21.15.69:0/4130074843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e01041a0 0x7f05e01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:12.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.926+0000 7f05e550a700 1 -- 172.21.15.69:0/4130074843 >> 172.21.15.69:0/4130074843 conn(0x7f05e007a6a0 msgr2=0x7f05e007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:12.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.926+0000 7f05e550a700 1 -- 172.21.15.69:0/4130074843 shutdown_connections 2024-05-14T16:46:12.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.926+0000 7f05e550a700 1 -- 172.21.15.69:0/4130074843 wait complete. 2024-05-14T16:46:12.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.926+0000 7f05e550a700 1 Processor -- start 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05e550a700 1 -- start start 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05e550a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e01037f0 0x7f05e019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05e550a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e01041a0 0x7f05e019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05e550a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05e01088f0 0x7f05e0199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05e550a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05e01136c0 con 0x7f05e01088f0 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05e550a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f05e0113540 con 0x7f05e01037f0 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05e550a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05e0113840 con 0x7f05e01041a0 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05df7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05e01088f0 0x7f05e0199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05deffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e01037f0 0x7f05e019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05df7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05e01088f0 0x7f05e0199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47152/0 (socket says 172.21.15.69:47152) 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05df7fe700 1 -- 172.21.15.69:0/2984516296 learned_addr learned my addr 172.21.15.69:0/2984516296 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:12.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05de7fc700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e01041a0 0x7f05e019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05deffd700 1 -- 172.21.15.69:0/2984516296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e01041a0 msgr2=0x7f05e019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05deffd700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e01041a0 0x7f05e019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05deffd700 1 -- 172.21.15.69:0/2984516296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05e01088f0 msgr2=0x7f05e0199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05deffd700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05e01088f0 0x7f05e0199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05deffd700 1 -- 172.21.15.69:0/2984516296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e0199fe0 con 0x7f05e01037f0 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.927+0000 7f05df7fe700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05e01088f0 0x7f05e0199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.928+0000 7f05deffd700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e01037f0 0x7f05e019f6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05d000e9f0 tx=0x7f05d000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.928+0000 7f05c7fff700 1 -- 172.21.15.69:0/2984516296 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d000cbd0 con 0x7f05e01037f0 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.928+0000 7f05c7fff700 1 -- 172.21.15.69:0/2984516296 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d000cd70 con 0x7f05e01037f0 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.928+0000 7f05e550a700 1 -- 172.21.15.69:0/2984516296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e019a2d0 con 0x7f05e01037f0 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.928+0000 7f05c7fff700 1 -- 172.21.15.69:0/2984516296 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0018680 con 0x7f05e01037f0 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.928+0000 7f05e550a700 1 -- 172.21.15.69:0/2984516296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e01a6590 con 0x7f05e01037f0 2024-05-14T16:46:12.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.929+0000 7f05e550a700 1 -- 172.21.15.69:0/2984516296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e006a230 con 0x7f05e01037f0 2024-05-14T16:46:12.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.930+0000 7f05c7fff700 1 -- 172.21.15.69:0/2984516296 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f05d0010430 con 0x7f05e01037f0 2024-05-14T16:46:12.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.931+0000 7f05c7fff700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f05c80b2880 0x7f05c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:12.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.931+0000 7f05de7fc700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f05c80b2880 0x7f05c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:12.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.931+0000 7f05c7fff700 1 -- 172.21.15.69:0/2984516296 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f05d00e1fb0 con 0x7f05e01037f0 2024-05-14T16:46:12.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.931+0000 7f05de7fc700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f05c80b2880 0x7f05c80b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f05d4005ea0 tx=0x7f05d4005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:12.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:12.935+0000 7f05c7fff700 1 -- 172.21.15.69:0/2984516296 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f05d00aa0d0 con 0x7f05e01037f0 2024-05-14T16:46:13.088 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:46:12 smithi203 ceph-mds[66277]: -- 172.21.15.203:0/1959739513 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557863289a40 con 0x55786325d000 2024-05-14T16:46:13.088 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:46:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[66273]: starting mds.c at 2024-05-14T16:46:13.088 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:46:12 smithi203 bash[66258]: 37afd030daf7762b90a58fb8f92d12f60d0d9b0992490c8e2f2ba980e9423d9f 2024-05-14T16:46:13.088 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:46:12 smithi203 systemd[1]: Started Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:46:13.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.186+0000 7f05e550a700 1 -- 172.21.15.69:0/2984516296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05e01a67c0 con 0x7f05e01037f0 2024-05-14T16:46:13.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.187+0000 7f05c7fff700 1 -- 172.21.15.69:0/2984516296 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6340 (secure 0 0 0) 0x7f05d00a99d0 con 0x7f05e01037f0 2024-05-14T16:46:13.187 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:46:13.187 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 -- 172.21.15.69:0/2984516296 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f05c80b2880 msgr2=0x7f05c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f05c80b2880 0x7f05c80b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f05d4005ea0 tx=0x7f05d4005e30 comp rx=0 tx=0).stop 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 -- 172.21.15.69:0/2984516296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e01037f0 msgr2=0x7f05e019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e01037f0 0x7f05e019f6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05d000e9f0 tx=0x7f05d000ed00 comp rx=0 tx=0).stop 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 -- 172.21.15.69:0/2984516296 shutdown_connections 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f05c80b2880 0x7f05c80b4d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05e01088f0 0x7f05e0199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e01037f0 0x7f05e019f6c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 --2- 172.21.15.69:0/2984516296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e01041a0 0x7f05e019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 -- 172.21.15.69:0/2984516296 >> 172.21.15.69:0/2984516296 conn(0x7f05e007a6a0 msgr2=0x7f05e0101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:13.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 -- 172.21.15.69:0/2984516296 shutdown_connections 2024-05-14T16:46:13.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:13.189+0000 7f05e550a700 1 -- 172.21.15.69:0/2984516296 wait complete. 2024-05-14T16:46:13.190 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 8 2024-05-14T16:46:13.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:13 smithi069 ceph-mon[31507]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:13.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:13 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:13.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:13 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:13.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:13 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:13.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:13 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:13.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:13 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:13.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:13 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:13.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:13 smithi203 ceph-mon[35787]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:13.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:13 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:13.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:13 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:13.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:13 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:13.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:13 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:13.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:13 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:13.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:13 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:13.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:13 smithi092 ceph-mon[35763]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:13.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:13 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:13.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:13 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:13.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:13 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:13.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:13 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:13.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:13 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:13.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:13 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:14.359 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:14 smithi203 ceph-mon[35787]: Deploying daemon mds.h on smithi092 2024-05-14T16:46:14.359 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:14 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6836/1824212740,v1:172.21.15.203:6837/1824212740] up:boot 2024-05-14T16:46:14.360 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:14 smithi203 ceph-mon[35787]: fsmap 7 up:standby 2024-05-14T16:46:14.360 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:14 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:46:14.360 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:14 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2984516296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:14.430 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:14 smithi069 ceph-mon[31507]: Deploying daemon mds.h on smithi092 2024-05-14T16:46:14.430 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:14 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6836/1824212740,v1:172.21.15.203:6837/1824212740] up:boot 2024-05-14T16:46:14.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:14 smithi069 ceph-mon[31507]: fsmap 7 up:standby 2024-05-14T16:46:14.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:14 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:46:14.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:14 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2984516296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:14.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:14 smithi092 ceph-mon[35763]: Deploying daemon mds.h on smithi092 2024-05-14T16:46:14.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:14 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6836/1824212740,v1:172.21.15.203:6837/1824212740] up:boot 2024-05-14T16:46:14.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:14 smithi092 ceph-mon[35763]: fsmap 7 up:standby 2024-05-14T16:46:14.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:14 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:46:14.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:14 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2984516296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:15.250 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 16:46:15 smithi092 systemd[1]: Starting Ceph mds.h for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:46:15.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:15 smithi092 ceph-mon[35763]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:15.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:15 smithi069 ceph-mon[31507]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:15.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:15 smithi203 ceph-mon[35787]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:15.697 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 16:46:15 smithi092 podman[65731]: 2024-05-14 16:46:15.279101703 +0000 UTC m=+0.016567145 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:46:15.697 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 16:46:15 smithi092 podman[65731]: 2024-05-14 16:46:15.394985127 +0000 UTC m=+0.132450599 container create 545cf6bb8536cc64134fa7e309609785e88d53f43fc2a4eba4d4004c6540c9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:46:15.697 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 16:46:15 smithi092 podman[65731]: 2024-05-14 16:46:15.697840294 +0000 UTC m=+0.435305719 container init 545cf6bb8536cc64134fa7e309609785e88d53f43fc2a4eba4d4004c6540c9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T16:46:15.814 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:46:16.000 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 16:46:15 smithi092 podman[65731]: 2024-05-14 16:46:15.700708006 +0000 UTC m=+0.438173443 container start 545cf6bb8536cc64134fa7e309609785e88d53f43fc2a4eba4d4004c6540c9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-14T16:46:16.000 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 16:46:15 smithi092 ceph-mds[65750]: -- 172.21.15.92:0/2107289855 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555dc9de1a40 con 0x555dc9db4c00 2024-05-14T16:46:16.001 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 16:46:15 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h[65746]: starting mds.h at 2024-05-14T16:46:16.001 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 16:46:15 smithi092 bash[65731]: 545cf6bb8536cc64134fa7e309609785e88d53f43fc2a4eba4d4004c6540c9c4 2024-05-14T16:46:16.001 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 16:46:15 smithi092 systemd[1]: Started Ceph mds.h for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:46:16.082 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:17.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:17.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:17.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:17.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:17.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:17.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:17.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:17.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: Deploying daemon mds.i on smithi203 2024-05-14T16:46:17.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] up:boot 2024-05-14T16:46:17.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: fsmap 8 up:standby 2024-05-14T16:46:17.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:16 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:46:17.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:17.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:17.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:17.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:17.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:17.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:17.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:17.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: Deploying daemon mds.i on smithi203 2024-05-14T16:46:17.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] up:boot 2024-05-14T16:46:17.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: fsmap 8 up:standby 2024-05-14T16:46:17.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:16 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:46:17.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:17.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:17.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:17.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:17.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:17.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:17.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:17.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: Deploying daemon mds.i on smithi203 2024-05-14T16:46:17.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] up:boot 2024-05-14T16:46:17.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: fsmap 8 up:standby 2024-05-14T16:46:17.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:16 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:46:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.707+0000 7f7f9039d700 1 -- 172.21.15.69:0/1646935126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f88102470 msgr2=0x7f7f881028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.707+0000 7f7f9039d700 1 --2- 172.21.15.69:0/1646935126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f88102470 0x7f7f881028f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009990 tx=0x7f7f78009ca0 comp rx=0 tx=0).stop 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.707+0000 7f7f9039d700 1 -- 172.21.15.69:0/1646935126 shutdown_connections 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.707+0000 7f7f9039d700 1 --2- 172.21.15.69:0/1646935126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f88102e30 0x7f7f8810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.707+0000 7f7f9039d700 1 --2- 172.21.15.69:0/1646935126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f881084b0 0x7f7f88108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.707+0000 7f7f9039d700 1 --2- 172.21.15.69:0/1646935126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f88102470 0x7f7f881028f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.707+0000 7f7f9039d700 1 -- 172.21.15.69:0/1646935126 >> 172.21.15.69:0/1646935126 conn(0x7f7f88079410 msgr2=0x7f7f88079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.707+0000 7f7f9039d700 1 -- 172.21.15.69:0/1646935126 shutdown_connections 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.707+0000 7f7f9039d700 1 -- 172.21.15.69:0/1646935126 wait complete. 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f9039d700 1 Processor -- start 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f9039d700 1 -- start start 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f9039d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f88102470 0x7f7f8819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f9039d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f88102e30 0x7f7f8819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f9039d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f881084b0 0x7f7f88199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:17.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f9039d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88113710 con 0x7f7f88102470 2024-05-14T16:46:17.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f9039d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88113590 con 0x7f7f881084b0 2024-05-14T16:46:17.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f9039d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88113890 con 0x7f7f88102e30 2024-05-14T16:46:17.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8e139700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f88102470 0x7f7f8819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:17.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8e139700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f88102470 0x7f7f8819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37324/0 (socket says 172.21.15.69:37324) 2024-05-14T16:46:17.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8e139700 1 -- 172.21.15.69:0/4184179594 learned_addr learned my addr 172.21.15.69:0/4184179594 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:17.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8d938700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f88102e30 0x7f7f8819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:17.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8e139700 1 -- 172.21.15.69:0/4184179594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f88102e30 msgr2=0x7f7f8819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:17.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8e139700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f88102e30 0x7f7f8819fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:17.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8e139700 1 -- 172.21.15.69:0/4184179594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f881084b0 msgr2=0x7f7f88199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8e139700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f881084b0 0x7f7f88199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8e139700 1 -- 172.21.15.69:0/4184179594 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8819a1f0 con 0x7f7f88102470 2024-05-14T16:46:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8d938700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f88102e30 0x7f7f8819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:46:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f8e139700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f88102470 0x7f7f8819f8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400d950 tx=0x7f7f8400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.708+0000 7f7f7f7fe700 1 -- 172.21.15.69:0/4184179594 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84016070 con 0x7f7f88102470 2024-05-14T16:46:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.709+0000 7f7f7f7fe700 1 -- 172.21.15.69:0/4184179594 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f84010460 con 0x7f7f88102470 2024-05-14T16:46:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.709+0000 7f7f9039d700 1 -- 172.21.15.69:0/4184179594 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8819a4e0 con 0x7f7f88102470 2024-05-14T16:46:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.709+0000 7f7f7f7fe700 1 -- 172.21.15.69:0/4184179594 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400f620 con 0x7f7f88102470 2024-05-14T16:46:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.709+0000 7f7f9039d700 1 -- 172.21.15.69:0/4184179594 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f8810f9b0 con 0x7f7f88102470 2024-05-14T16:46:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.709+0000 7f7f9039d700 1 -- 172.21.15.69:0/4184179594 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8806a230 con 0x7f7f88102470 2024-05-14T16:46:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.711+0000 7f7f7f7fe700 1 -- 172.21.15.69:0/4184179594 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7f84019070 con 0x7f7f88102470 2024-05-14T16:46:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.712+0000 7f7f7f7fe700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7f740b2880 0x7f7f740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.712+0000 7f7f8d938700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7f740b2880 0x7f7f740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.712+0000 7f7f7f7fe700 1 -- 172.21.15.69:0/4184179594 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7f840e18d0 con 0x7f7f88102470 2024-05-14T16:46:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.712+0000 7f7f8d938700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7f740b2880 0x7f7f740b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800b2e0 tx=0x7f7f78005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:17.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.715+0000 7f7f7f7fe700 1 -- 172.21.15.69:0/4184179594 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f840a9a90 con 0x7f7f88102470 2024-05-14T16:46:17.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.966+0000 7f7f9039d700 1 -- 172.21.15.69:0/4184179594 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f8802e560 con 0x7f7f88102470 2024-05-14T16:46:17.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.967+0000 7f7f7f7fe700 1 -- 172.21.15.69:0/4184179594 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7165 (secure 0 0 0) 0x7f7f840a9390 con 0x7f7f88102470 2024-05-14T16:46:17.968 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:46:17.968 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-14T16:46:17.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 -- 172.21.15.69:0/4184179594 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7f740b2880 msgr2=0x7f7f740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:17.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7f740b2880 0x7f7f740b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800b2e0 tx=0x7f7f78005e80 comp rx=0 tx=0).stop 2024-05-14T16:46:17.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 -- 172.21.15.69:0/4184179594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f88102470 msgr2=0x7f7f8819f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:17.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f88102470 0x7f7f8819f8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400d950 tx=0x7f7f8400dd10 comp rx=0 tx=0).stop 2024-05-14T16:46:17.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 -- 172.21.15.69:0/4184179594 shutdown_connections 2024-05-14T16:46:17.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7f740b2880 0x7f7f740b4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:17.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f88102470 0x7f7f8819f8d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:17.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f881084b0 0x7f7f88199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:17.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 --2- 172.21.15.69:0/4184179594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f88102e30 0x7f7f8819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:17.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 -- 172.21.15.69:0/4184179594 >> 172.21.15.69:0/4184179594 conn(0x7f7f88079410 msgr2=0x7f7f88100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:17.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 -- 172.21.15.69:0/4184179594 shutdown_connections 2024-05-14T16:46:17.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:17.969+0000 7f7f9039d700 1 -- 172.21.15.69:0/4184179594 wait complete. 2024-05-14T16:46:17.971 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 9 2024-05-14T16:46:18.209 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 16:46:18 smithi203 systemd[1]: Starting Ceph mds.i for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:46:18.666 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 16:46:18 smithi203 podman[66571]: 2024-05-14 16:46:18.195278874 +0000 UTC m=+0.017229022 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:46:18.666 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 16:46:18 smithi203 podman[66571]: 2024-05-14 16:46:18.314000117 +0000 UTC m=+0.135950252 container create 330076950df914426aa094ec224c454fad30ebc40b6cfdcc3867666c09d0028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-14T16:46:18.666 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 16:46:18 smithi203 podman[66571]: 2024-05-14 16:46:18.667492703 +0000 UTC m=+0.489442837 container init 330076950df914426aa094ec224c454fad30ebc40b6cfdcc3867666c09d0028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-14T16:46:18.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:18 smithi069 ceph-mon[31507]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:18.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:18 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/4184179594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:18.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:18 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:18.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:18 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:18.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:18 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:18.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:18 smithi203 ceph-mon[35787]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:18.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:18 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/4184179594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:18.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:18 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:18.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:18 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:18.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:18 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:18.960 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 16:46:18 smithi203 podman[66571]: 2024-05-14 16:46:18.670396614 +0000 UTC m=+0.492346754 container start 330076950df914426aa094ec224c454fad30ebc40b6cfdcc3867666c09d0028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-05-14T16:46:18.960 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 16:46:18 smithi203 ceph-mds[66590]: -- 172.21.15.203:0/1710700807 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fbbad93a40 con 0x55fbbad66c00 2024-05-14T16:46:18.960 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 16:46:18 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i[66586]: starting mds.i at 2024-05-14T16:46:18.960 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 16:46:18 smithi203 bash[66571]: 330076950df914426aa094ec224c454fad30ebc40b6cfdcc3867666c09d0028b 2024-05-14T16:46:18.960 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 16:46:18 smithi203 systemd[1]: Started Ceph mds.i for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:46:19.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:18 smithi092 ceph-mon[35763]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:19.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:18 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/4184179594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:19.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:18 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:19.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:18 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:19.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:18 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:20.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:20.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:20.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:20.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:19 smithi069 ceph-mon[31507]: Deploying daemon mds.g on smithi069 2024-05-14T16:46:20.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:19 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] up:boot 2024-05-14T16:46:20.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:19 smithi069 ceph-mon[31507]: fsmap 9 up:standby 2024-05-14T16:46:20.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:19 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:46:20.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:20.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:20.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:20.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:19 smithi203 ceph-mon[35787]: Deploying daemon mds.g on smithi069 2024-05-14T16:46:20.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:19 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] up:boot 2024-05-14T16:46:20.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:19 smithi203 ceph-mon[35787]: fsmap 9 up:standby 2024-05-14T16:46:20.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:19 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:46:20.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:20.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:20.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:20.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:19 smithi092 ceph-mon[35763]: Deploying daemon mds.g on smithi069 2024-05-14T16:46:20.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:19 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] up:boot 2024-05-14T16:46:20.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:19 smithi092 ceph-mon[35763]: fsmap 9 up:standby 2024-05-14T16:46:20.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:19 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:46:20.656 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:46:20.970 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:21.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:20 smithi069 ceph-mon[31507]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:21.102 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 16:46:20 smithi069 systemd[1]: Starting Ceph mds.g for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:46:21.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:20 smithi203 ceph-mon[35787]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:21.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:20 smithi092 ceph-mon[35763]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:22.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.561+0000 7ff21efdc700 1 -- 172.21.15.69:0/964130670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181027d0 msgr2=0x7ff218106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:22.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.561+0000 7ff21efdc700 1 --2- 172.21.15.69:0/964130670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181027d0 0x7ff218106820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff2040099c0 tx=0x7ff204009cd0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.561+0000 7ff21efdc700 1 -- 172.21.15.69:0/964130670 shutdown_connections 2024-05-14T16:46:22.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.561+0000 7ff21efdc700 1 --2- 172.21.15.69:0/964130670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff218106f00 0x7ff21810ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.561+0000 7ff21efdc700 1 --2- 172.21.15.69:0/964130670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff218101e20 0x7ff218102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.561+0000 7ff21efdc700 1 --2- 172.21.15.69:0/964130670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181027d0 0x7ff218106820 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.561+0000 7ff21efdc700 1 -- 172.21.15.69:0/964130670 >> 172.21.15.69:0/964130670 conn(0x7ff218078a40 msgr2=0x7ff2180ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:22.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.561+0000 7ff21efdc700 1 -- 172.21.15.69:0/964130670 shutdown_connections 2024-05-14T16:46:22.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.561+0000 7ff21efdc700 1 -- 172.21.15.69:0/964130670 wait complete. 2024-05-14T16:46:22.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff21efdc700 1 Processor -- start 2024-05-14T16:46:22.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff21efdc700 1 -- start start 2024-05-14T16:46:22.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff21efdc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff218101e20 0x7ff21819bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:22.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff21efdc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181027d0 0x7ff21819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:22.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff21efdc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff218106f00 0x7ff2181a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:22.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff21efdc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181126f0 con 0x7ff2181027d0 2024-05-14T16:46:22.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff21efdc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff218112570 con 0x7ff218101e20 2024-05-14T16:46:22.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff21efdc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff218112870 con 0x7ff218106f00 2024-05-14T16:46:22.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff217fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181027d0 0x7ff21819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:22.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff217fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181027d0 0x7ff21819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37340/0 (socket says 172.21.15.69:37340) 2024-05-14T16:46:22.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff217fff700 1 -- 172.21.15.69:0/21112606 learned_addr learned my addr 172.21.15.69:0/21112606 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff21cd78700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff218101e20 0x7ff21819bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff21d579700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff218106f00 0x7ff2181a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff217fff700 1 -- 172.21.15.69:0/21112606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff218106f00 msgr2=0x7ff2181a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff217fff700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff218106f00 0x7ff2181a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff217fff700 1 -- 172.21.15.69:0/21112606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff218101e20 msgr2=0x7ff21819bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.562+0000 7ff217fff700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff218101e20 0x7ff21819bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.563+0000 7ff217fff700 1 -- 172.21.15.69:0/21112606 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2181a0f30 con 0x7ff2181027d0 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.563+0000 7ff21d579700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff218106f00 0x7ff2181a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.563+0000 7ff217fff700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181027d0 0x7ff21819c460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff204009f50 tx=0x7ff2040037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.563+0000 7ff215ffb700 1 -- 172.21.15.69:0/21112606 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204027070 con 0x7ff2181027d0 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.563+0000 7ff215ffb700 1 -- 172.21.15.69:0/21112606 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff204004090 con 0x7ff2181027d0 2024-05-14T16:46:22.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.563+0000 7ff215ffb700 1 -- 172.21.15.69:0/21112606 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204022e60 con 0x7ff2181027d0 2024-05-14T16:46:22.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.563+0000 7ff21efdc700 1 -- 172.21.15.69:0/21112606 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2181a11c0 con 0x7ff2181027d0 2024-05-14T16:46:22.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.563+0000 7ff21efdc700 1 -- 172.21.15.69:0/21112606 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2181a8aa0 con 0x7ff2181027d0 2024-05-14T16:46:22.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.564+0000 7ff21efdc700 1 -- 172.21.15.69:0/21112606 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff218103300 con 0x7ff2181027d0 2024-05-14T16:46:22.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.566+0000 7ff215ffb700 1 -- 172.21.15.69:0/21112606 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff20402a070 con 0x7ff2181027d0 2024-05-14T16:46:22.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.567+0000 7ff215ffb700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2080b2900 0x7ff2080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:22.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.567+0000 7ff21cd78700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2080b2900 0x7ff2080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:22.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.567+0000 7ff215ffb700 1 -- 172.21.15.69:0/21112606 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff2040e2ba0 con 0x7ff2181027d0 2024-05-14T16:46:22.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.568+0000 7ff21cd78700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2080b2900 0x7ff2080b4dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009a10 tx=0x7ff20c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:22.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.572+0000 7ff215ffb700 1 -- 172.21.15.69:0/21112606 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff20405fdf0 con 0x7ff2181027d0 2024-05-14T16:46:22.824 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 16:46:22 smithi069 podman[69776]: 2024-05-14 16:46:22.566340805 +0000 UTC m=+0.097548129 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:46:22.824 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 16:46:22 smithi069 podman[69776]: 2024-05-14 16:46:22.693458182 +0000 UTC m=+0.224665486 container create 5a6ffcf4a6db964709fbc6d6943b02105aeaba74252756101de7afb8efc008f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-14T16:46:22.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.836+0000 7ff21efdc700 1 -- 172.21.15.69:0/21112606 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff21819d280 con 0x7ff2181027d0 2024-05-14T16:46:22.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.837+0000 7ff215ffb700 1 -- 172.21.15.69:0/21112606 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7992 (secure 0 0 0) 0x7ff204013070 con 0x7ff2181027d0 2024-05-14T16:46:22.837 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:46:22.837 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-14T16:46:22.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 -- 172.21.15.69:0/21112606 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2080b2900 msgr2=0x7ff2080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:22.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2080b2900 0x7ff2080b4dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009a10 tx=0x7ff20c009190 comp rx=0 tx=0).stop 2024-05-14T16:46:22.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 -- 172.21.15.69:0/21112606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181027d0 msgr2=0x7ff21819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:22.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181027d0 0x7ff21819c460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff204009f50 tx=0x7ff2040037c0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 -- 172.21.15.69:0/21112606 shutdown_connections 2024-05-14T16:46:22.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff218106f00 0x7ff2181a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff218101e20 0x7ff21819bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2080b2900 0x7ff2080b4dc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 --2- 172.21.15.69:0/21112606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181027d0 0x7ff21819c460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:22.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 -- 172.21.15.69:0/21112606 >> 172.21.15.69:0/21112606 conn(0x7ff218078a40 msgr2=0x7ff21810fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:22.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 -- 172.21.15.69:0/21112606 shutdown_connections 2024-05-14T16:46:22.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:22.839+0000 7ff21efdc700 1 -- 172.21.15.69:0/21112606 wait complete. 2024-05-14T16:46:22.840 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 10 2024-05-14T16:46:23.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:22 smithi069 ceph-mon[31507]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:23.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:22 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/21112606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:23.093 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 16:46:22 smithi069 podman[69776]: 2024-05-14 16:46:22.954166623 +0000 UTC m=+0.485373924 container init 5a6ffcf4a6db964709fbc6d6943b02105aeaba74252756101de7afb8efc008f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-g, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-14T16:46:23.093 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 16:46:22 smithi069 podman[69776]: 2024-05-14 16:46:22.95721578 +0000 UTC m=+0.488423090 container start 5a6ffcf4a6db964709fbc6d6943b02105aeaba74252756101de7afb8efc008f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-14T16:46:23.093 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 16:46:22 smithi069 ceph-mds[69828]: -- 172.21.15.69:0/4259130455 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7832bda40 con 0x55c783290c00 2024-05-14T16:46:23.093 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 16:46:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-g[69811]: starting mds.g at 2024-05-14T16:46:23.093 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 16:46:23 smithi069 bash[69776]: 5a6ffcf4a6db964709fbc6d6943b02105aeaba74252756101de7afb8efc008f6 2024-05-14T16:46:23.093 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 16:46:23 smithi069 systemd[1]: Started Ceph mds.g for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:46:23.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:22 smithi203 ceph-mon[35787]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:23.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:22 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/21112606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:23.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:22 smithi092 ceph-mon[35763]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:23.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:22 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/21112606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:24.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:24.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:24.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:24.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:24.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:24.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:24.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:24 smithi069 ceph-mon[31507]: Deploying daemon mds.a on smithi069 2024-05-14T16:46:24.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:24 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.69:6838/3026437817,v1:172.21.15.69:6839/3026437817] up:boot 2024-05-14T16:46:24.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:24 smithi069 ceph-mon[31507]: fsmap 10 up:standby 2024-05-14T16:46:24.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:24 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:46:24.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:24.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:24.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:24.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:24.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:24.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:24.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:24 smithi203 ceph-mon[35787]: Deploying daemon mds.a on smithi069 2024-05-14T16:46:24.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:24 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.69:6838/3026437817,v1:172.21.15.69:6839/3026437817] up:boot 2024-05-14T16:46:24.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:24 smithi203 ceph-mon[35787]: fsmap 10 up:standby 2024-05-14T16:46:24.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:24 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:46:24.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:24.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:24.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:24.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:24.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:24.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:24.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:24 smithi092 ceph-mon[35763]: Deploying daemon mds.a on smithi069 2024-05-14T16:46:24.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:24 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.69:6838/3026437817,v1:172.21.15.69:6839/3026437817] up:boot 2024-05-14T16:46:24.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:24 smithi092 ceph-mon[35763]: fsmap 10 up:standby 2024-05-14T16:46:24.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:24 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:46:25.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:25 smithi069 ceph-mon[31507]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:25.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:25 smithi203 ceph-mon[35787]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:25.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:25 smithi092 ceph-mon[35763]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:25.684 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:46:25.937 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:26.059 INFO:journalctl@ceph.mds.a.smithi069.stdout:May 14 16:46:25 smithi069 systemd[1]: Started Ceph mds.a for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:46:27.080 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:27.080 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:27.080 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:27.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:27.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:27.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:27.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:27.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: Deploying daemon mds.b on smithi092 2024-05-14T16:46:27.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.69:6840/3552362754,v1:172.21.15.69:6841/3552362754] up:boot 2024-05-14T16:46:27.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: fsmap 11 up:standby 2024-05-14T16:46:27.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:26 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: Deploying daemon mds.b on smithi092 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.69:6840/3552362754,v1:172.21.15.69:6841/3552362754] up:boot 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: fsmap 11 up:standby 2024-05-14T16:46:27.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:26 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:46:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:46:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:46:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: Deploying daemon mds.b on smithi092 2024-05-14T16:46:27.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.69:6840/3552362754,v1:172.21.15.69:6841/3552362754] up:boot 2024-05-14T16:46:27.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: fsmap 11 up:standby 2024-05-14T16:46:27.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:26 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:46:27.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.626+0000 7f2c48e98700 1 -- 172.21.15.69:0/3283235968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c441042f0 msgr2=0x7f2c441046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:27.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.626+0000 7f2c48e98700 1 --2- 172.21.15.69:0/3283235968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c441042f0 0x7f2c441046d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009990 tx=0x7f2c34009ca0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.626+0000 7f2c48e98700 1 -- 172.21.15.69:0/3283235968 shutdown_connections 2024-05-14T16:46:27.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.626+0000 7f2c48e98700 1 --2- 172.21.15.69:0/3283235968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c441042f0 0x7f2c441046d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.626+0000 7f2c48e98700 1 --2- 172.21.15.69:0/3283235968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c44102e30 0x7f2c4410f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.626+0000 7f2c48e98700 1 --2- 172.21.15.69:0/3283235968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c44102470 0x7f2c441028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.626+0000 7f2c48e98700 1 -- 172.21.15.69:0/3283235968 >> 172.21.15.69:0/3283235968 conn(0x7f2c440784b0 msgr2=0x7f2c440788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.626+0000 7f2c48e98700 1 -- 172.21.15.69:0/3283235968 shutdown_connections 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.626+0000 7f2c48e98700 1 -- 172.21.15.69:0/3283235968 wait complete. 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c48e98700 1 Processor -- start 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c48e98700 1 -- start start 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c48e98700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c44102470 0x7f2c4419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c48e98700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c44102e30 0x7f2c4419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c48e98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c441042f0 0x7f2c44199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c48e98700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441126f0 con 0x7f2c44102e30 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c48e98700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44112570 con 0x7f2c441042f0 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c48e98700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44112870 con 0x7f2c44102470 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c41d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c44102e30 0x7f2c4419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c41d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c44102e30 0x7f2c4419fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55364/0 (socket says 172.21.15.69:55364) 2024-05-14T16:46:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c41d9b700 1 -- 172.21.15.69:0/330120965 learned_addr learned my addr 172.21.15.69:0/330120965 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c42d9d700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c441042f0 0x7f2c44199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c4259c700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c44102470 0x7f2c4419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c41d9b700 1 -- 172.21.15.69:0/330120965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c44102470 msgr2=0x7f2c4419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c41d9b700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c44102470 0x7f2c4419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c41d9b700 1 -- 172.21.15.69:0/330120965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c441042f0 msgr2=0x7f2c44199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c41d9b700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c441042f0 0x7f2c44199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.627+0000 7f2c41d9b700 1 -- 172.21.15.69:0/330120965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4419a1c0 con 0x7f2c44102e30 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.628+0000 7f2c4259c700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c44102470 0x7f2c4419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.628+0000 7f2c41d9b700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c44102e30 0x7f2c4419fde0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c9b0 tx=0x7f2c3800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.628+0000 7f2c337fe700 1 -- 172.21.15.69:0/330120965 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38007b00 con 0x7f2c44102e30 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.628+0000 7f2c337fe700 1 -- 172.21.15.69:0/330120965 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c38007ca0 con 0x7f2c44102e30 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.628+0000 7f2c48e98700 1 -- 172.21.15.69:0/330120965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4419a4b0 con 0x7f2c44102e30 2024-05-14T16:46:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.628+0000 7f2c337fe700 1 -- 172.21.15.69:0/330120965 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38018660 con 0x7f2c44102e30 2024-05-14T16:46:27.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.628+0000 7f2c48e98700 1 -- 172.21.15.69:0/330120965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c441a6770 con 0x7f2c44102e30 2024-05-14T16:46:27.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.629+0000 7f2c48e98700 1 -- 172.21.15.69:0/330120965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4406a230 con 0x7f2c44102e30 2024-05-14T16:46:27.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.630+0000 7f2c337fe700 1 -- 172.21.15.69:0/330120965 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2c3800f450 con 0x7f2c44102e30 2024-05-14T16:46:27.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.631+0000 7f2c337fe700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c2c0b2880 0x7f2c2c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:27.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.631+0000 7f2c4259c700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c2c0b2880 0x7f2c2c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:27.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.631+0000 7f2c337fe700 1 -- 172.21.15.69:0/330120965 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f2c380e2010 con 0x7f2c44102e30 2024-05-14T16:46:27.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.631+0000 7f2c4259c700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c2c0b2880 0x7f2c2c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c34005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:27.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.635+0000 7f2c337fe700 1 -- 172.21.15.69:0/330120965 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c38014030 con 0x7f2c44102e30 2024-05-14T16:46:27.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.885+0000 7f2c48e98700 1 -- 172.21.15.69:0/330120965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c4419af90 con 0x7f2c44102e30 2024-05-14T16:46:27.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.885+0000 7f2c337fe700 1 -- 172.21.15.69:0/330120965 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7f2c4419af90 con 0x7f2c44102e30 2024-05-14T16:46:27.886 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:46:27.886 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-14T16:46:27.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 -- 172.21.15.69:0/330120965 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c2c0b2880 msgr2=0x7f2c2c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:27.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c2c0b2880 0x7f2c2c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c34005fd0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 -- 172.21.15.69:0/330120965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c44102e30 msgr2=0x7f2c4419fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:27.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c44102e30 0x7f2c4419fde0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c9b0 tx=0x7f2c3800ccc0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 -- 172.21.15.69:0/330120965 shutdown_connections 2024-05-14T16:46:27.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c441042f0 0x7f2c44199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c44102470 0x7f2c4419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c2c0b2880 0x7f2c2c0b4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 --2- 172.21.15.69:0/330120965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c44102e30 0x7f2c4419fde0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:27.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 -- 172.21.15.69:0/330120965 >> 172.21.15.69:0/330120965 conn(0x7f2c440784b0 msgr2=0x7f2c44101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:27.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 -- 172.21.15.69:0/330120965 shutdown_connections 2024-05-14T16:46:27.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:27.888+0000 7f2c48e98700 1 -- 172.21.15.69:0/330120965 wait complete. 2024-05-14T16:46:27.889 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 12 2024-05-14T16:46:28.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:27 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/330120965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:28.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:27 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/330120965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:28.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:27 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/330120965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:28.250 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 16:46:28 smithi092 systemd[1]: Starting Ceph mds.b for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:46:28.595 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 16:46:28 smithi092 podman[66036]: 2024-05-14 16:46:28.233987481 +0000 UTC m=+0.016746713 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:46:28.595 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 16:46:28 smithi092 podman[66036]: 2024-05-14 16:46:28.333606432 +0000 UTC m=+0.116365657 container create 9aa688b58ad46af9ba55a4c8ee64015abcb49777e1d6681b3c33e3a266fdbb5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-14T16:46:28.595 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 16:46:28 smithi092 podman[66036]: 2024-05-14 16:46:28.569813674 +0000 UTC m=+0.352572899 container init 9aa688b58ad46af9ba55a4c8ee64015abcb49777e1d6681b3c33e3a266fdbb5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-14T16:46:28.595 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 16:46:28 smithi092 podman[66036]: 2024-05-14 16:46:28.572362245 +0000 UTC m=+0.355121471 container start 9aa688b58ad46af9ba55a4c8ee64015abcb49777e1d6681b3c33e3a266fdbb5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:46:28.595 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 16:46:28 smithi092 ceph-mds[66055]: -- 172.21.15.92:0/4203885069 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556ff653fa40 con 0x556ff6513000 2024-05-14T16:46:28.919 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 16:46:28 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b[66051]: starting mds.b at 2024-05-14T16:46:28.920 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 16:46:28 smithi092 bash[66036]: 9aa688b58ad46af9ba55a4c8ee64015abcb49777e1d6681b3c33e3a266fdbb5c 2024-05-14T16:46:28.920 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 16:46:28 smithi092 systemd[1]: Started Ceph mds.b for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:46:29.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:28 smithi069 ceph-mon[31507]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:29.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:28 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:29.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:28 smithi203 ceph-mon[35787]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:29.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:28 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:29.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:28 smithi092 ceph-mon[35763]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:29.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:29.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:28 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:29.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:29 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] up:boot 2024-05-14T16:46:29.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:29 smithi203 ceph-mon[35787]: fsmap 12 up:standby 2024-05-14T16:46:29.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:29 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:46:29.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:29 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] up:boot 2024-05-14T16:46:29.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:29 smithi069 ceph-mon[31507]: fsmap 12 up:standby 2024-05-14T16:46:29.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:29 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:46:30.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:29 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] up:boot 2024-05-14T16:46:30.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:29 smithi092 ceph-mon[35763]: fsmap 12 up:standby 2024-05-14T16:46:30.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:29 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:46:30.630 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:46:30.881 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:31.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:31.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:31 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:31.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:31.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:31 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:31.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:31.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:31.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:31 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:32.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.552+0000 7f74b86f4700 1 -- 172.21.15.69:0/1587363692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b0102e30 msgr2=0x7f74b010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:32.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.552+0000 7f74b86f4700 1 --2- 172.21.15.69:0/1587363692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b0102e30 0x7f74b010f2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009990 tx=0x7f74a8009ca0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 -- 172.21.15.69:0/1587363692 shutdown_connections 2024-05-14T16:46:32.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 --2- 172.21.15.69:0/1587363692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b0102e30 0x7f74b010f2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 --2- 172.21.15.69:0/1587363692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01084b0 0x7f74b0108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 --2- 172.21.15.69:0/1587363692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b0102470 0x7f74b01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 -- 172.21.15.69:0/1587363692 >> 172.21.15.69:0/1587363692 conn(0x7f74b00793f0 msgr2=0x7f74b0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:32.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 -- 172.21.15.69:0/1587363692 shutdown_connections 2024-05-14T16:46:32.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 -- 172.21.15.69:0/1587363692 wait complete. 2024-05-14T16:46:32.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 Processor -- start 2024-05-14T16:46:32.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 -- start start 2024-05-14T16:46:32.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b0102470 0x7f74b019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:32.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b0102e30 0x7f74b019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:32.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b01084b0 0x7f74b01999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:32.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74b01125f0 con 0x7f74b01084b0 2024-05-14T16:46:32.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74b0112470 con 0x7f74b0102470 2024-05-14T16:46:32.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.553+0000 7f74b86f4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f74b0112770 con 0x7f74b0102e30 2024-05-14T16:46:32.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6c91700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b01084b0 0x7f74b01999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:32.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6c91700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b01084b0 0x7f74b01999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55380/0 (socket says 172.21.15.69:55380) 2024-05-14T16:46:32.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b5c8f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b0102e30 0x7f74b019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:32.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6c91700 1 -- 172.21.15.69:0/3436570096 learned_addr learned my addr 172.21.15.69:0/3436570096 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:32.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6490700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b0102470 0x7f74b019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:32.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6c91700 1 -- 172.21.15.69:0/3436570096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b0102e30 msgr2=0x7f74b019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:32.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6c91700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b0102e30 0x7f74b019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6c91700 1 -- 172.21.15.69:0/3436570096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b0102470 msgr2=0x7f74b019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:32.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6c91700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b0102470 0x7f74b019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6c91700 1 -- 172.21.15.69:0/3436570096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b019a150 con 0x7f74b01084b0 2024-05-14T16:46:32.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b5c8f700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b0102e30 0x7f74b019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:46:32.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6490700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b0102470 0x7f74b019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:46:32.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b6c91700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b01084b0 0x7f74b01999f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f74a800c010 tx=0x7f74a80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:32.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74af7fe700 1 -- 172.21.15.69:0/3436570096 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8027070 con 0x7f74b01084b0 2024-05-14T16:46:32.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74af7fe700 1 -- 172.21.15.69:0/3436570096 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a8022470 con 0x7f74b01084b0 2024-05-14T16:46:32.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b86f4700 1 -- 172.21.15.69:0/3436570096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b019a3e0 con 0x7f74b01084b0 2024-05-14T16:46:32.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74af7fe700 1 -- 172.21.15.69:0/3436570096 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8017650 con 0x7f74b01084b0 2024-05-14T16:46:32.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.554+0000 7f74b86f4700 1 -- 172.21.15.69:0/3436570096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b01a6700 con 0x7f74b01084b0 2024-05-14T16:46:32.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.555+0000 7f74b86f4700 1 -- 172.21.15.69:0/3436570096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b0103b60 con 0x7f74b01084b0 2024-05-14T16:46:32.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.557+0000 7f74af7fe700 1 -- 172.21.15.69:0/3436570096 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f74a802a070 con 0x7f74b01084b0 2024-05-14T16:46:32.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.558+0000 7f74af7fe700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74980b2900 0x7f74980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:32.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.558+0000 7f74b6490700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74980b2900 0x7f74980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:32.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.558+0000 7f74af7fe700 1 -- 172.21.15.69:0/3436570096 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f74a80e2f00 con 0x7f74b01084b0 2024-05-14T16:46:32.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.558+0000 7f74b6490700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74980b2900 0x7f74980b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f74b019ab30 tx=0x7f74a000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:32.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.561+0000 7f74af7fe700 1 -- 172.21.15.69:0/3436570096 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74a8013070 con 0x7f74b01084b0 2024-05-14T16:46:32.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.821+0000 7f74b86f4700 1 -- 172.21.15.69:0/3436570096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74b002e560 con 0x7f74b01084b0 2024-05-14T16:46:32.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.821+0000 7f74af7fe700 1 -- 172.21.15.69:0/3436570096 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7f74a80ab150 con 0x7f74b01084b0 2024-05-14T16:46:32.821 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:46:32.821 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24446,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-14T16:46:32.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 -- 172.21.15.69:0/3436570096 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74980b2900 msgr2=0x7f74980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:32.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74980b2900 0x7f74980b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f74b019ab30 tx=0x7f74a000b040 comp rx=0 tx=0).stop 2024-05-14T16:46:32.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 -- 172.21.15.69:0/3436570096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b01084b0 msgr2=0x7f74b01999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:32.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b01084b0 0x7f74b01999f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f74a800c010 tx=0x7f74a80047b0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 -- 172.21.15.69:0/3436570096 shutdown_connections 2024-05-14T16:46:32.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f74980b2900 0x7f74980b4dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b01084b0 0x7f74b01999f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b0102470 0x7f74b019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 --2- 172.21.15.69:0/3436570096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74b0102e30 0x7f74b019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:32.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 -- 172.21.15.69:0/3436570096 >> 172.21.15.69:0/3436570096 conn(0x7f74b00793f0 msgr2=0x7f74b006bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:32.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 -- 172.21.15.69:0/3436570096 shutdown_connections 2024-05-14T16:46:32.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:32.825+0000 7f74b86f4700 1 -- 172.21.15.69:0/3436570096 wait complete. 2024-05-14T16:46:32.826 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 13 2024-05-14T16:46:33.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:32 smithi069 ceph-mon[31507]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:33.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:32 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3436570096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:33.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:32 smithi203 ceph-mon[35787]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:33.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:32 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3436570096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:33.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:32 smithi092 ceph-mon[35763]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:33.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:33.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:32 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3436570096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:46:33.895 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-14T16:46:34.169 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:35.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:35 smithi092 ceph-mon[35763]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:35.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:35.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:35.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:35.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:35.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:35.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:35 smithi069 ceph-mon[31507]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:35.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:35.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:35.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:35.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:35.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:35.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:35 smithi203 ceph-mon[35787]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:35.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:35.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:35.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:35.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:35.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:46:35.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 -- 172.21.15.69:0/2803945841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ca4102470 msgr2=0x7f2ca41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:35.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 --2- 172.21.15.69:0/2803945841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ca4102470 0x7f2ca41028f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2c980099c0 tx=0x7f2c98009cd0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 -- 172.21.15.69:0/2803945841 shutdown_connections 2024-05-14T16:46:35.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 --2- 172.21.15.69:0/2803945841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca4102e30 0x7f2ca410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 --2- 172.21.15.69:0/2803945841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca4108550 0x7f2ca4108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 --2- 172.21.15.69:0/2803945841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ca4102470 0x7f2ca41028f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 -- 172.21.15.69:0/2803945841 >> 172.21.15.69:0/2803945841 conn(0x7f2ca4079480 msgr2=0x7f2ca4079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 -- 172.21.15.69:0/2803945841 shutdown_connections 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 -- 172.21.15.69:0/2803945841 wait complete. 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 Processor -- start 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.746+0000 7f2caa514700 1 -- start start 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2caa514700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca4102470 0x7f2ca419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2caa514700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ca4102e30 0x7f2ca419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2caa514700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca4108550 0x7f2ca4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2caa514700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41126a0 con 0x7f2ca4102e30 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2caa514700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca4112520 con 0x7f2ca4108550 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2caa514700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca4112820 con 0x7f2ca4102470 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ca4102e30 0x7f2ca419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ca4102e30 0x7f2ca419fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36204/0 (socket says 172.21.15.69:36204) 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca37fe700 1 -- 172.21.15.69:0/155899752 learned_addr learned my addr 172.21.15.69:0/155899752 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:35.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca3fff700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca4102470 0x7f2ca419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca8ab1700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca4108550 0x7f2ca4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca37fe700 1 -- 172.21.15.69:0/155899752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca4102470 msgr2=0x7f2ca419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca37fe700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca4102470 0x7f2ca419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca37fe700 1 -- 172.21.15.69:0/155899752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca4108550 msgr2=0x7f2ca4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca37fe700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca4108550 0x7f2ca4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca37fe700 1 -- 172.21.15.69:0/155899752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca419a1e0 con 0x7f2ca4102e30 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.747+0000 7f2ca37fe700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ca4102e30 0x7f2ca419fe00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800b580 tx=0x7f2c980037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.748+0000 7f2ca17fa700 1 -- 172.21.15.69:0/155899752 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98027070 con 0x7f2ca4102e30 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.748+0000 7f2caa514700 1 -- 172.21.15.69:0/155899752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca419a470 con 0x7f2ca4102e30 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.748+0000 7f2ca17fa700 1 -- 172.21.15.69:0/155899752 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c98004110 con 0x7f2ca4102e30 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.748+0000 7f2caa514700 1 -- 172.21.15.69:0/155899752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca41a6790 con 0x7f2ca4102e30 2024-05-14T16:46:35.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.748+0000 7f2ca17fa700 1 -- 172.21.15.69:0/155899752 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800f460 con 0x7f2ca4102e30 2024-05-14T16:46:35.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.749+0000 7f2caa514700 1 -- 172.21.15.69:0/155899752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca4103b90 con 0x7f2ca4102e30 2024-05-14T16:46:35.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.750+0000 7f2ca17fa700 1 -- 172.21.15.69:0/155899752 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2c9802a070 con 0x7f2ca4102e30 2024-05-14T16:46:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.751+0000 7f2ca17fa700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c8c0b2880 0x7f2c8c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.751+0000 7f2ca3fff700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c8c0b2880 0x7f2c8c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.751+0000 7f2ca17fa700 1 -- 172.21.15.69:0/155899752 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f2c980e3150 con 0x7f2ca4102e30 2024-05-14T16:46:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.751+0000 7f2ca3fff700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c8c0b2880 0x7f2c8c0b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c94006fd0 tx=0x7f2c94008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:35.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.754+0000 7f2ca17fa700 1 -- 172.21.15.69:0/155899752 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c98031070 con 0x7f2ca4102e30 2024-05-14T16:46:35.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.962+0000 7f2caa514700 1 -- 172.21.15.69:0/155899752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2ca402e560 con 0x7f2ca4102e30 2024-05-14T16:46:35.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.963+0000 7f2ca17fa700 1 -- 172.21.15.69:0/155899752 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f2c98013070 con 0x7f2ca4102e30 2024-05-14T16:46:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:46:35.963 INFO:teuthology.orchestra.run.smithi069.stdout:[ 2024-05-14T16:46:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T16:46:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: "poolnum": 1, 2024-05-14T16:46:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: "poolname": ".mgr" 2024-05-14T16:46:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:46:35.963 INFO:teuthology.orchestra.run.smithi069.stdout:] 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 -- 172.21.15.69:0/155899752 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c8c0b2880 msgr2=0x7f2c8c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c8c0b2880 0x7f2c8c0b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c94006fd0 tx=0x7f2c94008040 comp rx=0 tx=0).stop 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 -- 172.21.15.69:0/155899752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ca4102e30 msgr2=0x7f2ca419fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ca4102e30 0x7f2ca419fe00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800b580 tx=0x7f2c980037c0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 -- 172.21.15.69:0/155899752 shutdown_connections 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca4108550 0x7f2ca4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca4102470 0x7f2ca419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2c8c0b2880 0x7f2c8c0b4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 --2- 172.21.15.69:0/155899752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ca4102e30 0x7f2ca419fe00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 -- 172.21.15.69:0/155899752 >> 172.21.15.69:0/155899752 conn(0x7f2ca4079480 msgr2=0x7f2ca406bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 -- 172.21.15.69:0/155899752 shutdown_connections 2024-05-14T16:46:35.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:35.965+0000 7f2caa514700 1 -- 172.21.15.69:0/155899752 wait complete. 2024-05-14T16:46:36.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:36 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/155899752' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T16:46:36.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:36 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/155899752' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T16:46:36.702 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T16:46:36.702 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T16:46:36.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:36 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/155899752' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T16:46:36.944 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:37.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:37 smithi069 ceph-mon[31507]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:37.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:37 smithi092 ceph-mon[35763]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:37.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:37 smithi203 ceph-mon[35787]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:38.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.626+0000 7ff02833c700 1 -- 172.21.15.69:0/3957247054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff02006c150 msgr2=0x7ff02006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:38.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.626+0000 7ff02833c700 1 --2- 172.21.15.69:0/3957247054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff02006c150 0x7ff02006c5d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff010009990 tx=0x7ff010009ca0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.626+0000 7ff02833c700 1 -- 172.21.15.69:0/3957247054 shutdown_connections 2024-05-14T16:46:38.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.626+0000 7ff02833c700 1 --2- 172.21.15.69:0/3957247054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02010ff20 0x7ff020112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.626+0000 7ff02833c700 1 --2- 172.21.15.69:0/3957247054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff020104590 0x7ff02006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.626+0000 7ff02833c700 1 --2- 172.21.15.69:0/3957247054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff02006c150 0x7ff02006c5d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.626+0000 7ff02833c700 1 -- 172.21.15.69:0/3957247054 >> 172.21.15.69:0/3957247054 conn(0x7ff0200793f0 msgr2=0x7ff020079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:38.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.626+0000 7ff02833c700 1 -- 172.21.15.69:0/3957247054 shutdown_connections 2024-05-14T16:46:38.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.626+0000 7ff02833c700 1 -- 172.21.15.69:0/3957247054 wait complete. 2024-05-14T16:46:38.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.627+0000 7ff02833c700 1 Processor -- start 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.627+0000 7ff02833c700 1 -- start start 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.627+0000 7ff02833c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02006c150 0x7ff02019bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.627+0000 7ff02833c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff020104590 0x7ff02019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.627+0000 7ff02833c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff02010ff20 0x7ff0201a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.627+0000 7ff02833c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff0201159f0 con 0x7ff02010ff20 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.627+0000 7ff02833c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff020115870 con 0x7ff020104590 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.627+0000 7ff02833c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff020115b70 con 0x7ff02006c150 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.627+0000 7ff0268d9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff02010ff20 0x7ff0201a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0258d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff020104590 0x7ff02019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0268d9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff02010ff20 0x7ff0201a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36220/0 (socket says 172.21.15.69:36220) 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0268d9700 1 -- 172.21.15.69:0/4147558679 learned_addr learned my addr 172.21.15.69:0/4147558679 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0260d8700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02006c150 0x7ff02019bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0268d9700 1 -- 172.21.15.69:0/4147558679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02006c150 msgr2=0x7ff02019bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:38.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0268d9700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02006c150 0x7ff02019bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0268d9700 1 -- 172.21.15.69:0/4147558679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff020104590 msgr2=0x7ff02019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:38.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0268d9700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff020104590 0x7ff02019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0268d9700 1 -- 172.21.15.69:0/4147558679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0201a0f10 con 0x7ff02010ff20 2024-05-14T16:46:38.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0268d9700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff02010ff20 0x7ff0201a07d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff01800b9e0 tx=0x7ff01800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:38.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0177fe700 1 -- 172.21.15.69:0/4147558679 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01800d8e0 con 0x7ff02010ff20 2024-05-14T16:46:38.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0177fe700 1 -- 172.21.15.69:0/4147558679 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff018015420 con 0x7ff02010ff20 2024-05-14T16:46:38.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff02833c700 1 -- 172.21.15.69:0/4147558679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0201a1200 con 0x7ff02010ff20 2024-05-14T16:46:38.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff0177fe700 1 -- 172.21.15.69:0/4147558679 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018014630 con 0x7ff02010ff20 2024-05-14T16:46:38.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.628+0000 7ff02833c700 1 -- 172.21.15.69:0/4147558679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0201a8a80 con 0x7ff02010ff20 2024-05-14T16:46:38.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.630+0000 7ff02833c700 1 -- 172.21.15.69:0/4147558679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff020102a80 con 0x7ff02010ff20 2024-05-14T16:46:38.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.631+0000 7ff0177fe700 1 -- 172.21.15.69:0/4147558679 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff0180155d0 con 0x7ff02010ff20 2024-05-14T16:46:38.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.631+0000 7ff0177fe700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff00c0b2880 0x7ff00c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:38.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.631+0000 7ff0260d8700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff00c0b2880 0x7ff00c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:38.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.632+0000 7ff0177fe700 1 -- 172.21.15.69:0/4147558679 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff0180e16e0 con 0x7ff02010ff20 2024-05-14T16:46:38.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.632+0000 7ff0260d8700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff00c0b2880 0x7ff00c0b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009a10 tx=0x7ff01c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:38.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.636+0000 7ff0177fe700 1 -- 172.21.15.69:0/4147558679 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0180a98a0 con 0x7ff02010ff20 2024-05-14T16:46:38.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:38.838+0000 7ff02833c700 1 -- 172.21.15.69:0/4147558679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff0201a8c50 con 0x7ff02010ff20 2024-05-14T16:46:39.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.525+0000 7ff0177fe700 1 -- 172.21.15.69:0/4147558679 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7ff0180a91a0 con 0x7ff02010ff20 2024-05-14T16:46:39.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.528+0000 7ff02833c700 1 -- 172.21.15.69:0/4147558679 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff00c0b2880 msgr2=0x7ff00c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:39.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.528+0000 7ff02833c700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff00c0b2880 0x7ff00c0b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009a10 tx=0x7ff01c009190 comp rx=0 tx=0).stop 2024-05-14T16:46:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.528+0000 7ff02833c700 1 -- 172.21.15.69:0/4147558679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff02010ff20 msgr2=0x7ff0201a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.528+0000 7ff02833c700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff02010ff20 0x7ff0201a07d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff01800b9e0 tx=0x7ff01800bcf0 comp rx=0 tx=0).stop 2024-05-14T16:46:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.529+0000 7ff02833c700 1 -- 172.21.15.69:0/4147558679 shutdown_connections 2024-05-14T16:46:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.529+0000 7ff02833c700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff00c0b2880 0x7ff00c0b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.529+0000 7ff02833c700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff02010ff20 0x7ff0201a07d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.529+0000 7ff02833c700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02006c150 0x7ff02019bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.529+0000 7ff02833c700 1 --2- 172.21.15.69:0/4147558679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff020104590 0x7ff02019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.529+0000 7ff02833c700 1 -- 172.21.15.69:0/4147558679 >> 172.21.15.69:0/4147558679 conn(0x7ff0200793f0 msgr2=0x7ff020101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:39.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.529+0000 7ff02833c700 1 -- 172.21.15.69:0/4147558679 shutdown_connections 2024-05-14T16:46:39.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:39.529+0000 7ff02833c700 1 -- 172.21.15.69:0/4147558679 wait complete. 2024-05-14T16:46:39.531 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' created 2024-05-14T16:46:39.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:39 smithi069 ceph-mon[31507]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:39.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:39 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/4147558679' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T16:46:39.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:39 smithi203 ceph-mon[35787]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:39.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:39 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/4147558679' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T16:46:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:39 smithi092 ceph-mon[35763]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:39 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/4147558679' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T16:46:40.251 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T16:46:40.497 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:40 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/4147558679' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T16:46:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:40 smithi069 ceph-mon[31507]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T16:46:40.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:40 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/4147558679' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T16:46:40.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:40 smithi203 ceph-mon[35787]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T16:46:41.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:40 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/4147558679' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T16:46:41.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:40 smithi092 ceph-mon[35763]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T16:46:41.649 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:41 smithi069 ceph-mon[31507]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:41.649 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:41 smithi069 ceph-mon[31507]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T16:46:41.649 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:41 smithi069 ceph-mon[31507]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T16:46:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:41 smithi203 ceph-mon[35787]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:41 smithi203 ceph-mon[35787]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T16:46:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:41 smithi203 ceph-mon[35787]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T16:46:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:41 smithi092 ceph-mon[35763]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:41 smithi092 ceph-mon[35763]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T16:46:42.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:41 smithi092 ceph-mon[35763]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T16:46:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.170+0000 7fe74486d700 1 -- 172.21.15.69:0/145477090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe740102470 msgr2=0x7fe7401028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.170+0000 7fe74486d700 1 --2- 172.21.15.69:0/145477090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe740102470 0x7fe7401028d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe730009990 tx=0x7fe730009ca0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.170+0000 7fe74486d700 1 -- 172.21.15.69:0/145477090 shutdown_connections 2024-05-14T16:46:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.170+0000 7fe74486d700 1 --2- 172.21.15.69:0/145477090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe740102e10 0x7fe74010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.170+0000 7fe74486d700 1 --2- 172.21.15.69:0/145477090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7401084b0 0x7fe740108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.170+0000 7fe74486d700 1 --2- 172.21.15.69:0/145477090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe740102470 0x7fe7401028d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.170+0000 7fe74486d700 1 -- 172.21.15.69:0/145477090 >> 172.21.15.69:0/145477090 conn(0x7fe7400793e0 msgr2=0x7fe7400797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.170+0000 7fe74486d700 1 -- 172.21.15.69:0/145477090 shutdown_connections 2024-05-14T16:46:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.170+0000 7fe74486d700 1 -- 172.21.15.69:0/145477090 wait complete. 2024-05-14T16:46:42.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe74486d700 1 Processor -- start 2024-05-14T16:46:42.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe74486d700 1 -- start start 2024-05-14T16:46:42.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe74486d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe740102470 0x7fe74019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:42.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe74486d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe740102e10 0x7fe74019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe74486d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7401084b0 0x7fe740198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe74486d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe740112660 con 0x7fe740102470 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe74486d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401124e0 con 0x7fe7401084b0 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe74486d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401127e0 con 0x7fe740102e10 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe740102470 0x7fe74019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe740102470 0x7fe74019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36228/0 (socket says 172.21.15.69:36228) 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73e59c700 1 -- 172.21.15.69:0/1467631633 learned_addr learned my addr 172.21.15.69:0/1467631633 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73ed9d700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7401084b0 0x7fe740198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73dd9b700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe740102e10 0x7fe74019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73ed9d700 1 -- 172.21.15.69:0/1467631633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe740102e10 msgr2=0x7fe74019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73ed9d700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe740102e10 0x7fe74019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73ed9d700 1 -- 172.21.15.69:0/1467631633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe740102470 msgr2=0x7fe74019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73ed9d700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe740102470 0x7fe74019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73ed9d700 1 -- 172.21.15.69:0/1467631633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7401990e0 con 0x7fe7401084b0 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.171+0000 7fe73e59c700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe740102470 0x7fe74019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:46:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.172+0000 7fe73ed9d700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7401084b0 0x7fe740198980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe73800be20 tx=0x7fe738009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.172+0000 7fe7377fe700 1 -- 172.21.15.69:0/1467631633 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738019070 con 0x7fe7401084b0 2024-05-14T16:46:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.172+0000 7fe74486d700 1 -- 172.21.15.69:0/1467631633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7401993d0 con 0x7fe7401084b0 2024-05-14T16:46:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.172+0000 7fe7377fe700 1 -- 172.21.15.69:0/1467631633 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe73800cb90 con 0x7fe7401084b0 2024-05-14T16:46:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.172+0000 7fe7377fe700 1 -- 172.21.15.69:0/1467631633 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7380125a0 con 0x7fe7401084b0 2024-05-14T16:46:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.172+0000 7fe74486d700 1 -- 172.21.15.69:0/1467631633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74006bcd0 con 0x7fe7401084b0 2024-05-14T16:46:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.173+0000 7fe74486d700 1 -- 172.21.15.69:0/1467631633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe74006a230 con 0x7fe7401084b0 2024-05-14T16:46:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.174+0000 7fe7377fe700 1 -- 172.21.15.69:0/1467631633 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe73801c070 con 0x7fe7401084b0 2024-05-14T16:46:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.175+0000 7fe7377fe700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe72c0b2880 0x7fe72c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.175+0000 7fe73e59c700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe72c0b2880 0x7fe72c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.175+0000 7fe7377fe700 1 -- 172.21.15.69:0/1467631633 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7fe7380e1d80 con 0x7fe7401084b0 2024-05-14T16:46:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.175+0000 7fe73e59c700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe72c0b2880 0x7fe72c0b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe740199ac0 tx=0x7fe728010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:42.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.178+0000 7fe7377fe700 1 -- 172.21.15.69:0/1467631633 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe738023070 con 0x7fe7401084b0 2024-05-14T16:46:42.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.390+0000 7fe74486d700 1 -- 172.21.15.69:0/1467631633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe74006c760 con 0x7fe7401084b0 2024-05-14T16:46:42.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.549+0000 7fe7377fe700 1 -- 172.21.15.69:0/1467631633 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7fe7380a9d30 con 0x7fe7401084b0 2024-05-14T16:46:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.553+0000 7fe74486d700 1 -- 172.21.15.69:0/1467631633 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe72c0b2880 msgr2=0x7fe72c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.553+0000 7fe74486d700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe72c0b2880 0x7fe72c0b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe740199ac0 tx=0x7fe728010040 comp rx=0 tx=0).stop 2024-05-14T16:46:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.553+0000 7fe74486d700 1 -- 172.21.15.69:0/1467631633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7401084b0 msgr2=0x7fe740198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.553+0000 7fe74486d700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7401084b0 0x7fe740198980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe73800be20 tx=0x7fe738009ed0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.554+0000 7fe74486d700 1 -- 172.21.15.69:0/1467631633 shutdown_connections 2024-05-14T16:46:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.554+0000 7fe74486d700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe72c0b2880 0x7fe72c0b4d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.555+0000 7fe74486d700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe740102470 0x7fe74019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.555+0000 7fe74486d700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7401084b0 0x7fe740198980 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.555+0000 7fe74486d700 1 --2- 172.21.15.69:0/1467631633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe740102e10 0x7fe74019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.555+0000 7fe74486d700 1 -- 172.21.15.69:0/1467631633 >> 172.21.15.69:0/1467631633 conn(0x7fe7400793e0 msgr2=0x7fe7401005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:42.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.557+0000 7fe74486d700 1 -- 172.21.15.69:0/1467631633 shutdown_connections 2024-05-14T16:46:42.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:42.557+0000 7fe74486d700 1 -- 172.21.15.69:0/1467631633 wait complete. 2024-05-14T16:46:42.558 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' created 2024-05-14T16:46:42.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:42 smithi092 ceph-mon[35763]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T16:46:42.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:42 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1467631633' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T16:46:42.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:42 smithi092 ceph-mon[35763]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T16:46:42.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:42 smithi069 ceph-mon[31507]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T16:46:42.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:42 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1467631633' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T16:46:42.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:42 smithi069 ceph-mon[31507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T16:46:42.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:42 smithi203 ceph-mon[35787]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T16:46:42.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:42 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1467631633' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T16:46:42.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:42 smithi203 ceph-mon[35787]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T16:46:43.186 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T16:46:43.441 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:43.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:43 smithi069 ceph-mon[31507]: pgmap v279: 33 pgs: 28 unknown, 5 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:43.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:43 smithi069 ceph-mon[31507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T16:46:43.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:43 smithi069 ceph-mon[31507]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T16:46:43.861 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:43 smithi203 ceph-mon[35787]: pgmap v279: 33 pgs: 28 unknown, 5 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:43.861 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:43 smithi203 ceph-mon[35787]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T16:46:43.861 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:43 smithi203 ceph-mon[35787]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T16:46:44.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:43 smithi092 ceph-mon[35763]: pgmap v279: 33 pgs: 28 unknown, 5 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:44.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:43 smithi092 ceph-mon[35763]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T16:46:44.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:43 smithi092 ceph-mon[35763]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T16:46:44.848 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:44 smithi069 ceph-mon[31507]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T16:46:44.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:44 smithi203 ceph-mon[35787]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T16:46:45.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:44 smithi092 ceph-mon[35763]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T16:46:45.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.075+0000 7fe147ef1700 1 -- 172.21.15.69:0/1168082586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe140102df0 msgr2=0x7fe14010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:45.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.075+0000 7fe147ef1700 1 --2- 172.21.15.69:0/1168082586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe140102df0 0x7fe14010f2e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b0a0 tx=0x7fe13800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:46:45.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.075+0000 7fe147ef1700 1 -- 172.21.15.69:0/1168082586 shutdown_connections 2024-05-14T16:46:45.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.075+0000 7fe147ef1700 1 --2- 172.21.15.69:0/1168082586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe140102df0 0x7fe14010f2e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:45.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.075+0000 7fe147ef1700 1 --2- 172.21.15.69:0/1168082586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe140108510 0x7fe1401088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:45.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.075+0000 7fe147ef1700 1 --2- 172.21.15.69:0/1168082586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe140102430 0x7fe1401028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.075+0000 7fe147ef1700 1 -- 172.21.15.69:0/1168082586 >> 172.21.15.69:0/1168082586 conn(0x7fe1400793b0 msgr2=0x7fe1400797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.075+0000 7fe147ef1700 1 -- 172.21.15.69:0/1168082586 shutdown_connections 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.075+0000 7fe147ef1700 1 -- 172.21.15.69:0/1168082586 wait complete. 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe147ef1700 1 Processor -- start 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe147ef1700 1 -- start start 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe147ef1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe140102430 0x7fe14019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe147ef1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe140102df0 0x7fe14019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe147ef1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe140108510 0x7fe1401a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe147ef1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401126c0 con 0x7fe140108510 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe147ef1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe140112540 con 0x7fe140102430 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe147ef1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe140112840 con 0x7fe140102df0 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe14648e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe140108510 0x7fe1401a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe14648e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe140108510 0x7fe1401a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57200/0 (socket says 172.21.15.69:57200) 2024-05-14T16:46:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe14648e700 1 -- 172.21.15.69:0/1980126860 learned_addr learned my addr 172.21.15.69:0/1980126860 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe14548c700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe140102df0 0x7fe14019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.076+0000 7fe145c8d700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe140102430 0x7fe14019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe14648e700 1 -- 172.21.15.69:0/1980126860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe140102df0 msgr2=0x7fe14019c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe14648e700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe140102df0 0x7fe14019c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe14648e700 1 -- 172.21.15.69:0/1980126860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe140102430 msgr2=0x7fe14019bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe14648e700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe140102430 0x7fe14019bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe14648e700 1 -- 172.21.15.69:0/1980126860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1401a0f60 con 0x7fe140108510 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe14548c700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe140102df0 0x7fe14019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe14648e700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe140108510 0x7fe1401a0820 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe1380062a0 tx=0x7fe138004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe136ffd700 1 -- 172.21.15.69:0/1980126860 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138021070 con 0x7fe140108510 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe147ef1700 1 -- 172.21.15.69:0/1980126860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1401a11f0 con 0x7fe140108510 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe136ffd700 1 -- 172.21.15.69:0/1980126860 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe138007c80 con 0x7fe140108510 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe136ffd700 1 -- 172.21.15.69:0/1980126860 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13801d450 con 0x7fe140108510 2024-05-14T16:46:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.077+0000 7fe147ef1700 1 -- 172.21.15.69:0/1980126860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1401a8ad0 con 0x7fe140108510 2024-05-14T16:46:45.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.078+0000 7fe147ef1700 1 -- 172.21.15.69:0/1980126860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe14006a230 con 0x7fe140108510 2024-05-14T16:46:45.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.080+0000 7fe136ffd700 1 -- 172.21.15.69:0/1980126860 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe138004000 con 0x7fe140108510 2024-05-14T16:46:45.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.081+0000 7fe136ffd700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe12c0b2880 0x7fe12c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:45.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.081+0000 7fe145c8d700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe12c0b2880 0x7fe12c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:45.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.081+0000 7fe136ffd700 1 -- 172.21.15.69:0/1980126860 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7fe1380e3270 con 0x7fe140108510 2024-05-14T16:46:45.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.081+0000 7fe145c8d700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe12c0b2880 0x7fe12c0b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe130009a10 tx=0x7fe130009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:45.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.085+0000 7fe136ffd700 1 -- 172.21.15.69:0/1980126860 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe1380ab1d0 con 0x7fe140108510 2024-05-14T16:46:45.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.331+0000 7fe147ef1700 1 -- 172.21.15.69:0/1980126860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe1401a8ca0 con 0x7fe140108510 2024-05-14T16:46:45.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.560+0000 7fe136ffd700 1 -- 172.21.15.69:0/1980126860 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fe1380aaad0 con 0x7fe140108510 2024-05-14T16:46:45.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 -- 172.21.15.69:0/1980126860 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe12c0b2880 msgr2=0x7fe12c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:45.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe12c0b2880 0x7fe12c0b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe130009a10 tx=0x7fe130009190 comp rx=0 tx=0).stop 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 -- 172.21.15.69:0/1980126860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe140108510 msgr2=0x7fe1401a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe140108510 0x7fe1401a0820 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe1380062a0 tx=0x7fe138004e20 comp rx=0 tx=0).stop 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 -- 172.21.15.69:0/1980126860 shutdown_connections 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe12c0b2880 0x7fe12c0b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe140108510 0x7fe1401a0820 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe140102430 0x7fe14019bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 --2- 172.21.15.69:0/1980126860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe140102df0 0x7fe14019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 -- 172.21.15.69:0/1980126860 >> 172.21.15.69:0/1980126860 conn(0x7fe1400793b0 msgr2=0x7fe14006bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 -- 172.21.15.69:0/1980126860 shutdown_connections 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:45.564+0000 7fe147ef1700 1 -- 172.21.15.69:0/1980126860 wait complete. 2024-05-14T16:46:45.565 INFO:teuthology.orchestra.run.smithi069.stderr:new fs with metadata pool 2 and data pool 3 2024-05-14T16:46:45.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:45 smithi069 ceph-mon[31507]: pgmap v282: 97 pgs: 2 creating+activating, 19 creating+peering, 39 active+clean, 37 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:45.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:45 smithi069 ceph-mon[31507]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T16:46:45.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:45 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1980126860' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T16:46:45.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:45 smithi069 ceph-mon[31507]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T16:46:45.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:45 smithi203 ceph-mon[35787]: pgmap v282: 97 pgs: 2 creating+activating, 19 creating+peering, 39 active+clean, 37 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:45.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:45 smithi203 ceph-mon[35787]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T16:46:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:45 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1980126860' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T16:46:45.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:45 smithi203 ceph-mon[35787]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T16:46:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:45 smithi092 ceph-mon[35763]: pgmap v282: 97 pgs: 2 creating+activating, 19 creating+peering, 39 active+clean, 37 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:45 smithi092 ceph-mon[35763]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T16:46:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:45 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1980126860' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T16:46:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:45 smithi092 ceph-mon[35763]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T16:46:46.245 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:46:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:46.853 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:46 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1980126860' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T16:46:46.854 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:46 smithi069 ceph-mon[31507]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T16:46:46.854 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:46 smithi069 ceph-mon[31507]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T16:46:46.854 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:46 smithi069 ceph-mon[31507]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T16:46:46.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:46 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1980126860' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T16:46:46.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:46 smithi203 ceph-mon[35787]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T16:46:46.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:46 smithi203 ceph-mon[35787]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T16:46:46.961 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:46 smithi203 ceph-mon[35787]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T16:46:47.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:46 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1980126860' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T16:46:47.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:46 smithi092 ceph-mon[35763]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T16:46:47.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:46 smithi092 ceph-mon[35763]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T16:46:47.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:46 smithi092 ceph-mon[35763]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T16:46:47.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:47 smithi069 ceph-mon[31507]: pgmap v285: 97 pgs: 2 creating+activating, 19 creating+peering, 56 active+clean, 20 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:47.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:47 smithi069 ceph-mon[31507]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T16:46:47.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:47 smithi069 ceph-mon[31507]: Cluster is now healthy 2024-05-14T16:46:47.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:47 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] up:active 2024-05-14T16:46:47.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:47 smithi069 ceph-mon[31507]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:46:47.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:47 smithi069 ceph-mon[31507]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:46:47.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:47 smithi203 ceph-mon[35787]: pgmap v285: 97 pgs: 2 creating+activating, 19 creating+peering, 56 active+clean, 20 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:47.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:47 smithi203 ceph-mon[35787]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T16:46:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:47 smithi203 ceph-mon[35787]: Cluster is now healthy 2024-05-14T16:46:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:47 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] up:active 2024-05-14T16:46:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:47 smithi203 ceph-mon[35787]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:46:47.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:47 smithi203 ceph-mon[35787]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:46:48.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:47 smithi092 ceph-mon[35763]: pgmap v285: 97 pgs: 2 creating+activating, 19 creating+peering, 56 active+clean, 20 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:46:48.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:47 smithi092 ceph-mon[35763]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T16:46:48.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:47 smithi092 ceph-mon[35763]: Cluster is now healthy 2024-05-14T16:46:48.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:47 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] up:active 2024-05-14T16:46:48.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:47 smithi092 ceph-mon[35763]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:46:48.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:47 smithi092 ceph-mon[35763]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:46:48.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.072+0000 7f8728898700 1 -- 172.21.15.69:0/3333394952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87241088f0 msgr2=0x7f8724110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:48.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.072+0000 7f8728898700 1 --2- 172.21.15.69:0/3333394952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87241088f0 0x7f8724110470 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f871c00b0d0 tx=0x7f871c00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:46:48.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.072+0000 7f8728898700 1 -- 172.21.15.69:0/3333394952 shutdown_connections 2024-05-14T16:46:48.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.072+0000 7f8728898700 1 --2- 172.21.15.69:0/3333394952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87241088f0 0x7f8724110470 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:48.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.072+0000 7f8728898700 1 --2- 172.21.15.69:0/3333394952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87241037f0 0x7f8724103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:48.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.072+0000 7f8728898700 1 --2- 172.21.15.69:0/3333394952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87241041a0 0x7f87241081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:48.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.072+0000 7f8728898700 1 -- 172.21.15.69:0/3333394952 >> 172.21.15.69:0/3333394952 conn(0x7f872407a6a0 msgr2=0x7f872407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:48.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.072+0000 7f8728898700 1 -- 172.21.15.69:0/3333394952 shutdown_connections 2024-05-14T16:46:48.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.072+0000 7f8728898700 1 -- 172.21.15.69:0/3333394952 wait complete. 2024-05-14T16:46:48.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8728898700 1 Processor -- start 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8728898700 1 -- start start 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8728898700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87241037f0 0x7f8724075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8728898700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87241041a0 0x7f8724075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8728898700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87241088f0 0x7f87240762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8728898700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87241136e0 con 0x7f87241041a0 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8728898700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8724113560 con 0x7f87241037f0 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8728898700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8724113860 con 0x7f87241088f0 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8721d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87241041a0 0x7f8724075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8721d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87241041a0 0x7f8724075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57208/0 (socket says 172.21.15.69:57208) 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f8721d9b700 1 -- 172.21.15.69:0/853836872 learned_addr learned my addr 172.21.15.69:0/853836872 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.073+0000 7f872259c700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87241037f0 0x7f8724075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f8722d9d700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87241088f0 0x7f87240762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:48.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f8721d9b700 1 -- 172.21.15.69:0/853836872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87241088f0 msgr2=0x7f87240762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f8721d9b700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87241088f0 0x7f87240762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f8721d9b700 1 -- 172.21.15.69:0/853836872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87241037f0 msgr2=0x7f8724075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f8721d9b700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87241037f0 0x7f8724075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f8721d9b700 1 -- 172.21.15.69:0/853836872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87241a8080 con 0x7f87241041a0 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f8722d9d700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87241088f0 0x7f87240762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f8721d9b700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87241041a0 0x7f8724075d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f871800d950 tx=0x7f871800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f87137fe700 1 -- 172.21.15.69:0/853836872 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718016070 con 0x7f87241041a0 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f87137fe700 1 -- 172.21.15.69:0/853836872 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8718009df0 con 0x7f87241041a0 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f8728898700 1 -- 172.21.15.69:0/853836872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87241a8370 con 0x7f87241041a0 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f87137fe700 1 -- 172.21.15.69:0/853836872 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871800f3c0 con 0x7f87241041a0 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.074+0000 7f8728898700 1 -- 172.21.15.69:0/853836872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87241a8900 con 0x7f87241041a0 2024-05-14T16:46:48.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.075+0000 7f8728898700 1 -- 172.21.15.69:0/853836872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872410bac0 con 0x7f87241041a0 2024-05-14T16:46:48.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.076+0000 7f87137fe700 1 -- 172.21.15.69:0/853836872 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8718019070 con 0x7f87241041a0 2024-05-14T16:46:48.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.077+0000 7f87137fe700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f870c0b2900 0x7f870c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:48.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.077+0000 7f872259c700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f870c0b2900 0x7f870c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:48.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.077+0000 7f87137fe700 1 -- 172.21.15.69:0/853836872 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f87180e1bc0 con 0x7f87241041a0 2024-05-14T16:46:48.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.078+0000 7f872259c700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f870c0b2900 0x7f870c0b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8714009c60 tx=0x7f8714009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:48.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.080+0000 7f87137fe700 1 -- 172.21.15.69:0/853836872 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87180a9a30 con 0x7f87241041a0 2024-05-14T16:46:48.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.289+0000 7f8728898700 1 -- 172.21.15.69:0/853836872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f872410b4a0 con 0x7f87241041a0 2024-05-14T16:46:48.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.290+0000 7f87137fe700 1 -- 172.21.15.69:0/853836872 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22183 (secure 0 0 0) 0x7f87180a9330 con 0x7f87241041a0 2024-05-14T16:46:48.291 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:46:48.291 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":78,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T16:46:45.333145+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:46:38.843325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:46:42.395401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:46:48.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 -- 172.21.15.69:0/853836872 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f870c0b2900 msgr2=0x7f870c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f870c0b2900 0x7f870c0b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8714009c60 tx=0x7f8714009260 comp rx=0 tx=0).stop 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 -- 172.21.15.69:0/853836872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87241041a0 msgr2=0x7f8724075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87241041a0 0x7f8724075d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f871800d950 tx=0x7f871800dc60 comp rx=0 tx=0).stop 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 -- 172.21.15.69:0/853836872 shutdown_connections 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87241088f0 0x7f87240762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87241037f0 0x7f8724075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f870c0b2900 0x7f870c0b4dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 --2- 172.21.15.69:0/853836872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87241041a0 0x7f8724075d80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 -- 172.21.15.69:0/853836872 >> 172.21.15.69:0/853836872 conn(0x7f872407a6a0 msgr2=0x7f87241017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 -- 172.21.15.69:0/853836872 shutdown_connections 2024-05-14T16:46:48.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:48.293+0000 7f8728898700 1 -- 172.21.15.69:0/853836872 wait complete. 2024-05-14T16:46:48.932 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:48 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/853836872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:46:48.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:48 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/853836872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:46:48.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:48 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/853836872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:46:48.970 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:46:49.208 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:49.250 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 16:46:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b[66051]: 2024-05-14T16:46:48.932+0000 7f558e94d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T16:46:49.582 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:49 smithi069 ceph-mon[31507]: pgmap v286: 97 pgs: 2 creating+activating, 6 creating+peering, 89 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 385 B/s wr, 1 op/s 2024-05-14T16:46:49.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:49 smithi203 ceph-mon[35787]: pgmap v286: 97 pgs: 2 creating+activating, 6 creating+peering, 89 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 385 B/s wr, 1 op/s 2024-05-14T16:46:50.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:49 smithi092 ceph-mon[35763]: pgmap v286: 97 pgs: 2 creating+activating, 6 creating+peering, 89 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 385 B/s wr, 1 op/s 2024-05-14T16:46:50.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.823+0000 7f7c6e056700 1 -- 172.21.15.69:0/1947425351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6806b790 msgr2=0x7f7c6806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.824+0000 7f7c6e056700 1 --2- 172.21.15.69:0/1947425351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6806b790 0x7f7c6806bbf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009990 tx=0x7f7c5c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.824+0000 7f7c6e056700 1 -- 172.21.15.69:0/1947425351 shutdown_connections 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.824+0000 7f7c6e056700 1 --2- 172.21.15.69:0/1947425351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68105f80 0x7f7c68106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.824+0000 7f7c6e056700 1 --2- 172.21.15.69:0/1947425351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c68108470 0x7f7c6806b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.824+0000 7f7c6e056700 1 --2- 172.21.15.69:0/1947425351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6806b790 0x7f7c6806bbf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.824+0000 7f7c6e056700 1 -- 172.21.15.69:0/1947425351 >> 172.21.15.69:0/1947425351 conn(0x7f7c680793f0 msgr2=0x7f7c68079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.824+0000 7f7c6e056700 1 -- 172.21.15.69:0/1947425351 shutdown_connections 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.824+0000 7f7c6e056700 1 -- 172.21.15.69:0/1947425351 wait complete. 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.824+0000 7f7c6e056700 1 Processor -- start 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.824+0000 7f7c6e056700 1 -- start start 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c6e056700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6806b790 0x7f7c681a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c6e056700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68105f80 0x7f7c681a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c6e056700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c68108470 0x7f7c681a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c6e056700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68119d30 con 0x7f7c6806b790 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c6e056700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68119bb0 con 0x7f7c68108470 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c6e056700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68119eb0 con 0x7f7c68105f80 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6806b790 0x7f7c681a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6806b790 0x7f7c681a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57226/0 (socket says 172.21.15.69:57226) 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c677fe700 1 -- 172.21.15.69:0/460970613 learned_addr learned my addr 172.21.15.69:0/460970613 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c66ffd700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68105f80 0x7f7c681a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c677fe700 1 -- 172.21.15.69:0/460970613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68105f80 msgr2=0x7f7c681a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c677fe700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68105f80 0x7f7c681a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c677fe700 1 -- 172.21.15.69:0/460970613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c68108470 msgr2=0x7f7c681a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c677fe700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c68108470 0x7f7c681a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c677fe700 1 -- 172.21.15.69:0/460970613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c681a5230 con 0x7f7c6806b790 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c66ffd700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68105f80 0x7f7c681a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c677fe700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6806b790 0x7f7c681a02c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800b530 tx=0x7f7c5800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c64ff9700 1 -- 172.21.15.69:0/460970613 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58015020 con 0x7f7c6806b790 2024-05-14T16:46:50.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c64ff9700 1 -- 172.21.15.69:0/460970613 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c58011ce0 con 0x7f7c6806b790 2024-05-14T16:46:50.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c6e056700 1 -- 172.21.15.69:0/460970613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c681a5520 con 0x7f7c6806b790 2024-05-14T16:46:50.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c64ff9700 1 -- 172.21.15.69:0/460970613 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58010d20 con 0x7f7c6806b790 2024-05-14T16:46:50.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.825+0000 7f7c6e056700 1 -- 172.21.15.69:0/460970613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c681acda0 con 0x7f7c6806b790 2024-05-14T16:46:50.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.826+0000 7f7c6e056700 1 -- 172.21.15.69:0/460970613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c6806a230 con 0x7f7c6806b790 2024-05-14T16:46:50.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.828+0000 7f7c64ff9700 1 -- 172.21.15.69:0/460970613 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7c58018020 con 0x7f7c6806b790 2024-05-14T16:46:50.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.828+0000 7f7c64ff9700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7c500b2880 0x7f7c500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:50.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.829+0000 7f7c66ffd700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7c500b2880 0x7f7c500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:50.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.829+0000 7f7c64ff9700 1 -- 172.21.15.69:0/460970613 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f7c580e1e70 con 0x7f7c6806b790 2024-05-14T16:46:50.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.829+0000 7f7c66ffd700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7c500b2880 0x7f7c500b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000c00 tx=0x7f7c5c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:50.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:50.832+0000 7f7c64ff9700 1 -- 172.21.15.69:0/460970613 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c5801f020 con 0x7f7c6806b790 2024-05-14T16:46:51.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.037+0000 7f7c6e056700 1 -- 172.21.15.69:0/460970613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c681a1620 con 0x7f7c6806b790 2024-05-14T16:46:51.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.038+0000 7f7c64ff9700 1 -- 172.21.15.69:0/460970613 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22183 (secure 0 0 0) 0x7f7c681a1620 con 0x7f7c6806b790 2024-05-14T16:46:51.038 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:46:51.038 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":78,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T16:46:45.333145+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:46:38.843325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:46:42.395401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:46:51.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.040+0000 7f7c6e056700 1 -- 172.21.15.69:0/460970613 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7c500b2880 msgr2=0x7f7c500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:51.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.040+0000 7f7c6e056700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7c500b2880 0x7f7c500b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000c00 tx=0x7f7c5c005fb0 comp rx=0 tx=0).stop 2024-05-14T16:46:51.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.040+0000 7f7c6e056700 1 -- 172.21.15.69:0/460970613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6806b790 msgr2=0x7f7c681a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:51.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.040+0000 7f7c6e056700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6806b790 0x7f7c681a02c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800b530 tx=0x7f7c5800b8f0 comp rx=0 tx=0).stop 2024-05-14T16:46:51.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.040+0000 7f7c6e056700 1 -- 172.21.15.69:0/460970613 shutdown_connections 2024-05-14T16:46:51.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.040+0000 7f7c6e056700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f7c500b2880 0x7f7c500b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:51.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.040+0000 7f7c6e056700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6806b790 0x7f7c681a02c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:51.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.040+0000 7f7c6e056700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c68108470 0x7f7c681a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:51.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.040+0000 7f7c6e056700 1 --2- 172.21.15.69:0/460970613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68105f80 0x7f7c681a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:51.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.040+0000 7f7c6e056700 1 -- 172.21.15.69:0/460970613 >> 172.21.15.69:0/460970613 conn(0x7f7c680793f0 msgr2=0x7f7c68104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:51.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.041+0000 7f7c6e056700 1 -- 172.21.15.69:0/460970613 shutdown_connections 2024-05-14T16:46:51.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:51.041+0000 7f7c6e056700 1 -- 172.21.15.69:0/460970613 wait complete. 2024-05-14T16:46:51.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:51 smithi069 ceph-mon[31507]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-14T16:46:51.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:51 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/460970613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:46:51.854 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-14T16:46:51.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:51 smithi203 ceph-mon[35787]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-14T16:46:51.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:51 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/460970613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:46:52.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:51 smithi092 ceph-mon[35763]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-14T16:46:52.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:51 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/460970613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:46:52.095 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:53.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 -- 172.21.15.69:0/52395046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c06bab0 msgr2=0x7f5c2c06bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:53.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 --2- 172.21.15.69:0/52395046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c06bab0 0x7f5c2c06bf30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009990 tx=0x7f5c1c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:46:53.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 -- 172.21.15.69:0/52395046 shutdown_connections 2024-05-14T16:46:53.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 --2- 172.21.15.69:0/52395046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c2c06c470 0x7f5c2c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:53.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 --2- 172.21.15.69:0/52395046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2c1095a0 0x7f5c2c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:53.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 --2- 172.21.15.69:0/52395046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c06bab0 0x7f5c2c06bf30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:53.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 -- 172.21.15.69:0/52395046 >> 172.21.15.69:0/52395046 conn(0x7f5c2c0793b0 msgr2=0x7f5c2c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:53.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 -- 172.21.15.69:0/52395046 shutdown_connections 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 -- 172.21.15.69:0/52395046 wait complete. 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 Processor -- start 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.658+0000 7f5c33fba700 1 -- start start 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c33fba700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2c06bab0 0x7f5c2c102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c33fba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c2c06c470 0x7f5c2c102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c33fba700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1095a0 0x7f5c2c102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c33fba700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c113730 con 0x7f5c2c1095a0 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c33fba700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1135b0 con 0x7f5c2c06c470 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c33fba700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1138b0 con 0x7f5c2c06bab0 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c32557700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1095a0 0x7f5c2c102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c32557700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1095a0 0x7f5c2c102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57246/0 (socket says 172.21.15.69:57246) 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c32557700 1 -- 172.21.15.69:0/1747634407 learned_addr learned my addr 172.21.15.69:0/1747634407 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c31d56700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2c06bab0 0x7f5c2c102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:53.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c31555700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c2c06c470 0x7f5c2c102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c32557700 1 -- 172.21.15.69:0/1747634407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2c06bab0 msgr2=0x7f5c2c102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c32557700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2c06bab0 0x7f5c2c102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c32557700 1 -- 172.21.15.69:0/1747634407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c2c06c470 msgr2=0x7f5c2c102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c32557700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c2c06c470 0x7f5c2c102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.659+0000 7f5c32557700 1 -- 172.21.15.69:0/1747634407 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c1a8190 con 0x7f5c2c1095a0 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.660+0000 7f5c32557700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1095a0 0x7f5c2c102f80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5c24007ee0 tx=0x7f5c2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.660+0000 7f5c22ffd700 1 -- 172.21.15.69:0/1747634407 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24004bb0 con 0x7f5c2c1095a0 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.660+0000 7f5c22ffd700 1 -- 172.21.15.69:0/1747634407 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c24004d50 con 0x7f5c2c1095a0 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.660+0000 7f5c33fba700 1 -- 172.21.15.69:0/1747634407 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c1a8480 con 0x7f5c2c1095a0 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.660+0000 7f5c22ffd700 1 -- 172.21.15.69:0/1747634407 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24005920 con 0x7f5c2c1095a0 2024-05-14T16:46:53.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.660+0000 7f5c33fba700 1 -- 172.21.15.69:0/1747634407 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c1a8940 con 0x7f5c2c1095a0 2024-05-14T16:46:53.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.661+0000 7f5c33fba700 1 -- 172.21.15.69:0/1747634407 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2c10ba80 con 0x7f5c2c1095a0 2024-05-14T16:46:53.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.662+0000 7f5c22ffd700 1 -- 172.21.15.69:0/1747634407 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5c2401a020 con 0x7f5c2c1095a0 2024-05-14T16:46:53.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.663+0000 7f5c22ffd700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c180b2880 0x7f5c180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:53.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.663+0000 7f5c31d56700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c180b2880 0x7f5c180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:53.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.663+0000 7f5c22ffd700 1 -- 172.21.15.69:0/1747634407 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f5c240e1db0 con 0x7f5c2c1095a0 2024-05-14T16:46:53.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.663+0000 7f5c31d56700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c180b2880 0x7f5c180b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009a10 tx=0x7f5c28009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:53.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.666+0000 7f5c22ffd700 1 -- 172.21.15.69:0/1747634407 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5c24010070 con 0x7f5c2c1095a0 2024-05-14T16:46:53.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:53.922+0000 7f5c33fba700 1 -- 172.21.15.69:0/1747634407 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5c2c10b460 con 0x7f5c2c1095a0 2024-05-14T16:46:53.923 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:53 smithi069 ceph-mon[31507]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-14T16:46:53.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:53 smithi203 ceph-mon[35787]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-14T16:46:54.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:53 smithi092 ceph-mon[35763]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-14T16:46:54.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.590+0000 7f5c22ffd700 1 -- 172.21.15.69:0/1747634407 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f5c240a9c20 con 0x7f5c2c1095a0 2024-05-14T16:46:54.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 -- 172.21.15.69:0/1747634407 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c180b2880 msgr2=0x7f5c180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:54.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c180b2880 0x7f5c180b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009a10 tx=0x7f5c28009190 comp rx=0 tx=0).stop 2024-05-14T16:46:54.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 -- 172.21.15.69:0/1747634407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1095a0 msgr2=0x7f5c2c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:54.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1095a0 0x7f5c2c102f80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5c24007ee0 tx=0x7f5c2400d3b0 comp rx=0 tx=0).stop 2024-05-14T16:46:54.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 -- 172.21.15.69:0/1747634407 shutdown_connections 2024-05-14T16:46:54.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c180b2880 0x7f5c180b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:54.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1095a0 0x7f5c2c102f80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:54.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2c06bab0 0x7f5c2c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:54.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 --2- 172.21.15.69:0/1747634407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c2c06c470 0x7f5c2c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:54.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 -- 172.21.15.69:0/1747634407 >> 172.21.15.69:0/1747634407 conn(0x7f5c2c0793b0 msgr2=0x7f5c2c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:54.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 -- 172.21.15.69:0/1747634407 shutdown_connections 2024-05-14T16:46:54.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:54.593+0000 7f5c33fba700 1 -- 172.21.15.69:0/1747634407 wait complete. 2024-05-14T16:46:54.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:54 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1747634407' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T16:46:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:54 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1747634407' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T16:46:55.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:54 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1747634407' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T16:46:55.386 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-14T16:46:55.386 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-14T16:46:55.631 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:55.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:55 smithi069 ceph-mon[31507]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-14T16:46:55.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:55 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1747634407' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T16:46:55.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:55 smithi069 ceph-mon[31507]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:46:55.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:55 smithi203 ceph-mon[35787]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-14T16:46:55.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:55 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1747634407' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T16:46:55.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:55 smithi203 ceph-mon[35787]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:46:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:55 smithi092 ceph-mon[35763]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-14T16:46:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:55 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1747634407' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T16:46:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:55 smithi092 ceph-mon[35763]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:46:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.430+0000 7f7594a7e700 1 -- 172.21.15.69:0/3397283272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901041a0 msgr2=0x7f75901081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.430+0000 7f7594a7e700 1 --2- 172.21.15.69:0/3397283272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901041a0 0x7f75901081c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f75800099c0 tx=0x7f7580009cd0 comp rx=0 tx=0).stop 2024-05-14T16:46:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.430+0000 7f7594a7e700 1 -- 172.21.15.69:0/3397283272 shutdown_connections 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.430+0000 7f7594a7e700 1 --2- 172.21.15.69:0/3397283272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75901088f0 0x7f7590110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.430+0000 7f7594a7e700 1 --2- 172.21.15.69:0/3397283272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75901037f0 0x7f7590103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.430+0000 7f7594a7e700 1 --2- 172.21.15.69:0/3397283272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901041a0 0x7f75901081c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.430+0000 7f7594a7e700 1 -- 172.21.15.69:0/3397283272 >> 172.21.15.69:0/3397283272 conn(0x7f759007a6a0 msgr2=0x7f759007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.430+0000 7f7594a7e700 1 -- 172.21.15.69:0/3397283272 shutdown_connections 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.430+0000 7f7594a7e700 1 -- 172.21.15.69:0/3397283272 wait complete. 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f7594a7e700 1 Processor -- start 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f7594a7e700 1 -- start start 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f7594a7e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901037f0 0x7f759019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f7594a7e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75901041a0 0x7f759019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f7594a7e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75901088f0 0x7f7590199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f7594a7e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75901136d0 con 0x7f75901037f0 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f7594a7e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7590113550 con 0x7f75901041a0 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f7594a7e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7590113850 con 0x7f75901088f0 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f758e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901037f0 0x7f759019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f758e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901037f0 0x7f759019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52504/0 (socket says 172.21.15.69:52504) 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f758e59c700 1 -- 172.21.15.69:0/2754178116 learned_addr learned my addr 172.21.15.69:0/2754178116 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f758e59c700 1 -- 172.21.15.69:0/2754178116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75901088f0 msgr2=0x7f7590199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f758e59c700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75901088f0 0x7f7590199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f758e59c700 1 -- 172.21.15.69:0/2754178116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75901041a0 msgr2=0x7f759019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f758e59c700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75901041a0 0x7f759019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f758e59c700 1 -- 172.21.15.69:0/2754178116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759019a0d0 con 0x7f75901037f0 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.431+0000 7f758e59c700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901037f0 0x7f759019f710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f758400c9b0 tx=0x7f758400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.432+0000 7f757f7fe700 1 -- 172.21.15.69:0/2754178116 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584007b00 con 0x7f75901037f0 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.432+0000 7f7594a7e700 1 -- 172.21.15.69:0/2754178116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759019a3c0 con 0x7f75901037f0 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.432+0000 7f757f7fe700 1 -- 172.21.15.69:0/2754178116 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7584007ca0 con 0x7f75901037f0 2024-05-14T16:46:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.432+0000 7f757f7fe700 1 -- 172.21.15.69:0/2754178116 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584018730 con 0x7f75901037f0 2024-05-14T16:46:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.432+0000 7f7594a7e700 1 -- 172.21.15.69:0/2754178116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75901a65e0 con 0x7f75901037f0 2024-05-14T16:46:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.433+0000 7f7594a7e700 1 -- 172.21.15.69:0/2754178116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759006a230 con 0x7f75901037f0 2024-05-14T16:46:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.434+0000 7f757f7fe700 1 -- 172.21.15.69:0/2754178116 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f758400f450 con 0x7f75901037f0 2024-05-14T16:46:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.435+0000 7f757f7fe700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75780b2880 0x7f75780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.435+0000 7f758dd9b700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75780b2880 0x7f75780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:57.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.435+0000 7f757f7fe700 1 -- 172.21.15.69:0/2754178116 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f75840e2140 con 0x7f75901037f0 2024-05-14T16:46:57.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.435+0000 7f758dd9b700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75780b2880 0x7f75780b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7580009fd0 tx=0x7f7580000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:57.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.439+0000 7f757f7fe700 1 -- 172.21.15.69:0/2754178116 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7584014030 con 0x7f75901037f0 2024-05-14T16:46:57.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:57.691+0000 7f7594a7e700 1 -- 172.21.15.69:0/2754178116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f759002e560 con 0x7f75901037f0 2024-05-14T16:46:57.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:57 smithi069 ceph-mon[31507]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-14T16:46:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:57 smithi203 ceph-mon[35787]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-14T16:46:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:57 smithi092 ceph-mon[35763]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-14T16:46:58.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.519+0000 7f757f7fe700 1 -- 172.21.15.69:0/2754178116 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f75840a9fb0 con 0x7f75901037f0 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 -- 172.21.15.69:0/2754178116 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75780b2880 msgr2=0x7f75780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75780b2880 0x7f75780b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7580009fd0 tx=0x7f7580000bc0 comp rx=0 tx=0).stop 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 -- 172.21.15.69:0/2754178116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901037f0 msgr2=0x7f759019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901037f0 0x7f759019f710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f758400c9b0 tx=0x7f758400cd70 comp rx=0 tx=0).stop 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 -- 172.21.15.69:0/2754178116 shutdown_connections 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75780b2880 0x7f75780b4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901037f0 0x7f759019f710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75901088f0 0x7f7590199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 --2- 172.21.15.69:0/2754178116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75901041a0 0x7f759019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 -- 172.21.15.69:0/2754178116 >> 172.21.15.69:0/2754178116 conn(0x7f759007a6a0 msgr2=0x7f75901017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 -- 172.21.15.69:0/2754178116 shutdown_connections 2024-05-14T16:46:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:46:58.523+0000 7f7594a7e700 1 -- 172.21.15.69:0/2754178116 wait complete. 2024-05-14T16:46:58.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:58 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2754178116' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T16:46:58.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:58 smithi069 ceph-mon[31507]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T16:46:58.922 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:58 smithi069 ceph-mon[31507]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T16:46:58.922 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:58 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2754178116' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T16:46:58.922 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:58 smithi069 ceph-mon[31507]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-14T16:46:58.922 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:58 smithi069 ceph-mon[31507]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-14T16:46:58.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:58 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2754178116' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T16:46:58.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:58 smithi203 ceph-mon[35787]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T16:46:58.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:58 smithi203 ceph-mon[35787]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T16:46:58.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:58 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2754178116' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T16:46:58.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:58 smithi203 ceph-mon[35787]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-14T16:46:58.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:58 smithi203 ceph-mon[35787]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-14T16:46:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:58 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2754178116' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T16:46:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:58 smithi092 ceph-mon[35763]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T16:46:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:58 smithi092 ceph-mon[35763]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T16:46:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:58 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2754178116' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T16:46:59.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:58 smithi092 ceph-mon[35763]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-14T16:46:59.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:58 smithi092 ceph-mon[35763]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-14T16:46:59.221 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-14T16:46:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:59.925 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:59 smithi069 ceph-mon[31507]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T16:46:59.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:59 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] up:active 2024-05-14T16:46:59.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:59 smithi069 ceph-mon[31507]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T16:46:59.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:59 smithi069 ceph-mon[31507]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T16:46:59.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:59 smithi069 ceph-mon[31507]: Cluster is now healthy 2024-05-14T16:46:59.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:59 smithi069 ceph-mon[31507]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-14T16:46:59.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:59 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-14T16:46:59.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:46:59 smithi069 ceph-mon[31507]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-14T16:46:59.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:59 smithi203 ceph-mon[35787]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T16:46:59.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:59 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] up:active 2024-05-14T16:46:59.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:59 smithi203 ceph-mon[35787]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T16:46:59.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:59 smithi203 ceph-mon[35787]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T16:46:59.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:59 smithi203 ceph-mon[35787]: Cluster is now healthy 2024-05-14T16:46:59.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:59 smithi203 ceph-mon[35787]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-14T16:46:59.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:59 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-14T16:46:59.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:46:59 smithi203 ceph-mon[35787]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-14T16:47:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:59 smithi092 ceph-mon[35763]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T16:47:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:59 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] up:active 2024-05-14T16:47:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:59 smithi092 ceph-mon[35763]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T16:47:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:59 smithi092 ceph-mon[35763]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T16:47:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:59 smithi092 ceph-mon[35763]: Cluster is now healthy 2024-05-14T16:47:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:59 smithi092 ceph-mon[35763]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-14T16:47:00.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:59 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-14T16:47:00.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:46:59 smithi092 ceph-mon[35763]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-14T16:47:01.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.098+0000 7f938d11a700 1 -- 172.21.15.69:0/3190391528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938806c470 msgr2=0x7f93881103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:01.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.098+0000 7f938d11a700 1 --2- 172.21.15.69:0/3190391528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938806c470 0x7f93881103c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f938000b0a0 tx=0x7f938000b3b0 comp rx=0 tx=0).stop 2024-05-14T16:47:01.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.098+0000 7f938d11a700 1 -- 172.21.15.69:0/3190391528 shutdown_connections 2024-05-14T16:47:01.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.098+0000 7f938d11a700 1 --2- 172.21.15.69:0/3190391528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938806c470 0x7f93881103c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:01.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.098+0000 7f938d11a700 1 --2- 172.21.15.69:0/3190391528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93881095a0 0x7f9388109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:01.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.098+0000 7f938d11a700 1 --2- 172.21.15.69:0/3190391528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938806bab0 0x7f938806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:01.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.098+0000 7f938d11a700 1 -- 172.21.15.69:0/3190391528 >> 172.21.15.69:0/3190391528 conn(0x7f93880793b0 msgr2=0x7f93880797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:01.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.098+0000 7f938d11a700 1 -- 172.21.15.69:0/3190391528 shutdown_connections 2024-05-14T16:47:01.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.098+0000 7f938d11a700 1 -- 172.21.15.69:0/3190391528 wait complete. 2024-05-14T16:47:01.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f938d11a700 1 Processor -- start 2024-05-14T16:47:01.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f938d11a700 1 -- start start 2024-05-14T16:47:01.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f938d11a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938806bab0 0x7f9388199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f938d11a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938806c470 0x7f938819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f938d11a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93881095a0 0x7f938819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f938d11a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93881137b0 con 0x7f938806c470 2024-05-14T16:47:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f938d11a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9388113630 con 0x7f938806bab0 2024-05-14T16:47:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f938d11a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9388113930 con 0x7f93881095a0 2024-05-14T16:47:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f937e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938806c470 0x7f938819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f937e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938806c470 0x7f938819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52512/0 (socket says 172.21.15.69:52512) 2024-05-14T16:47:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f937e59c700 1 -- 172.21.15.69:0/377803387 learned_addr learned my addr 172.21.15.69:0/377803387 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f9386d9d700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938806bab0 0x7f9388199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.099+0000 7f938759e700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93881095a0 0x7f938819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f937e59c700 1 -- 172.21.15.69:0/377803387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93881095a0 msgr2=0x7f938819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f937e59c700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93881095a0 0x7f938819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f937e59c700 1 -- 172.21.15.69:0/377803387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938806bab0 msgr2=0x7f9388199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f937e59c700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938806bab0 0x7f9388199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f937e59c700 1 -- 172.21.15.69:0/377803387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938819ed50 con 0x7f938806c470 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f938759e700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93881095a0 0x7f938819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f937e59c700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938806c470 0x7f938819a280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f937000b530 tx=0x7f937000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f9384d99700 1 -- 172.21.15.69:0/377803387 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370015020 con 0x7f938806c470 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f9384d99700 1 -- 172.21.15.69:0/377803387 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9370011ce0 con 0x7f938806c470 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f938d11a700 1 -- 172.21.15.69:0/377803387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938819f040 con 0x7f938806c470 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f9384d99700 1 -- 172.21.15.69:0/377803387 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370010e40 con 0x7f938806c470 2024-05-14T16:47:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.100+0000 7f938d11a700 1 -- 172.21.15.69:0/377803387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93881a68c0 con 0x7f938806c470 2024-05-14T16:47:01.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.101+0000 7f938d11a700 1 -- 172.21.15.69:0/377803387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938810ba80 con 0x7f938806c470 2024-05-14T16:47:01.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.102+0000 7f9384d99700 1 -- 172.21.15.69:0/377803387 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9370018020 con 0x7f938806c470 2024-05-14T16:47:01.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.103+0000 7f9384d99700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f93740b2880 0x7f93740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:01.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.103+0000 7f9386d9d700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f93740b2880 0x7f93740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:01.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.103+0000 7f9384d99700 1 -- 172.21.15.69:0/377803387 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f93700e1e70 con 0x7f938806c470 2024-05-14T16:47:01.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.103+0000 7f9386d9d700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f93740b2880 0x7f93740b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9378009a10 tx=0x7f9378009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:01.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.106+0000 7f9384d99700 1 -- 172.21.15.69:0/377803387 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f937001f020 con 0x7f938806c470 2024-05-14T16:47:01.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.359+0000 7f938d11a700 1 -- 172.21.15.69:0/377803387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f938806a230 con 0x7f938806c470 2024-05-14T16:47:01.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.604+0000 7f9384d99700 1 -- 172.21.15.69:0/377803387 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f9370010510 con 0x7f938806c470 2024-05-14T16:47:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.606+0000 7f938d11a700 1 -- 172.21.15.69:0/377803387 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f93740b2880 msgr2=0x7f93740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.606+0000 7f938d11a700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f93740b2880 0x7f93740b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9378009a10 tx=0x7f9378009190 comp rx=0 tx=0).stop 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.606+0000 7f938d11a700 1 -- 172.21.15.69:0/377803387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938806c470 msgr2=0x7f938819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.606+0000 7f938d11a700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938806c470 0x7f938819a280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f937000b530 tx=0x7f937000b840 comp rx=0 tx=0).stop 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.606+0000 7f938d11a700 1 -- 172.21.15.69:0/377803387 shutdown_connections 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.606+0000 7f938d11a700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93881095a0 0x7f938819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.606+0000 7f938d11a700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f938806bab0 0x7f9388199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.606+0000 7f938d11a700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f93740b2880 0x7f93740b4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.606+0000 7f938d11a700 1 --2- 172.21.15.69:0/377803387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938806c470 0x7f938819a280 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.606+0000 7f938d11a700 1 -- 172.21.15.69:0/377803387 >> 172.21.15.69:0/377803387 conn(0x7f93880793b0 msgr2=0x7f9388101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.607+0000 7f938d11a700 1 -- 172.21.15.69:0/377803387 shutdown_connections 2024-05-14T16:47:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:01.607+0000 7f938d11a700 1 -- 172.21.15.69:0/377803387 wait complete. 2024-05-14T16:47:01.618 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:01 smithi069 ceph-mon[31507]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-14T16:47:01.618 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:01 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] up:active 2024-05-14T16:47:01.618 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:01 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-14T16:47:01.619 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:01 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/377803387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T16:47:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:01 smithi203 ceph-mon[35787]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-14T16:47:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:01 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] up:active 2024-05-14T16:47:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:01 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-14T16:47:01.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:01 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/377803387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T16:47:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:01 smithi092 ceph-mon[35763]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-14T16:47:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:01 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] up:active 2024-05-14T16:47:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:01 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-14T16:47:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:01 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/377803387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T16:47:02.625 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-14T16:47:02.871 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:02.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:02 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/377803387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T16:47:02.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:02 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-14T16:47:02.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:02 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:47:02.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:02 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/377803387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T16:47:02.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:02 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-14T16:47:02.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:02 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:47:03.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:02 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/377803387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T16:47:03.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:02 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-14T16:47:03.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:02 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:47:03.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:03 smithi069 ceph-mon[31507]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 255 B/s wr, 2 op/s 2024-05-14T16:47:03.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:03 smithi203 ceph-mon[35787]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 255 B/s wr, 2 op/s 2024-05-14T16:47:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:03 smithi092 ceph-mon[35763]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 255 B/s wr, 2 op/s 2024-05-14T16:47:04.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.607+0000 7fdec746f700 1 -- 172.21.15.69:0/1600301082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec0102440 msgr2=0x7fdec01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:04.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.607+0000 7fdec746f700 1 --2- 172.21.15.69:0/1600301082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec0102440 0x7fdec01028c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009990 tx=0x7fdeb4009ca0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.607+0000 7fdec746f700 1 -- 172.21.15.69:0/1600301082 shutdown_connections 2024-05-14T16:47:04.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.607+0000 7fdec746f700 1 --2- 172.21.15.69:0/1600301082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec0102e00 0x7fdec010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.607+0000 7fdec746f700 1 --2- 172.21.15.69:0/1600301082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec0108520 0x7fdec0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.607+0000 7fdec746f700 1 --2- 172.21.15.69:0/1600301082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec0102440 0x7fdec01028c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.607+0000 7fdec746f700 1 -- 172.21.15.69:0/1600301082 >> 172.21.15.69:0/1600301082 conn(0x7fdec00793f0 msgr2=0x7fdec0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:04.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.607+0000 7fdec746f700 1 -- 172.21.15.69:0/1600301082 shutdown_connections 2024-05-14T16:47:04.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.607+0000 7fdec746f700 1 -- 172.21.15.69:0/1600301082 wait complete. 2024-05-14T16:47:04.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec746f700 1 Processor -- start 2024-05-14T16:47:04.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec746f700 1 -- start start 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec746f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec0102440 0x7fdec0075060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec746f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec0102e00 0x7fdec006d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec746f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec0108520 0x7fdec006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec746f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdec0112650 con 0x7fdec0108520 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec746f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdec01124d0 con 0x7fdec0102e00 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec746f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdec01127d0 con 0x7fdec0102440 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec5a0c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec0108520 0x7fdec006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec5a0c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec0108520 0x7fdec006d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39842/0 (socket says 172.21.15.69:39842) 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec5a0c700 1 -- 172.21.15.69:0/729264960 learned_addr learned my addr 172.21.15.69:0/729264960 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec4a0a700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec0102e00 0x7fdec006d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.608+0000 7fdec520b700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec0102440 0x7fdec0075060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdec5a0c700 1 -- 172.21.15.69:0/729264960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec0102440 msgr2=0x7fdec0075060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdec5a0c700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec0102440 0x7fdec0075060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdec5a0c700 1 -- 172.21.15.69:0/729264960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec0102e00 msgr2=0x7fdec006d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdec5a0c700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec0102e00 0x7fdec006d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdec5a0c700 1 -- 172.21.15.69:0/729264960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec006dde0 con 0x7fdec0108520 2024-05-14T16:47:04.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdec5a0c700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec0108520 0x7fdec006d570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8007ee0 tx=0x7fdeb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:04.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdeb27fc700 1 -- 172.21.15.69:0/729264960 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb800dbd0 con 0x7fdec0108520 2024-05-14T16:47:04.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdec746f700 1 -- 172.21.15.69:0/729264960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec01b0ce0 con 0x7fdec0108520 2024-05-14T16:47:04.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdeb27fc700 1 -- 172.21.15.69:0/729264960 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb800dd70 con 0x7fdec0108520 2024-05-14T16:47:04.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdeb27fc700 1 -- 172.21.15.69:0/729264960 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8005990 con 0x7fdec0108520 2024-05-14T16:47:04.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.609+0000 7fdec746f700 1 -- 172.21.15.69:0/729264960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec01b11a0 con 0x7fdec0108520 2024-05-14T16:47:04.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.610+0000 7fdec746f700 1 -- 172.21.15.69:0/729264960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec0103b60 con 0x7fdec0108520 2024-05-14T16:47:04.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.612+0000 7fdeb27fc700 1 -- 172.21.15.69:0/729264960 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdeb801a020 con 0x7fdec0108520 2024-05-14T16:47:04.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.612+0000 7fdeb27fc700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fdeac0b2880 0x7fdeac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:04.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.613+0000 7fdec520b700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fdeac0b2880 0x7fdeac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:04.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.613+0000 7fdeb27fc700 1 -- 172.21.15.69:0/729264960 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fdeb80e1f50 con 0x7fdec0108520 2024-05-14T16:47:04.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.613+0000 7fdec520b700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fdeac0b2880 0x7fdeac0b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdebc006fd0 tx=0x7fdebc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:04.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.616+0000 7fdeb27fc700 1 -- 172.21.15.69:0/729264960 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdeb8010070 con 0x7fdec0108520 2024-05-14T16:47:04.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:04.865+0000 7fdec746f700 1 -- 172.21.15.69:0/729264960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fdec01b13d0 con 0x7fdec0108520 2024-05-14T16:47:05.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.618+0000 7fdeb27fc700 1 -- 172.21.15.69:0/729264960 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fdeb80a9dc0 con 0x7fdec0108520 2024-05-14T16:47:05.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.621+0000 7fdec746f700 1 -- 172.21.15.69:0/729264960 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fdeac0b2880 msgr2=0x7fdeac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:05.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.621+0000 7fdec746f700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fdeac0b2880 0x7fdeac0b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdebc006fd0 tx=0x7fdebc008040 comp rx=0 tx=0).stop 2024-05-14T16:47:05.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.621+0000 7fdec746f700 1 -- 172.21.15.69:0/729264960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec0108520 msgr2=0x7fdec006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:05.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.621+0000 7fdec746f700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec0108520 0x7fdec006d570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8007ee0 tx=0x7fdeb800d3b0 comp rx=0 tx=0).stop 2024-05-14T16:47:05.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.621+0000 7fdec746f700 1 -- 172.21.15.69:0/729264960 shutdown_connections 2024-05-14T16:47:05.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.621+0000 7fdec746f700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fdeac0b2880 0x7fdeac0b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:05.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.622+0000 7fdec746f700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec0108520 0x7fdec006d570 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:05.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.622+0000 7fdec746f700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec0102440 0x7fdec0075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:05.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.622+0000 7fdec746f700 1 --2- 172.21.15.69:0/729264960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec0102e00 0x7fdec006d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:05.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.622+0000 7fdec746f700 1 -- 172.21.15.69:0/729264960 >> 172.21.15.69:0/729264960 conn(0x7fdec00793f0 msgr2=0x7fdec006b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:05.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.622+0000 7fdec746f700 1 -- 172.21.15.69:0/729264960 shutdown_connections 2024-05-14T16:47:05.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:05.622+0000 7fdec746f700 1 -- 172.21.15.69:0/729264960 wait complete. 2024-05-14T16:47:05.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:05 smithi069 ceph-mon[31507]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:47:05.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:05 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/729264960' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T16:47:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:05 smithi203 ceph-mon[35787]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:47:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:05 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/729264960' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T16:47:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:05 smithi092 ceph-mon[35763]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:47:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:05 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/729264960' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T16:47:06.397 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-14T16:47:06.397 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-14T16:47:06.643 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:06.884 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:06 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/729264960' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T16:47:06.885 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:06 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:47:06.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:06 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/729264960' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T16:47:06.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:06 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:47:07.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:06 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/729264960' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T16:47:07.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:06 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:47:07.878 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:07 smithi069 ceph-mon[31507]: pgmap v295: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:47:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:07 smithi203 ceph-mon[35787]: pgmap v295: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:47:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:07 smithi092 ceph-mon[35763]: pgmap v295: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:47:08.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.265+0000 7f5428f07700 1 -- 172.21.15.69:0/4146844458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54241088f0 msgr2=0x7f5424110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:08.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.265+0000 7f5428f07700 1 --2- 172.21.15.69:0/4146844458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54241088f0 0x7f5424110470 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f541c00b0d0 tx=0x7f541c00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.265+0000 7f5428f07700 1 -- 172.21.15.69:0/4146844458 shutdown_connections 2024-05-14T16:47:08.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.265+0000 7f5428f07700 1 --2- 172.21.15.69:0/4146844458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54241088f0 0x7f5424110470 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.265+0000 7f5428f07700 1 --2- 172.21.15.69:0/4146844458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54241037f0 0x7f5424103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.265+0000 7f5428f07700 1 --2- 172.21.15.69:0/4146844458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54241041a0 0x7f54241081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.265+0000 7f5428f07700 1 -- 172.21.15.69:0/4146844458 >> 172.21.15.69:0/4146844458 conn(0x7f542407a6a0 msgr2=0x7f542407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:08.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.266+0000 7f5428f07700 1 -- 172.21.15.69:0/4146844458 shutdown_connections 2024-05-14T16:47:08.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.266+0000 7f5428f07700 1 -- 172.21.15.69:0/4146844458 wait complete. 2024-05-14T16:47:08.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.266+0000 7f5428f07700 1 Processor -- start 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.266+0000 7f5428f07700 1 -- start start 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5428f07700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54241037f0 0x7f542419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5428f07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54241041a0 0x7f542419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5428f07700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54241088f0 0x7f5424199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5428f07700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f54241137b0 con 0x7f54241088f0 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5428f07700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5424113630 con 0x7f54241041a0 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5428f07700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5424113930 con 0x7f54241037f0 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5422d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54241088f0 0x7f5424199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5422d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54241088f0 0x7f5424199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39860/0 (socket says 172.21.15.69:39860) 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5422d9d700 1 -- 172.21.15.69:0/1530112309 learned_addr learned my addr 172.21.15.69:0/1530112309 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f542259c700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54241037f0 0x7f542419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5422d9d700 1 -- 172.21.15.69:0/1530112309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54241037f0 msgr2=0x7f542419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5422d9d700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54241037f0 0x7f542419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5422d9d700 1 -- 172.21.15.69:0/1530112309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54241041a0 msgr2=0x7f542419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5421d9b700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54241041a0 0x7f542419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5422d9d700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54241041a0 0x7f542419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5422d9d700 1 -- 172.21.15.69:0/1530112309 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542419a1b0 con 0x7f54241088f0 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f542259c700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54241037f0 0x7f542419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5422d9d700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54241088f0 0x7f5424199a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f541c009fd0 tx=0x7f541c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f54137fe700 1 -- 172.21.15.69:0/1530112309 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c01f070 con 0x7f54241088f0 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5428f07700 1 -- 172.21.15.69:0/1530112309 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542419a440 con 0x7f54241088f0 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f54137fe700 1 -- 172.21.15.69:0/1530112309 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541c003c00 con 0x7f54241088f0 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f5428f07700 1 -- 172.21.15.69:0/1530112309 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54241a6760 con 0x7f54241088f0 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.267+0000 7f54137fe700 1 -- 172.21.15.69:0/1530112309 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c01b8e0 con 0x7f54241088f0 2024-05-14T16:47:08.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.268+0000 7f5428f07700 1 -- 172.21.15.69:0/1530112309 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f542406a230 con 0x7f54241088f0 2024-05-14T16:47:08.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.270+0000 7f54137fe700 1 -- 172.21.15.69:0/1530112309 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f541c022070 con 0x7f54241088f0 2024-05-14T16:47:08.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.270+0000 7f54137fe700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f540c0b2900 0x7f540c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:08.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.270+0000 7f542259c700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f540c0b2900 0x7f540c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:08.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.270+0000 7f54137fe700 1 -- 172.21.15.69:0/1530112309 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f541c00f020 con 0x7f54241088f0 2024-05-14T16:47:08.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.271+0000 7f542259c700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f540c0b2900 0x7f540c0b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f542419ab90 tx=0x7f5414008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:08.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.274+0000 7f54137fe700 1 -- 172.21.15.69:0/1530112309 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f541c029020 con 0x7f54241088f0 2024-05-14T16:47:08.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.510+0000 7f5428f07700 1 -- 172.21.15.69:0/1530112309 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5424063980 con 0x7f540c0b2900 2024-05-14T16:47:08.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.530+0000 7f54137fe700 1 -- 172.21.15.69:0/1530112309 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5424063980 con 0x7f540c0b2900 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 -- 172.21.15.69:0/1530112309 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f540c0b2900 msgr2=0x7f540c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f540c0b2900 0x7f540c0b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f542419ab90 tx=0x7f5414008040 comp rx=0 tx=0).stop 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 -- 172.21.15.69:0/1530112309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54241088f0 msgr2=0x7f5424199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54241088f0 0x7f5424199a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f541c009fd0 tx=0x7f541c007ad0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 -- 172.21.15.69:0/1530112309 shutdown_connections 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f540c0b2900 0x7f540c0b4dc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54241088f0 0x7f5424199a50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54241037f0 0x7f542419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 --2- 172.21.15.69:0/1530112309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54241041a0 0x7f542419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 -- 172.21.15.69:0/1530112309 >> 172.21.15.69:0/1530112309 conn(0x7f542407a6a0 msgr2=0x7f5424101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 -- 172.21.15.69:0/1530112309 shutdown_connections 2024-05-14T16:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:08.533+0000 7f5428f07700 1 -- 172.21.15.69:0/1530112309 wait complete. 2024-05-14T16:47:08.910 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:47:08 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:47:08.515+0000 7fbc31c02700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:47:08.910 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:47:08 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:47:08.515+0000 7fbc31c02700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:47:08.910 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:47:08 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:47:08.515+0000 7fbc31c02700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:47:08.910 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:47:08 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:47:08.515+0000 7fbc31c02700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:47:08.910 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:47:08 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:47:08.515+0000 7fbc31c02700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:47:09.251 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-14T16:47:09.489 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:09.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:09 smithi069 ceph-mon[31507]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T16:47:09.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:09 smithi069 ceph-mon[31507]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:09.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:09 smithi203 ceph-mon[35787]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T16:47:09.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:09 smithi203 ceph-mon[35787]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:09 smithi092 ceph-mon[35763]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T16:47:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:09 smithi092 ceph-mon[35763]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:11.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.184+0000 7f0980481700 1 -- 172.21.15.69:0/1344561358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781037f0 msgr2=0x7f0978103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:11.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.184+0000 7f0980481700 1 --2- 172.21.15.69:0/1344561358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781037f0 0x7f0978103bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f096c0099c0 tx=0x7f096c009cd0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.184+0000 7f0980481700 1 -- 172.21.15.69:0/1344561358 shutdown_connections 2024-05-14T16:47:11.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.184+0000 7f0980481700 1 --2- 172.21.15.69:0/1344561358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781037f0 0x7f0978103bd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.184+0000 7f0980481700 1 --2- 172.21.15.69:0/1344561358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09781088f0 0x7f0978110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.184+0000 7f0980481700 1 --2- 172.21.15.69:0/1344561358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09781041a0 0x7f09781081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.184+0000 7f0980481700 1 -- 172.21.15.69:0/1344561358 >> 172.21.15.69:0/1344561358 conn(0x7f097807a6a0 msgr2=0x7f097807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:11.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.185+0000 7f0980481700 1 -- 172.21.15.69:0/1344561358 shutdown_connections 2024-05-14T16:47:11.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.185+0000 7f0980481700 1 -- 172.21.15.69:0/1344561358 wait complete. 2024-05-14T16:47:11.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.185+0000 7f0980481700 1 Processor -- start 2024-05-14T16:47:11.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.185+0000 7f0980481700 1 -- start start 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.185+0000 7f0980481700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09781037f0 0x7f0978075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.185+0000 7f0980481700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09781041a0 0x7f0978075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.185+0000 7f0980481700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781088f0 0x7f09780762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.185+0000 7f0980481700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09781136c0 con 0x7f09781088f0 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.185+0000 7f0980481700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0978113540 con 0x7f09781037f0 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.185+0000 7f0980481700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0978113840 con 0x7f09781041a0 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097ea1e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781088f0 0x7f09780762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097ea1e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781088f0 0x7f09780762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39894/0 (socket says 172.21.15.69:39894) 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097ea1e700 1 -- 172.21.15.69:0/1274514642 learned_addr learned my addr 172.21.15.69:0/1274514642 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097e21d700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09781037f0 0x7f0978075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097da1c700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09781041a0 0x7f0978075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097ea1e700 1 -- 172.21.15.69:0/1274514642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09781041a0 msgr2=0x7f0978075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:11.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097ea1e700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09781041a0 0x7f0978075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097ea1e700 1 -- 172.21.15.69:0/1274514642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09781037f0 msgr2=0x7f0978075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097ea1e700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09781037f0 0x7f0978075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097ea1e700 1 -- 172.21.15.69:0/1274514642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09781a8060 con 0x7f09781088f0 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097e21d700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09781037f0 0x7f0978075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f097ea1e700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781088f0 0x7f09780762c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0970007ee0 tx=0x7f097000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f096b7fe700 1 -- 172.21.15.69:0/1274514642 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097000dbd0 con 0x7f09781088f0 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f096b7fe700 1 -- 172.21.15.69:0/1274514642 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f097000dd70 con 0x7f09781088f0 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f0980481700 1 -- 172.21.15.69:0/1274514642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09781a8350 con 0x7f09781088f0 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f096b7fe700 1 -- 172.21.15.69:0/1274514642 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970005990 con 0x7f09781088f0 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.186+0000 7f0980481700 1 -- 172.21.15.69:0/1274514642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09781a8840 con 0x7f09781088f0 2024-05-14T16:47:11.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.187+0000 7f0980481700 1 -- 172.21.15.69:0/1274514642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097810bb30 con 0x7f09781088f0 2024-05-14T16:47:11.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.189+0000 7f096b7fe700 1 -- 172.21.15.69:0/1274514642 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f097001a020 con 0x7f09781088f0 2024-05-14T16:47:11.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.189+0000 7f096b7fe700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09640b2910 0x7f09640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:11.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.189+0000 7f097e21d700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09640b2910 0x7f09640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:11.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.189+0000 7f096b7fe700 1 -- 172.21.15.69:0/1274514642 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f09700e2070 con 0x7f09781088f0 2024-05-14T16:47:11.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.190+0000 7f097e21d700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09640b2910 0x7f09640b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f096c005b40 tx=0x7f096c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:11.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.193+0000 7f096b7fe700 1 -- 172.21.15.69:0/1274514642 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0970010070 con 0x7f09781088f0 2024-05-14T16:47:11.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.427+0000 7f0980481700 1 -- 172.21.15.69:0/1274514642 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0978063980 con 0x7f09640b2910 2024-05-14T16:47:11.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.454+0000 7f096b7fe700 1 -- 172.21.15.69:0/1274514642 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0978063980 con 0x7f09640b2910 2024-05-14T16:47:11.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 -- 172.21.15.69:0/1274514642 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09640b2910 msgr2=0x7f09640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09640b2910 0x7f09640b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f096c005b40 tx=0x7f096c01a040 comp rx=0 tx=0).stop 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 -- 172.21.15.69:0/1274514642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781088f0 msgr2=0x7f09780762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781088f0 0x7f09780762c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0970007ee0 tx=0x7f097000d3b0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 -- 172.21.15.69:0/1274514642 shutdown_connections 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f09640b2910 0x7f09640b4dd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781088f0 0x7f09780762c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09781037f0 0x7f0978075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 --2- 172.21.15.69:0/1274514642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09781041a0 0x7f0978075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 -- 172.21.15.69:0/1274514642 >> 172.21.15.69:0/1274514642 conn(0x7f097807a6a0 msgr2=0x7f0978109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 -- 172.21.15.69:0/1274514642 shutdown_connections 2024-05-14T16:47:11.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:11.457+0000 7f0980481700 1 -- 172.21.15.69:0/1274514642 wait complete. 2024-05-14T16:47:11.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:11 smithi069 ceph-mon[31507]: pgmap v297: 97 pgs: 97 active+clean; 588 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-14T16:47:11.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:11 smithi069 ceph-mon[31507]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-14T16:47:11.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:11 smithi203 ceph-mon[35787]: pgmap v297: 97 pgs: 97 active+clean; 588 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-14T16:47:11.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:11 smithi203 ceph-mon[35787]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-14T16:47:12.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:11 smithi092 ceph-mon[35763]: pgmap v297: 97 pgs: 97 active+clean; 588 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-14T16:47:12.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:11 smithi092 ceph-mon[35763]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-14T16:47:12.102 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-14T16:47:12.342 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:12.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:12 smithi069 ceph-mon[31507]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:12.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:12 smithi203 ceph-mon[35787]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:13.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:12 smithi092 ceph-mon[35763]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:13.805 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:13 smithi069 ceph-mon[31507]: pgmap v298: 97 pgs: 97 active+clean; 588 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-14T16:47:13.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:13 smithi203 ceph-mon[35787]: pgmap v298: 97 pgs: 97 active+clean; 588 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-14T16:47:14.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:13 smithi092 ceph-mon[35763]: pgmap v298: 97 pgs: 97 active+clean; 588 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.060+0000 7f5b8f877700 1 -- 172.21.15.69:0/360740023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b881037f0 msgr2=0x7f5b88103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.060+0000 7f5b8f877700 1 --2- 172.21.15.69:0/360740023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b881037f0 0x7f5b88103bd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b840099c0 tx=0x7f5b84009cd0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.060+0000 7f5b8f877700 1 -- 172.21.15.69:0/360740023 shutdown_connections 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.060+0000 7f5b8f877700 1 --2- 172.21.15.69:0/360740023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b881037f0 0x7f5b88103bd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.060+0000 7f5b8f877700 1 --2- 172.21.15.69:0/360740023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b881088f0 0x7f5b88110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.060+0000 7f5b8f877700 1 --2- 172.21.15.69:0/360740023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b881041a0 0x7f5b881081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.060+0000 7f5b8f877700 1 -- 172.21.15.69:0/360740023 >> 172.21.15.69:0/360740023 conn(0x7f5b8807a6a0 msgr2=0x7f5b8807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.060+0000 7f5b8f877700 1 -- 172.21.15.69:0/360740023 shutdown_connections 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.060+0000 7f5b8f877700 1 -- 172.21.15.69:0/360740023 wait complete. 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.060+0000 7f5b8f877700 1 Processor -- start 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8f877700 1 -- start start 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8f877700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b881037f0 0x7f5b8819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:14.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8f877700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b881041a0 0x7f5b8819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8f877700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b881088f0 0x7f5b88199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8f877700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5b88113740 con 0x7f5b881088f0 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8f877700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5b881135c0 con 0x7f5b881037f0 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8f877700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b881138c0 con 0x7f5b881041a0 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8de14700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b881088f0 0x7f5b88199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8de14700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b881088f0 0x7f5b88199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39918/0 (socket says 172.21.15.69:39918) 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8de14700 1 -- 172.21.15.69:0/3332794507 learned_addr learned my addr 172.21.15.69:0/3332794507 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8ce12700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b881041a0 0x7f5b8819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8d613700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b881037f0 0x7f5b8819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8de14700 1 -- 172.21.15.69:0/3332794507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b881041a0 msgr2=0x7f5b8819fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8de14700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b881041a0 0x7f5b8819fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8de14700 1 -- 172.21.15.69:0/3332794507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b881037f0 msgr2=0x7f5b8819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8de14700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b881037f0 0x7f5b8819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8de14700 1 -- 172.21.15.69:0/3332794507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8819a160 con 0x7f5b881088f0 2024-05-14T16:47:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.061+0000 7f5b8de14700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b881088f0 0x7f5b88199a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000b970 tx=0x7f5b8000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:14.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.062+0000 7f5b7e7fc700 1 -- 172.21.15.69:0/3332794507 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80019070 con 0x7f5b881088f0 2024-05-14T16:47:14.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.062+0000 7f5b8f877700 1 -- 172.21.15.69:0/3332794507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8819a450 con 0x7f5b881088f0 2024-05-14T16:47:14.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.062+0000 7f5b7e7fc700 1 -- 172.21.15.69:0/3332794507 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b80014440 con 0x7f5b881088f0 2024-05-14T16:47:14.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.062+0000 7f5b7e7fc700 1 -- 172.21.15.69:0/3332794507 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8000ce60 con 0x7f5b881088f0 2024-05-14T16:47:14.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.062+0000 7f5b8f877700 1 -- 172.21.15.69:0/3332794507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b881a6710 con 0x7f5b881088f0 2024-05-14T16:47:14.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.063+0000 7f5b8f877700 1 -- 172.21.15.69:0/3332794507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8810bb30 con 0x7f5b881088f0 2024-05-14T16:47:14.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.064+0000 7f5b7e7fc700 1 -- 172.21.15.69:0/3332794507 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5b8001c070 con 0x7f5b881088f0 2024-05-14T16:47:14.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.065+0000 7f5b7e7fc700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5b740b2990 0x7f5b740b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:14.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.065+0000 7f5b8d613700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5b740b2990 0x7f5b740b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:14.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.065+0000 7f5b7e7fc700 1 -- 172.21.15.69:0/3332794507 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f5b800e15f0 con 0x7f5b881088f0 2024-05-14T16:47:14.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.065+0000 7f5b8d613700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5b740b2990 0x7f5b740b4e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b84000c00 tx=0x7f5b84005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:14.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.069+0000 7f5b7e7fc700 1 -- 172.21.15.69:0/3332794507 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b800a9370 con 0x7f5b881088f0 2024-05-14T16:47:14.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.319+0000 7f5b8f877700 1 -- 172.21.15.69:0/3332794507 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f5b88063980 con 0x7f5b740b2990 2024-05-14T16:47:14.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.325+0000 7f5b7e7fc700 1 -- 172.21.15.69:0/3332794507 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5b88063980 con 0x7f5b740b2990 2024-05-14T16:47:14.325 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/qa/sv_0/e4b4d05c-3ef8-4ac8-aa0c-8f9188561329 2024-05-14T16:47:14.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 -- 172.21.15.69:0/3332794507 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5b740b2990 msgr2=0x7f5b740b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:14.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5b740b2990 0x7f5b740b4e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b84000c00 tx=0x7f5b84005fb0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 -- 172.21.15.69:0/3332794507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b881088f0 msgr2=0x7f5b88199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:14.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b881088f0 0x7f5b88199a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000b970 tx=0x7f5b8000bd30 comp rx=0 tx=0).stop 2024-05-14T16:47:14.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 -- 172.21.15.69:0/3332794507 shutdown_connections 2024-05-14T16:47:14.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5b740b2990 0x7f5b740b4e50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b881088f0 0x7f5b88199a00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b881037f0 0x7f5b8819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 --2- 172.21.15.69:0/3332794507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b881041a0 0x7f5b8819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:14.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 -- 172.21.15.69:0/3332794507 >> 172.21.15.69:0/3332794507 conn(0x7f5b8807a6a0 msgr2=0x7f5b881018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:14.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 -- 172.21.15.69:0/3332794507 shutdown_connections 2024-05-14T16:47:14.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:14.327+0000 7f5b8f877700 1 -- 172.21.15.69:0/3332794507 wait complete. 2024-05-14T16:47:15.129 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-14T16:47:15.376 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:15.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:15 smithi069 ceph-mon[31507]: pgmap v299: 97 pgs: 97 active+clean; 608 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-05-14T16:47:15.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:15 smithi069 ceph-mon[31507]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:15.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:15 smithi203 ceph-mon[35787]: pgmap v299: 97 pgs: 97 active+clean; 608 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-05-14T16:47:15.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:15 smithi203 ceph-mon[35787]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:16.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:15 smithi092 ceph-mon[35763]: pgmap v299: 97 pgs: 97 active+clean; 608 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-05-14T16:47:16.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:15 smithi092 ceph-mon[35763]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:17.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.265+0000 7f2456db7700 1 -- 172.21.15.69:0/4086394996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f245006bc70 msgr2=0x7f245006c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:17.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.265+0000 7f2456db7700 1 --2- 172.21.15.69:0/4086394996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f245006bc70 0x7f245006c0f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2440009990 tx=0x7f2440009ca0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.265+0000 7f2456db7700 1 -- 172.21.15.69:0/4086394996 shutdown_connections 2024-05-14T16:47:17.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.265+0000 7f2456db7700 1 --2- 172.21.15.69:0/4086394996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f245006c630 0x7f24501103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.265+0000 7f2456db7700 1 --2- 172.21.15.69:0/4086394996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24501095b0 0x7f2450109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.265+0000 7f2456db7700 1 --2- 172.21.15.69:0/4086394996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f245006bc70 0x7f245006c0f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.265+0000 7f2456db7700 1 -- 172.21.15.69:0/4086394996 >> 172.21.15.69:0/4086394996 conn(0x7f24500793f0 msgr2=0x7f2450079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.265+0000 7f2456db7700 1 -- 172.21.15.69:0/4086394996 shutdown_connections 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.265+0000 7f2456db7700 1 -- 172.21.15.69:0/4086394996 wait complete. 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2456db7700 1 Processor -- start 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2456db7700 1 -- start start 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2456db7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f245006bc70 0x7f2450199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2456db7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f245006c630 0x7f245019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2456db7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24501095b0 0x7f245019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2456db7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2450113740 con 0x7f245006c630 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2456db7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f24501135c0 con 0x7f245006bc70 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2456db7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f24501138c0 con 0x7f24501095b0 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2447fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f245006c630 0x7f245019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2447fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f245006c630 0x7f245019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53892/0 (socket says 172.21.15.69:53892) 2024-05-14T16:47:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2447fff700 1 -- 172.21.15.69:0/3176270829 learned_addr learned my addr 172.21.15.69:0/3176270829 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2454b53700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f245006bc70 0x7f2450199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2455354700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24501095b0 0x7f245019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2447fff700 1 -- 172.21.15.69:0/3176270829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24501095b0 msgr2=0x7f245019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2447fff700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24501095b0 0x7f245019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2447fff700 1 -- 172.21.15.69:0/3176270829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f245006bc70 msgr2=0x7f2450199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2447fff700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f245006bc70 0x7f2450199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2447fff700 1 -- 172.21.15.69:0/3176270829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245019ed20 con 0x7f245006c630 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.266+0000 7f2447fff700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f245006c630 0x7f245019a250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2440000c00 tx=0x7f24400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.267+0000 7f2445ffb700 1 -- 172.21.15.69:0/3176270829 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440027070 con 0x7f245006c630 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.267+0000 7f2445ffb700 1 -- 172.21.15.69:0/3176270829 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24400040e0 con 0x7f245006c630 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.267+0000 7f2456db7700 1 -- 172.21.15.69:0/3176270829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245019efb0 con 0x7f245006c630 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.267+0000 7f2445ffb700 1 -- 172.21.15.69:0/3176270829 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244000f460 con 0x7f245006c630 2024-05-14T16:47:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.267+0000 7f2456db7700 1 -- 172.21.15.69:0/3176270829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24501a6890 con 0x7f245006c630 2024-05-14T16:47:17.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.268+0000 7f2456db7700 1 -- 172.21.15.69:0/3176270829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245006a230 con 0x7f245006c630 2024-05-14T16:47:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.269+0000 7f2445ffb700 1 -- 172.21.15.69:0/3176270829 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f244002a070 con 0x7f245006c630 2024-05-14T16:47:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.270+0000 7f2445ffb700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24380b2990 0x7f24380b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.270+0000 7f2454b53700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24380b2990 0x7f24380b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.270+0000 7f2445ffb700 1 -- 172.21.15.69:0/3176270829 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f24400e35f0 con 0x7f245006c630 2024-05-14T16:47:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.270+0000 7f2454b53700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24380b2990 0x7f24380b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f244c005950 tx=0x7f244c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:17.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.273+0000 7f2445ffb700 1 -- 172.21.15.69:0/3176270829 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2440031070 con 0x7f245006c630 2024-05-14T16:47:17.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.504+0000 7f2456db7700 1 -- 172.21.15.69:0/3176270829 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f245019b070 con 0x7f24380b2990 2024-05-14T16:47:17.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.530+0000 7f2445ffb700 1 -- 172.21.15.69:0/3176270829 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f245019b070 con 0x7f24380b2990 2024-05-14T16:47:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.532+0000 7f2456db7700 1 -- 172.21.15.69:0/3176270829 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24380b2990 msgr2=0x7f24380b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.532+0000 7f2456db7700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24380b2990 0x7f24380b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f244c005950 tx=0x7f244c00a300 comp rx=0 tx=0).stop 2024-05-14T16:47:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.532+0000 7f2456db7700 1 -- 172.21.15.69:0/3176270829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f245006c630 msgr2=0x7f245019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.532+0000 7f2456db7700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f245006c630 0x7f245019a250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2440000c00 tx=0x7f24400047b0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.532+0000 7f2456db7700 1 -- 172.21.15.69:0/3176270829 shutdown_connections 2024-05-14T16:47:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.532+0000 7f2456db7700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24501095b0 0x7f245019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.532+0000 7f2456db7700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f245006bc70 0x7f2450199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.532+0000 7f2456db7700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24380b2990 0x7f24380b4e50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.532+0000 7f2456db7700 1 --2- 172.21.15.69:0/3176270829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f245006c630 0x7f245019a250 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.533+0000 7f2456db7700 1 -- 172.21.15.69:0/3176270829 >> 172.21.15.69:0/3176270829 conn(0x7f24500793f0 msgr2=0x7f2450101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.533+0000 7f2456db7700 1 -- 172.21.15.69:0/3176270829 shutdown_connections 2024-05-14T16:47:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:17.533+0000 7f2456db7700 1 -- 172.21.15.69:0/3176270829 wait complete. 2024-05-14T16:47:17.893 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:17 smithi069 ceph-mon[31507]: pgmap v300: 97 pgs: 97 active+clean; 608 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-14T16:47:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:17 smithi203 ceph-mon[35787]: pgmap v300: 97 pgs: 97 active+clean; 608 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-14T16:47:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:17 smithi092 ceph-mon[35763]: pgmap v300: 97 pgs: 97 active+clean; 608 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-14T16:47:18.221 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-14T16:47:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:18.847 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:18 smithi069 ceph-mon[31507]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:18.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:18 smithi203 ceph-mon[35787]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:19.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:18 smithi092 ceph-mon[35763]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:19.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:19 smithi069 ceph-mon[31507]: pgmap v301: 97 pgs: 97 active+clean; 608 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-14T16:47:19.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:19 smithi203 ceph-mon[35787]: pgmap v301: 97 pgs: 97 active+clean; 608 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-14T16:47:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:19 smithi092 ceph-mon[35763]: pgmap v301: 97 pgs: 97 active+clean; 608 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-14T16:47:20.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 -- 172.21.15.69:0/2489363797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b181035d0 msgr2=0x7f2b18103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:20.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 --2- 172.21.15.69:0/2489363797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b181035d0 0x7f2b18103a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b040099c0 tx=0x7f2b04009cd0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 -- 172.21.15.69:0/2489363797 shutdown_connections 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 --2- 172.21.15.69:0/2489363797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b18103f90 0x7f2b18110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 --2- 172.21.15.69:0/2489363797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b18109670 0x7f2b18109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 --2- 172.21.15.69:0/2489363797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b181035d0 0x7f2b18103a50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 -- 172.21.15.69:0/2489363797 >> 172.21.15.69:0/2489363797 conn(0x7f2b1807a730 msgr2=0x7f2b1807ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 -- 172.21.15.69:0/2489363797 shutdown_connections 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 -- 172.21.15.69:0/2489363797 wait complete. 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 Processor -- start 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.271+0000 7f2b1f16a700 1 -- start start 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b1f16a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b181035d0 0x7f2b1819fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b1f16a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b18103f90 0x7f2b1819ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b1f16a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b18109670 0x7f2b18199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b1f16a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181137d0 con 0x7f2b18103f90 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b1f16a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18113650 con 0x7f2b181035d0 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b1f16a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18113950 con 0x7f2b18109670 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b17fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b18103f90 0x7f2b1819ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b17fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b18103f90 0x7f2b1819ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53910/0 (socket says 172.21.15.69:53910) 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b17fff700 1 -- 172.21.15.69:0/3395553361 learned_addr learned my addr 172.21.15.69:0/3395553361 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b1cf06700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b181035d0 0x7f2b1819fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b1d707700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b18109670 0x7f2b18199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b17fff700 1 -- 172.21.15.69:0/3395553361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b18109670 msgr2=0x7f2b18199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b17fff700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b18109670 0x7f2b18199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b17fff700 1 -- 172.21.15.69:0/3395553361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b181035d0 msgr2=0x7f2b1819fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b17fff700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b181035d0 0x7f2b1819fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b17fff700 1 -- 172.21.15.69:0/3395553361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1819a3f0 con 0x7f2b18103f90 2024-05-14T16:47:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b1d707700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b18109670 0x7f2b18199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b17fff700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b18103f90 0x7f2b1819ff80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400b580 tx=0x7f2b040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.272+0000 7f2b15ffb700 1 -- 172.21.15.69:0/3395553361 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04027070 con 0x7f2b18103f90 2024-05-14T16:47:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.273+0000 7f2b1f16a700 1 -- 172.21.15.69:0/3395553361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1819a680 con 0x7f2b18103f90 2024-05-14T16:47:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.273+0000 7f2b15ffb700 1 -- 172.21.15.69:0/3395553361 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b040040a0 con 0x7f2b18103f90 2024-05-14T16:47:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.273+0000 7f2b15ffb700 1 -- 172.21.15.69:0/3395553361 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0400f460 con 0x7f2b18103f90 2024-05-14T16:47:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.273+0000 7f2b1f16a700 1 -- 172.21.15.69:0/3395553361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b181a67f0 con 0x7f2b18103f90 2024-05-14T16:47:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.274+0000 7f2b1f16a700 1 -- 172.21.15.69:0/3395553361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b1810bb00 con 0x7f2b18103f90 2024-05-14T16:47:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.275+0000 7f2b15ffb700 1 -- 172.21.15.69:0/3395553361 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2b0402a070 con 0x7f2b18103f90 2024-05-14T16:47:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.276+0000 7f2b15ffb700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2b080b2910 0x7f2b080b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.276+0000 7f2b1cf06700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2b080b2910 0x7f2b080b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.276+0000 7f2b15ffb700 1 -- 172.21.15.69:0/3395553361 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f2b040e3760 con 0x7f2b18103f90 2024-05-14T16:47:20.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.277+0000 7f2b1cf06700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2b080b2910 0x7f2b080b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00a400 tx=0x7f2b0c00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:20.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.280+0000 7f2b15ffb700 1 -- 172.21.15.69:0/3395553361 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2b04031070 con 0x7f2b18103f90 2024-05-14T16:47:20.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.518+0000 7f2b1f16a700 1 -- 172.21.15.69:0/3395553361 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f2b1819afd0 con 0x7f2b080b2910 2024-05-14T16:47:20.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.523+0000 7f2b15ffb700 1 -- 172.21.15.69:0/3395553361 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2b1819afd0 con 0x7f2b080b2910 2024-05-14T16:47:20.523 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/qa/sv_1/265d528f-6ed7-4d79-b5c4-99c32193f887 2024-05-14T16:47:20.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 -- 172.21.15.69:0/3395553361 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2b080b2910 msgr2=0x7f2b080b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:20.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2b080b2910 0x7f2b080b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00a400 tx=0x7f2b0c00d040 comp rx=0 tx=0).stop 2024-05-14T16:47:20.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 -- 172.21.15.69:0/3395553361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b18103f90 msgr2=0x7f2b1819ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:20.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b18103f90 0x7f2b1819ff80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400b580 tx=0x7f2b040047b0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 -- 172.21.15.69:0/3395553361 shutdown_connections 2024-05-14T16:47:20.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b18109670 0x7f2b18199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b181035d0 0x7f2b1819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f2b080b2910 0x7f2b080b4dd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 --2- 172.21.15.69:0/3395553361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b18103f90 0x7f2b1819ff80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:20.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 -- 172.21.15.69:0/3395553361 >> 172.21.15.69:0/3395553361 conn(0x7f2b1807a730 msgr2=0x7f2b181022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:20.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 -- 172.21.15.69:0/3395553361 shutdown_connections 2024-05-14T16:47:20.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:20.525+0000 7f2b1f16a700 1 -- 172.21.15.69:0/3395553361 wait complete. 2024-05-14T16:47:21.145 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:47:21.384 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:21.739 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:21 smithi069 ceph-mon[31507]: pgmap v302: 97 pgs: 97 active+clean; 632 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-14T16:47:21.740 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:21 smithi069 ceph-mon[31507]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:21.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:21 smithi203 ceph-mon[35787]: pgmap v302: 97 pgs: 97 active+clean; 632 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-14T16:47:21.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:21 smithi203 ceph-mon[35787]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:22.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:21 smithi092 ceph-mon[35763]: pgmap v302: 97 pgs: 97 active+clean; 632 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-14T16:47:22.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:21 smithi092 ceph-mon[35763]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.107+0000 7f40954d1700 1 -- 172.21.15.69:0/2890452048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4090103f50 msgr2=0x7f4090110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.107+0000 7f40954d1700 1 --2- 172.21.15.69:0/2890452048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4090103f50 0x7f4090110430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f408800b0a0 tx=0x7f408800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.107+0000 7f40954d1700 1 -- 172.21.15.69:0/2890452048 shutdown_connections 2024-05-14T16:47:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.107+0000 7f40954d1700 1 --2- 172.21.15.69:0/2890452048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4090103f50 0x7f4090110430 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.107+0000 7f40954d1700 1 --2- 172.21.15.69:0/2890452048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40901096a0 0x7f4090109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.107+0000 7f40954d1700 1 --2- 172.21.15.69:0/2890452048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4090103590 0x7f4090103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.107+0000 7f40954d1700 1 -- 172.21.15.69:0/2890452048 >> 172.21.15.69:0/2890452048 conn(0x7f409007a6f0 msgr2=0x7f409007ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.107+0000 7f40954d1700 1 -- 172.21.15.69:0/2890452048 shutdown_connections 2024-05-14T16:47:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.107+0000 7f40954d1700 1 -- 172.21.15.69:0/2890452048 wait complete. 2024-05-14T16:47:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f40954d1700 1 Processor -- start 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f40954d1700 1 -- start start 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f40954d1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4090103590 0x7f409019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f40954d1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4090103f50 0x7f409019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f40954d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40901096a0 0x7f40901999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f40954d1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4090113770 con 0x7f4090103590 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f40954d1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f40901135f0 con 0x7f40901096a0 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f40954d1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f40901138f0 con 0x7f4090103f50 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f408effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4090103590 0x7f409019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f408effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4090103590 0x7f409019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53938/0 (socket says 172.21.15.69:53938) 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f408effd700 1 -- 172.21.15.69:0/901454092 learned_addr learned my addr 172.21.15.69:0/901454092 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f40867fc700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4090103f50 0x7f409019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f408f7fe700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40901096a0 0x7f40901999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f408effd700 1 -- 172.21.15.69:0/901454092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4090103f50 msgr2=0x7f409019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f408effd700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4090103f50 0x7f409019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f408effd700 1 -- 172.21.15.69:0/901454092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40901096a0 msgr2=0x7f40901999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f408effd700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40901096a0 0x7f40901999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f408effd700 1 -- 172.21.15.69:0/901454092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409019a130 con 0x7f4090103590 2024-05-14T16:47:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.108+0000 7f408f7fe700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40901096a0 0x7f40901999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.109+0000 7f408effd700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4090103590 0x7f409019f810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f408000b530 tx=0x7f408000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.109+0000 7f408cff9700 1 -- 172.21.15.69:0/901454092 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080015020 con 0x7f4090103590 2024-05-14T16:47:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.109+0000 7f40954d1700 1 -- 172.21.15.69:0/901454092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409019a420 con 0x7f4090103590 2024-05-14T16:47:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.109+0000 7f408cff9700 1 -- 172.21.15.69:0/901454092 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4080011ce0 con 0x7f4090103590 2024-05-14T16:47:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.109+0000 7f408cff9700 1 -- 172.21.15.69:0/901454092 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080010e40 con 0x7f4090103590 2024-05-14T16:47:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.109+0000 7f40954d1700 1 -- 172.21.15.69:0/901454092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40901a66e0 con 0x7f4090103590 2024-05-14T16:47:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.110+0000 7f40954d1700 1 -- 172.21.15.69:0/901454092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409006a230 con 0x7f4090103590 2024-05-14T16:47:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.111+0000 7f408cff9700 1 -- 172.21.15.69:0/901454092 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4080018020 con 0x7f4090103590 2024-05-14T16:47:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.112+0000 7f408cff9700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f407c0b2910 0x7f407c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.112+0000 7f408cff9700 1 -- 172.21.15.69:0/901454092 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f40800e1ed0 con 0x7f4090103590 2024-05-14T16:47:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.112+0000 7f40867fc700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f407c0b2910 0x7f407c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.112+0000 7f40867fc700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f407c0b2910 0x7f407c0b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f409007ac50 tx=0x7f407800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:23.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.115+0000 7f408cff9700 1 -- 172.21.15.69:0/901454092 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f408001f020 con 0x7f4090103590 2024-05-14T16:47:23.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.364+0000 7f40954d1700 1 -- 172.21.15.69:0/901454092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f409019af00 con 0x7f4090103590 2024-05-14T16:47:23.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.365+0000 7f408cff9700 1 -- 172.21.15.69:0/901454092 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7f40800a9d50 con 0x7f4090103590 2024-05-14T16:47:23.365 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:47:23.365 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:47:04.867362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24433,24446,24422]},"id":1}]} 2024-05-14T16:47:23.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.367+0000 7f40954d1700 1 -- 172.21.15.69:0/901454092 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f407c0b2910 msgr2=0x7f407c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:23.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.367+0000 7f40954d1700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f407c0b2910 0x7f407c0b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f409007ac50 tx=0x7f407800a380 comp rx=0 tx=0).stop 2024-05-14T16:47:23.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.367+0000 7f40954d1700 1 -- 172.21.15.69:0/901454092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4090103590 msgr2=0x7f409019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.367+0000 7f40954d1700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4090103590 0x7f409019f810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f408000b530 tx=0x7f408000b840 comp rx=0 tx=0).stop 2024-05-14T16:47:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.368+0000 7f40954d1700 1 -- 172.21.15.69:0/901454092 shutdown_connections 2024-05-14T16:47:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.368+0000 7f40954d1700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f407c0b2910 0x7f407c0b4dd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.368+0000 7f40954d1700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4090103590 0x7f409019f810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.368+0000 7f40954d1700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40901096a0 0x7f40901999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.368+0000 7f40954d1700 1 --2- 172.21.15.69:0/901454092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4090103f50 0x7f409019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.368+0000 7f40954d1700 1 -- 172.21.15.69:0/901454092 >> 172.21.15.69:0/901454092 conn(0x7f409007a6f0 msgr2=0x7f4090101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.368+0000 7f40954d1700 1 -- 172.21.15.69:0/901454092 shutdown_connections 2024-05-14T16:47:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:23.368+0000 7f40954d1700 1 -- 172.21.15.69:0/901454092 wait complete. 2024-05-14T16:47:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-05-14T16:47:23.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:23 smithi069 ceph-mon[31507]: pgmap v303: 97 pgs: 97 active+clean; 632 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-05-14T16:47:23.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:23 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/901454092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:47:23.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:23 smithi203 ceph-mon[35787]: pgmap v303: 97 pgs: 97 active+clean; 632 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-05-14T16:47:23.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:23 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/901454092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:47:23.996 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:47:24.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:23 smithi092 ceph-mon[35763]: pgmap v303: 97 pgs: 97 active+clean; 632 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-05-14T16:47:24.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:23 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/901454092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:47:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:25.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2406942820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10ec108730 msgr2=0x7f10ec108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:25.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 --2- 172.21.15.69:0/2406942820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10ec108730 0x7f10ec108b10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f10e00099c0 tx=0x7f10e0009cd0 comp rx=0 tx=0).stop 2024-05-14T16:47:25.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2406942820 shutdown_connections 2024-05-14T16:47:25.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 --2- 172.21.15.69:0/2406942820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10ec108730 0x7f10ec108b10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:25.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 --2- 172.21.15.69:0/2406942820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec1039d0 0x7f10ec10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 --2- 172.21.15.69:0/2406942820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec103010 0x7f10ec103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2406942820 >> 172.21.15.69:0/2406942820 conn(0x7f10ec077780 msgr2=0x7f10ec077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2406942820 shutdown_connections 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2406942820 wait complete. 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 Processor -- start 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 -- start start 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.898+0000 7f10f2cc0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec103010 0x7f10ec199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10f2cc0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10ec1039d0 0x7f10ec19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10f2cc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec108730 0x7f10ec19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10f2cc0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec1127b0 con 0x7f10ec1039d0 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10f2cc0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec112630 con 0x7f10ec108730 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10f2cc0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec112930 con 0x7f10ec103010 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10ebfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10ec1039d0 0x7f10ec19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:25.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10ebfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10ec1039d0 0x7f10ec19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35240/0 (socket says 172.21.15.69:35240) 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10ebfff700 1 -- 172.21.15.69:0/2123218336 learned_addr learned my addr 172.21.15.69:0/2123218336 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10f125d700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec108730 0x7f10ec19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10f0a5c700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec103010 0x7f10ec199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10ebfff700 1 -- 172.21.15.69:0/2123218336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec103010 msgr2=0x7f10ec199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10ebfff700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec103010 0x7f10ec199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10ebfff700 1 -- 172.21.15.69:0/2123218336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec108730 msgr2=0x7f10ec19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10ebfff700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec108730 0x7f10ec19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10ebfff700 1 -- 172.21.15.69:0/2123218336 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec19ede0 con 0x7f10ec1039d0 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10f0a5c700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec103010 0x7f10ec199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10ebfff700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10ec1039d0 0x7f10ec19a310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10d800b530 tx=0x7f10d800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10e9ffb700 1 -- 172.21.15.69:0/2123218336 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8015020 con 0x7f10ec1039d0 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10e9ffb700 1 -- 172.21.15.69:0/2123218336 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d8011ce0 con 0x7f10ec1039d0 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2123218336 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec19f0d0 con 0x7f10ec1039d0 2024-05-14T16:47:25.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.899+0000 7f10e9ffb700 1 -- 172.21.15.69:0/2123218336 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8010d20 con 0x7f10ec1039d0 2024-05-14T16:47:25.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.900+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2123218336 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec1a6950 con 0x7f10ec1039d0 2024-05-14T16:47:25.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.900+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2123218336 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ec06a230 con 0x7f10ec1039d0 2024-05-14T16:47:25.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.902+0000 7f10e9ffb700 1 -- 172.21.15.69:0/2123218336 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f10d8018020 con 0x7f10ec1039d0 2024-05-14T16:47:25.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.902+0000 7f10e9ffb700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f10dc0b2910 0x7f10dc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:25.903 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:25 smithi069 ceph-mon[31507]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-05-14T16:47:25.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.902+0000 7f10f0a5c700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f10dc0b2910 0x7f10dc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:25.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.902+0000 7f10e9ffb700 1 -- 172.21.15.69:0/2123218336 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f10d80e1920 con 0x7f10ec1039d0 2024-05-14T16:47:25.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.903+0000 7f10f0a5c700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f10dc0b2910 0x7f10dc0b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f10e0003800 tx=0x7f10e0009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:25.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:25.909+0000 7f10e9ffb700 1 -- 172.21.15.69:0/2123218336 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10d801f020 con 0x7f10ec1039d0 2024-05-14T16:47:25.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:25 smithi203 ceph-mon[35787]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-05-14T16:47:26.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:25 smithi092 ceph-mon[35763]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-05-14T16:47:26.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.113+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2123218336 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10ec19b130 con 0x7f10ec1039d0 2024-05-14T16:47:26.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.114+0000 7f10e9ffb700 1 -- 172.21.15.69:0/2123218336 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22183 (secure 0 0 0) 0x7f10d80a97a0 con 0x7f10ec1039d0 2024-05-14T16:47:26.114 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:47:26.118 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":78,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T16:46:45.333145+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:46:38.843325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:46:42.395401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:47:26.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2123218336 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f10dc0b2910 msgr2=0x7f10dc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:26.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f10dc0b2910 0x7f10dc0b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f10e0003800 tx=0x7f10e0009f10 comp rx=0 tx=0).stop 2024-05-14T16:47:26.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2123218336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10ec1039d0 msgr2=0x7f10ec19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:26.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10ec1039d0 0x7f10ec19a310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10d800b530 tx=0x7f10d800b8f0 comp rx=0 tx=0).stop 2024-05-14T16:47:26.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2123218336 shutdown_connections 2024-05-14T16:47:26.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec108730 0x7f10ec19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:26.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec103010 0x7f10ec199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:26.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f10dc0b2910 0x7f10dc0b4dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:26.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 --2- 172.21.15.69:0/2123218336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10ec1039d0 0x7f10ec19a310 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:26.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2123218336 >> 172.21.15.69:0/2123218336 conn(0x7f10ec077780 msgr2=0x7f10ec1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:26.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2123218336 shutdown_connections 2024-05-14T16:47:26.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:26.116+0000 7f10f2cc0700 1 -- 172.21.15.69:0/2123218336 wait complete. 2024-05-14T16:47:26.753 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:26 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2123218336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:47:26.781 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T16:47:26.781 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:47:26.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:26 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2123218336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:47:27.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:26 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2123218336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:47:27.020 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:27.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:27 smithi069 ceph-mon[31507]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-05-14T16:47:27.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:27 smithi203 ceph-mon[35787]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-05-14T16:47:28.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:27 smithi092 ceph-mon[35763]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-05-14T16:47:28.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.659+0000 7ff9cba6f700 1 -- 172.21.15.69:0/2183307750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9c4102440 msgr2=0x7ff9c41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:28.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.659+0000 7ff9cba6f700 1 --2- 172.21.15.69:0/2183307750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9c4102440 0x7ff9c41028c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009990 tx=0x7ff9b4009ca0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.659+0000 7ff9cba6f700 1 -- 172.21.15.69:0/2183307750 shutdown_connections 2024-05-14T16:47:28.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 --2- 172.21.15.69:0/2183307750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c4102e00 0x7ff9c410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 --2- 172.21.15.69:0/2183307750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c4108520 0x7ff9c4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 --2- 172.21.15.69:0/2183307750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9c4102440 0x7ff9c41028c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 -- 172.21.15.69:0/2183307750 >> 172.21.15.69:0/2183307750 conn(0x7ff9c40793f0 msgr2=0x7ff9c4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:28.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 -- 172.21.15.69:0/2183307750 shutdown_connections 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 -- 172.21.15.69:0/2183307750 wait complete. 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 Processor -- start 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 -- start start 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c4102440 0x7ff9c419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9c4102e00 0x7ff9c419afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c4108520 0x7ff9c41a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41125f0 con 0x7ff9c4102e00 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4112470 con 0x7ff9c4108520 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.660+0000 7ff9cba6f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4112770 con 0x7ff9c4102440 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c900a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9c4102e00 0x7ff9c419afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c900a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9c4102e00 0x7ff9c419afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35270/0 (socket says 172.21.15.69:35270) 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c900a700 1 -- 172.21.15.69:0/3340589537 learned_addr learned my addr 172.21.15.69:0/3340589537 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c980b700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c4102440 0x7ff9c419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:28.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9ca00c700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c4108520 0x7ff9c41a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c900a700 1 -- 172.21.15.69:0/3340589537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c4102440 msgr2=0x7ff9c419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c900a700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c4102440 0x7ff9c419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c900a700 1 -- 172.21.15.69:0/3340589537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c4108520 msgr2=0x7ff9c41a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c900a700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c4108520 0x7ff9c41a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c900a700 1 -- 172.21.15.69:0/3340589537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c41a6030 con 0x7ff9c4102e00 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9ca00c700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c4108520 0x7ff9c41a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c980b700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c4102440 0x7ff9c419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9c900a700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9c4102e00 0x7ff9c419afb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4000c00 tx=0x7ff9b4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9baffd700 1 -- 172.21.15.69:0/3340589537 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4027070 con 0x7ff9c4102e00 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9baffd700 1 -- 172.21.15.69:0/3340589537 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b4022470 con 0x7ff9c4102e00 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9baffd700 1 -- 172.21.15.69:0/3340589537 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4017650 con 0x7ff9c4102e00 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9cba6f700 1 -- 172.21.15.69:0/3340589537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c41a6200 con 0x7ff9c4102e00 2024-05-14T16:47:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.661+0000 7ff9cba6f700 1 -- 172.21.15.69:0/3340589537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41a66b0 con 0x7ff9c4102e00 2024-05-14T16:47:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.662+0000 7ff9cba6f700 1 -- 172.21.15.69:0/3340589537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c406a230 con 0x7ff9c4102e00 2024-05-14T16:47:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.664+0000 7ff9baffd700 1 -- 172.21.15.69:0/3340589537 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff9b402a070 con 0x7ff9c4102e00 2024-05-14T16:47:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.664+0000 7ff9baffd700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff9b00b2990 0x7ff9b00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:28.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.664+0000 7ff9c980b700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff9b00b2990 0x7ff9b00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:28.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.664+0000 7ff9baffd700 1 -- 172.21.15.69:0/3340589537 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7ff9b40e35c0 con 0x7ff9c4102e00 2024-05-14T16:47:28.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.665+0000 7ff9c980b700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff9b00b2990 0x7ff9b00b4e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005e10 tx=0x7ff9c0005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:28.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.668+0000 7ff9baffd700 1 -- 172.21.15.69:0/3340589537 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9b4013070 con 0x7ff9c4102e00 2024-05-14T16:47:28.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.870+0000 7ff9cba6f700 1 -- 172.21.15.69:0/3340589537 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9c406b810 con 0x7ff9b00b2990 2024-05-14T16:47:28.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.878+0000 7ff9baffd700 1 -- 172.21.15.69:0/3340589537 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258830 (secure 0 0 0) 0x7ff9c406b810 con 0x7ff9b00b2990 2024-05-14T16:47:28.883 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:47:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 -- 172.21.15.69:0/3340589537 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff9b00b2990 msgr2=0x7ff9b00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff9b00b2990 0x7ff9b00b4e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005e10 tx=0x7ff9c0005d60 comp rx=0 tx=0).stop 2024-05-14T16:47:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 -- 172.21.15.69:0/3340589537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9c4102e00 msgr2=0x7ff9c419afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9c4102e00 0x7ff9c419afb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4000c00 tx=0x7ff9b4009580 comp rx=0 tx=0).stop 2024-05-14T16:47:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 -- 172.21.15.69:0/3340589537 shutdown_connections 2024-05-14T16:47:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c4108520 0x7ff9c41a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c4102440 0x7ff9c419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff9b00b2990 0x7ff9b00b4e50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 --2- 172.21.15.69:0/3340589537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9c4102e00 0x7ff9c419afb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 -- 172.21.15.69:0/3340589537 >> 172.21.15.69:0/3340589537 conn(0x7ff9c40793f0 msgr2=0x7ff9c4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 -- 172.21.15.69:0/3340589537 shutdown_connections 2024-05-14T16:47:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:28.881+0000 7ff9cba6f700 1 -- 172.21.15.69:0/3340589537 wait complete. 2024-05-14T16:47:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-05-14T16:47:29.539 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-05-14T16:47:27.868248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651705,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":246,"num_read_kb":211,"num_write":284,"num_write_kb":2258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351180,"kb_used_data":10348,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590644,"statfs":{"total":1151940427776,"available":1151580819456,"internally_reserved":0,"allocated":10596352,"data_stored":6834847,"data_compressed":107202,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29527,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":69,"num_read_kb":58,"num_write":21,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002903"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572834+0000","last_change":"2024-05-14T16:46:43.572738+0000","last_active":"2024-05-14T16:46:45.572834+0000","last_peered":"2024-05-14T16:46:45.572834+0000","last_clean":"2024-05-14T16:46:45.572834+0000","last_became_active":"2024-05-14T16:46:43.572400+0000","last_became_peered":"2024-05-14T16:46:43.572400+0000","last_unstale":"2024-05-14T16:46:45.572834+0000","last_undegraded":"2024-05-14T16:46:45.572834+0000","last_fullsized":"2024-05-14T16:46:45.572834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:44.332862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572468+0000","last_change":"2024-05-14T16:46:43.566193+0000","last_active":"2024-05-14T16:46:45.572468+0000","last_peered":"2024-05-14T16:46:45.572468+0000","last_clean":"2024-05-14T16:46:45.572468+0000","last_became_active":"2024-05-14T16:46:43.565853+0000","last_became_peered":"2024-05-14T16:46:43.565853+0000","last_unstale":"2024-05-14T16:46:45.572468+0000","last_undegraded":"2024-05-14T16:46:45.572468+0000","last_fullsized":"2024-05-14T16:46:45.572468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:19:34.481225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573087+0000","last_change":"2024-05-14T16:46:43.574937+0000","last_active":"2024-05-14T16:46:45.573087+0000","last_peered":"2024-05-14T16:46:45.573087+0000","last_clean":"2024-05-14T16:46:45.573087+0000","last_became_active":"2024-05-14T16:46:43.574317+0000","last_became_peered":"2024-05-14T16:46:43.574317+0000","last_unstale":"2024-05-14T16:46:45.573087+0000","last_undegraded":"2024-05-14T16:46:45.573087+0000","last_fullsized":"2024-05-14T16:46:45.573087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:35:49.933658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.565158+0000","last_change":"2024-05-14T16:46:43.577130+0000","last_active":"2024-05-14T16:46:45.565158+0000","last_peered":"2024-05-14T16:46:45.565158+0000","last_clean":"2024-05-14T16:46:45.565158+0000","last_became_active":"2024-05-14T16:46:43.576725+0000","last_became_peered":"2024-05-14T16:46:43.576725+0000","last_unstale":"2024-05-14T16:46:45.565158+0000","last_undegraded":"2024-05-14T16:46:45.565158+0000","last_fullsized":"2024-05-14T16:46:45.565158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:24:55.724298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573648+0000","last_change":"2024-05-14T16:46:43.567104+0000","last_active":"2024-05-14T16:46:45.573648+0000","last_peered":"2024-05-14T16:46:45.573648+0000","last_clean":"2024-05-14T16:46:45.573648+0000","last_became_active":"2024-05-14T16:46:43.566762+0000","last_became_peered":"2024-05-14T16:46:43.566762+0000","last_unstale":"2024-05-14T16:46:45.573648+0000","last_undegraded":"2024-05-14T16:46:45.573648+0000","last_fullsized":"2024-05-14T16:46:45.573648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:08.819856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.570819+0000","last_change":"2024-05-14T16:46:43.565055+0000","last_active":"2024-05-14T16:46:45.570819+0000","last_peered":"2024-05-14T16:46:45.570819+0000","last_clean":"2024-05-14T16:46:45.570819+0000","last_became_active":"2024-05-14T16:46:43.564699+0000","last_became_peered":"2024-05-14T16:46:43.564699+0000","last_unstale":"2024-05-14T16:46:45.570819+0000","last_undegraded":"2024-05-14T16:46:45.570819+0000","last_fullsized":"2024-05-14T16:46:45.570819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:01.903025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571963+0000","last_change":"2024-05-14T16:46:43.568607+0000","last_active":"2024-05-14T16:46:45.571963+0000","last_peered":"2024-05-14T16:46:45.571963+0000","last_clean":"2024-05-14T16:46:45.571963+0000","last_became_active":"2024-05-14T16:46:43.568195+0000","last_became_peered":"2024-05-14T16:46:43.568195+0000","last_unstale":"2024-05-14T16:46:45.571963+0000","last_undegraded":"2024-05-14T16:46:45.571963+0000","last_fullsized":"2024-05-14T16:46:45.571963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:31:58.526198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571533+0000","last_change":"2024-05-14T16:46:43.564470+0000","last_active":"2024-05-14T16:46:45.571533+0000","last_peered":"2024-05-14T16:46:45.571533+0000","last_clean":"2024-05-14T16:46:45.571533+0000","last_became_active":"2024-05-14T16:46:43.564169+0000","last_became_peered":"2024-05-14T16:46:43.564169+0000","last_unstale":"2024-05-14T16:46:45.571533+0000","last_undegraded":"2024-05-14T16:46:45.571533+0000","last_fullsized":"2024-05-14T16:46:45.571533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:38.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.577057+0000","last_change":"2024-05-14T16:46:43.580988+0000","last_active":"2024-05-14T16:46:45.577057+0000","last_peered":"2024-05-14T16:46:45.577057+0000","last_clean":"2024-05-14T16:46:45.577057+0000","last_became_active":"2024-05-14T16:46:43.580657+0000","last_became_peered":"2024-05-14T16:46:43.580657+0000","last_unstale":"2024-05-14T16:46:45.577057+0000","last_undegraded":"2024-05-14T16:46:45.577057+0000","last_fullsized":"2024-05-14T16:46:45.577057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:39:01.568822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572643+0000","last_change":"2024-05-14T16:46:43.565742+0000","last_active":"2024-05-14T16:46:45.572643+0000","last_peered":"2024-05-14T16:46:45.572643+0000","last_clean":"2024-05-14T16:46:45.572643+0000","last_became_active":"2024-05-14T16:46:43.565260+0000","last_became_peered":"2024-05-14T16:46:43.565260+0000","last_unstale":"2024-05-14T16:46:45.572643+0000","last_undegraded":"2024-05-14T16:46:45.572643+0000","last_fullsized":"2024-05-14T16:46:45.572643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:25.718174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571632+0000","last_change":"2024-05-14T16:46:43.576268+0000","last_active":"2024-05-14T16:46:45.571632+0000","last_peered":"2024-05-14T16:46:45.571632+0000","last_clean":"2024-05-14T16:46:45.571632+0000","last_became_active":"2024-05-14T16:46:43.575856+0000","last_became_peered":"2024-05-14T16:46:43.575856+0000","last_unstale":"2024-05-14T16:46:45.571632+0000","last_undegraded":"2024-05-14T16:46:45.571632+0000","last_fullsized":"2024-05-14T16:46:45.571632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:12:16.549127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.565360+0000","last_change":"2024-05-14T16:46:43.573315+0000","last_active":"2024-05-14T16:46:45.565360+0000","last_peered":"2024-05-14T16:46:45.565360+0000","last_clean":"2024-05-14T16:46:45.565360+0000","last_became_active":"2024-05-14T16:46:43.572981+0000","last_became_peered":"2024-05-14T16:46:43.572981+0000","last_unstale":"2024-05-14T16:46:45.565360+0000","last_undegraded":"2024-05-14T16:46:45.565360+0000","last_fullsized":"2024-05-14T16:46:45.565360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:07.757539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564359+0000","last_change":"2024-05-14T16:46:43.569388+0000","last_active":"2024-05-14T16:46:45.564359+0000","last_peered":"2024-05-14T16:46:45.564359+0000","last_clean":"2024-05-14T16:46:45.564359+0000","last_became_active":"2024-05-14T16:46:43.568992+0000","last_became_peered":"2024-05-14T16:46:43.568992+0000","last_unstale":"2024-05-14T16:46:45.564359+0000","last_undegraded":"2024-05-14T16:46:45.564359+0000","last_fullsized":"2024-05-14T16:46:45.564359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:08:06.236797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572162+0000","last_change":"2024-05-14T16:46:43.564137+0000","last_active":"2024-05-14T16:46:45.572162+0000","last_peered":"2024-05-14T16:46:45.572162+0000","last_clean":"2024-05-14T16:46:45.572162+0000","last_became_active":"2024-05-14T16:46:43.563831+0000","last_became_peered":"2024-05-14T16:46:43.563831+0000","last_unstale":"2024-05-14T16:46:45.572162+0000","last_undegraded":"2024-05-14T16:46:45.572162+0000","last_fullsized":"2024-05-14T16:46:45.572162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:58.473863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571169+0000","last_change":"2024-05-14T16:46:43.571789+0000","last_active":"2024-05-14T16:46:45.571169+0000","last_peered":"2024-05-14T16:46:45.571169+0000","last_clean":"2024-05-14T16:46:45.571169+0000","last_became_active":"2024-05-14T16:46:43.571521+0000","last_became_peered":"2024-05-14T16:46:43.571521+0000","last_unstale":"2024-05-14T16:46:45.571169+0000","last_undegraded":"2024-05-14T16:46:45.571169+0000","last_fullsized":"2024-05-14T16:46:45.571169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:23.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575910+0000","last_change":"2024-05-14T16:46:43.569532+0000","last_active":"2024-05-14T16:46:45.575910+0000","last_peered":"2024-05-14T16:46:45.575910+0000","last_clean":"2024-05-14T16:46:45.575910+0000","last_became_active":"2024-05-14T16:46:43.569132+0000","last_became_peered":"2024-05-14T16:46:43.569132+0000","last_unstale":"2024-05-14T16:46:45.575910+0000","last_undegraded":"2024-05-14T16:46:45.575910+0000","last_fullsized":"2024-05-14T16:46:45.575910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:09.328241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573018+0000","last_change":"2024-05-14T16:46:43.574451+0000","last_active":"2024-05-14T16:46:45.573018+0000","last_peered":"2024-05-14T16:46:45.573018+0000","last_clean":"2024-05-14T16:46:45.573018+0000","last_became_active":"2024-05-14T16:46:43.574143+0000","last_became_peered":"2024-05-14T16:46:43.574143+0000","last_unstale":"2024-05-14T16:46:45.573018+0000","last_undegraded":"2024-05-14T16:46:45.573018+0000","last_fullsized":"2024-05-14T16:46:45.573018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:01:42.831262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575400+0000","last_change":"2024-05-14T16:46:43.568799+0000","last_active":"2024-05-14T16:46:45.575400+0000","last_peered":"2024-05-14T16:46:45.575400+0000","last_clean":"2024-05-14T16:46:45.575400+0000","last_became_active":"2024-05-14T16:46:43.568467+0000","last_became_peered":"2024-05-14T16:46:43.568467+0000","last_unstale":"2024-05-14T16:46:45.575400+0000","last_undegraded":"2024-05-14T16:46:45.575400+0000","last_fullsized":"2024-05-14T16:46:45.575400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:59.036372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572053+0000","last_change":"2024-05-14T16:46:43.566879+0000","last_active":"2024-05-14T16:46:45.572053+0000","last_peered":"2024-05-14T16:46:45.572053+0000","last_clean":"2024-05-14T16:46:45.572053+0000","last_became_active":"2024-05-14T16:46:43.566346+0000","last_became_peered":"2024-05-14T16:46:43.566346+0000","last_unstale":"2024-05-14T16:46:45.572053+0000","last_undegraded":"2024-05-14T16:46:45.572053+0000","last_fullsized":"2024-05-14T16:46:45.572053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:11.027173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.537507+0000","last_change":"2024-05-14T16:46:40.546990+0000","last_active":"2024-05-14T16:46:59.537507+0000","last_peered":"2024-05-14T16:46:59.537507+0000","last_clean":"2024-05-14T16:46:59.537507+0000","last_became_active":"2024-05-14T16:46:40.546646+0000","last_became_peered":"2024-05-14T16:46:40.546646+0000","last_unstale":"2024-05-14T16:46:59.537507+0000","last_undegraded":"2024-05-14T16:46:59.537507+0000","last_fullsized":"2024-05-14T16:46:59.537507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:47.001760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571591+0000","last_change":"2024-05-14T16:46:43.570352+0000","last_active":"2024-05-14T16:46:45.571591+0000","last_peered":"2024-05-14T16:46:45.571591+0000","last_clean":"2024-05-14T16:46:45.571591+0000","last_became_active":"2024-05-14T16:46:43.569748+0000","last_became_peered":"2024-05-14T16:46:43.569748+0000","last_unstale":"2024-05-14T16:46:45.571591+0000","last_undegraded":"2024-05-14T16:46:45.571591+0000","last_fullsized":"2024-05-14T16:46:45.571591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:24.707586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575687+0000","last_change":"2024-05-14T16:46:40.855034+0000","last_active":"2024-05-14T16:46:45.575687+0000","last_peered":"2024-05-14T16:46:45.575687+0000","last_clean":"2024-05-14T16:46:45.575687+0000","last_became_active":"2024-05-14T16:46:40.854686+0000","last_became_peered":"2024-05-14T16:46:40.854686+0000","last_unstale":"2024-05-14T16:46:45.575687+0000","last_undegraded":"2024-05-14T16:46:45.575687+0000","last_fullsized":"2024-05-14T16:46:45.575687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:56.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573497+0000","last_change":"2024-05-14T16:46:43.574682+0000","last_active":"2024-05-14T16:46:45.573497+0000","last_peered":"2024-05-14T16:46:45.573497+0000","last_clean":"2024-05-14T16:46:45.573497+0000","last_became_active":"2024-05-14T16:46:43.574322+0000","last_became_peered":"2024-05-14T16:46:43.574322+0000","last_unstale":"2024-05-14T16:46:45.573497+0000","last_undegraded":"2024-05-14T16:46:45.573497+0000","last_fullsized":"2024-05-14T16:46:45.573497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:20.169428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.537610+0000","last_change":"2024-05-14T16:46:40.551307+0000","last_active":"2024-05-14T16:46:59.537610+0000","last_peered":"2024-05-14T16:46:59.537610+0000","last_clean":"2024-05-14T16:46:59.537610+0000","last_became_active":"2024-05-14T16:46:40.550987+0000","last_became_peered":"2024-05-14T16:46:40.550987+0000","last_unstale":"2024-05-14T16:46:59.537610+0000","last_undegraded":"2024-05-14T16:46:59.537610+0000","last_fullsized":"2024-05-14T16:46:59.537610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:27.819457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575151+0000","last_change":"2024-05-14T16:46:43.575200+0000","last_active":"2024-05-14T16:46:45.575151+0000","last_peered":"2024-05-14T16:46:45.575151+0000","last_clean":"2024-05-14T16:46:45.575151+0000","last_became_active":"2024-05-14T16:46:43.574864+0000","last_became_peered":"2024-05-14T16:46:43.574864+0000","last_unstale":"2024-05-14T16:46:45.575151+0000","last_undegraded":"2024-05-14T16:46:45.575151+0000","last_fullsized":"2024-05-14T16:46:45.575151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:48.630101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.578732+0000","last_change":"2024-05-14T16:46:40.548397+0000","last_active":"2024-05-14T16:46:45.578732+0000","last_peered":"2024-05-14T16:46:45.578732+0000","last_clean":"2024-05-14T16:46:45.578732+0000","last_became_active":"2024-05-14T16:46:40.547803+0000","last_became_peered":"2024-05-14T16:46:40.547803+0000","last_unstale":"2024-05-14T16:46:45.578732+0000","last_undegraded":"2024-05-14T16:46:45.578732+0000","last_fullsized":"2024-05-14T16:46:45.578732+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:21:19.812413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564491+0000","last_change":"2024-05-14T16:46:43.578301+0000","last_active":"2024-05-14T16:46:45.564491+0000","last_peered":"2024-05-14T16:46:45.564491+0000","last_clean":"2024-05-14T16:46:45.564491+0000","last_became_active":"2024-05-14T16:46:43.577796+0000","last_became_peered":"2024-05-14T16:46:43.577796+0000","last_unstale":"2024-05-14T16:46:45.564491+0000","last_undegraded":"2024-05-14T16:46:45.564491+0000","last_fullsized":"2024-05-14T16:46:45.564491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:13.159570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:25.656114+0000","last_change":"2024-05-14T16:46:40.547818+0000","last_active":"2024-05-14T16:47:25.656114+0000","last_peered":"2024-05-14T16:47:25.656114+0000","last_clean":"2024-05-14T16:47:25.656114+0000","last_became_active":"2024-05-14T16:46:40.547454+0000","last_became_peered":"2024-05-14T16:46:40.547454+0000","last_unstale":"2024-05-14T16:47:25.656114+0000","last_undegraded":"2024-05-14T16:47:25.656114+0000","last_fullsized":"2024-05-14T16:47:25.656114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:16.362415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573546+0000","last_change":"2024-05-14T16:46:43.570743+0000","last_active":"2024-05-14T16:46:45.573546+0000","last_peered":"2024-05-14T16:46:45.573546+0000","last_clean":"2024-05-14T16:46:45.573546+0000","last_became_active":"2024-05-14T16:46:43.570427+0000","last_became_peered":"2024-05-14T16:46:43.570427+0000","last_unstale":"2024-05-14T16:46:45.573546+0000","last_undegraded":"2024-05-14T16:46:45.573546+0000","last_fullsized":"2024-05-14T16:46:45.573546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:04:11.091285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.536351+0000","last_change":"2024-05-14T16:46:40.546675+0000","last_active":"2024-05-14T16:46:59.536351+0000","last_peered":"2024-05-14T16:46:59.536351+0000","last_clean":"2024-05-14T16:46:59.536351+0000","last_became_active":"2024-05-14T16:46:40.546250+0000","last_became_peered":"2024-05-14T16:46:40.546250+0000","last_unstale":"2024-05-14T16:46:59.536351+0000","last_undegraded":"2024-05-14T16:46:59.536351+0000","last_fullsized":"2024-05-14T16:46:59.536351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:42:52.367719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564362+0000","last_change":"2024-05-14T16:46:43.579119+0000","last_active":"2024-05-14T16:46:45.564362+0000","last_peered":"2024-05-14T16:46:45.564362+0000","last_clean":"2024-05-14T16:46:45.564362+0000","last_became_active":"2024-05-14T16:46:43.578808+0000","last_became_peered":"2024-05-14T16:46:43.578808+0000","last_unstale":"2024-05-14T16:46:45.564362+0000","last_undegraded":"2024-05-14T16:46:45.564362+0000","last_fullsized":"2024-05-14T16:46:45.564362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:57.852982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.537856+0000","last_change":"2024-05-14T16:46:40.538827+0000","last_active":"2024-05-14T16:46:59.537856+0000","last_peered":"2024-05-14T16:46:59.537856+0000","last_clean":"2024-05-14T16:46:59.537856+0000","last_became_active":"2024-05-14T16:46:40.538369+0000","last_became_peered":"2024-05-14T16:46:40.538369+0000","last_unstale":"2024-05-14T16:46:59.537856+0000","last_undegraded":"2024-05-14T16:46:59.537856+0000","last_fullsized":"2024-05-14T16:46:59.537856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:48:03.599457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571807+0000","last_change":"2024-05-14T16:46:43.558411+0000","last_active":"2024-05-14T16:46:45.571807+0000","last_peered":"2024-05-14T16:46:45.571807+0000","last_clean":"2024-05-14T16:46:45.571807+0000","last_became_active":"2024-05-14T16:46:43.558101+0000","last_became_peered":"2024-05-14T16:46:43.558101+0000","last_unstale":"2024-05-14T16:46:45.571807+0000","last_undegraded":"2024-05-14T16:46:45.571807+0000","last_fullsized":"2024-05-14T16:46:45.571807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:10.595833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564008+0000","last_change":"2024-05-14T16:46:40.544858+0000","last_active":"2024-05-14T16:46:45.564008+0000","last_peered":"2024-05-14T16:46:45.564008+0000","last_clean":"2024-05-14T16:46:45.564008+0000","last_became_active":"2024-05-14T16:46:40.544197+0000","last_became_peered":"2024-05-14T16:46:40.544197+0000","last_unstale":"2024-05-14T16:46:45.564008+0000","last_undegraded":"2024-05-14T16:46:45.564008+0000","last_fullsized":"2024-05-14T16:46:45.564008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:16:45.249626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.574782+0000","last_change":"2024-05-14T16:46:43.567019+0000","last_active":"2024-05-14T16:46:45.574782+0000","last_peered":"2024-05-14T16:46:45.574782+0000","last_clean":"2024-05-14T16:46:45.574782+0000","last_became_active":"2024-05-14T16:46:43.566700+0000","last_became_peered":"2024-05-14T16:46:43.566700+0000","last_unstale":"2024-05-14T16:46:45.574782+0000","last_undegraded":"2024-05-14T16:46:45.574782+0000","last_fullsized":"2024-05-14T16:46:45.574782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:43.373467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":47,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:21.649503+0000","last_change":"2024-05-14T16:46:40.541876+0000","last_active":"2024-05-14T16:47:21.649503+0000","last_peered":"2024-05-14T16:47:21.649503+0000","last_clean":"2024-05-14T16:47:21.649503+0000","last_became_active":"2024-05-14T16:46:40.541460+0000","last_became_peered":"2024-05-14T16:46:40.541460+0000","last_unstale":"2024-05-14T16:47:21.649503+0000","last_undegraded":"2024-05-14T16:47:21.649503+0000","last_fullsized":"2024-05-14T16:47:21.649503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:07:22.536429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575557+0000","last_change":"2024-05-14T16:46:43.577015+0000","last_active":"2024-05-14T16:46:45.575557+0000","last_peered":"2024-05-14T16:46:45.575557+0000","last_clean":"2024-05-14T16:46:45.575557+0000","last_became_active":"2024-05-14T16:46:43.576657+0000","last_became_peered":"2024-05-14T16:46:43.576657+0000","last_unstale":"2024-05-14T16:46:45.575557+0000","last_undegraded":"2024-05-14T16:46:45.575557+0000","last_fullsized":"2024-05-14T16:46:45.575557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:29.242445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.531704+0000","last_change":"2024-05-14T16:46:40.550727+0000","last_active":"2024-05-14T16:46:58.531704+0000","last_peered":"2024-05-14T16:46:58.531704+0000","last_clean":"2024-05-14T16:46:58.531704+0000","last_became_active":"2024-05-14T16:46:40.550310+0000","last_became_peered":"2024-05-14T16:46:40.550310+0000","last_unstale":"2024-05-14T16:46:58.531704+0000","last_undegraded":"2024-05-14T16:46:58.531704+0000","last_fullsized":"2024-05-14T16:46:58.531704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:04.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573717+0000","last_change":"2024-05-14T16:46:43.573934+0000","last_active":"2024-05-14T16:46:45.573717+0000","last_peered":"2024-05-14T16:46:45.573717+0000","last_clean":"2024-05-14T16:46:45.573717+0000","last_became_active":"2024-05-14T16:46:43.573611+0000","last_became_peered":"2024-05-14T16:46:43.573611+0000","last_unstale":"2024-05-14T16:46:45.573717+0000","last_undegraded":"2024-05-14T16:46:45.573717+0000","last_fullsized":"2024-05-14T16:46:45.573717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:55:32.518517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.531604+0000","last_change":"2024-05-14T16:46:40.855680+0000","last_active":"2024-05-14T16:46:58.531604+0000","last_peered":"2024-05-14T16:46:58.531604+0000","last_clean":"2024-05-14T16:46:58.531604+0000","last_became_active":"2024-05-14T16:46:40.855265+0000","last_became_peered":"2024-05-14T16:46:40.855265+0000","last_unstale":"2024-05-14T16:46:58.531604+0000","last_undegraded":"2024-05-14T16:46:58.531604+0000","last_fullsized":"2024-05-14T16:46:58.531604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:26.065298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575390+0000","last_change":"2024-05-14T16:46:43.579423+0000","last_active":"2024-05-14T16:46:45.575390+0000","last_peered":"2024-05-14T16:46:45.575390+0000","last_clean":"2024-05-14T16:46:45.575390+0000","last_became_active":"2024-05-14T16:46:43.579023+0000","last_became_peered":"2024-05-14T16:46:43.579023+0000","last_unstale":"2024-05-14T16:46:45.575390+0000","last_undegraded":"2024-05-14T16:46:45.575390+0000","last_fullsized":"2024-05-14T16:46:45.575390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:21.311018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":102,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:23.603568+0000","last_change":"2024-05-14T16:46:40.550871+0000","last_active":"2024-05-14T16:47:23.603568+0000","last_peered":"2024-05-14T16:47:23.603568+0000","last_clean":"2024-05-14T16:47:23.603568+0000","last_became_active":"2024-05-14T16:46:40.550419+0000","last_became_peered":"2024-05-14T16:46:40.550419+0000","last_unstale":"2024-05-14T16:47:23.603568+0000","last_undegraded":"2024-05-14T16:47:23.603568+0000","last_fullsized":"2024-05-14T16:47:23.603568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:53:09.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57493,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":51,"num_read_kb":76,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.574565+0000","last_change":"2024-05-14T16:46:43.569996+0000","last_active":"2024-05-14T16:46:45.574565+0000","last_peered":"2024-05-14T16:46:45.574565+0000","last_clean":"2024-05-14T16:46:45.574565+0000","last_became_active":"2024-05-14T16:46:43.569501+0000","last_became_peered":"2024-05-14T16:46:43.569501+0000","last_unstale":"2024-05-14T16:46:45.574565+0000","last_undegraded":"2024-05-14T16:46:45.574565+0000","last_fullsized":"2024-05-14T16:46:45.574565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:55.135103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.537448+0000","last_change":"2024-05-14T16:46:40.547684+0000","last_active":"2024-05-14T16:46:59.537448+0000","last_peered":"2024-05-14T16:46:59.537448+0000","last_clean":"2024-05-14T16:46:59.537448+0000","last_became_active":"2024-05-14T16:46:40.547076+0000","last_became_peered":"2024-05-14T16:46:40.547076+0000","last_unstale":"2024-05-14T16:46:59.537448+0000","last_undegraded":"2024-05-14T16:46:59.537448+0000","last_fullsized":"2024-05-14T16:46:59.537448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:44:52.349456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564119+0000","last_change":"2024-05-14T16:46:43.562773+0000","last_active":"2024-05-14T16:46:45.564119+0000","last_peered":"2024-05-14T16:46:45.564119+0000","last_clean":"2024-05-14T16:46:45.564119+0000","last_became_active":"2024-05-14T16:46:43.562449+0000","last_became_peered":"2024-05-14T16:46:43.562449+0000","last_unstale":"2024-05-14T16:46:45.564119+0000","last_undegraded":"2024-05-14T16:46:45.564119+0000","last_fullsized":"2024-05-14T16:46:45.564119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:45.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.532204+0000","last_change":"2024-05-14T16:46:40.545882+0000","last_active":"2024-05-14T16:46:58.532204+0000","last_peered":"2024-05-14T16:46:58.532204+0000","last_clean":"2024-05-14T16:46:58.532204+0000","last_became_active":"2024-05-14T16:46:40.545439+0000","last_became_peered":"2024-05-14T16:46:40.545439+0000","last_unstale":"2024-05-14T16:46:58.532204+0000","last_undegraded":"2024-05-14T16:46:58.532204+0000","last_fullsized":"2024-05-14T16:46:58.532204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:07.983329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571795+0000","last_change":"2024-05-14T16:46:43.568109+0000","last_active":"2024-05-14T16:46:45.571795+0000","last_peered":"2024-05-14T16:46:45.571795+0000","last_clean":"2024-05-14T16:46:45.571795+0000","last_became_active":"2024-05-14T16:46:43.567106+0000","last_became_peered":"2024-05-14T16:46:43.567106+0000","last_unstale":"2024-05-14T16:46:45.571795+0000","last_undegraded":"2024-05-14T16:46:45.571795+0000","last_fullsized":"2024-05-14T16:46:45.571795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:29.705387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:21.649543+0000","last_change":"2024-05-14T16:46:40.542204+0000","last_active":"2024-05-14T16:47:21.649543+0000","last_peered":"2024-05-14T16:47:21.649543+0000","last_clean":"2024-05-14T16:47:21.649543+0000","last_became_active":"2024-05-14T16:46:40.541807+0000","last_became_peered":"2024-05-14T16:46:40.541807+0000","last_unstale":"2024-05-14T16:47:21.649543+0000","last_undegraded":"2024-05-14T16:47:21.649543+0000","last_fullsized":"2024-05-14T16:47:21.649543+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:30:45.115647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571360+0000","last_change":"2024-05-14T16:46:43.573546+0000","last_active":"2024-05-14T16:46:45.571360+0000","last_peered":"2024-05-14T16:46:45.571360+0000","last_clean":"2024-05-14T16:46:45.571360+0000","last_became_active":"2024-05-14T16:46:43.573183+0000","last_became_peered":"2024-05-14T16:46:43.573183+0000","last_unstale":"2024-05-14T16:46:45.571360+0000","last_undegraded":"2024-05-14T16:46:45.571360+0000","last_fullsized":"2024-05-14T16:46:45.571360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:49.873097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.538139+0000","last_change":"2024-05-14T16:46:40.855713+0000","last_active":"2024-05-14T16:46:59.538139+0000","last_peered":"2024-05-14T16:46:59.538139+0000","last_clean":"2024-05-14T16:46:59.538139+0000","last_became_active":"2024-05-14T16:46:40.855292+0000","last_became_peered":"2024-05-14T16:46:40.855292+0000","last_unstale":"2024-05-14T16:46:59.538139+0000","last_undegraded":"2024-05-14T16:46:59.538139+0000","last_fullsized":"2024-05-14T16:46:59.538139+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:49:47.045820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564109+0000","last_change":"2024-05-14T16:46:43.579059+0000","last_active":"2024-05-14T16:46:45.564109+0000","last_peered":"2024-05-14T16:46:45.564109+0000","last_clean":"2024-05-14T16:46:45.564109+0000","last_became_active":"2024-05-14T16:46:43.578748+0000","last_became_peered":"2024-05-14T16:46:43.578748+0000","last_unstale":"2024-05-14T16:46:45.564109+0000","last_undegraded":"2024-05-14T16:46:45.564109+0000","last_fullsized":"2024-05-14T16:46:45.564109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:42.446422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:00.532869+0000","last_change":"2024-05-14T16:46:40.549047+0000","last_active":"2024-05-14T16:47:00.532869+0000","last_peered":"2024-05-14T16:47:00.532869+0000","last_clean":"2024-05-14T16:47:00.532869+0000","last_became_active":"2024-05-14T16:46:40.548648+0000","last_became_peered":"2024-05-14T16:46:40.548648+0000","last_unstale":"2024-05-14T16:47:00.532869+0000","last_undegraded":"2024-05-14T16:47:00.532869+0000","last_fullsized":"2024-05-14T16:47:00.532869+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:52:56.370871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572329+0000","last_change":"2024-05-14T16:46:43.569624+0000","last_active":"2024-05-14T16:46:45.572329+0000","last_peered":"2024-05-14T16:46:45.572329+0000","last_clean":"2024-05-14T16:46:45.572329+0000","last_became_active":"2024-05-14T16:46:43.569077+0000","last_became_peered":"2024-05-14T16:46:43.569077+0000","last_unstale":"2024-05-14T16:46:45.572329+0000","last_undegraded":"2024-05-14T16:46:45.572329+0000","last_fullsized":"2024-05-14T16:46:45.572329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:45:50.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.574873+0000","last_change":"2024-05-14T16:46:40.856302+0000","last_active":"2024-05-14T16:46:45.574873+0000","last_peered":"2024-05-14T16:46:45.574873+0000","last_clean":"2024-05-14T16:46:45.574873+0000","last_became_active":"2024-05-14T16:46:40.855964+0000","last_became_peered":"2024-05-14T16:46:40.855964+0000","last_unstale":"2024-05-14T16:46:45.574873+0000","last_undegraded":"2024-05-14T16:46:45.574873+0000","last_fullsized":"2024-05-14T16:46:45.574873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:38.501418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571807+0000","last_change":"2024-05-14T16:46:43.562847+0000","last_active":"2024-05-14T16:46:45.571807+0000","last_peered":"2024-05-14T16:46:45.571807+0000","last_clean":"2024-05-14T16:46:45.571807+0000","last_became_active":"2024-05-14T16:46:43.560978+0000","last_became_peered":"2024-05-14T16:46:43.560978+0000","last_unstale":"2024-05-14T16:46:45.571807+0000","last_undegraded":"2024-05-14T16:46:45.571807+0000","last_fullsized":"2024-05-14T16:46:45.571807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:12:33.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571711+0000","last_change":"2024-05-14T16:46:40.544017+0000","last_active":"2024-05-14T16:46:45.571711+0000","last_peered":"2024-05-14T16:46:45.571711+0000","last_clean":"2024-05-14T16:46:45.571711+0000","last_became_active":"2024-05-14T16:46:40.543176+0000","last_became_peered":"2024-05-14T16:46:40.543176+0000","last_unstale":"2024-05-14T16:46:45.571711+0000","last_undegraded":"2024-05-14T16:46:45.571711+0000","last_fullsized":"2024-05-14T16:46:45.571711+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:09.817891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.570976+0000","last_change":"2024-05-14T16:46:43.564395+0000","last_active":"2024-05-14T16:46:45.570976+0000","last_peered":"2024-05-14T16:46:45.570976+0000","last_clean":"2024-05-14T16:46:45.570976+0000","last_became_active":"2024-05-14T16:46:43.564065+0000","last_became_peered":"2024-05-14T16:46:43.564065+0000","last_unstale":"2024-05-14T16:46:45.570976+0000","last_undegraded":"2024-05-14T16:46:45.570976+0000","last_fullsized":"2024-05-14T16:46:45.570976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:52.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564551+0000","last_change":"2024-05-14T16:46:43.577067+0000","last_active":"2024-05-14T16:46:45.564551+0000","last_peered":"2024-05-14T16:46:45.564551+0000","last_clean":"2024-05-14T16:46:45.564551+0000","last_became_active":"2024-05-14T16:46:43.576750+0000","last_became_peered":"2024-05-14T16:46:43.576750+0000","last_unstale":"2024-05-14T16:46:45.564551+0000","last_undegraded":"2024-05-14T16:46:45.564551+0000","last_fullsized":"2024-05-14T16:46:45.564551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:25.961124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:25.654849+0000","last_change":"2024-05-14T16:46:40.855073+0000","last_active":"2024-05-14T16:47:25.654849+0000","last_peered":"2024-05-14T16:47:25.654849+0000","last_clean":"2024-05-14T16:47:25.654849+0000","last_became_active":"2024-05-14T16:46:40.854641+0000","last_became_peered":"2024-05-14T16:46:40.854641+0000","last_unstale":"2024-05-14T16:47:25.654849+0000","last_undegraded":"2024-05-14T16:47:25.654849+0000","last_fullsized":"2024-05-14T16:47:25.654849+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:58:11.670926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572335+0000","last_change":"2024-05-14T16:46:43.569030+0000","last_active":"2024-05-14T16:46:45.572335+0000","last_peered":"2024-05-14T16:46:45.572335+0000","last_clean":"2024-05-14T16:46:45.572335+0000","last_became_active":"2024-05-14T16:46:43.568646+0000","last_became_peered":"2024-05-14T16:46:43.568646+0000","last_unstale":"2024-05-14T16:46:45.572335+0000","last_undegraded":"2024-05-14T16:46:45.572335+0000","last_fullsized":"2024-05-14T16:46:45.572335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:38.086818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:01.615659+0000","last_change":"2024-05-14T16:46:40.542624+0000","last_active":"2024-05-14T16:47:01.615659+0000","last_peered":"2024-05-14T16:47:01.615659+0000","last_clean":"2024-05-14T16:47:01.615659+0000","last_became_active":"2024-05-14T16:46:40.542230+0000","last_became_peered":"2024-05-14T16:46:40.542230+0000","last_unstale":"2024-05-14T16:47:01.615659+0000","last_undegraded":"2024-05-14T16:47:01.615659+0000","last_fullsized":"2024-05-14T16:47:01.615659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:54:50.083362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571922+0000","last_change":"2024-05-14T16:45:15.724994+0000","last_active":"2024-05-14T16:46:45.571922+0000","last_peered":"2024-05-14T16:46:45.571922+0000","last_clean":"2024-05-14T16:46:45.571922+0000","last_became_active":"2024-05-14T16:45:15.720080+0000","last_became_peered":"2024-05-14T16:45:15.720080+0000","last_unstale":"2024-05-14T16:46:45.571922+0000","last_undegraded":"2024-05-14T16:46:45.571922+0000","last_fullsized":"2024-05-14T16:46:45.571922+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_clean_scrub_stamp":"2024-05-14T16:41:54.615675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:32:43.569611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564202+0000","last_change":"2024-05-14T16:46:43.567143+0000","last_active":"2024-05-14T16:46:45.564202+0000","last_peered":"2024-05-14T16:46:45.564202+0000","last_clean":"2024-05-14T16:46:45.564202+0000","last_became_active":"2024-05-14T16:46:43.564717+0000","last_became_peered":"2024-05-14T16:46:43.564717+0000","last_unstale":"2024-05-14T16:46:45.564202+0000","last_undegraded":"2024-05-14T16:46:45.564202+0000","last_fullsized":"2024-05-14T16:46:45.564202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:59.025279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.535914+0000","last_change":"2024-05-14T16:46:40.546709+0000","last_active":"2024-05-14T16:46:59.535914+0000","last_peered":"2024-05-14T16:46:59.535914+0000","last_clean":"2024-05-14T16:46:59.535914+0000","last_became_active":"2024-05-14T16:46:40.546295+0000","last_became_peered":"2024-05-14T16:46:40.546295+0000","last_unstale":"2024-05-14T16:46:59.535914+0000","last_undegraded":"2024-05-14T16:46:59.535914+0000","last_fullsized":"2024-05-14T16:46:59.535914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:42:50.255289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.531349+0000","last_change":"2024-05-14T16:46:40.856822+0000","last_active":"2024-05-14T16:46:58.531349+0000","last_peered":"2024-05-14T16:46:58.531349+0000","last_clean":"2024-05-14T16:46:58.531349+0000","last_became_active":"2024-05-14T16:46:40.856434+0000","last_became_peered":"2024-05-14T16:46:40.856434+0000","last_unstale":"2024-05-14T16:46:58.531349+0000","last_undegraded":"2024-05-14T16:46:58.531349+0000","last_fullsized":"2024-05-14T16:46:58.531349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:38.997679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564282+0000","last_change":"2024-05-14T16:46:43.563918+0000","last_active":"2024-05-14T16:46:45.564282+0000","last_peered":"2024-05-14T16:46:45.564282+0000","last_clean":"2024-05-14T16:46:45.564282+0000","last_became_active":"2024-05-14T16:46:43.563527+0000","last_became_peered":"2024-05-14T16:46:43.563527+0000","last_unstale":"2024-05-14T16:46:45.564282+0000","last_undegraded":"2024-05-14T16:46:45.564282+0000","last_fullsized":"2024-05-14T16:46:45.564282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:58:52.984695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'9","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:17.573474+0000","last_change":"2024-05-14T16:46:40.547480+0000","last_active":"2024-05-14T16:47:17.573474+0000","last_peered":"2024-05-14T16:47:17.573474+0000","last_clean":"2024-05-14T16:47:17.573474+0000","last_became_active":"2024-05-14T16:46:40.547062+0000","last_became_peered":"2024-05-14T16:46:40.547062+0000","last_unstale":"2024-05-14T16:47:17.573474+0000","last_undegraded":"2024-05-14T16:47:17.573474+0000","last_fullsized":"2024-05-14T16:47:17.573474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:30.211421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573020+0000","last_change":"2024-05-14T16:46:43.563870+0000","last_active":"2024-05-14T16:46:45.573020+0000","last_peered":"2024-05-14T16:46:45.573020+0000","last_clean":"2024-05-14T16:46:45.573020+0000","last_became_active":"2024-05-14T16:46:43.563542+0000","last_became_peered":"2024-05-14T16:46:43.563542+0000","last_unstale":"2024-05-14T16:46:45.573020+0000","last_undegraded":"2024-05-14T16:46:45.573020+0000","last_fullsized":"2024-05-14T16:46:45.573020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:50.699601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.531082+0000","last_change":"2024-05-14T16:46:40.665192+0000","last_active":"2024-05-14T16:46:58.531082+0000","last_peered":"2024-05-14T16:46:58.531082+0000","last_clean":"2024-05-14T16:46:58.531082+0000","last_became_active":"2024-05-14T16:46:40.664827+0000","last_became_peered":"2024-05-14T16:46:40.664827+0000","last_unstale":"2024-05-14T16:46:58.531082+0000","last_undegraded":"2024-05-14T16:46:58.531082+0000","last_fullsized":"2024-05-14T16:46:58.531082+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:41:47.179677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572828+0000","last_change":"2024-05-14T16:46:43.572596+0000","last_active":"2024-05-14T16:46:45.572828+0000","last_peered":"2024-05-14T16:46:45.572828+0000","last_clean":"2024-05-14T16:46:45.572828+0000","last_became_active":"2024-05-14T16:46:43.572271+0000","last_became_peered":"2024-05-14T16:46:43.572271+0000","last_unstale":"2024-05-14T16:46:45.572828+0000","last_undegraded":"2024-05-14T16:46:45.572828+0000","last_fullsized":"2024-05-14T16:46:45.572828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:39:38.132462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.532196+0000","last_change":"2024-05-14T16:46:40.855458+0000","last_active":"2024-05-14T16:46:58.532196+0000","last_peered":"2024-05-14T16:46:58.532196+0000","last_clean":"2024-05-14T16:46:58.532196+0000","last_became_active":"2024-05-14T16:46:40.855122+0000","last_became_peered":"2024-05-14T16:46:40.855122+0000","last_unstale":"2024-05-14T16:46:58.532196+0000","last_undegraded":"2024-05-14T16:46:58.532196+0000","last_fullsized":"2024-05-14T16:46:58.532196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:20:43.435860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573451+0000","last_change":"2024-05-14T16:46:43.566482+0000","last_active":"2024-05-14T16:46:45.573451+0000","last_peered":"2024-05-14T16:46:45.573451+0000","last_clean":"2024-05-14T16:46:45.573451+0000","last_became_active":"2024-05-14T16:46:43.566145+0000","last_became_peered":"2024-05-14T16:46:43.566145+0000","last_unstale":"2024-05-14T16:46:45.573451+0000","last_undegraded":"2024-05-14T16:46:45.573451+0000","last_fullsized":"2024-05-14T16:46:45.573451+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:14:22.731629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.530741+0000","last_change":"2024-05-14T16:46:40.541655+0000","last_active":"2024-05-14T16:46:58.530741+0000","last_peered":"2024-05-14T16:46:58.530741+0000","last_clean":"2024-05-14T16:46:58.530741+0000","last_became_active":"2024-05-14T16:46:40.541331+0000","last_became_peered":"2024-05-14T16:46:40.541331+0000","last_unstale":"2024-05-14T16:46:58.530741+0000","last_undegraded":"2024-05-14T16:46:58.530741+0000","last_fullsized":"2024-05-14T16:46:58.530741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:47:27.950532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572475+0000","last_change":"2024-05-14T16:46:43.564216+0000","last_active":"2024-05-14T16:46:45.572475+0000","last_peered":"2024-05-14T16:46:45.572475+0000","last_clean":"2024-05-14T16:46:45.572475+0000","last_became_active":"2024-05-14T16:46:43.563894+0000","last_became_peered":"2024-05-14T16:46:43.563894+0000","last_unstale":"2024-05-14T16:46:45.572475+0000","last_undegraded":"2024-05-14T16:46:45.572475+0000","last_fullsized":"2024-05-14T16:46:45.572475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:16:09.256909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571604+0000","last_change":"2024-05-14T16:46:40.551034+0000","last_active":"2024-05-14T16:46:45.571604+0000","last_peered":"2024-05-14T16:46:45.571604+0000","last_clean":"2024-05-14T16:46:45.571604+0000","last_became_active":"2024-05-14T16:46:40.550211+0000","last_became_peered":"2024-05-14T16:46:40.550211+0000","last_unstale":"2024-05-14T16:46:45.571604+0000","last_undegraded":"2024-05-14T16:46:45.571604+0000","last_fullsized":"2024-05-14T16:46:45.571604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:15.720942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572750+0000","last_change":"2024-05-14T16:46:43.558264+0000","last_active":"2024-05-14T16:46:45.572750+0000","last_peered":"2024-05-14T16:46:45.572750+0000","last_clean":"2024-05-14T16:46:45.572750+0000","last_became_active":"2024-05-14T16:46:43.557907+0000","last_became_peered":"2024-05-14T16:46:43.557907+0000","last_unstale":"2024-05-14T16:46:45.572750+0000","last_undegraded":"2024-05-14T16:46:45.572750+0000","last_fullsized":"2024-05-14T16:46:45.572750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:20:52.752482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.533416+0000","last_change":"2024-05-14T16:46:40.551013+0000","last_active":"2024-05-14T16:46:58.533416+0000","last_peered":"2024-05-14T16:46:58.533416+0000","last_clean":"2024-05-14T16:46:58.533416+0000","last_became_active":"2024-05-14T16:46:40.550658+0000","last_became_peered":"2024-05-14T16:46:40.550658+0000","last_unstale":"2024-05-14T16:46:58.533416+0000","last_undegraded":"2024-05-14T16:46:58.533416+0000","last_fullsized":"2024-05-14T16:46:58.533416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:36.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571905+0000","last_change":"2024-05-14T16:46:43.572270+0000","last_active":"2024-05-14T16:46:45.571905+0000","last_peered":"2024-05-14T16:46:45.571905+0000","last_clean":"2024-05-14T16:46:45.571905+0000","last_became_active":"2024-05-14T16:46:43.571886+0000","last_became_peered":"2024-05-14T16:46:43.571886+0000","last_unstale":"2024-05-14T16:46:45.571905+0000","last_undegraded":"2024-05-14T16:46:45.571905+0000","last_fullsized":"2024-05-14T16:46:45.571905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:11.890881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.537950+0000","last_change":"2024-05-14T16:46:40.855707+0000","last_active":"2024-05-14T16:46:59.537950+0000","last_peered":"2024-05-14T16:46:59.537950+0000","last_clean":"2024-05-14T16:46:59.537950+0000","last_became_active":"2024-05-14T16:46:40.855335+0000","last_became_peered":"2024-05-14T16:46:40.855335+0000","last_unstale":"2024-05-14T16:46:59.537950+0000","last_undegraded":"2024-05-14T16:46:59.537950+0000","last_fullsized":"2024-05-14T16:46:59.537950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:59:47.192246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572675+0000","last_change":"2024-05-14T16:46:43.572455+0000","last_active":"2024-05-14T16:46:45.572675+0000","last_peered":"2024-05-14T16:46:45.572675+0000","last_clean":"2024-05-14T16:46:45.572675+0000","last_became_active":"2024-05-14T16:46:43.571930+0000","last_became_peered":"2024-05-14T16:46:43.571930+0000","last_unstale":"2024-05-14T16:46:45.572675+0000","last_undegraded":"2024-05-14T16:46:45.572675+0000","last_fullsized":"2024-05-14T16:46:45.572675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:35:39.643239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575557+0000","last_change":"2024-05-14T16:46:40.541708+0000","last_active":"2024-05-14T16:46:45.575557+0000","last_peered":"2024-05-14T16:46:45.575557+0000","last_clean":"2024-05-14T16:46:45.575557+0000","last_became_active":"2024-05-14T16:46:40.541202+0000","last_became_peered":"2024-05-14T16:46:40.541202+0000","last_unstale":"2024-05-14T16:46:45.575557+0000","last_undegraded":"2024-05-14T16:46:45.575557+0000","last_fullsized":"2024-05-14T16:46:45.575557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:50:01.821013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573320+0000","last_change":"2024-05-14T16:46:43.574039+0000","last_active":"2024-05-14T16:46:45.573320+0000","last_peered":"2024-05-14T16:46:45.573320+0000","last_clean":"2024-05-14T16:46:45.573320+0000","last_became_active":"2024-05-14T16:46:43.573237+0000","last_became_peered":"2024-05-14T16:46:43.573237+0000","last_unstale":"2024-05-14T16:46:45.573320+0000","last_undegraded":"2024-05-14T16:46:45.573320+0000","last_fullsized":"2024-05-14T16:46:45.573320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:13.234495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.570397+0000","last_change":"2024-05-14T16:46:40.549519+0000","last_active":"2024-05-14T16:46:45.570397+0000","last_peered":"2024-05-14T16:46:45.570397+0000","last_clean":"2024-05-14T16:46:45.570397+0000","last_became_active":"2024-05-14T16:46:40.549169+0000","last_became_peered":"2024-05-14T16:46:40.549169+0000","last_unstale":"2024-05-14T16:46:45.570397+0000","last_undegraded":"2024-05-14T16:46:45.570397+0000","last_fullsized":"2024-05-14T16:46:45.570397+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:22.559724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572368+0000","last_change":"2024-05-14T16:46:43.567674+0000","last_active":"2024-05-14T16:46:45.572368+0000","last_peered":"2024-05-14T16:46:45.572368+0000","last_clean":"2024-05-14T16:46:45.572368+0000","last_became_active":"2024-05-14T16:46:43.567337+0000","last_became_peered":"2024-05-14T16:46:43.567337+0000","last_unstale":"2024-05-14T16:46:45.572368+0000","last_undegraded":"2024-05-14T16:46:45.572368+0000","last_fullsized":"2024-05-14T16:46:45.572368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:02.321955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572683+0000","last_change":"2024-05-14T16:46:43.567627+0000","last_active":"2024-05-14T16:46:45.572683+0000","last_peered":"2024-05-14T16:46:45.572683+0000","last_clean":"2024-05-14T16:46:45.572683+0000","last_became_active":"2024-05-14T16:46:43.567313+0000","last_became_peered":"2024-05-14T16:46:43.567313+0000","last_unstale":"2024-05-14T16:46:45.572683+0000","last_undegraded":"2024-05-14T16:46:45.572683+0000","last_fullsized":"2024-05-14T16:46:45.572683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:47.396923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571179+0000","last_change":"2024-05-14T16:46:43.571872+0000","last_active":"2024-05-14T16:46:45.571179+0000","last_peered":"2024-05-14T16:46:45.571179+0000","last_clean":"2024-05-14T16:46:45.571179+0000","last_became_active":"2024-05-14T16:46:43.571495+0000","last_became_peered":"2024-05-14T16:46:43.571495+0000","last_unstale":"2024-05-14T16:46:45.571179+0000","last_undegraded":"2024-05-14T16:46:45.571179+0000","last_fullsized":"2024-05-14T16:46:45.571179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:34.639950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571419+0000","last_change":"2024-05-14T16:46:43.572671+0000","last_active":"2024-05-14T16:46:45.571419+0000","last_peered":"2024-05-14T16:46:45.571419+0000","last_clean":"2024-05-14T16:46:45.571419+0000","last_became_active":"2024-05-14T16:46:43.572357+0000","last_became_peered":"2024-05-14T16:46:43.572357+0000","last_unstale":"2024-05-14T16:46:45.571419+0000","last_undegraded":"2024-05-14T16:46:45.571419+0000","last_fullsized":"2024-05-14T16:46:45.571419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:57.597636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572931+0000","last_change":"2024-05-14T16:46:43.572168+0000","last_active":"2024-05-14T16:46:45.572931+0000","last_peered":"2024-05-14T16:46:45.572931+0000","last_clean":"2024-05-14T16:46:45.572931+0000","last_became_active":"2024-05-14T16:46:43.571822+0000","last_became_peered":"2024-05-14T16:46:43.571822+0000","last_unstale":"2024-05-14T16:46:45.572931+0000","last_undegraded":"2024-05-14T16:46:45.572931+0000","last_fullsized":"2024-05-14T16:46:45.572931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:57.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564338+0000","last_change":"2024-05-14T16:46:43.570935+0000","last_active":"2024-05-14T16:46:45.564338+0000","last_peered":"2024-05-14T16:46:45.564338+0000","last_clean":"2024-05-14T16:46:45.564338+0000","last_became_active":"2024-05-14T16:46:43.570605+0000","last_became_peered":"2024-05-14T16:46:43.570605+0000","last_unstale":"2024-05-14T16:46:45.564338+0000","last_undegraded":"2024-05-14T16:46:45.564338+0000","last_fullsized":"2024-05-14T16:46:45.564338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:17:55.161195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572218+0000","last_change":"2024-05-14T16:46:43.568442+0000","last_active":"2024-05-14T16:46:45.572218+0000","last_peered":"2024-05-14T16:46:45.572218+0000","last_clean":"2024-05-14T16:46:45.572218+0000","last_became_active":"2024-05-14T16:46:43.568118+0000","last_became_peered":"2024-05-14T16:46:43.568118+0000","last_unstale":"2024-05-14T16:46:45.572218+0000","last_undegraded":"2024-05-14T16:46:45.572218+0000","last_fullsized":"2024-05-14T16:46:45.572218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:52.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572315+0000","last_change":"2024-05-14T16:46:43.576174+0000","last_active":"2024-05-14T16:46:45.572315+0000","last_peered":"2024-05-14T16:46:45.572315+0000","last_clean":"2024-05-14T16:46:45.572315+0000","last_became_active":"2024-05-14T16:46:43.575737+0000","last_became_peered":"2024-05-14T16:46:43.575737+0000","last_unstale":"2024-05-14T16:46:45.572315+0000","last_undegraded":"2024-05-14T16:46:45.572315+0000","last_fullsized":"2024-05-14T16:46:45.572315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:08.453070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572904+0000","last_change":"2024-05-14T16:46:43.566256+0000","last_active":"2024-05-14T16:46:45.572904+0000","last_peered":"2024-05-14T16:46:45.572904+0000","last_clean":"2024-05-14T16:46:45.572904+0000","last_became_active":"2024-05-14T16:46:43.565886+0000","last_became_peered":"2024-05-14T16:46:43.565886+0000","last_unstale":"2024-05-14T16:46:45.572904+0000","last_undegraded":"2024-05-14T16:46:45.572904+0000","last_fullsized":"2024-05-14T16:46:45.572904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:50.021286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564042+0000","last_change":"2024-05-14T16:46:43.569819+0000","last_active":"2024-05-14T16:46:45.564042+0000","last_peered":"2024-05-14T16:46:45.564042+0000","last_clean":"2024-05-14T16:46:45.564042+0000","last_became_active":"2024-05-14T16:46:43.569461+0000","last_became_peered":"2024-05-14T16:46:43.569461+0000","last_unstale":"2024-05-14T16:46:45.564042+0000","last_undegraded":"2024-05-14T16:46:45.564042+0000","last_fullsized":"2024-05-14T16:46:45.564042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:52.013604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572023+0000","last_change":"2024-05-14T16:46:43.567569+0000","last_active":"2024-05-14T16:46:45.572023+0000","last_peered":"2024-05-14T16:46:45.572023+0000","last_clean":"2024-05-14T16:46:45.572023+0000","last_became_active":"2024-05-14T16:46:43.564808+0000","last_became_peered":"2024-05-14T16:46:43.564808+0000","last_unstale":"2024-05-14T16:46:45.572023+0000","last_undegraded":"2024-05-14T16:46:45.572023+0000","last_fullsized":"2024-05-14T16:46:45.572023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:13.744730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.576092+0000","last_change":"2024-05-14T16:46:43.578417+0000","last_active":"2024-05-14T16:46:45.576092+0000","last_peered":"2024-05-14T16:46:45.576092+0000","last_clean":"2024-05-14T16:46:45.576092+0000","last_became_active":"2024-05-14T16:46:43.577687+0000","last_became_peered":"2024-05-14T16:46:43.577687+0000","last_unstale":"2024-05-14T16:46:45.576092+0000","last_undegraded":"2024-05-14T16:46:45.576092+0000","last_fullsized":"2024-05-14T16:46:45.576092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:32.060934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564376+0000","last_change":"2024-05-14T16:46:43.564114+0000","last_active":"2024-05-14T16:46:45.564376+0000","last_peered":"2024-05-14T16:46:45.564376+0000","last_clean":"2024-05-14T16:46:45.564376+0000","last_became_active":"2024-05-14T16:46:43.563734+0000","last_became_peered":"2024-05-14T16:46:43.563734+0000","last_unstale":"2024-05-14T16:46:45.564376+0000","last_undegraded":"2024-05-14T16:46:45.564376+0000","last_fullsized":"2024-05-14T16:46:45.564376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:19.016741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571845+0000","last_change":"2024-05-14T16:46:43.566490+0000","last_active":"2024-05-14T16:46:45.571845+0000","last_peered":"2024-05-14T16:46:45.571845+0000","last_clean":"2024-05-14T16:46:45.571845+0000","last_became_active":"2024-05-14T16:46:43.566086+0000","last_became_peered":"2024-05-14T16:46:43.566086+0000","last_unstale":"2024-05-14T16:46:45.571845+0000","last_undegraded":"2024-05-14T16:46:45.571845+0000","last_fullsized":"2024-05-14T16:46:45.571845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:39:09.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245266,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61103,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":165,"num_read_kb":142,"num_write":159,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":183309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710744,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":408028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906972,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":1052192,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070498,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":856064,"data_stored":408264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266727,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":999723,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":407653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":464939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":408924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":461479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":407964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":847872,"data_stored":408090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":999407,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:47:29.544 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:47:29.785 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:29.907 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:29 smithi069 ceph-mon[31507]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-14T16:47:29.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:29 smithi203 ceph-mon[35787]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-14T16:47:30.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:29 smithi092 ceph-mon[35763]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-14T16:47:30.878 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:30 smithi069 ceph-mon[31507]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:47:30.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:30 smithi203 ceph-mon[35787]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:47:31.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:30 smithi092 ceph-mon[35763]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:47:31.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.425+0000 7f76041ab700 1 -- 172.21.15.69:0/2101469065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75fc1039b0 msgr2=0x7f75fc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:31.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.425+0000 7f76041ab700 1 --2- 172.21.15.69:0/2101469065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75fc1039b0 0x7f75fc10f3c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f75f400b0d0 tx=0x7f75f400b3e0 comp rx=0 tx=0).stop 2024-05-14T16:47:31.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.425+0000 7f76041ab700 1 -- 172.21.15.69:0/2101469065 shutdown_connections 2024-05-14T16:47:31.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.425+0000 7f76041ab700 1 --2- 172.21.15.69:0/2101469065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75fc1039b0 0x7f75fc10f3c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:31.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.425+0000 7f76041ab700 1 --2- 172.21.15.69:0/2101469065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75fc108710 0x7f75fc108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:31.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.425+0000 7f76041ab700 1 --2- 172.21.15.69:0/2101469065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75fc102ff0 0x7f75fc103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:31.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.425+0000 7f76041ab700 1 -- 172.21.15.69:0/2101469065 >> 172.21.15.69:0/2101469065 conn(0x7f75fc077760 msgr2=0x7f75fc077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:31.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.425+0000 7f76041ab700 1 -- 172.21.15.69:0/2101469065 shutdown_connections 2024-05-14T16:47:31.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.425+0000 7f76041ab700 1 -- 172.21.15.69:0/2101469065 wait complete. 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.425+0000 7f76041ab700 1 Processor -- start 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f76041ab700 1 -- start start 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f76041ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75fc102ff0 0x7f75fc19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f76041ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75fc1039b0 0x7f75fc19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f76041ab700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75fc108710 0x7f75fc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f76041ab700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc112790 con 0x7f75fc108710 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f76041ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc112610 con 0x7f75fc1039b0 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f76041ab700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc112910 con 0x7f75fc102ff0 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7602748700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75fc108710 0x7f75fc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7602748700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75fc108710 0x7f75fc1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35288/0 (socket says 172.21.15.69:35288) 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7602748700 1 -- 172.21.15.69:0/3501604552 learned_addr learned my addr 172.21.15.69:0/3501604552 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7601746700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75fc1039b0 0x7f75fc19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:31.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7601f47700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75fc102ff0 0x7f75fc19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7602748700 1 -- 172.21.15.69:0/3501604552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75fc102ff0 msgr2=0x7f75fc19be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7602748700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75fc102ff0 0x7f75fc19be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7602748700 1 -- 172.21.15.69:0/3501604552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75fc1039b0 msgr2=0x7f75fc19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7602748700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75fc1039b0 0x7f75fc19c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7602748700 1 -- 172.21.15.69:0/3501604552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc1a0e30 con 0x7f75fc108710 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f7602748700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75fc108710 0x7f75fc1a06f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009fd0 tx=0x7f75f4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f75f2ffd700 1 -- 172.21.15.69:0/3501604552 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f401f070 con 0x7f75fc108710 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.426+0000 7f75f2ffd700 1 -- 172.21.15.69:0/3501604552 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75f4003c00 con 0x7f75fc108710 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.427+0000 7f75f2ffd700 1 -- 172.21.15.69:0/3501604552 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f401b6a0 con 0x7f75fc108710 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.427+0000 7f76041ab700 1 -- 172.21.15.69:0/3501604552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc1a10c0 con 0x7f75fc108710 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.427+0000 7f76041ab700 1 -- 172.21.15.69:0/3501604552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc1a89a0 con 0x7f75fc108710 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.427+0000 7f76041ab700 1 -- 172.21.15.69:0/3501604552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75fc06b890 con 0x7f75fc108710 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.429+0000 7f75f2ffd700 1 -- 172.21.15.69:0/3501604552 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f75f4022070 con 0x7f75fc108710 2024-05-14T16:47:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.429+0000 7f75f2ffd700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75e80b2910 0x7f75e80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:31.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.429+0000 7f7601f47700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75e80b2910 0x7f75e80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:31.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.430+0000 7f75f2ffd700 1 -- 172.21.15.69:0/3501604552 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f75f400f020 con 0x7f75fc108710 2024-05-14T16:47:31.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.430+0000 7f7601f47700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75e80b2910 0x7f75e80b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f75f8005ea0 tx=0x7f75f8005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:31.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.433+0000 7f75f2ffd700 1 -- 172.21.15.69:0/3501604552 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f75f4029020 con 0x7f75fc108710 2024-05-14T16:47:31.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.637+0000 7f76041ab700 1 -- 172.21.15.69:0/3501604552 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75fc19d180 con 0x7f75e80b2910 2024-05-14T16:47:31.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.644+0000 7f75f2ffd700 1 -- 172.21.15.69:0/3501604552 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258834 (secure 0 0 0) 0x7f75fc19d180 con 0x7f75e80b2910 2024-05-14T16:47:31.647 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:47:31.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 -- 172.21.15.69:0/3501604552 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75e80b2910 msgr2=0x7f75e80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:31.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75e80b2910 0x7f75e80b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f75f8005ea0 tx=0x7f75f8005e30 comp rx=0 tx=0).stop 2024-05-14T16:47:31.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 -- 172.21.15.69:0/3501604552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75fc108710 msgr2=0x7f75fc1a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:31.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75fc108710 0x7f75fc1a06f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009fd0 tx=0x7f75f4007a20 comp rx=0 tx=0).stop 2024-05-14T16:47:31.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 -- 172.21.15.69:0/3501604552 shutdown_connections 2024-05-14T16:47:31.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f75e80b2910 0x7f75e80b4dd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:31.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75fc108710 0x7f75fc1a06f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:31.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75fc102ff0 0x7f75fc19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:31.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 --2- 172.21.15.69:0/3501604552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75fc1039b0 0x7f75fc19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:31.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 -- 172.21.15.69:0/3501604552 >> 172.21.15.69:0/3501604552 conn(0x7f75fc077760 msgr2=0x7f75fc10d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:31.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 -- 172.21.15.69:0/3501604552 shutdown_connections 2024-05-14T16:47:31.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:31.648+0000 7f76041ab700 1 -- 172.21.15.69:0/3501604552 wait complete. 2024-05-14T16:47:31.650 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-05-14T16:47:31.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:31 smithi069 ceph-mon[31507]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-05-14T16:47:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:31 smithi203 ceph-mon[35787]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-05-14T16:47:32.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:31 smithi092 ceph-mon[35763]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-05-14T16:47:32.358 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-05-14T16:47:29.868726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655962,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":267,"num_read_kb":226,"num_write":289,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351188,"kb_used_data":10356,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590636,"statfs":{"total":1151940427776,"available":1151580811264,"internally_reserved":0,"allocated":10604544,"data_stored":6844467,"data_compressed":107202,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33667,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":68,"num_write":25,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003003"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572834+0000","last_change":"2024-05-14T16:46:43.572738+0000","last_active":"2024-05-14T16:46:45.572834+0000","last_peered":"2024-05-14T16:46:45.572834+0000","last_clean":"2024-05-14T16:46:45.572834+0000","last_became_active":"2024-05-14T16:46:43.572400+0000","last_became_peered":"2024-05-14T16:46:43.572400+0000","last_unstale":"2024-05-14T16:46:45.572834+0000","last_undegraded":"2024-05-14T16:46:45.572834+0000","last_fullsized":"2024-05-14T16:46:45.572834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:44.332862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572468+0000","last_change":"2024-05-14T16:46:43.566193+0000","last_active":"2024-05-14T16:46:45.572468+0000","last_peered":"2024-05-14T16:46:45.572468+0000","last_clean":"2024-05-14T16:46:45.572468+0000","last_became_active":"2024-05-14T16:46:43.565853+0000","last_became_peered":"2024-05-14T16:46:43.565853+0000","last_unstale":"2024-05-14T16:46:45.572468+0000","last_undegraded":"2024-05-14T16:46:45.572468+0000","last_fullsized":"2024-05-14T16:46:45.572468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:19:34.481225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573087+0000","last_change":"2024-05-14T16:46:43.574937+0000","last_active":"2024-05-14T16:46:45.573087+0000","last_peered":"2024-05-14T16:46:45.573087+0000","last_clean":"2024-05-14T16:46:45.573087+0000","last_became_active":"2024-05-14T16:46:43.574317+0000","last_became_peered":"2024-05-14T16:46:43.574317+0000","last_unstale":"2024-05-14T16:46:45.573087+0000","last_undegraded":"2024-05-14T16:46:45.573087+0000","last_fullsized":"2024-05-14T16:46:45.573087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:35:49.933658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.565158+0000","last_change":"2024-05-14T16:46:43.577130+0000","last_active":"2024-05-14T16:46:45.565158+0000","last_peered":"2024-05-14T16:46:45.565158+0000","last_clean":"2024-05-14T16:46:45.565158+0000","last_became_active":"2024-05-14T16:46:43.576725+0000","last_became_peered":"2024-05-14T16:46:43.576725+0000","last_unstale":"2024-05-14T16:46:45.565158+0000","last_undegraded":"2024-05-14T16:46:45.565158+0000","last_fullsized":"2024-05-14T16:46:45.565158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:24:55.724298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573648+0000","last_change":"2024-05-14T16:46:43.567104+0000","last_active":"2024-05-14T16:46:45.573648+0000","last_peered":"2024-05-14T16:46:45.573648+0000","last_clean":"2024-05-14T16:46:45.573648+0000","last_became_active":"2024-05-14T16:46:43.566762+0000","last_became_peered":"2024-05-14T16:46:43.566762+0000","last_unstale":"2024-05-14T16:46:45.573648+0000","last_undegraded":"2024-05-14T16:46:45.573648+0000","last_fullsized":"2024-05-14T16:46:45.573648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:08.819856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.570819+0000","last_change":"2024-05-14T16:46:43.565055+0000","last_active":"2024-05-14T16:46:45.570819+0000","last_peered":"2024-05-14T16:46:45.570819+0000","last_clean":"2024-05-14T16:46:45.570819+0000","last_became_active":"2024-05-14T16:46:43.564699+0000","last_became_peered":"2024-05-14T16:46:43.564699+0000","last_unstale":"2024-05-14T16:46:45.570819+0000","last_undegraded":"2024-05-14T16:46:45.570819+0000","last_fullsized":"2024-05-14T16:46:45.570819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:01.903025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571963+0000","last_change":"2024-05-14T16:46:43.568607+0000","last_active":"2024-05-14T16:46:45.571963+0000","last_peered":"2024-05-14T16:46:45.571963+0000","last_clean":"2024-05-14T16:46:45.571963+0000","last_became_active":"2024-05-14T16:46:43.568195+0000","last_became_peered":"2024-05-14T16:46:43.568195+0000","last_unstale":"2024-05-14T16:46:45.571963+0000","last_undegraded":"2024-05-14T16:46:45.571963+0000","last_fullsized":"2024-05-14T16:46:45.571963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:31:58.526198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571533+0000","last_change":"2024-05-14T16:46:43.564470+0000","last_active":"2024-05-14T16:46:45.571533+0000","last_peered":"2024-05-14T16:46:45.571533+0000","last_clean":"2024-05-14T16:46:45.571533+0000","last_became_active":"2024-05-14T16:46:43.564169+0000","last_became_peered":"2024-05-14T16:46:43.564169+0000","last_unstale":"2024-05-14T16:46:45.571533+0000","last_undegraded":"2024-05-14T16:46:45.571533+0000","last_fullsized":"2024-05-14T16:46:45.571533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:38.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.577057+0000","last_change":"2024-05-14T16:46:43.580988+0000","last_active":"2024-05-14T16:46:45.577057+0000","last_peered":"2024-05-14T16:46:45.577057+0000","last_clean":"2024-05-14T16:46:45.577057+0000","last_became_active":"2024-05-14T16:46:43.580657+0000","last_became_peered":"2024-05-14T16:46:43.580657+0000","last_unstale":"2024-05-14T16:46:45.577057+0000","last_undegraded":"2024-05-14T16:46:45.577057+0000","last_fullsized":"2024-05-14T16:46:45.577057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:39:01.568822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572643+0000","last_change":"2024-05-14T16:46:43.565742+0000","last_active":"2024-05-14T16:46:45.572643+0000","last_peered":"2024-05-14T16:46:45.572643+0000","last_clean":"2024-05-14T16:46:45.572643+0000","last_became_active":"2024-05-14T16:46:43.565260+0000","last_became_peered":"2024-05-14T16:46:43.565260+0000","last_unstale":"2024-05-14T16:46:45.572643+0000","last_undegraded":"2024-05-14T16:46:45.572643+0000","last_fullsized":"2024-05-14T16:46:45.572643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:25.718174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571632+0000","last_change":"2024-05-14T16:46:43.576268+0000","last_active":"2024-05-14T16:46:45.571632+0000","last_peered":"2024-05-14T16:46:45.571632+0000","last_clean":"2024-05-14T16:46:45.571632+0000","last_became_active":"2024-05-14T16:46:43.575856+0000","last_became_peered":"2024-05-14T16:46:43.575856+0000","last_unstale":"2024-05-14T16:46:45.571632+0000","last_undegraded":"2024-05-14T16:46:45.571632+0000","last_fullsized":"2024-05-14T16:46:45.571632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:12:16.549127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.565360+0000","last_change":"2024-05-14T16:46:43.573315+0000","last_active":"2024-05-14T16:46:45.565360+0000","last_peered":"2024-05-14T16:46:45.565360+0000","last_clean":"2024-05-14T16:46:45.565360+0000","last_became_active":"2024-05-14T16:46:43.572981+0000","last_became_peered":"2024-05-14T16:46:43.572981+0000","last_unstale":"2024-05-14T16:46:45.565360+0000","last_undegraded":"2024-05-14T16:46:45.565360+0000","last_fullsized":"2024-05-14T16:46:45.565360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:07.757539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564359+0000","last_change":"2024-05-14T16:46:43.569388+0000","last_active":"2024-05-14T16:46:45.564359+0000","last_peered":"2024-05-14T16:46:45.564359+0000","last_clean":"2024-05-14T16:46:45.564359+0000","last_became_active":"2024-05-14T16:46:43.568992+0000","last_became_peered":"2024-05-14T16:46:43.568992+0000","last_unstale":"2024-05-14T16:46:45.564359+0000","last_undegraded":"2024-05-14T16:46:45.564359+0000","last_fullsized":"2024-05-14T16:46:45.564359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:08:06.236797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572162+0000","last_change":"2024-05-14T16:46:43.564137+0000","last_active":"2024-05-14T16:46:45.572162+0000","last_peered":"2024-05-14T16:46:45.572162+0000","last_clean":"2024-05-14T16:46:45.572162+0000","last_became_active":"2024-05-14T16:46:43.563831+0000","last_became_peered":"2024-05-14T16:46:43.563831+0000","last_unstale":"2024-05-14T16:46:45.572162+0000","last_undegraded":"2024-05-14T16:46:45.572162+0000","last_fullsized":"2024-05-14T16:46:45.572162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:58.473863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571169+0000","last_change":"2024-05-14T16:46:43.571789+0000","last_active":"2024-05-14T16:46:45.571169+0000","last_peered":"2024-05-14T16:46:45.571169+0000","last_clean":"2024-05-14T16:46:45.571169+0000","last_became_active":"2024-05-14T16:46:43.571521+0000","last_became_peered":"2024-05-14T16:46:43.571521+0000","last_unstale":"2024-05-14T16:46:45.571169+0000","last_undegraded":"2024-05-14T16:46:45.571169+0000","last_fullsized":"2024-05-14T16:46:45.571169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:23.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575910+0000","last_change":"2024-05-14T16:46:43.569532+0000","last_active":"2024-05-14T16:46:45.575910+0000","last_peered":"2024-05-14T16:46:45.575910+0000","last_clean":"2024-05-14T16:46:45.575910+0000","last_became_active":"2024-05-14T16:46:43.569132+0000","last_became_peered":"2024-05-14T16:46:43.569132+0000","last_unstale":"2024-05-14T16:46:45.575910+0000","last_undegraded":"2024-05-14T16:46:45.575910+0000","last_fullsized":"2024-05-14T16:46:45.575910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:09.328241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573018+0000","last_change":"2024-05-14T16:46:43.574451+0000","last_active":"2024-05-14T16:46:45.573018+0000","last_peered":"2024-05-14T16:46:45.573018+0000","last_clean":"2024-05-14T16:46:45.573018+0000","last_became_active":"2024-05-14T16:46:43.574143+0000","last_became_peered":"2024-05-14T16:46:43.574143+0000","last_unstale":"2024-05-14T16:46:45.573018+0000","last_undegraded":"2024-05-14T16:46:45.573018+0000","last_fullsized":"2024-05-14T16:46:45.573018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:01:42.831262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575400+0000","last_change":"2024-05-14T16:46:43.568799+0000","last_active":"2024-05-14T16:46:45.575400+0000","last_peered":"2024-05-14T16:46:45.575400+0000","last_clean":"2024-05-14T16:46:45.575400+0000","last_became_active":"2024-05-14T16:46:43.568467+0000","last_became_peered":"2024-05-14T16:46:43.568467+0000","last_unstale":"2024-05-14T16:46:45.575400+0000","last_undegraded":"2024-05-14T16:46:45.575400+0000","last_fullsized":"2024-05-14T16:46:45.575400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:59.036372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572053+0000","last_change":"2024-05-14T16:46:43.566879+0000","last_active":"2024-05-14T16:46:45.572053+0000","last_peered":"2024-05-14T16:46:45.572053+0000","last_clean":"2024-05-14T16:46:45.572053+0000","last_became_active":"2024-05-14T16:46:43.566346+0000","last_became_peered":"2024-05-14T16:46:43.566346+0000","last_unstale":"2024-05-14T16:46:45.572053+0000","last_undegraded":"2024-05-14T16:46:45.572053+0000","last_fullsized":"2024-05-14T16:46:45.572053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:11.027173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.537507+0000","last_change":"2024-05-14T16:46:40.546990+0000","last_active":"2024-05-14T16:46:59.537507+0000","last_peered":"2024-05-14T16:46:59.537507+0000","last_clean":"2024-05-14T16:46:59.537507+0000","last_became_active":"2024-05-14T16:46:40.546646+0000","last_became_peered":"2024-05-14T16:46:40.546646+0000","last_unstale":"2024-05-14T16:46:59.537507+0000","last_undegraded":"2024-05-14T16:46:59.537507+0000","last_fullsized":"2024-05-14T16:46:59.537507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:47.001760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571591+0000","last_change":"2024-05-14T16:46:43.570352+0000","last_active":"2024-05-14T16:46:45.571591+0000","last_peered":"2024-05-14T16:46:45.571591+0000","last_clean":"2024-05-14T16:46:45.571591+0000","last_became_active":"2024-05-14T16:46:43.569748+0000","last_became_peered":"2024-05-14T16:46:43.569748+0000","last_unstale":"2024-05-14T16:46:45.571591+0000","last_undegraded":"2024-05-14T16:46:45.571591+0000","last_fullsized":"2024-05-14T16:46:45.571591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:24.707586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575687+0000","last_change":"2024-05-14T16:46:40.855034+0000","last_active":"2024-05-14T16:46:45.575687+0000","last_peered":"2024-05-14T16:46:45.575687+0000","last_clean":"2024-05-14T16:46:45.575687+0000","last_became_active":"2024-05-14T16:46:40.854686+0000","last_became_peered":"2024-05-14T16:46:40.854686+0000","last_unstale":"2024-05-14T16:46:45.575687+0000","last_undegraded":"2024-05-14T16:46:45.575687+0000","last_fullsized":"2024-05-14T16:46:45.575687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:56.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573497+0000","last_change":"2024-05-14T16:46:43.574682+0000","last_active":"2024-05-14T16:46:45.573497+0000","last_peered":"2024-05-14T16:46:45.573497+0000","last_clean":"2024-05-14T16:46:45.573497+0000","last_became_active":"2024-05-14T16:46:43.574322+0000","last_became_peered":"2024-05-14T16:46:43.574322+0000","last_unstale":"2024-05-14T16:46:45.573497+0000","last_undegraded":"2024-05-14T16:46:45.573497+0000","last_fullsized":"2024-05-14T16:46:45.573497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:20.169428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.537610+0000","last_change":"2024-05-14T16:46:40.551307+0000","last_active":"2024-05-14T16:46:59.537610+0000","last_peered":"2024-05-14T16:46:59.537610+0000","last_clean":"2024-05-14T16:46:59.537610+0000","last_became_active":"2024-05-14T16:46:40.550987+0000","last_became_peered":"2024-05-14T16:46:40.550987+0000","last_unstale":"2024-05-14T16:46:59.537610+0000","last_undegraded":"2024-05-14T16:46:59.537610+0000","last_fullsized":"2024-05-14T16:46:59.537610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:27.819457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575151+0000","last_change":"2024-05-14T16:46:43.575200+0000","last_active":"2024-05-14T16:46:45.575151+0000","last_peered":"2024-05-14T16:46:45.575151+0000","last_clean":"2024-05-14T16:46:45.575151+0000","last_became_active":"2024-05-14T16:46:43.574864+0000","last_became_peered":"2024-05-14T16:46:43.574864+0000","last_unstale":"2024-05-14T16:46:45.575151+0000","last_undegraded":"2024-05-14T16:46:45.575151+0000","last_fullsized":"2024-05-14T16:46:45.575151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:48.630101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.578732+0000","last_change":"2024-05-14T16:46:40.548397+0000","last_active":"2024-05-14T16:46:45.578732+0000","last_peered":"2024-05-14T16:46:45.578732+0000","last_clean":"2024-05-14T16:46:45.578732+0000","last_became_active":"2024-05-14T16:46:40.547803+0000","last_became_peered":"2024-05-14T16:46:40.547803+0000","last_unstale":"2024-05-14T16:46:45.578732+0000","last_undegraded":"2024-05-14T16:46:45.578732+0000","last_fullsized":"2024-05-14T16:46:45.578732+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:21:19.812413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564491+0000","last_change":"2024-05-14T16:46:43.578301+0000","last_active":"2024-05-14T16:46:45.564491+0000","last_peered":"2024-05-14T16:46:45.564491+0000","last_clean":"2024-05-14T16:46:45.564491+0000","last_became_active":"2024-05-14T16:46:43.577796+0000","last_became_peered":"2024-05-14T16:46:43.577796+0000","last_unstale":"2024-05-14T16:46:45.564491+0000","last_undegraded":"2024-05-14T16:46:45.564491+0000","last_fullsized":"2024-05-14T16:46:45.564491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:13.159570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:25.656114+0000","last_change":"2024-05-14T16:46:40.547818+0000","last_active":"2024-05-14T16:47:25.656114+0000","last_peered":"2024-05-14T16:47:25.656114+0000","last_clean":"2024-05-14T16:47:25.656114+0000","last_became_active":"2024-05-14T16:46:40.547454+0000","last_became_peered":"2024-05-14T16:46:40.547454+0000","last_unstale":"2024-05-14T16:47:25.656114+0000","last_undegraded":"2024-05-14T16:47:25.656114+0000","last_fullsized":"2024-05-14T16:47:25.656114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:16.362415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573546+0000","last_change":"2024-05-14T16:46:43.570743+0000","last_active":"2024-05-14T16:46:45.573546+0000","last_peered":"2024-05-14T16:46:45.573546+0000","last_clean":"2024-05-14T16:46:45.573546+0000","last_became_active":"2024-05-14T16:46:43.570427+0000","last_became_peered":"2024-05-14T16:46:43.570427+0000","last_unstale":"2024-05-14T16:46:45.573546+0000","last_undegraded":"2024-05-14T16:46:45.573546+0000","last_fullsized":"2024-05-14T16:46:45.573546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:04:11.091285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.536351+0000","last_change":"2024-05-14T16:46:40.546675+0000","last_active":"2024-05-14T16:46:59.536351+0000","last_peered":"2024-05-14T16:46:59.536351+0000","last_clean":"2024-05-14T16:46:59.536351+0000","last_became_active":"2024-05-14T16:46:40.546250+0000","last_became_peered":"2024-05-14T16:46:40.546250+0000","last_unstale":"2024-05-14T16:46:59.536351+0000","last_undegraded":"2024-05-14T16:46:59.536351+0000","last_fullsized":"2024-05-14T16:46:59.536351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:42:52.367719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564362+0000","last_change":"2024-05-14T16:46:43.579119+0000","last_active":"2024-05-14T16:46:45.564362+0000","last_peered":"2024-05-14T16:46:45.564362+0000","last_clean":"2024-05-14T16:46:45.564362+0000","last_became_active":"2024-05-14T16:46:43.578808+0000","last_became_peered":"2024-05-14T16:46:43.578808+0000","last_unstale":"2024-05-14T16:46:45.564362+0000","last_undegraded":"2024-05-14T16:46:45.564362+0000","last_fullsized":"2024-05-14T16:46:45.564362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:57.852982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.537856+0000","last_change":"2024-05-14T16:46:40.538827+0000","last_active":"2024-05-14T16:46:59.537856+0000","last_peered":"2024-05-14T16:46:59.537856+0000","last_clean":"2024-05-14T16:46:59.537856+0000","last_became_active":"2024-05-14T16:46:40.538369+0000","last_became_peered":"2024-05-14T16:46:40.538369+0000","last_unstale":"2024-05-14T16:46:59.537856+0000","last_undegraded":"2024-05-14T16:46:59.537856+0000","last_fullsized":"2024-05-14T16:46:59.537856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:48:03.599457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571807+0000","last_change":"2024-05-14T16:46:43.558411+0000","last_active":"2024-05-14T16:46:45.571807+0000","last_peered":"2024-05-14T16:46:45.571807+0000","last_clean":"2024-05-14T16:46:45.571807+0000","last_became_active":"2024-05-14T16:46:43.558101+0000","last_became_peered":"2024-05-14T16:46:43.558101+0000","last_unstale":"2024-05-14T16:46:45.571807+0000","last_undegraded":"2024-05-14T16:46:45.571807+0000","last_fullsized":"2024-05-14T16:46:45.571807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:10.595833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564008+0000","last_change":"2024-05-14T16:46:40.544858+0000","last_active":"2024-05-14T16:46:45.564008+0000","last_peered":"2024-05-14T16:46:45.564008+0000","last_clean":"2024-05-14T16:46:45.564008+0000","last_became_active":"2024-05-14T16:46:40.544197+0000","last_became_peered":"2024-05-14T16:46:40.544197+0000","last_unstale":"2024-05-14T16:46:45.564008+0000","last_undegraded":"2024-05-14T16:46:45.564008+0000","last_fullsized":"2024-05-14T16:46:45.564008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:16:45.249626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.574782+0000","last_change":"2024-05-14T16:46:43.567019+0000","last_active":"2024-05-14T16:46:45.574782+0000","last_peered":"2024-05-14T16:46:45.574782+0000","last_clean":"2024-05-14T16:46:45.574782+0000","last_became_active":"2024-05-14T16:46:43.566700+0000","last_became_peered":"2024-05-14T16:46:43.566700+0000","last_unstale":"2024-05-14T16:46:45.574782+0000","last_undegraded":"2024-05-14T16:46:45.574782+0000","last_fullsized":"2024-05-14T16:46:45.574782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:43.373467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":52,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:26.656820+0000","last_change":"2024-05-14T16:46:40.541876+0000","last_active":"2024-05-14T16:47:26.656820+0000","last_peered":"2024-05-14T16:47:26.656820+0000","last_clean":"2024-05-14T16:47:26.656820+0000","last_became_active":"2024-05-14T16:46:40.541460+0000","last_became_peered":"2024-05-14T16:46:40.541460+0000","last_unstale":"2024-05-14T16:47:26.656820+0000","last_undegraded":"2024-05-14T16:47:26.656820+0000","last_fullsized":"2024-05-14T16:47:26.656820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:07:22.536429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575557+0000","last_change":"2024-05-14T16:46:43.577015+0000","last_active":"2024-05-14T16:46:45.575557+0000","last_peered":"2024-05-14T16:46:45.575557+0000","last_clean":"2024-05-14T16:46:45.575557+0000","last_became_active":"2024-05-14T16:46:43.576657+0000","last_became_peered":"2024-05-14T16:46:43.576657+0000","last_unstale":"2024-05-14T16:46:45.575557+0000","last_undegraded":"2024-05-14T16:46:45.575557+0000","last_fullsized":"2024-05-14T16:46:45.575557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:29.242445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.531704+0000","last_change":"2024-05-14T16:46:40.550727+0000","last_active":"2024-05-14T16:46:58.531704+0000","last_peered":"2024-05-14T16:46:58.531704+0000","last_clean":"2024-05-14T16:46:58.531704+0000","last_became_active":"2024-05-14T16:46:40.550310+0000","last_became_peered":"2024-05-14T16:46:40.550310+0000","last_unstale":"2024-05-14T16:46:58.531704+0000","last_undegraded":"2024-05-14T16:46:58.531704+0000","last_fullsized":"2024-05-14T16:46:58.531704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:04.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573717+0000","last_change":"2024-05-14T16:46:43.573934+0000","last_active":"2024-05-14T16:46:45.573717+0000","last_peered":"2024-05-14T16:46:45.573717+0000","last_clean":"2024-05-14T16:46:45.573717+0000","last_became_active":"2024-05-14T16:46:43.573611+0000","last_became_peered":"2024-05-14T16:46:43.573611+0000","last_unstale":"2024-05-14T16:46:45.573717+0000","last_undegraded":"2024-05-14T16:46:45.573717+0000","last_fullsized":"2024-05-14T16:46:45.573717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:55:32.518517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.531604+0000","last_change":"2024-05-14T16:46:40.855680+0000","last_active":"2024-05-14T16:46:58.531604+0000","last_peered":"2024-05-14T16:46:58.531604+0000","last_clean":"2024-05-14T16:46:58.531604+0000","last_became_active":"2024-05-14T16:46:40.855265+0000","last_became_peered":"2024-05-14T16:46:40.855265+0000","last_unstale":"2024-05-14T16:46:58.531604+0000","last_undegraded":"2024-05-14T16:46:58.531604+0000","last_fullsized":"2024-05-14T16:46:58.531604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:26.065298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575390+0000","last_change":"2024-05-14T16:46:43.579423+0000","last_active":"2024-05-14T16:46:45.575390+0000","last_peered":"2024-05-14T16:46:45.575390+0000","last_clean":"2024-05-14T16:46:45.575390+0000","last_became_active":"2024-05-14T16:46:43.579023+0000","last_became_peered":"2024-05-14T16:46:43.579023+0000","last_unstale":"2024-05-14T16:46:45.575390+0000","last_undegraded":"2024-05-14T16:46:45.575390+0000","last_fullsized":"2024-05-14T16:46:45.575390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:21.311018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'30","reported_seq":114,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:27.659269+0000","last_change":"2024-05-14T16:46:40.550871+0000","last_active":"2024-05-14T16:47:27.659269+0000","last_peered":"2024-05-14T16:47:27.659269+0000","last_clean":"2024-05-14T16:47:27.659269+0000","last_became_active":"2024-05-14T16:46:40.550419+0000","last_became_peered":"2024-05-14T16:46:40.550419+0000","last_unstale":"2024-05-14T16:47:27.659269+0000","last_undegraded":"2024-05-14T16:47:27.659269+0000","last_fullsized":"2024-05-14T16:47:27.659269+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:53:09.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":62,"num_read_kb":86,"num_write":31,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.574565+0000","last_change":"2024-05-14T16:46:43.569996+0000","last_active":"2024-05-14T16:46:45.574565+0000","last_peered":"2024-05-14T16:46:45.574565+0000","last_clean":"2024-05-14T16:46:45.574565+0000","last_became_active":"2024-05-14T16:46:43.569501+0000","last_became_peered":"2024-05-14T16:46:43.569501+0000","last_unstale":"2024-05-14T16:46:45.574565+0000","last_undegraded":"2024-05-14T16:46:45.574565+0000","last_fullsized":"2024-05-14T16:46:45.574565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:55.135103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.537448+0000","last_change":"2024-05-14T16:46:40.547684+0000","last_active":"2024-05-14T16:46:59.537448+0000","last_peered":"2024-05-14T16:46:59.537448+0000","last_clean":"2024-05-14T16:46:59.537448+0000","last_became_active":"2024-05-14T16:46:40.547076+0000","last_became_peered":"2024-05-14T16:46:40.547076+0000","last_unstale":"2024-05-14T16:46:59.537448+0000","last_undegraded":"2024-05-14T16:46:59.537448+0000","last_fullsized":"2024-05-14T16:46:59.537448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:44:52.349456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564119+0000","last_change":"2024-05-14T16:46:43.562773+0000","last_active":"2024-05-14T16:46:45.564119+0000","last_peered":"2024-05-14T16:46:45.564119+0000","last_clean":"2024-05-14T16:46:45.564119+0000","last_became_active":"2024-05-14T16:46:43.562449+0000","last_became_peered":"2024-05-14T16:46:43.562449+0000","last_unstale":"2024-05-14T16:46:45.564119+0000","last_undegraded":"2024-05-14T16:46:45.564119+0000","last_fullsized":"2024-05-14T16:46:45.564119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:45.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.532204+0000","last_change":"2024-05-14T16:46:40.545882+0000","last_active":"2024-05-14T16:46:58.532204+0000","last_peered":"2024-05-14T16:46:58.532204+0000","last_clean":"2024-05-14T16:46:58.532204+0000","last_became_active":"2024-05-14T16:46:40.545439+0000","last_became_peered":"2024-05-14T16:46:40.545439+0000","last_unstale":"2024-05-14T16:46:58.532204+0000","last_undegraded":"2024-05-14T16:46:58.532204+0000","last_fullsized":"2024-05-14T16:46:58.532204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:07.983329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571795+0000","last_change":"2024-05-14T16:46:43.568109+0000","last_active":"2024-05-14T16:46:45.571795+0000","last_peered":"2024-05-14T16:46:45.571795+0000","last_clean":"2024-05-14T16:46:45.571795+0000","last_became_active":"2024-05-14T16:46:43.567106+0000","last_became_peered":"2024-05-14T16:46:43.567106+0000","last_unstale":"2024-05-14T16:46:45.571795+0000","last_undegraded":"2024-05-14T16:46:45.571795+0000","last_fullsized":"2024-05-14T16:46:45.571795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:29.705387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:26.656987+0000","last_change":"2024-05-14T16:46:40.542204+0000","last_active":"2024-05-14T16:47:26.656987+0000","last_peered":"2024-05-14T16:47:26.656987+0000","last_clean":"2024-05-14T16:47:26.656987+0000","last_became_active":"2024-05-14T16:46:40.541807+0000","last_became_peered":"2024-05-14T16:46:40.541807+0000","last_unstale":"2024-05-14T16:47:26.656987+0000","last_undegraded":"2024-05-14T16:47:26.656987+0000","last_fullsized":"2024-05-14T16:47:26.656987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:30:45.115647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571360+0000","last_change":"2024-05-14T16:46:43.573546+0000","last_active":"2024-05-14T16:46:45.571360+0000","last_peered":"2024-05-14T16:46:45.571360+0000","last_clean":"2024-05-14T16:46:45.571360+0000","last_became_active":"2024-05-14T16:46:43.573183+0000","last_became_peered":"2024-05-14T16:46:43.573183+0000","last_unstale":"2024-05-14T16:46:45.571360+0000","last_undegraded":"2024-05-14T16:46:45.571360+0000","last_fullsized":"2024-05-14T16:46:45.571360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:49.873097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.538139+0000","last_change":"2024-05-14T16:46:40.855713+0000","last_active":"2024-05-14T16:46:59.538139+0000","last_peered":"2024-05-14T16:46:59.538139+0000","last_clean":"2024-05-14T16:46:59.538139+0000","last_became_active":"2024-05-14T16:46:40.855292+0000","last_became_peered":"2024-05-14T16:46:40.855292+0000","last_unstale":"2024-05-14T16:46:59.538139+0000","last_undegraded":"2024-05-14T16:46:59.538139+0000","last_fullsized":"2024-05-14T16:46:59.538139+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:49:47.045820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564109+0000","last_change":"2024-05-14T16:46:43.579059+0000","last_active":"2024-05-14T16:46:45.564109+0000","last_peered":"2024-05-14T16:46:45.564109+0000","last_clean":"2024-05-14T16:46:45.564109+0000","last_became_active":"2024-05-14T16:46:43.578748+0000","last_became_peered":"2024-05-14T16:46:43.578748+0000","last_unstale":"2024-05-14T16:46:45.564109+0000","last_undegraded":"2024-05-14T16:46:45.564109+0000","last_fullsized":"2024-05-14T16:46:45.564109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:42.446422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:00.532869+0000","last_change":"2024-05-14T16:46:40.549047+0000","last_active":"2024-05-14T16:47:00.532869+0000","last_peered":"2024-05-14T16:47:00.532869+0000","last_clean":"2024-05-14T16:47:00.532869+0000","last_became_active":"2024-05-14T16:46:40.548648+0000","last_became_peered":"2024-05-14T16:46:40.548648+0000","last_unstale":"2024-05-14T16:47:00.532869+0000","last_undegraded":"2024-05-14T16:47:00.532869+0000","last_fullsized":"2024-05-14T16:47:00.532869+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:52:56.370871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572329+0000","last_change":"2024-05-14T16:46:43.569624+0000","last_active":"2024-05-14T16:46:45.572329+0000","last_peered":"2024-05-14T16:46:45.572329+0000","last_clean":"2024-05-14T16:46:45.572329+0000","last_became_active":"2024-05-14T16:46:43.569077+0000","last_became_peered":"2024-05-14T16:46:43.569077+0000","last_unstale":"2024-05-14T16:46:45.572329+0000","last_undegraded":"2024-05-14T16:46:45.572329+0000","last_fullsized":"2024-05-14T16:46:45.572329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:45:50.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.574873+0000","last_change":"2024-05-14T16:46:40.856302+0000","last_active":"2024-05-14T16:46:45.574873+0000","last_peered":"2024-05-14T16:46:45.574873+0000","last_clean":"2024-05-14T16:46:45.574873+0000","last_became_active":"2024-05-14T16:46:40.855964+0000","last_became_peered":"2024-05-14T16:46:40.855964+0000","last_unstale":"2024-05-14T16:46:45.574873+0000","last_undegraded":"2024-05-14T16:46:45.574873+0000","last_fullsized":"2024-05-14T16:46:45.574873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:38.501418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571807+0000","last_change":"2024-05-14T16:46:43.562847+0000","last_active":"2024-05-14T16:46:45.571807+0000","last_peered":"2024-05-14T16:46:45.571807+0000","last_clean":"2024-05-14T16:46:45.571807+0000","last_became_active":"2024-05-14T16:46:43.560978+0000","last_became_peered":"2024-05-14T16:46:43.560978+0000","last_unstale":"2024-05-14T16:46:45.571807+0000","last_undegraded":"2024-05-14T16:46:45.571807+0000","last_fullsized":"2024-05-14T16:46:45.571807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:12:33.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571711+0000","last_change":"2024-05-14T16:46:40.544017+0000","last_active":"2024-05-14T16:46:45.571711+0000","last_peered":"2024-05-14T16:46:45.571711+0000","last_clean":"2024-05-14T16:46:45.571711+0000","last_became_active":"2024-05-14T16:46:40.543176+0000","last_became_peered":"2024-05-14T16:46:40.543176+0000","last_unstale":"2024-05-14T16:46:45.571711+0000","last_undegraded":"2024-05-14T16:46:45.571711+0000","last_fullsized":"2024-05-14T16:46:45.571711+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:09.817891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.570976+0000","last_change":"2024-05-14T16:46:43.564395+0000","last_active":"2024-05-14T16:46:45.570976+0000","last_peered":"2024-05-14T16:46:45.570976+0000","last_clean":"2024-05-14T16:46:45.570976+0000","last_became_active":"2024-05-14T16:46:43.564065+0000","last_became_peered":"2024-05-14T16:46:43.564065+0000","last_unstale":"2024-05-14T16:46:45.570976+0000","last_undegraded":"2024-05-14T16:46:45.570976+0000","last_fullsized":"2024-05-14T16:46:45.570976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:52.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564551+0000","last_change":"2024-05-14T16:46:43.577067+0000","last_active":"2024-05-14T16:46:45.564551+0000","last_peered":"2024-05-14T16:46:45.564551+0000","last_clean":"2024-05-14T16:46:45.564551+0000","last_became_active":"2024-05-14T16:46:43.576750+0000","last_became_peered":"2024-05-14T16:46:43.576750+0000","last_unstale":"2024-05-14T16:46:45.564551+0000","last_undegraded":"2024-05-14T16:46:45.564551+0000","last_fullsized":"2024-05-14T16:46:45.564551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:25.961124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:25.654849+0000","last_change":"2024-05-14T16:46:40.855073+0000","last_active":"2024-05-14T16:47:25.654849+0000","last_peered":"2024-05-14T16:47:25.654849+0000","last_clean":"2024-05-14T16:47:25.654849+0000","last_became_active":"2024-05-14T16:46:40.854641+0000","last_became_peered":"2024-05-14T16:46:40.854641+0000","last_unstale":"2024-05-14T16:47:25.654849+0000","last_undegraded":"2024-05-14T16:47:25.654849+0000","last_fullsized":"2024-05-14T16:47:25.654849+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:58:11.670926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572335+0000","last_change":"2024-05-14T16:46:43.569030+0000","last_active":"2024-05-14T16:46:45.572335+0000","last_peered":"2024-05-14T16:46:45.572335+0000","last_clean":"2024-05-14T16:46:45.572335+0000","last_became_active":"2024-05-14T16:46:43.568646+0000","last_became_peered":"2024-05-14T16:46:43.568646+0000","last_unstale":"2024-05-14T16:46:45.572335+0000","last_undegraded":"2024-05-14T16:46:45.572335+0000","last_fullsized":"2024-05-14T16:46:45.572335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:38.086818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:01.615659+0000","last_change":"2024-05-14T16:46:40.542624+0000","last_active":"2024-05-14T16:47:01.615659+0000","last_peered":"2024-05-14T16:47:01.615659+0000","last_clean":"2024-05-14T16:47:01.615659+0000","last_became_active":"2024-05-14T16:46:40.542230+0000","last_became_peered":"2024-05-14T16:46:40.542230+0000","last_unstale":"2024-05-14T16:47:01.615659+0000","last_undegraded":"2024-05-14T16:47:01.615659+0000","last_fullsized":"2024-05-14T16:47:01.615659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:54:50.083362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571922+0000","last_change":"2024-05-14T16:45:15.724994+0000","last_active":"2024-05-14T16:46:45.571922+0000","last_peered":"2024-05-14T16:46:45.571922+0000","last_clean":"2024-05-14T16:46:45.571922+0000","last_became_active":"2024-05-14T16:45:15.720080+0000","last_became_peered":"2024-05-14T16:45:15.720080+0000","last_unstale":"2024-05-14T16:46:45.571922+0000","last_undegraded":"2024-05-14T16:46:45.571922+0000","last_fullsized":"2024-05-14T16:46:45.571922+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_clean_scrub_stamp":"2024-05-14T16:41:54.615675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:32:43.569611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564202+0000","last_change":"2024-05-14T16:46:43.567143+0000","last_active":"2024-05-14T16:46:45.564202+0000","last_peered":"2024-05-14T16:46:45.564202+0000","last_clean":"2024-05-14T16:46:45.564202+0000","last_became_active":"2024-05-14T16:46:43.564717+0000","last_became_peered":"2024-05-14T16:46:43.564717+0000","last_unstale":"2024-05-14T16:46:45.564202+0000","last_undegraded":"2024-05-14T16:46:45.564202+0000","last_fullsized":"2024-05-14T16:46:45.564202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:59.025279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.535914+0000","last_change":"2024-05-14T16:46:40.546709+0000","last_active":"2024-05-14T16:46:59.535914+0000","last_peered":"2024-05-14T16:46:59.535914+0000","last_clean":"2024-05-14T16:46:59.535914+0000","last_became_active":"2024-05-14T16:46:40.546295+0000","last_became_peered":"2024-05-14T16:46:40.546295+0000","last_unstale":"2024-05-14T16:46:59.535914+0000","last_undegraded":"2024-05-14T16:46:59.535914+0000","last_fullsized":"2024-05-14T16:46:59.535914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:42:50.255289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.531349+0000","last_change":"2024-05-14T16:46:40.856822+0000","last_active":"2024-05-14T16:46:58.531349+0000","last_peered":"2024-05-14T16:46:58.531349+0000","last_clean":"2024-05-14T16:46:58.531349+0000","last_became_active":"2024-05-14T16:46:40.856434+0000","last_became_peered":"2024-05-14T16:46:40.856434+0000","last_unstale":"2024-05-14T16:46:58.531349+0000","last_undegraded":"2024-05-14T16:46:58.531349+0000","last_fullsized":"2024-05-14T16:46:58.531349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:38.997679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564282+0000","last_change":"2024-05-14T16:46:43.563918+0000","last_active":"2024-05-14T16:46:45.564282+0000","last_peered":"2024-05-14T16:46:45.564282+0000","last_clean":"2024-05-14T16:46:45.564282+0000","last_became_active":"2024-05-14T16:46:43.563527+0000","last_became_peered":"2024-05-14T16:46:43.563527+0000","last_unstale":"2024-05-14T16:46:45.564282+0000","last_undegraded":"2024-05-14T16:46:45.564282+0000","last_fullsized":"2024-05-14T16:46:45.564282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:58:52.984695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'11","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:47:23.574304+0000","last_change":"2024-05-14T16:46:40.547480+0000","last_active":"2024-05-14T16:47:23.574304+0000","last_peered":"2024-05-14T16:47:23.574304+0000","last_clean":"2024-05-14T16:47:23.574304+0000","last_became_active":"2024-05-14T16:46:40.547062+0000","last_became_peered":"2024-05-14T16:46:40.547062+0000","last_unstale":"2024-05-14T16:47:23.574304+0000","last_undegraded":"2024-05-14T16:47:23.574304+0000","last_fullsized":"2024-05-14T16:47:23.574304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:30.211421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573020+0000","last_change":"2024-05-14T16:46:43.563870+0000","last_active":"2024-05-14T16:46:45.573020+0000","last_peered":"2024-05-14T16:46:45.573020+0000","last_clean":"2024-05-14T16:46:45.573020+0000","last_became_active":"2024-05-14T16:46:43.563542+0000","last_became_peered":"2024-05-14T16:46:43.563542+0000","last_unstale":"2024-05-14T16:46:45.573020+0000","last_undegraded":"2024-05-14T16:46:45.573020+0000","last_fullsized":"2024-05-14T16:46:45.573020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:50.699601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.531082+0000","last_change":"2024-05-14T16:46:40.665192+0000","last_active":"2024-05-14T16:46:58.531082+0000","last_peered":"2024-05-14T16:46:58.531082+0000","last_clean":"2024-05-14T16:46:58.531082+0000","last_became_active":"2024-05-14T16:46:40.664827+0000","last_became_peered":"2024-05-14T16:46:40.664827+0000","last_unstale":"2024-05-14T16:46:58.531082+0000","last_undegraded":"2024-05-14T16:46:58.531082+0000","last_fullsized":"2024-05-14T16:46:58.531082+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:41:47.179677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572828+0000","last_change":"2024-05-14T16:46:43.572596+0000","last_active":"2024-05-14T16:46:45.572828+0000","last_peered":"2024-05-14T16:46:45.572828+0000","last_clean":"2024-05-14T16:46:45.572828+0000","last_became_active":"2024-05-14T16:46:43.572271+0000","last_became_peered":"2024-05-14T16:46:43.572271+0000","last_unstale":"2024-05-14T16:46:45.572828+0000","last_undegraded":"2024-05-14T16:46:45.572828+0000","last_fullsized":"2024-05-14T16:46:45.572828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:39:38.132462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.532196+0000","last_change":"2024-05-14T16:46:40.855458+0000","last_active":"2024-05-14T16:46:58.532196+0000","last_peered":"2024-05-14T16:46:58.532196+0000","last_clean":"2024-05-14T16:46:58.532196+0000","last_became_active":"2024-05-14T16:46:40.855122+0000","last_became_peered":"2024-05-14T16:46:40.855122+0000","last_unstale":"2024-05-14T16:46:58.532196+0000","last_undegraded":"2024-05-14T16:46:58.532196+0000","last_fullsized":"2024-05-14T16:46:58.532196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:20:43.435860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573451+0000","last_change":"2024-05-14T16:46:43.566482+0000","last_active":"2024-05-14T16:46:45.573451+0000","last_peered":"2024-05-14T16:46:45.573451+0000","last_clean":"2024-05-14T16:46:45.573451+0000","last_became_active":"2024-05-14T16:46:43.566145+0000","last_became_peered":"2024-05-14T16:46:43.566145+0000","last_unstale":"2024-05-14T16:46:45.573451+0000","last_undegraded":"2024-05-14T16:46:45.573451+0000","last_fullsized":"2024-05-14T16:46:45.573451+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:14:22.731629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.530741+0000","last_change":"2024-05-14T16:46:40.541655+0000","last_active":"2024-05-14T16:46:58.530741+0000","last_peered":"2024-05-14T16:46:58.530741+0000","last_clean":"2024-05-14T16:46:58.530741+0000","last_became_active":"2024-05-14T16:46:40.541331+0000","last_became_peered":"2024-05-14T16:46:40.541331+0000","last_unstale":"2024-05-14T16:46:58.530741+0000","last_undegraded":"2024-05-14T16:46:58.530741+0000","last_fullsized":"2024-05-14T16:46:58.530741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:47:27.950532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572475+0000","last_change":"2024-05-14T16:46:43.564216+0000","last_active":"2024-05-14T16:46:45.572475+0000","last_peered":"2024-05-14T16:46:45.572475+0000","last_clean":"2024-05-14T16:46:45.572475+0000","last_became_active":"2024-05-14T16:46:43.563894+0000","last_became_peered":"2024-05-14T16:46:43.563894+0000","last_unstale":"2024-05-14T16:46:45.572475+0000","last_undegraded":"2024-05-14T16:46:45.572475+0000","last_fullsized":"2024-05-14T16:46:45.572475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:16:09.256909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571604+0000","last_change":"2024-05-14T16:46:40.551034+0000","last_active":"2024-05-14T16:46:45.571604+0000","last_peered":"2024-05-14T16:46:45.571604+0000","last_clean":"2024-05-14T16:46:45.571604+0000","last_became_active":"2024-05-14T16:46:40.550211+0000","last_became_peered":"2024-05-14T16:46:40.550211+0000","last_unstale":"2024-05-14T16:46:45.571604+0000","last_undegraded":"2024-05-14T16:46:45.571604+0000","last_fullsized":"2024-05-14T16:46:45.571604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:15.720942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572750+0000","last_change":"2024-05-14T16:46:43.558264+0000","last_active":"2024-05-14T16:46:45.572750+0000","last_peered":"2024-05-14T16:46:45.572750+0000","last_clean":"2024-05-14T16:46:45.572750+0000","last_became_active":"2024-05-14T16:46:43.557907+0000","last_became_peered":"2024-05-14T16:46:43.557907+0000","last_unstale":"2024-05-14T16:46:45.572750+0000","last_undegraded":"2024-05-14T16:46:45.572750+0000","last_fullsized":"2024-05-14T16:46:45.572750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:20:52.752482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:58.533416+0000","last_change":"2024-05-14T16:46:40.551013+0000","last_active":"2024-05-14T16:46:58.533416+0000","last_peered":"2024-05-14T16:46:58.533416+0000","last_clean":"2024-05-14T16:46:58.533416+0000","last_became_active":"2024-05-14T16:46:40.550658+0000","last_became_peered":"2024-05-14T16:46:40.550658+0000","last_unstale":"2024-05-14T16:46:58.533416+0000","last_undegraded":"2024-05-14T16:46:58.533416+0000","last_fullsized":"2024-05-14T16:46:58.533416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:36.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571905+0000","last_change":"2024-05-14T16:46:43.572270+0000","last_active":"2024-05-14T16:46:45.571905+0000","last_peered":"2024-05-14T16:46:45.571905+0000","last_clean":"2024-05-14T16:46:45.571905+0000","last_became_active":"2024-05-14T16:46:43.571886+0000","last_became_peered":"2024-05-14T16:46:43.571886+0000","last_unstale":"2024-05-14T16:46:45.571905+0000","last_undegraded":"2024-05-14T16:46:45.571905+0000","last_fullsized":"2024-05-14T16:46:45.571905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:11.890881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:59.537950+0000","last_change":"2024-05-14T16:46:40.855707+0000","last_active":"2024-05-14T16:46:59.537950+0000","last_peered":"2024-05-14T16:46:59.537950+0000","last_clean":"2024-05-14T16:46:59.537950+0000","last_became_active":"2024-05-14T16:46:40.855335+0000","last_became_peered":"2024-05-14T16:46:40.855335+0000","last_unstale":"2024-05-14T16:46:59.537950+0000","last_undegraded":"2024-05-14T16:46:59.537950+0000","last_fullsized":"2024-05-14T16:46:59.537950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:59:47.192246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572675+0000","last_change":"2024-05-14T16:46:43.572455+0000","last_active":"2024-05-14T16:46:45.572675+0000","last_peered":"2024-05-14T16:46:45.572675+0000","last_clean":"2024-05-14T16:46:45.572675+0000","last_became_active":"2024-05-14T16:46:43.571930+0000","last_became_peered":"2024-05-14T16:46:43.571930+0000","last_unstale":"2024-05-14T16:46:45.572675+0000","last_undegraded":"2024-05-14T16:46:45.572675+0000","last_fullsized":"2024-05-14T16:46:45.572675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:35:39.643239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.575557+0000","last_change":"2024-05-14T16:46:40.541708+0000","last_active":"2024-05-14T16:46:45.575557+0000","last_peered":"2024-05-14T16:46:45.575557+0000","last_clean":"2024-05-14T16:46:45.575557+0000","last_became_active":"2024-05-14T16:46:40.541202+0000","last_became_peered":"2024-05-14T16:46:40.541202+0000","last_unstale":"2024-05-14T16:46:45.575557+0000","last_undegraded":"2024-05-14T16:46:45.575557+0000","last_fullsized":"2024-05-14T16:46:45.575557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:50:01.821013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.573320+0000","last_change":"2024-05-14T16:46:43.574039+0000","last_active":"2024-05-14T16:46:45.573320+0000","last_peered":"2024-05-14T16:46:45.573320+0000","last_clean":"2024-05-14T16:46:45.573320+0000","last_became_active":"2024-05-14T16:46:43.573237+0000","last_became_peered":"2024-05-14T16:46:43.573237+0000","last_unstale":"2024-05-14T16:46:45.573320+0000","last_undegraded":"2024-05-14T16:46:45.573320+0000","last_fullsized":"2024-05-14T16:46:45.573320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:13.234495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.570397+0000","last_change":"2024-05-14T16:46:40.549519+0000","last_active":"2024-05-14T16:46:45.570397+0000","last_peered":"2024-05-14T16:46:45.570397+0000","last_clean":"2024-05-14T16:46:45.570397+0000","last_became_active":"2024-05-14T16:46:40.549169+0000","last_became_peered":"2024-05-14T16:46:40.549169+0000","last_unstale":"2024-05-14T16:46:45.570397+0000","last_undegraded":"2024-05-14T16:46:45.570397+0000","last_fullsized":"2024-05-14T16:46:45.570397+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:22.559724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572368+0000","last_change":"2024-05-14T16:46:43.567674+0000","last_active":"2024-05-14T16:46:45.572368+0000","last_peered":"2024-05-14T16:46:45.572368+0000","last_clean":"2024-05-14T16:46:45.572368+0000","last_became_active":"2024-05-14T16:46:43.567337+0000","last_became_peered":"2024-05-14T16:46:43.567337+0000","last_unstale":"2024-05-14T16:46:45.572368+0000","last_undegraded":"2024-05-14T16:46:45.572368+0000","last_fullsized":"2024-05-14T16:46:45.572368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:02.321955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572683+0000","last_change":"2024-05-14T16:46:43.567627+0000","last_active":"2024-05-14T16:46:45.572683+0000","last_peered":"2024-05-14T16:46:45.572683+0000","last_clean":"2024-05-14T16:46:45.572683+0000","last_became_active":"2024-05-14T16:46:43.567313+0000","last_became_peered":"2024-05-14T16:46:43.567313+0000","last_unstale":"2024-05-14T16:46:45.572683+0000","last_undegraded":"2024-05-14T16:46:45.572683+0000","last_fullsized":"2024-05-14T16:46:45.572683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:47.396923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571179+0000","last_change":"2024-05-14T16:46:43.571872+0000","last_active":"2024-05-14T16:46:45.571179+0000","last_peered":"2024-05-14T16:46:45.571179+0000","last_clean":"2024-05-14T16:46:45.571179+0000","last_became_active":"2024-05-14T16:46:43.571495+0000","last_became_peered":"2024-05-14T16:46:43.571495+0000","last_unstale":"2024-05-14T16:46:45.571179+0000","last_undegraded":"2024-05-14T16:46:45.571179+0000","last_fullsized":"2024-05-14T16:46:45.571179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:34.639950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571419+0000","last_change":"2024-05-14T16:46:43.572671+0000","last_active":"2024-05-14T16:46:45.571419+0000","last_peered":"2024-05-14T16:46:45.571419+0000","last_clean":"2024-05-14T16:46:45.571419+0000","last_became_active":"2024-05-14T16:46:43.572357+0000","last_became_peered":"2024-05-14T16:46:43.572357+0000","last_unstale":"2024-05-14T16:46:45.571419+0000","last_undegraded":"2024-05-14T16:46:45.571419+0000","last_fullsized":"2024-05-14T16:46:45.571419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:57.597636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572931+0000","last_change":"2024-05-14T16:46:43.572168+0000","last_active":"2024-05-14T16:46:45.572931+0000","last_peered":"2024-05-14T16:46:45.572931+0000","last_clean":"2024-05-14T16:46:45.572931+0000","last_became_active":"2024-05-14T16:46:43.571822+0000","last_became_peered":"2024-05-14T16:46:43.571822+0000","last_unstale":"2024-05-14T16:46:45.572931+0000","last_undegraded":"2024-05-14T16:46:45.572931+0000","last_fullsized":"2024-05-14T16:46:45.572931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:57.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564338+0000","last_change":"2024-05-14T16:46:43.570935+0000","last_active":"2024-05-14T16:46:45.564338+0000","last_peered":"2024-05-14T16:46:45.564338+0000","last_clean":"2024-05-14T16:46:45.564338+0000","last_became_active":"2024-05-14T16:46:43.570605+0000","last_became_peered":"2024-05-14T16:46:43.570605+0000","last_unstale":"2024-05-14T16:46:45.564338+0000","last_undegraded":"2024-05-14T16:46:45.564338+0000","last_fullsized":"2024-05-14T16:46:45.564338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:17:55.161195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572218+0000","last_change":"2024-05-14T16:46:43.568442+0000","last_active":"2024-05-14T16:46:45.572218+0000","last_peered":"2024-05-14T16:46:45.572218+0000","last_clean":"2024-05-14T16:46:45.572218+0000","last_became_active":"2024-05-14T16:46:43.568118+0000","last_became_peered":"2024-05-14T16:46:43.568118+0000","last_unstale":"2024-05-14T16:46:45.572218+0000","last_undegraded":"2024-05-14T16:46:45.572218+0000","last_fullsized":"2024-05-14T16:46:45.572218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:52.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572315+0000","last_change":"2024-05-14T16:46:43.576174+0000","last_active":"2024-05-14T16:46:45.572315+0000","last_peered":"2024-05-14T16:46:45.572315+0000","last_clean":"2024-05-14T16:46:45.572315+0000","last_became_active":"2024-05-14T16:46:43.575737+0000","last_became_peered":"2024-05-14T16:46:43.575737+0000","last_unstale":"2024-05-14T16:46:45.572315+0000","last_undegraded":"2024-05-14T16:46:45.572315+0000","last_fullsized":"2024-05-14T16:46:45.572315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:08.453070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572904+0000","last_change":"2024-05-14T16:46:43.566256+0000","last_active":"2024-05-14T16:46:45.572904+0000","last_peered":"2024-05-14T16:46:45.572904+0000","last_clean":"2024-05-14T16:46:45.572904+0000","last_became_active":"2024-05-14T16:46:43.565886+0000","last_became_peered":"2024-05-14T16:46:43.565886+0000","last_unstale":"2024-05-14T16:46:45.572904+0000","last_undegraded":"2024-05-14T16:46:45.572904+0000","last_fullsized":"2024-05-14T16:46:45.572904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:50.021286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564042+0000","last_change":"2024-05-14T16:46:43.569819+0000","last_active":"2024-05-14T16:46:45.564042+0000","last_peered":"2024-05-14T16:46:45.564042+0000","last_clean":"2024-05-14T16:46:45.564042+0000","last_became_active":"2024-05-14T16:46:43.569461+0000","last_became_peered":"2024-05-14T16:46:43.569461+0000","last_unstale":"2024-05-14T16:46:45.564042+0000","last_undegraded":"2024-05-14T16:46:45.564042+0000","last_fullsized":"2024-05-14T16:46:45.564042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:52.013604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.572023+0000","last_change":"2024-05-14T16:46:43.567569+0000","last_active":"2024-05-14T16:46:45.572023+0000","last_peered":"2024-05-14T16:46:45.572023+0000","last_clean":"2024-05-14T16:46:45.572023+0000","last_became_active":"2024-05-14T16:46:43.564808+0000","last_became_peered":"2024-05-14T16:46:43.564808+0000","last_unstale":"2024-05-14T16:46:45.572023+0000","last_undegraded":"2024-05-14T16:46:45.572023+0000","last_fullsized":"2024-05-14T16:46:45.572023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:13.744730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.576092+0000","last_change":"2024-05-14T16:46:43.578417+0000","last_active":"2024-05-14T16:46:45.576092+0000","last_peered":"2024-05-14T16:46:45.576092+0000","last_clean":"2024-05-14T16:46:45.576092+0000","last_became_active":"2024-05-14T16:46:43.577687+0000","last_became_peered":"2024-05-14T16:46:43.577687+0000","last_unstale":"2024-05-14T16:46:45.576092+0000","last_undegraded":"2024-05-14T16:46:45.576092+0000","last_fullsized":"2024-05-14T16:46:45.576092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:32.060934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.564376+0000","last_change":"2024-05-14T16:46:43.564114+0000","last_active":"2024-05-14T16:46:45.564376+0000","last_peered":"2024-05-14T16:46:45.564376+0000","last_clean":"2024-05-14T16:46:45.564376+0000","last_became_active":"2024-05-14T16:46:43.563734+0000","last_became_peered":"2024-05-14T16:46:43.563734+0000","last_unstale":"2024-05-14T16:46:45.564376+0000","last_undegraded":"2024-05-14T16:46:45.564376+0000","last_fullsized":"2024-05-14T16:46:45.564376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:19.016741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:46:45.571845+0000","last_change":"2024-05-14T16:46:43.566490+0000","last_active":"2024-05-14T16:46:45.571845+0000","last_peered":"2024-05-14T16:46:45.571845+0000","last_clean":"2024-05-14T16:46:45.571845+0000","last_became_active":"2024-05-14T16:46:43.566086+0000","last_became_peered":"2024-05-14T16:46:43.566086+0000","last_unstale":"2024-05-14T16:46:45.571845+0000","last_undegraded":"2024-05-14T16:46:45.571845+0000","last_fullsized":"2024-05-14T16:46:45.571845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:39:09.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245266,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65360,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":186,"num_read_kb":157,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":191823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710744,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":408028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":1057002,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070499,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":856064,"data_stored":408264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266728,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":999723,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":407653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":464939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":408924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":466289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":407964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":847872,"data_stored":408090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":999407,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:47:32.364 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T16:47:32.364 INFO:tasks.cephadm:Setting up client nodes... 2024-05-14T16:47:32.364 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-14T16:47:32.642 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:32.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:32 smithi069 ceph-mon[31507]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:47:32.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-14T16:47:32.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-14T16:47:32.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-14T16:47:32.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-05-14T16:47:32.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-14T16:47:32.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-14T16:47:32.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-14T16:47:32.701 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-14T16:47:32.701 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:32 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-14T16:47:32.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:32 smithi203 ceph-mon[35787]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:47:32.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-14T16:47:32.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-14T16:47:32.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-14T16:47:32.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-05-14T16:47:32.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-14T16:47:32.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-14T16:47:32.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-14T16:47:32.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-14T16:47:32.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:32 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-14T16:47:33.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:32 smithi092 ceph-mon[35763]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:47:33.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-14T16:47:33.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-14T16:47:33.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-14T16:47:33.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-05-14T16:47:33.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-14T16:47:33.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-14T16:47:33.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-14T16:47:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-14T16:47:33.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:32 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-14T16:47:33.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:33 smithi203 ceph-mon[35787]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T16:47:33.966 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-14T16:47:33.966 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-14T16:47:33.966 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-14T16:47:33.966 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-14T16:47:33.966 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-05-14T16:47:33.966 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-14T16:47:33.966 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-14T16:47:33.966 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-14T16:47:33.966 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-14T16:47:33.967 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-14T16:47:33.967 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:33 smithi069 ceph-mon[31507]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T16:47:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-14T16:47:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-14T16:47:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-14T16:47:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-14T16:47:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-05-14T16:47:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-14T16:47:34.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-14T16:47:34.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-14T16:47:34.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-14T16:47:34.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-14T16:47:34.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:33 smithi092 ceph-mon[35763]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T16:47:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.370+0000 7f971c148700 1 -- 172.21.15.69:0/2597514869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9714104080 msgr2=0x7f971410a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.370+0000 7f971c148700 1 --2- 172.21.15.69:0/2597514869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9714104080 0x7f971410a930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b0d0 tx=0x7f970c00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.370+0000 7f971c148700 1 -- 172.21.15.69:0/2597514869 shutdown_connections 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.370+0000 7f971c148700 1 --2- 172.21.15.69:0/2597514869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9714104080 0x7f971410a930 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.370+0000 7f971c148700 1 --2- 172.21.15.69:0/2597514869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97141024e0 0x7f9714102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.370+0000 7f971c148700 1 --2- 172.21.15.69:0/2597514869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141036e0 0x7f9714103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.370+0000 7f971c148700 1 -- 172.21.15.69:0/2597514869 >> 172.21.15.69:0/2597514869 conn(0x7f97140fda60 msgr2=0x7f97140ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.370+0000 7f971c148700 1 -- 172.21.15.69:0/2597514869 shutdown_connections 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.370+0000 7f971c148700 1 -- 172.21.15.69:0/2597514869 wait complete. 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f971c148700 1 Processor -- start 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f971c148700 1 -- start start 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f971c148700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141024e0 0x7f9714199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f971c148700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97141036e0 0x7f9714199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f971c148700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9714104080 0x7f97141a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f971c148700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f971410df60 con 0x7f97141036e0 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f971c148700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f971410dde0 con 0x7f9714104080 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f971c148700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f971410e0e0 con 0x7f97141024e0 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f97196e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97141036e0 0x7f9714199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f97196e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97141036e0 0x7f9714199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35306/0 (socket says 172.21.15.69:35306) 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f97196e3700 1 -- 172.21.15.69:0/350157295 learned_addr learned my addr 172.21.15.69:0/350157295 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f9719ee4700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141024e0 0x7f9714199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f971a6e5700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9714104080 0x7f97141a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f97196e3700 1 -- 172.21.15.69:0/350157295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141024e0 msgr2=0x7f9714199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f97196e3700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141024e0 0x7f9714199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f97196e3700 1 -- 172.21.15.69:0/350157295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9714104080 msgr2=0x7f97141a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f97196e3700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9714104080 0x7f97141a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f97196e3700 1 -- 172.21.15.69:0/350157295 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97141a11a0 con 0x7f97141036e0 2024-05-14T16:47:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.371+0000 7f97196e3700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97141036e0 0x7f9714199ad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f970400c950 tx=0x7f970400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.372+0000 7f970affd700 1 -- 172.21.15.69:0/350157295 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97040078d0 con 0x7f97141036e0 2024-05-14T16:47:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.372+0000 7f970affd700 1 -- 172.21.15.69:0/350157295 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f970400f450 con 0x7f97141036e0 2024-05-14T16:47:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.372+0000 7f971c148700 1 -- 172.21.15.69:0/350157295 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97141a1490 con 0x7f97141036e0 2024-05-14T16:47:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.372+0000 7f970affd700 1 -- 172.21.15.69:0/350157295 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704018660 con 0x7f97141036e0 2024-05-14T16:47:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.372+0000 7f971c148700 1 -- 172.21.15.69:0/350157295 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97141a1a20 con 0x7f97141036e0 2024-05-14T16:47:34.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.373+0000 7f971c148700 1 -- 172.21.15.69:0/350157295 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9714051a80 con 0x7f97141036e0 2024-05-14T16:47:34.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.374+0000 7f970affd700 1 -- 172.21.15.69:0/350157295 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f970400f8e0 con 0x7f97141036e0 2024-05-14T16:47:34.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.375+0000 7f970affd700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f97000b2a30 0x7f97000b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:34.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.375+0000 7f9719ee4700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f97000b2a30 0x7f97000b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:34.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.375+0000 7f970affd700 1 -- 172.21.15.69:0/350157295 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f97040e2230 con 0x7f97141036e0 2024-05-14T16:47:34.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.376+0000 7f9719ee4700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f97000b2a30 0x7f97000b4ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9714103540 tx=0x7f971000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:34.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.379+0000 7f970affd700 1 -- 172.21.15.69:0/350157295 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9704014030 con 0x7f97141036e0 2024-05-14T16:47:34.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.666+0000 7f971c148700 1 -- 172.21.15.69:0/350157295 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f97141a1ea0 con 0x7f97141036e0 2024-05-14T16:47:34.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.668+0000 7f970affd700 1 -- 172.21.15.69:0/350157295 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f97040a9ec0 con 0x7f97141036e0 2024-05-14T16:47:34.668 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2024-05-14T16:47:34.668 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQCmlUNm9CjOJxAAVE0xaFGtpDVzU6T8k233Ag== 2024-05-14T16:47:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 -- 172.21.15.69:0/350157295 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f97000b2a30 msgr2=0x7f97000b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f97000b2a30 0x7f97000b4ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9714103540 tx=0x7f971000b410 comp rx=0 tx=0).stop 2024-05-14T16:47:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 -- 172.21.15.69:0/350157295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97141036e0 msgr2=0x7f9714199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97141036e0 0x7f9714199ad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f970400c950 tx=0x7f970400cc60 comp rx=0 tx=0).stop 2024-05-14T16:47:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 -- 172.21.15.69:0/350157295 shutdown_connections 2024-05-14T16:47:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9714104080 0x7f97141a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141024e0 0x7f9714199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f97000b2a30 0x7f97000b4ef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 --2- 172.21.15.69:0/350157295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97141036e0 0x7f9714199ad0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 -- 172.21.15.69:0/350157295 >> 172.21.15.69:0/350157295 conn(0x7f97140fda60 msgr2=0x7f97140ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:34.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 -- 172.21.15.69:0/350157295 shutdown_connections 2024-05-14T16:47:34.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:34.671+0000 7f971c148700 1 -- 172.21.15.69:0/350157295 wait complete. 2024-05-14T16:47:34.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:34 smithi069 ceph-mon[31507]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T16:47:34.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:34 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/350157295' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T16:47:34.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:34 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/350157295' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T16:47:35.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:34 smithi092 ceph-mon[35763]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T16:47:35.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:34 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/350157295' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T16:47:35.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:34 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/350157295' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T16:47:35.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:34 smithi203 ceph-mon[35787]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T16:47:35.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:34 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/350157295' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T16:47:35.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:34 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/350157295' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T16:47:35.358 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:47:35.358 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-14T16:47:35.358 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-14T16:47:35.395 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-14T16:47:35.395 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-14T16:47:35.395 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-14T16:47:35.690 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:35.992 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:35 smithi092 ceph-mon[35763]: pgmap v311: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 639 B/s wr, 8 op/s 2024-05-14T16:47:35.992 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:35 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:36.069 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:35 smithi069 ceph-mon[31507]: pgmap v311: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 639 B/s wr, 8 op/s 2024-05-14T16:47:36.069 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:35 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:36.108 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:35 smithi203 ceph-mon[35787]: pgmap v311: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 639 B/s wr, 8 op/s 2024-05-14T16:47:36.108 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:35 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:36.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:36 smithi069 ceph-mon[31507]: pgmap v312: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 6 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:37.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:36 smithi092 ceph-mon[35763]: pgmap v312: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 6 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:37.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:36 smithi203 ceph-mon[35787]: pgmap v312: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 6 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:37.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:37 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:37.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:37 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:37.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:37 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:37.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:37 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:38.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.035+0000 7fd8b3e40700 1 -- 172.21.15.69:0/936771745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8ac102440 msgr2=0x7fd8ac1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:38.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.035+0000 7fd8b3e40700 1 --2- 172.21.15.69:0/936771745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8ac102440 0x7fd8ac1028c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009990 tx=0x7fd8a0009ca0 comp rx=0 tx=0).stop 2024-05-14T16:47:38.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.035+0000 7fd8b3e40700 1 -- 172.21.15.69:0/936771745 shutdown_connections 2024-05-14T16:47:38.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.035+0000 7fd8b3e40700 1 --2- 172.21.15.69:0/936771745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8ac102e00 0x7fd8ac10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:38.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.035+0000 7fd8b3e40700 1 --2- 172.21.15.69:0/936771745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac108520 0x7fd8ac108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:38.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.035+0000 7fd8b3e40700 1 --2- 172.21.15.69:0/936771745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8ac102440 0x7fd8ac1028c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:38.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.035+0000 7fd8b3e40700 1 -- 172.21.15.69:0/936771745 >> 172.21.15.69:0/936771745 conn(0x7fd8ac0793f0 msgr2=0x7fd8ac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:38.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.035+0000 7fd8b3e40700 1 -- 172.21.15.69:0/936771745 shutdown_connections 2024-05-14T16:47:38.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.035+0000 7fd8b3e40700 1 -- 172.21.15.69:0/936771745 wait complete. 2024-05-14T16:47:38.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b3e40700 1 Processor -- start 2024-05-14T16:47:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b3e40700 1 -- start start 2024-05-14T16:47:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b3e40700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8ac102440 0x7fd8ac19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b3e40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac102e00 0x7fd8ac19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b3e40700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8ac108520 0x7fd8ac1a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b3e40700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac1125f0 con 0x7fd8ac102440 2024-05-14T16:47:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b3e40700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac112470 con 0x7fd8ac102e00 2024-05-14T16:47:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b3e40700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac112770 con 0x7fd8ac108520 2024-05-14T16:47:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b1bdc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8ac102440 0x7fd8ac19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b1bdc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8ac102440 0x7fd8ac19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48558/0 (socket says 172.21.15.69:48558) 2024-05-14T16:47:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b1bdc700 1 -- 172.21.15.69:0/3538785229 learned_addr learned my addr 172.21.15.69:0/3538785229 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b23dd700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8ac108520 0x7fd8ac1a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.036+0000 7fd8b13db700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac102e00 0x7fd8ac19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8b1bdc700 1 -- 172.21.15.69:0/3538785229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8ac108520 msgr2=0x7fd8ac1a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8b1bdc700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8ac108520 0x7fd8ac1a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8b1bdc700 1 -- 172.21.15.69:0/3538785229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac102e00 msgr2=0x7fd8ac19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8b1bdc700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac102e00 0x7fd8ac19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8b1bdc700 1 -- 172.21.15.69:0/3538785229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac1a6030 con 0x7fd8ac102440 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8b13db700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac102e00 0x7fd8ac19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8b1bdc700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8ac102440 0x7fd8ac19aa70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400d980 tx=0x7fd8a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8aaffd700 1 -- 172.21.15.69:0/3538785229 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4016070 con 0x7fd8ac102440 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8aaffd700 1 -- 172.21.15.69:0/3538785229 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a4004830 con 0x7fd8ac102440 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8b3e40700 1 -- 172.21.15.69:0/3538785229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac1a6260 con 0x7fd8ac102440 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8aaffd700 1 -- 172.21.15.69:0/3538785229 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40054c0 con 0x7fd8ac102440 2024-05-14T16:47:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.037+0000 7fd8b3e40700 1 -- 172.21.15.69:0/3538785229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac1a6770 con 0x7fd8ac102440 2024-05-14T16:47:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.038+0000 7fd8b3e40700 1 -- 172.21.15.69:0/3538785229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ac06a230 con 0x7fd8ac102440 2024-05-14T16:47:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.039+0000 7fd8aaffd700 1 -- 172.21.15.69:0/3538785229 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd8a4004220 con 0x7fd8ac102440 2024-05-14T16:47:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.040+0000 7fd8aaffd700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8940b2910 0x7fd8940b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:38.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.040+0000 7fd8b13db700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8940b2910 0x7fd8940b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:38.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.040+0000 7fd8aaffd700 1 -- 172.21.15.69:0/3538785229 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd8a40e2630 con 0x7fd8ac102440 2024-05-14T16:47:38.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.040+0000 7fd8b13db700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8940b2910 0x7fd8940b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0006010 tx=0x7fd8a000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:38.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.043+0000 7fd8aaffd700 1 -- 172.21.15.69:0/3538785229 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8a401f020 con 0x7fd8ac102440 2024-05-14T16:47:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:37 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:37 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:37 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:37 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:38.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:37 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:38.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:37 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:38.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:37 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:38.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:37 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:38.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.329+0000 7fd8b3e40700 1 -- 172.21.15.69:0/3538785229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd8ac02e720 con 0x7fd8ac102440 2024-05-14T16:47:38.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.338+0000 7fd8aaffd700 1 -- 172.21.15.69:0/3538785229 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288645 (secure 0 0 0) 0x7fd8a40aa1f0 con 0x7fd8ac102440 2024-05-14T16:47:38.341 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:47:38.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 -- 172.21.15.69:0/3538785229 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8940b2910 msgr2=0x7fd8940b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:38.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8940b2910 0x7fd8940b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0006010 tx=0x7fd8a000b540 comp rx=0 tx=0).stop 2024-05-14T16:47:38.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 -- 172.21.15.69:0/3538785229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8ac102440 msgr2=0x7fd8ac19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8ac102440 0x7fd8ac19aa70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400d980 tx=0x7fd8a400dc90 comp rx=0 tx=0).stop 2024-05-14T16:47:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 -- 172.21.15.69:0/3538785229 shutdown_connections 2024-05-14T16:47:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8940b2910 0x7fd8940b4dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8ac102440 0x7fd8ac19aa70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8ac108520 0x7fd8ac1a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 --2- 172.21.15.69:0/3538785229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac102e00 0x7fd8ac19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 -- 172.21.15.69:0/3538785229 >> 172.21.15.69:0/3538785229 conn(0x7fd8ac0793f0 msgr2=0x7fd8ac100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 -- 172.21.15.69:0/3538785229 shutdown_connections 2024-05-14T16:47:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:38.343+0000 7fd8b3e40700 1 -- 172.21.15.69:0/3538785229 wait complete. 2024-05-14T16:47:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:38 smithi069 ceph-mon[31507]: pgmap v313: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:38 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3538785229' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T16:47:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:38 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:39.002 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":4012740750},{"type":"v1","addr":"172.21.15.69:6801","nonce":4012740750}]},"active_addr":"172.21.15.69:6801/4012740750","active_change":"2024-05-14T16:39:31.768166+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.69:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":1931804604}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":70559253}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2944165482}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3176577502}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3112349441}]}]} 2024-05-14T16:47:39.008 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-14T16:47:39.008 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-14T16:47:39.008 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:47:39.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:38 smithi203 ceph-mon[35787]: pgmap v313: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:39.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:38 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3538785229' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T16:47:39.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:39.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:38 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:39.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:38 smithi092 ceph-mon[35763]: pgmap v313: 97 pgs: 4 peering, 93 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:39.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:38 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3538785229' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T16:47:39.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:39.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:38 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:39.256 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:39.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:39 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:39.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:39 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:39.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:39 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:40.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:39 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:40.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:39 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:40.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:39 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:40.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:39 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:40.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:39 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:40.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:39 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:47:40.937 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:40 smithi069 ceph-mon[31507]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:40.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.937+0000 7fb028a9d700 1 -- 172.21.15.69:0/491980164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb024108860 msgr2=0x7fb024110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:40.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.937+0000 7fb028a9d700 1 --2- 172.21.15.69:0/491980164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb024108860 0x7fb024110470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00b0d0 tx=0x7fb01c00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:47:40.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.937+0000 7fb028a9d700 1 -- 172.21.15.69:0/491980164 shutdown_connections 2024-05-14T16:47:40.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.937+0000 7fb028a9d700 1 --2- 172.21.15.69:0/491980164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb024108860 0x7fb024110470 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:40.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.937+0000 7fb028a9d700 1 --2- 172.21.15.69:0/491980164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb02406bf10 0x7fb02406c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:40.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.937+0000 7fb028a9d700 1 --2- 172.21.15.69:0/491980164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02406c8c0 0x7fb024108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:40.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.937+0000 7fb028a9d700 1 -- 172.21.15.69:0/491980164 >> 172.21.15.69:0/491980164 conn(0x7fb024078490 msgr2=0x7fb0240788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.937+0000 7fb028a9d700 1 -- 172.21.15.69:0/491980164 shutdown_connections 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.937+0000 7fb028a9d700 1 -- 172.21.15.69:0/491980164 wait complete. 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb028a9d700 1 Processor -- start 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb028a9d700 1 -- start start 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb028a9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb02406bf10 0x7fb02419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb028a9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb02406c8c0 0x7fb02419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb028a9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024108860 0x7fb024199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb028a9d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241137d0 con 0x7fb02406c8c0 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb028a9d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb024113650 con 0x7fb024108860 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb028a9d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb024113950 con 0x7fb02406bf10 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb021d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb02406c8c0 0x7fb02419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb021d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb02406c8c0 0x7fb02419fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48574/0 (socket says 172.21.15.69:48574) 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb021d9b700 1 -- 172.21.15.69:0/2192852997 learned_addr learned my addr 172.21.15.69:0/2192852997 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:40.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb022d9d700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024108860 0x7fb024199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb02259c700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb02406bf10 0x7fb02419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb021d9b700 1 -- 172.21.15.69:0/2192852997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb02406bf10 msgr2=0x7fb02419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb021d9b700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb02406bf10 0x7fb02419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb021d9b700 1 -- 172.21.15.69:0/2192852997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024108860 msgr2=0x7fb024199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb021d9b700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024108860 0x7fb024199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb021d9b700 1 -- 172.21.15.69:0/2192852997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02419a210 con 0x7fb02406c8c0 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb022d9d700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024108860 0x7fb024199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.938+0000 7fb021d9b700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb02406c8c0 0x7fb02419fde0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb01400c970 tx=0x7fb01400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.939+0000 7fb0137fe700 1 -- 172.21.15.69:0/2192852997 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014007b00 con 0x7fb02406c8c0 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.939+0000 7fb0137fe700 1 -- 172.21.15.69:0/2192852997 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb014004ce0 con 0x7fb02406c8c0 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.939+0000 7fb028a9d700 1 -- 172.21.15.69:0/2192852997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02419a500 con 0x7fb02406c8c0 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.939+0000 7fb0137fe700 1 -- 172.21.15.69:0/2192852997 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140055a0 con 0x7fb02406c8c0 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.939+0000 7fb028a9d700 1 -- 172.21.15.69:0/2192852997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0241a6770 con 0x7fb02406c8c0 2024-05-14T16:47:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.940+0000 7fb028a9d700 1 -- 172.21.15.69:0/2192852997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02406a230 con 0x7fb02406c8c0 2024-05-14T16:47:40.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.941+0000 7fb0137fe700 1 -- 172.21.15.69:0/2192852997 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb014007ca0 con 0x7fb02406c8c0 2024-05-14T16:47:40.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.942+0000 7fb0137fe700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb00c0b2910 0x7fb00c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:40.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.942+0000 7fb02259c700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb00c0b2910 0x7fb00c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:40.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.942+0000 7fb0137fe700 1 -- 172.21.15.69:0/2192852997 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb0140e1d00 con 0x7fb02406c8c0 2024-05-14T16:47:40.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.942+0000 7fb02259c700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb00c0b2910 0x7fb00c0b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb018006fd0 tx=0x7fb018008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:40.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:40.946+0000 7fb0137fe700 1 -- 172.21.15.69:0/2192852997 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb014014030 con 0x7fb02406c8c0 2024-05-14T16:47:41.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.150+0000 7fb028a9d700 1 -- 172.21.15.69:0/2192852997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb02419af90 con 0x7fb02406c8c0 2024-05-14T16:47:41.151 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:40 smithi092 ceph-mon[35763]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:41.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.152+0000 7fb0137fe700 1 -- 172.21.15.69:0/2192852997 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22664 (secure 0 0 0) 0x7fb0140a98c0 con 0x7fb02406c8c0 2024-05-14T16:47:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:47:41.152 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T16:47:33.698176+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:46:38.843325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:46:42.395401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:47:41.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 -- 172.21.15.69:0/2192852997 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb00c0b2910 msgr2=0x7fb00c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:41.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb00c0b2910 0x7fb00c0b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb018006fd0 tx=0x7fb018008040 comp rx=0 tx=0).stop 2024-05-14T16:47:41.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 -- 172.21.15.69:0/2192852997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb02406c8c0 msgr2=0x7fb02419fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:41.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb02406c8c0 0x7fb02419fde0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb01400c970 tx=0x7fb01400cd30 comp rx=0 tx=0).stop 2024-05-14T16:47:41.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 -- 172.21.15.69:0/2192852997 shutdown_connections 2024-05-14T16:47:41.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024108860 0x7fb024199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:41.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb02406bf10 0x7fb02419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:41.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb00c0b2910 0x7fb00c0b4dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:41.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 --2- 172.21.15.69:0/2192852997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb02406c8c0 0x7fb02419fde0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:41.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 -- 172.21.15.69:0/2192852997 >> 172.21.15.69:0/2192852997 conn(0x7fb024078490 msgr2=0x7fb024100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:41.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 -- 172.21.15.69:0/2192852997 shutdown_connections 2024-05-14T16:47:41.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:41.154+0000 7fb028a9d700 1 -- 172.21.15.69:0/2192852997 wait complete. 2024-05-14T16:47:41.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:40 smithi203 ceph-mon[35787]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 90 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:41.832 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-14T16:47:41.833 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:47:42.075 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:42.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:41 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2192852997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:47:42.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:41 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2192852997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:47:42.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:41 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2192852997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:47:43.125 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:42 smithi203 ceph-mon[35787]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 79 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:42 smithi069 ceph-mon[31507]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 79 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:43.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:42 smithi092 ceph-mon[35763]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 79 B/s, 1 keys/s, 1 objects/s recovering 2024-05-14T16:47:43.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.788+0000 7f695ee7a700 1 -- 172.21.15.69:0/1475555371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69581095b0 msgr2=0x7f6958109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:43.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.788+0000 7f695ee7a700 1 --2- 172.21.15.69:0/1475555371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69581095b0 0x7f6958109990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6950009990 tx=0x7f6950009ca0 comp rx=0 tx=0).stop 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.788+0000 7f695ee7a700 1 -- 172.21.15.69:0/1475555371 shutdown_connections 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.788+0000 7f695ee7a700 1 --2- 172.21.15.69:0/1475555371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69581095b0 0x7f6958109990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.788+0000 7f695ee7a700 1 --2- 172.21.15.69:0/1475555371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f695806bbe0 0x7f69581103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.788+0000 7f695ee7a700 1 --2- 172.21.15.69:0/1475555371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f695806b220 0x7f695806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.788+0000 7f695ee7a700 1 -- 172.21.15.69:0/1475555371 >> 172.21.15.69:0/1475555371 conn(0x7f69580793f0 msgr2=0x7f6958079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.788+0000 7f695ee7a700 1 -- 172.21.15.69:0/1475555371 shutdown_connections 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.788+0000 7f695ee7a700 1 -- 172.21.15.69:0/1475555371 wait complete. 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695ee7a700 1 Processor -- start 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695ee7a700 1 -- start start 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695ee7a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695806b220 0x7f695819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695ee7a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f695806bbe0 0x7f695819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695ee7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69581095b0 0x7f6958199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:43.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695ee7a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6958113760 con 0x7f695806b220 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695ee7a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f69581135e0 con 0x7f69581095b0 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695ee7a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69581138e0 con 0x7f695806bbe0 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695cc16700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695806b220 0x7f695819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695cc16700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695806b220 0x7f695819f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48582/0 (socket says 172.21.15.69:48582) 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695cc16700 1 -- 172.21.15.69:0/2693483138 learned_addr learned my addr 172.21.15.69:0/2693483138 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f6957fff700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f695806bbe0 0x7f695819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695cc16700 1 -- 172.21.15.69:0/2693483138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f695806bbe0 msgr2=0x7f695819fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695cc16700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f695806bbe0 0x7f695819fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695cc16700 1 -- 172.21.15.69:0/2693483138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69581095b0 msgr2=0x7f6958199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695cc16700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69581095b0 0x7f6958199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695cc16700 1 -- 172.21.15.69:0/2693483138 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695819a1e0 con 0x7f695806b220 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f6957fff700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f695806bbe0 0x7f695819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:43.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f695cc16700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695806b220 0x7f695819f8c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6950000c00 tx=0x7f69500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:43.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.789+0000 7f6955ffb700 1 -- 172.21.15.69:0/2693483138 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950027070 con 0x7f695806b220 2024-05-14T16:47:43.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.790+0000 7f695ee7a700 1 -- 172.21.15.69:0/2693483138 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695819a470 con 0x7f695806b220 2024-05-14T16:47:43.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.790+0000 7f6955ffb700 1 -- 172.21.15.69:0/2693483138 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6950003fa0 con 0x7f695806b220 2024-05-14T16:47:43.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.790+0000 7f6955ffb700 1 -- 172.21.15.69:0/2693483138 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950022db0 con 0x7f695806b220 2024-05-14T16:47:43.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.790+0000 7f695ee7a700 1 -- 172.21.15.69:0/2693483138 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69581a6790 con 0x7f695806b220 2024-05-14T16:47:43.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.791+0000 7f695ee7a700 1 -- 172.21.15.69:0/2693483138 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f695810ba90 con 0x7f695806b220 2024-05-14T16:47:43.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.792+0000 7f6955ffb700 1 -- 172.21.15.69:0/2693483138 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f695002a070 con 0x7f695806b220 2024-05-14T16:47:43.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.793+0000 7f6955ffb700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f69440b2910 0x7f69440b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:43.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.793+0000 7f6957fff700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f69440b2910 0x7f69440b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:43.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.793+0000 7f6955ffb700 1 -- 172.21.15.69:0/2693483138 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f69500e38f0 con 0x7f695806b220 2024-05-14T16:47:43.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.794+0000 7f6957fff700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f69440b2910 0x7f69440b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6958079ad0 tx=0x7f6948009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:43.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:43.797+0000 7f6955ffb700 1 -- 172.21.15.69:0/2693483138 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6950031070 con 0x7f695806b220 2024-05-14T16:47:44.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.003+0000 7f695ee7a700 1 -- 172.21.15.69:0/2693483138 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f695819afb0 con 0x7f695806b220 2024-05-14T16:47:44.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.004+0000 7f6955ffb700 1 -- 172.21.15.69:0/2693483138 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22664 (secure 0 0 0) 0x7f6950013070 con 0x7f695806b220 2024-05-14T16:47:44.004 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:47:44.005 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T16:47:33.698176+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:46:38.843325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:46:42.395401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:47:44.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 -- 172.21.15.69:0/2693483138 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f69440b2910 msgr2=0x7f69440b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:44.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f69440b2910 0x7f69440b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6958079ad0 tx=0x7f6948009570 comp rx=0 tx=0).stop 2024-05-14T16:47:44.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 -- 172.21.15.69:0/2693483138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695806b220 msgr2=0x7f695819f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:44.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695806b220 0x7f695819f8c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6950000c00 tx=0x7f69500047b0 comp rx=0 tx=0).stop 2024-05-14T16:47:44.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 -- 172.21.15.69:0/2693483138 shutdown_connections 2024-05-14T16:47:44.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f69440b2910 0x7f69440b4dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:44.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695806b220 0x7f695819f8c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:44.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69581095b0 0x7f6958199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:44.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 --2- 172.21.15.69:0/2693483138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f695806bbe0 0x7f695819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:44.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 -- 172.21.15.69:0/2693483138 >> 172.21.15.69:0/2693483138 conn(0x7f69580793f0 msgr2=0x7f6958101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:44.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 -- 172.21.15.69:0/2693483138 shutdown_connections 2024-05-14T16:47:44.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:44.007+0000 7f695ee7a700 1 -- 172.21.15.69:0/2693483138 wait complete. 2024-05-14T16:47:44.685 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-14T16:47:44.685 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-14T16:47:44.686 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-14T16:47:44.686 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-14T16:47:44.687 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-14T16:47:44.687 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-14T16:47:44.687 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-14T16:47:44.688 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-14T16:47:44.688 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-14T16:47:44.689 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-14T16:47:44.689 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-14T16:47:44.689 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-14T16:47:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:44 smithi069 ceph-mon[31507]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 71 B/s, 0 keys/s, 1 objects/s recovering 2024-05-14T16:47:45.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:44 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2693483138' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:47:45.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:44 smithi203 ceph-mon[35787]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 71 B/s, 0 keys/s, 1 objects/s recovering 2024-05-14T16:47:45.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:44 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2693483138' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:47:45.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:44 smithi092 ceph-mon[35763]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 71 B/s, 0 keys/s, 1 objects/s recovering 2024-05-14T16:47:45.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:44 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2693483138' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:47:45.308 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:45.405 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:46.124 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:46.125 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:46.128 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:46.128 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:46.129 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:46.129 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:46.134 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:46.136 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:46.144 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:46.144 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:47.058 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:46 smithi069 ceph-mon[31507]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 60 B/s, 0 keys/s, 1 objects/s recovering 2024-05-14T16:47:47.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:46 smithi203 ceph-mon[35787]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 60 B/s, 0 keys/s, 1 objects/s recovering 2024-05-14T16:47:47.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:46 smithi092 ceph-mon[35763]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 60 B/s, 0 keys/s, 1 objects/s recovering 2024-05-14T16:47:48.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.559+0000 7fba8be42700 1 -- 172.21.15.69:0/3789373325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba84102e30 msgr2=0x7fba8410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:48.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.559+0000 7fba8be42700 1 --2- 172.21.15.69:0/3789373325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba84102e30 0x7fba8410f2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00b0a0 tx=0x7fba7c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.559+0000 7fba8be42700 1 -- 172.21.15.69:0/3789373325 shutdown_connections 2024-05-14T16:47:48.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.559+0000 7fba8be42700 1 --2- 172.21.15.69:0/3789373325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba84102e30 0x7fba8410f2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.559+0000 7fba8be42700 1 --2- 172.21.15.69:0/3789373325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba841084b0 0x7fba84108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.559+0000 7fba8be42700 1 --2- 172.21.15.69:0/3789373325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba84102470 0x7fba841028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.559+0000 7fba8be42700 1 -- 172.21.15.69:0/3789373325 >> 172.21.15.69:0/3789373325 conn(0x7fba840793f0 msgr2=0x7fba84079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:48.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.559+0000 7fba8be42700 1 -- 172.21.15.69:0/3789373325 shutdown_connections 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.559+0000 7fba8be42700 1 -- 172.21.15.69:0/3789373325 wait complete. 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba8be42700 1 Processor -- start 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba8be42700 1 -- start start 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba8be42700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba84102470 0x7fba8419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba8be42700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba84102e30 0x7fba8419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba8be42700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba841084b0 0x7fba84199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba8be42700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba84112660 con 0x7fba84102470 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba8be42700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba841124e0 con 0x7fba841084b0 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba8be42700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba841127e0 con 0x7fba84102e30 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba89bde700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba84102470 0x7fba8419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba89bde700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba84102470 0x7fba8419f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33408/0 (socket says 172.21.15.69:33408) 2024-05-14T16:47:48.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba89bde700 1 -- 172.21.15.69:0/3956160556 learned_addr learned my addr 172.21.15.69:0/3956160556 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:48.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba893dd700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba84102e30 0x7fba8419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:48.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.560+0000 7fba8a3df700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba841084b0 0x7fba84199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba89bde700 1 -- 172.21.15.69:0/3956160556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba84102e30 msgr2=0x7fba8419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba89bde700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba84102e30 0x7fba8419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba89bde700 1 -- 172.21.15.69:0/3956160556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba841084b0 msgr2=0x7fba84199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba89bde700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba841084b0 0x7fba84199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba89bde700 1 -- 172.21.15.69:0/3956160556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8419a1f0 con 0x7fba84102470 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba89bde700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba84102470 0x7fba8419f880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba8000b530 tx=0x7fba8000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba7affd700 1 -- 172.21.15.69:0/3956160556 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80015020 con 0x7fba84102470 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba7affd700 1 -- 172.21.15.69:0/3956160556 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba80011ce0 con 0x7fba84102470 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8419a4e0 con 0x7fba84102470 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba7affd700 1 -- 172.21.15.69:0/3956160556 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80010dd0 con 0x7fba84102470 2024-05-14T16:47:48.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.561+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba841a6750 con 0x7fba84102470 2024-05-14T16:47:48.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.562+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba8406bfc0 con 0x7fba84102470 2024-05-14T16:47:48.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.563+0000 7fba7affd700 1 -- 172.21.15.69:0/3956160556 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fba80018020 con 0x7fba84102470 2024-05-14T16:47:48.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.564+0000 7fba7affd700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fba700b2990 0x7fba700b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:48.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.564+0000 7fba893dd700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fba700b2990 0x7fba700b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:48.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.564+0000 7fba7affd700 1 -- 172.21.15.69:0/3956160556 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fba800e1f90 con 0x7fba84102470 2024-05-14T16:47:48.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.564+0000 7fba7affd700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] conn(0x7fba700bc550 0x7fba700be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:48.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.564+0000 7fba7affd700 1 -- 172.21.15.69:0/3956160556 --> [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba8000fec0 con 0x7fba700bc550 2024-05-14T16:47:48.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.564+0000 7fba7affd700 1 -- 172.21.15.69:0/3956160556 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fba800e2470 con 0x7fba84102470 2024-05-14T16:47:48.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.564+0000 7fba893dd700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fba700b2990 0x7fba700b4e50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fba8406cd50 tx=0x7fba7400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:48.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.564+0000 7fba8a3df700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] conn(0x7fba700bc550 0x7fba700be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:48.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.565+0000 7fba8a3df700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] conn(0x7fba700bc550 0x7fba700be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:48.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.566+0000 7fba7affd700 1 -- 172.21.15.69:0/3956160556 <== osd.5 v2:172.21.15.203:6808/3198746082 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba8000fec0 con 0x7fba700bc550 2024-05-14T16:47:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.582+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 --> [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fba84103b60 con 0x7fba700bc550 2024-05-14T16:47:48.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.583+0000 7fba7affd700 1 -- 172.21.15.69:0/3956160556 <== osd.5 v2:172.21.15.203:6808/3198746082 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fba84103b60 con 0x7fba700bc550 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.583+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 >> [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] conn(0x7fba700bc550 msgr2=0x7fba700be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.583+0000 7fba8be42700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] conn(0x7fba700bc550 0x7fba700be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.583+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fba700b2990 msgr2=0x7fba700b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.583+0000 7fba8be42700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fba700b2990 0x7fba700b4e50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fba8406cd50 tx=0x7fba7400b3c0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.583+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba84102470 msgr2=0x7fba8419f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.583+0000 7fba8be42700 1 --2- 172.21.15.69:0/3956160556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba84102470 0x7fba8419f880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba8000b530 tx=0x7fba8000b840 comp rx=0 tx=0).stop 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.583+0000 7fba8a3df700 1 -- 172.21.15.69:0/3956160556 reap_dead start 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.584+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 shutdown_connections 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.584+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 >> 172.21.15.69:0/3956160556 conn(0x7fba840793f0 msgr2=0x7fba84106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.584+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 shutdown_connections 2024-05-14T16:47:48.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:48.584+0000 7fba8be42700 1 -- 172.21.15.69:0/3956160556 wait complete. 2024-05-14T16:47:49.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:48 smithi069 ceph-mon[31507]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2024-05-14T16:47:49.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:48 smithi203 ceph-mon[35787]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2024-05-14T16:47:49.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:48 smithi092 ceph-mon[35763]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2024-05-14T16:47:50.068 INFO:teuthology.orchestra.run.smithi069.stdout:154618822716 2024-05-14T16:47:50.068 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-14T16:47:51.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:50 smithi069 ceph-mon[31507]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-05-14T16:47:51.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:50 smithi203 ceph-mon[35787]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-05-14T16:47:51.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:50 smithi092 ceph-mon[35763]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-05-14T16:47:53.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:52 smithi069 ceph-mon[31507]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:47:53.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:52 smithi203 ceph-mon[35787]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:47:53.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:52 smithi092 ceph-mon[35763]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:47:55.150 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:54 smithi069 ceph-mon[31507]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:47:55.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:54 smithi203 ceph-mon[35787]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:47:55.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:54 smithi092 ceph-mon[35763]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:47:57.150 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:56 smithi069 ceph-mon[31507]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:47:57.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:56 smithi203 ceph-mon[35787]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:47:57.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:56 smithi092 ceph-mon[35763]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:47:57.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.377+0000 7f099a00f700 1 -- 172.21.15.69:0/4185310733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0994102e90 msgr2=0x7f099410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:57.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.377+0000 7f099a00f700 1 --2- 172.21.15.69:0/4185310733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0994102e90 0x7f099410f310 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f098c00b0a0 tx=0x7f098c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.377+0000 7f099a00f700 1 -- 172.21.15.69:0/4185310733 shutdown_connections 2024-05-14T16:47:57.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.377+0000 7f099a00f700 1 --2- 172.21.15.69:0/4185310733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0994102e90 0x7f099410f310 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.377+0000 7f099a00f700 1 --2- 172.21.15.69:0/4185310733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09941084d0 0x7f09941088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.377+0000 7f099a00f700 1 --2- 172.21.15.69:0/4185310733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941024d0 0x7f0994102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.377+0000 7f099a00f700 1 -- 172.21.15.69:0/4185310733 >> 172.21.15.69:0/4185310733 conn(0x7f0994078490 msgr2=0x7f09940788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.377+0000 7f099a00f700 1 -- 172.21.15.69:0/4185310733 shutdown_connections 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.377+0000 7f099a00f700 1 -- 172.21.15.69:0/4185310733 wait complete. 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.378+0000 7f099a00f700 1 Processor -- start 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.378+0000 7f099a00f700 1 -- start start 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.378+0000 7f099a00f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941024d0 0x7f099419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.378+0000 7f099a00f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0994102e90 0x7f099419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f099a00f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09941084d0 0x7f09941a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f099a00f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0994112680 con 0x7f0994102e90 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f099a00f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0994112500 con 0x7f09941024d0 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f099a00f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0994112800 con 0x7f09941084d0 2024-05-14T16:47:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0992ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0994102e90 0x7f099419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0992ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0994102e90 0x7f099419af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34850/0 (socket says 172.21.15.69:34850) 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0992ffd700 1 -- 172.21.15.69:0/1438326755 learned_addr learned my addr 172.21.15.69:0/1438326755 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0993fff700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09941084d0 0x7f09941a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0992ffd700 1 -- 172.21.15.69:0/1438326755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09941084d0 msgr2=0x7f09941a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0992ffd700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09941084d0 0x7f09941a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f09937fe700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941024d0 0x7f099419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0992ffd700 1 -- 172.21.15.69:0/1438326755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941024d0 msgr2=0x7f099419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0992ffd700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941024d0 0x7f099419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0992ffd700 1 -- 172.21.15.69:0/1438326755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09941a5fe0 con 0x7f0994102e90 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0993fff700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09941084d0 0x7f09941a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f09937fe700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941024d0 0x7f099419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0992ffd700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0994102e90 0x7f099419af60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f098800b530 tx=0x7f098800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0990ff9700 1 -- 172.21.15.69:0/1438326755 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988015020 con 0x7f0994102e90 2024-05-14T16:47:57.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0990ff9700 1 -- 172.21.15.69:0/1438326755 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0988011ce0 con 0x7f0994102e90 2024-05-14T16:47:57.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f0990ff9700 1 -- 172.21.15.69:0/1438326755 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988010d20 con 0x7f0994102e90 2024-05-14T16:47:57.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09941a6210 con 0x7f0994102e90 2024-05-14T16:47:57.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.379+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09941a6750 con 0x7f0994102e90 2024-05-14T16:47:57.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.380+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0994063c50 con 0x7f0994102e90 2024-05-14T16:47:57.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.382+0000 7f0990ff9700 1 -- 172.21.15.69:0/1438326755 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0988018020 con 0x7f0994102e90 2024-05-14T16:47:57.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.382+0000 7f0990ff9700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f097c0b2910 0x7f097c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:57.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.382+0000 7f09937fe700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f097c0b2910 0x7f097c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:57.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.382+0000 7f0990ff9700 1 -- 172.21.15.69:0/1438326755 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f09880e2260 con 0x7f0994102e90 2024-05-14T16:47:57.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.383+0000 7f0990ff9700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] conn(0x7f097c0bc4d0 0x7f097c0be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:57.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.383+0000 7f0990ff9700 1 -- 172.21.15.69:0/1438326755 --> [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f098800fec0 con 0x7f097c0bc4d0 2024-05-14T16:47:57.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.383+0000 7f0990ff9700 1 -- 172.21.15.69:0/1438326755 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f09880e2690 con 0x7f0994102e90 2024-05-14T16:47:57.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.383+0000 7f09937fe700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f097c0b2910 0x7f097c0b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0984005fd0 tx=0x7f0984005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:57.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.383+0000 7f0993fff700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] conn(0x7f097c0bc4d0 0x7f097c0be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:57.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.383+0000 7f0993fff700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] conn(0x7f097c0bc4d0 0x7f097c0be970 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:57.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.384+0000 7f0990ff9700 1 -- 172.21.15.69:0/1438326755 <== osd.11 v2:172.21.15.203:6824/3400874682 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f098800fec0 con 0x7f097c0bc4d0 2024-05-14T16:47:57.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.400+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 --> [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09940640d0 con 0x7f097c0bc4d0 2024-05-14T16:47:57.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.401+0000 7f0990ff9700 1 -- 172.21.15.69:0/1438326755 <== osd.11 v2:172.21.15.203:6824/3400874682 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f09940640d0 con 0x7f097c0bc4d0 2024-05-14T16:47:57.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.401+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 >> [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] conn(0x7f097c0bc4d0 msgr2=0x7f097c0be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:57.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.401+0000 7f099a00f700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] conn(0x7f097c0bc4d0 0x7f097c0be970 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.401+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f097c0b2910 msgr2=0x7f097c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:57.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.401+0000 7f099a00f700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f097c0b2910 0x7f097c0b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0984005fd0 tx=0x7f0984005dc0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.401+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0994102e90 msgr2=0x7f099419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:57.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.401+0000 7f099a00f700 1 --2- 172.21.15.69:0/1438326755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0994102e90 0x7f099419af60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f098800b530 tx=0x7f098800b8f0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.401+0000 7f0993fff700 1 -- 172.21.15.69:0/1438326755 reap_dead start 2024-05-14T16:47:57.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.402+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 shutdown_connections 2024-05-14T16:47:57.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.402+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 >> 172.21.15.69:0/1438326755 conn(0x7f0994078490 msgr2=0x7f09941004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:57.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.402+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 shutdown_connections 2024-05-14T16:47:57.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.402+0000 7f099a00f700 1 -- 172.21.15.69:0/1438326755 wait complete. 2024-05-14T16:47:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.979+0000 7f17da30e700 1 -- 172.21.15.69:0/3995997644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d406b220 msgr2=0x7f17d406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.979+0000 7f17da30e700 1 --2- 172.21.15.69:0/3995997644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d406b220 0x7f17d406b6a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009990 tx=0x7f17bc009ca0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.979+0000 7f17da30e700 1 -- 172.21.15.69:0/3995997644 shutdown_connections 2024-05-14T16:47:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.979+0000 7f17da30e700 1 --2- 172.21.15.69:0/3995997644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d406bbe0 0x7f17d41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.979+0000 7f17da30e700 1 --2- 172.21.15.69:0/3995997644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17d41095b0 0x7f17d4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.979+0000 7f17da30e700 1 --2- 172.21.15.69:0/3995997644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d406b220 0x7f17d406b6a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.979+0000 7f17da30e700 1 -- 172.21.15.69:0/3995997644 >> 172.21.15.69:0/3995997644 conn(0x7f17d40793f0 msgr2=0x7f17d4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.979+0000 7f17da30e700 1 -- 172.21.15.69:0/3995997644 shutdown_connections 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.979+0000 7f17da30e700 1 -- 172.21.15.69:0/3995997644 wait complete. 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.979+0000 7f17da30e700 1 Processor -- start 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17da30e700 1 -- start start 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17da30e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d406b220 0x7f17d419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17da30e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d406bbe0 0x7f17d419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17da30e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17d41095b0 0x7f17d4199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17da30e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4113760 con 0x7f17d406bbe0 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17da30e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41135e0 con 0x7f17d406b220 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17da30e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41138e0 con 0x7f17d41095b0 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17cbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d406bbe0 0x7f17d419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17cbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d406bbe0 0x7f17d419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34856/0 (socket says 172.21.15.69:34856) 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17cbfff700 1 -- 172.21.15.69:0/468043680 learned_addr learned my addr 172.21.15.69:0/468043680 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17d88ab700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17d41095b0 0x7f17d4199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17d3fff700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d406b220 0x7f17d419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17cbfff700 1 -- 172.21.15.69:0/468043680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17d41095b0 msgr2=0x7f17d4199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17cbfff700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17d41095b0 0x7f17d4199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17cbfff700 1 -- 172.21.15.69:0/468043680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d406b220 msgr2=0x7f17d419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17cbfff700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17d406b220 0x7f17d419f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17cbfff700 1 -- 172.21.15.69:0/468043680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d419a240 con 0x7f17d406bbe0 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17cbfff700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d406bbe0 0x7f17d419fe10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000c00 tx=0x7f17bc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17d1ffb700 1 -- 172.21.15.69:0/468043680 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc027070 con 0x7f17d406bbe0 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.980+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d419a4d0 con 0x7f17d406bbe0 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.981+0000 7f17d1ffb700 1 -- 172.21.15.69:0/468043680 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17bc003fa0 con 0x7f17d406bbe0 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.981+0000 7f17d1ffb700 1 -- 172.21.15.69:0/468043680 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc022db0 con 0x7f17d406bbe0 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.981+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d41a67a0 con 0x7f17d406bbe0 2024-05-14T16:47:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.981+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17d4063cd0 con 0x7f17d406bbe0 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.983+0000 7f17d1ffb700 1 -- 172.21.15.69:0/468043680 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f17bc02a070 con 0x7f17d406bbe0 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.983+0000 7f17d1ffb700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f17c00b2910 0x7f17c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.983+0000 7f17d3fff700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f17c00b2910 0x7f17c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.984+0000 7f17d1ffb700 1 -- 172.21.15.69:0/468043680 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f17bc0e3720 con 0x7f17d406bbe0 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.984+0000 7f17d1ffb700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] conn(0x7f17c00bc3a0 0x7f17c00be840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.984+0000 7f17d1ffb700 1 -- 172.21.15.69:0/468043680 --> [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17bc023e50 con 0x7f17c00bc3a0 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.984+0000 7f17d1ffb700 1 -- 172.21.15.69:0/468043680 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f17bc0e3b50 con 0x7f17d406bbe0 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.984+0000 7f17d3fff700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f17c00b2910 0x7f17c00b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f17c4005950 tx=0x7f17c40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.984+0000 7f17d88ab700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] conn(0x7f17c00bc3a0 0x7f17c00be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.984+0000 7f17d88ab700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] conn(0x7f17c00bc3a0 0x7f17c00be840 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:57.985+0000 7f17d1ffb700 1 -- 172.21.15.69:0/468043680 <== osd.2 v2:172.21.15.203:6800/2458206770 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17bc023e50 con 0x7f17c00bc3a0 2024-05-14T16:47:57.991 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:58.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.010+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 --> [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f17d40640d0 con 0x7f17c00bc3a0 2024-05-14T16:47:58.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.010+0000 7f17d1ffb700 1 -- 172.21.15.69:0/468043680 <== osd.2 v2:172.21.15.203:6800/2458206770 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f17d40640d0 con 0x7f17c00bc3a0 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 >> [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] conn(0x7f17c00bc3a0 msgr2=0x7f17c00be840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17da30e700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] conn(0x7f17c00bc3a0 0x7f17c00be840 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f17c00b2910 msgr2=0x7f17c00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17da30e700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f17c00b2910 0x7f17c00b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f17c4005950 tx=0x7f17c40058e0 comp rx=0 tx=0).stop 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d406bbe0 msgr2=0x7f17d419fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17da30e700 1 --2- 172.21.15.69:0/468043680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d406bbe0 0x7f17d419fe10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000c00 tx=0x7f17bc0047b0 comp rx=0 tx=0).stop 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17d88ab700 1 -- 172.21.15.69:0/468043680 reap_dead start 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 shutdown_connections 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 >> 172.21.15.69:0/468043680 conn(0x7f17d40793f0 msgr2=0x7f17d4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 shutdown_connections 2024-05-14T16:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:58.011+0000 7f17da30e700 1 -- 172.21.15.69:0/468043680 wait complete. 2024-05-14T16:47:59.136 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:47:58 smithi069 ceph-mon[31507]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:47:59.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:47:58 smithi203 ceph-mon[35787]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:47:59.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.240+0000 7f204b7e3700 1 -- 172.21.15.69:0/871404146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2044102430 msgr2=0x7f20441028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:59.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.240+0000 7f204b7e3700 1 --2- 172.21.15.69:0/871404146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2044102430 0x7f20441028b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2034009990 tx=0x7f2034009ca0 comp rx=0 tx=0).stop 2024-05-14T16:47:59.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.240+0000 7f204b7e3700 1 -- 172.21.15.69:0/871404146 shutdown_connections 2024-05-14T16:47:59.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.240+0000 7f204b7e3700 1 --2- 172.21.15.69:0/871404146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2044102df0 0x7f204410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:59.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.240+0000 7f204b7e3700 1 --2- 172.21.15.69:0/871404146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2044108510 0x7f20441088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:59.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.240+0000 7f204b7e3700 1 --2- 172.21.15.69:0/871404146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2044102430 0x7f20441028b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:59.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.240+0000 7f204b7e3700 1 -- 172.21.15.69:0/871404146 >> 172.21.15.69:0/871404146 conn(0x7f20440793b0 msgr2=0x7f20440797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:59.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.240+0000 7f204b7e3700 1 -- 172.21.15.69:0/871404146 shutdown_connections 2024-05-14T16:47:59.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.240+0000 7f204b7e3700 1 -- 172.21.15.69:0/871404146 wait complete. 2024-05-14T16:47:59.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f204b7e3700 1 Processor -- start 2024-05-14T16:47:59.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f204b7e3700 1 -- start start 2024-05-14T16:47:59.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f204b7e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2044102430 0x7f204419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:59.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f204b7e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2044102df0 0x7f204419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f204b7e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2044108510 0x7f2044199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f204b7e3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20441125c0 con 0x7f2044108510 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f204b7e3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2044112440 con 0x7f2044102df0 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f204b7e3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2044112740 con 0x7f2044102430 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f2049d80700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2044108510 0x7f2044199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f2049d80700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2044108510 0x7f2044199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34880/0 (socket says 172.21.15.69:34880) 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f2049d80700 1 -- 172.21.15.69:0/3453060264 learned_addr learned my addr 172.21.15.69:0/3453060264 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f204957f700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2044102430 0x7f204419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f2048d7e700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2044102df0 0x7f204419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f2049d80700 1 -- 172.21.15.69:0/3453060264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2044102430 msgr2=0x7f204419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f2049d80700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2044102430 0x7f204419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f2049d80700 1 -- 172.21.15.69:0/3453060264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2044102df0 msgr2=0x7f204419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f2049d80700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2044102df0 0x7f204419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:59.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f2049d80700 1 -- 172.21.15.69:0/3453060264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204419a0e0 con 0x7f2044108510 2024-05-14T16:47:59.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.241+0000 7f2049d80700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2044108510 0x7f2044199980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f203c007d70 tx=0x7f203c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:59.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.242+0000 7f203a7fc700 1 -- 172.21.15.69:0/3453060264 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c016020 con 0x7f2044108510 2024-05-14T16:47:59.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.242+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204419a3d0 con 0x7f2044108510 2024-05-14T16:47:59.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.242+0000 7f203a7fc700 1 -- 172.21.15.69:0/3453060264 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f203c00ad40 con 0x7f2044108510 2024-05-14T16:47:59.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.242+0000 7f203a7fc700 1 -- 172.21.15.69:0/3453060264 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c0115e0 con 0x7f2044108510 2024-05-14T16:47:59.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.242+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20441a6690 con 0x7f2044108510 2024-05-14T16:47:59.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.243+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2044063cd0 con 0x7f2044108510 2024-05-14T16:47:59.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.244+0000 7f203a7fc700 1 -- 172.21.15.69:0/3453060264 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f203c019020 con 0x7f2044108510 2024-05-14T16:47:59.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.245+0000 7f203a7fc700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f20300b2910 0x7f20300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:59.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.245+0000 7f204957f700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f20300b2910 0x7f20300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:59.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.245+0000 7f203a7fc700 1 -- 172.21.15.69:0/3453060264 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f203c0e28f0 con 0x7f2044108510 2024-05-14T16:47:59.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.245+0000 7f203a7fc700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] conn(0x7f20300bc4d0 0x7f20300be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:59.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.245+0000 7f203a7fc700 1 -- 172.21.15.69:0/3453060264 --> [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f203c01ce50 con 0x7f20300bc4d0 2024-05-14T16:47:59.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.245+0000 7f203a7fc700 1 -- 172.21.15.69:0/3453060264 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f203c0e2d20 con 0x7f2044108510 2024-05-14T16:47:59.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.245+0000 7f204957f700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f20300b2910 0x7f20300b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2040005950 tx=0x7f204000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:59.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.245+0000 7f2048d7e700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] conn(0x7f20300bc4d0 0x7f20300be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:59.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.246+0000 7f2048d7e700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] conn(0x7f20300bc4d0 0x7f20300be970 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:59.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.247+0000 7f203a7fc700 1 -- 172.21.15.69:0/3453060264 <== osd.8 v2:172.21.15.203:6816/1294563875 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f203c01ce50 con 0x7f20300bc4d0 2024-05-14T16:47:59.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:47:58 smithi092 ceph-mon[35763]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:47:59.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.263+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 --> [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2044064150 con 0x7f20300bc4d0 2024-05-14T16:47:59.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f203a7fc700 1 -- 172.21.15.69:0/3453060264 <== osd.8 v2:172.21.15.203:6816/1294563875 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2044064150 con 0x7f20300bc4d0 2024-05-14T16:47:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 >> [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] conn(0x7f20300bc4d0 msgr2=0x7f20300be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f204b7e3700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] conn(0x7f20300bc4d0 0x7f20300be970 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f20300b2910 msgr2=0x7f20300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f204b7e3700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f20300b2910 0x7f20300b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2040005950 tx=0x7f204000a300 comp rx=0 tx=0).stop 2024-05-14T16:47:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2044108510 msgr2=0x7f2044199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:59.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f204b7e3700 1 --2- 172.21.15.69:0/3453060264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2044108510 0x7f2044199980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f203c007d70 tx=0x7f203c00a340 comp rx=0 tx=0).stop 2024-05-14T16:47:59.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f2049d80700 1 -- 172.21.15.69:0/3453060264 reap_dead start 2024-05-14T16:47:59.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 shutdown_connections 2024-05-14T16:47:59.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 >> 172.21.15.69:0/3453060264 conn(0x7f20440793b0 msgr2=0x7f204406bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:59.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 shutdown_connections 2024-05-14T16:47:59.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:47:59.264+0000 7f204b7e3700 1 -- 172.21.15.69:0/3453060264 wait complete. 2024-05-14T16:48:00.985 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:00 smithi069 ceph-mon[31507]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:01.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:00 smithi203 ceph-mon[35787]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:01.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:00 smithi092 ceph-mon[35763]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.576+0000 7fea9e16e700 1 -- 172.21.15.69:0/90971078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea981095b0 msgr2=0x7fea98109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.576+0000 7fea9e16e700 1 --2- 172.21.15.69:0/90971078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea981095b0 0x7fea98109990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea88009990 tx=0x7fea88009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.576+0000 7fea9e16e700 1 -- 172.21.15.69:0/90971078 shutdown_connections 2024-05-14T16:48:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.576+0000 7fea9e16e700 1 --2- 172.21.15.69:0/90971078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea981095b0 0x7fea98109990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.576+0000 7fea9e16e700 1 --2- 172.21.15.69:0/90971078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9806c630 0x7fea981103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.576+0000 7fea9e16e700 1 --2- 172.21.15.69:0/90971078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea9806bc70 0x7fea9806c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.576+0000 7fea9e16e700 1 -- 172.21.15.69:0/90971078 >> 172.21.15.69:0/90971078 conn(0x7fea980793f0 msgr2=0x7fea98079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.576+0000 7fea9e16e700 1 -- 172.21.15.69:0/90971078 shutdown_connections 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.576+0000 7fea9e16e700 1 -- 172.21.15.69:0/90971078 wait complete. 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.577+0000 7fea9e16e700 1 Processor -- start 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.577+0000 7fea9e16e700 1 -- start start 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.577+0000 7fea9e16e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9806bc70 0x7fea9819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.577+0000 7fea9e16e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea9806c630 0x7fea9819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.577+0000 7fea9e16e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981095b0 0x7fea98199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.577+0000 7fea9e16e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea98113740 con 0x7fea9806c630 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.577+0000 7fea9e16e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fea981135c0 con 0x7fea981095b0 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.577+0000 7fea9e16e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea981138c0 con 0x7fea9806bc70 2024-05-14T16:48:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea96ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea9806c630 0x7fea9819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea96ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea9806c630 0x7fea9819fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34894/0 (socket says 172.21.15.69:34894) 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea96ffd700 1 -- 172.21.15.69:0/411047391 learned_addr learned my addr 172.21.15.69:0/411047391 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea977fe700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9806bc70 0x7fea9819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea97fff700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981095b0 0x7fea98199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea96ffd700 1 -- 172.21.15.69:0/411047391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9806bc70 msgr2=0x7fea9819f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea96ffd700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea9806bc70 0x7fea9819f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea96ffd700 1 -- 172.21.15.69:0/411047391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981095b0 msgr2=0x7fea98199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea96ffd700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981095b0 0x7fea98199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea96ffd700 1 -- 172.21.15.69:0/411047391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9819a220 con 0x7fea9806c630 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea96ffd700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea9806c630 0x7fea9819fdf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00c920 tx=0x7fea8c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea94ff9700 1 -- 172.21.15.69:0/411047391 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c007b00 con 0x7fea9806c630 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea94ff9700 1 -- 172.21.15.69:0/411047391 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c004ce0 con 0x7fea9806c630 2024-05-14T16:48:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9819a510 con 0x7fea9806c630 2024-05-14T16:48:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea94ff9700 1 -- 172.21.15.69:0/411047391 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c005650 con 0x7fea9806c630 2024-05-14T16:48:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.578+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea981a6780 con 0x7fea9806c630 2024-05-14T16:48:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.579+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea98063cd0 con 0x7fea9806c630 2024-05-14T16:48:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.580+0000 7fea94ff9700 1 -- 172.21.15.69:0/411047391 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fea8c004750 con 0x7fea9806c630 2024-05-14T16:48:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.581+0000 7fea94ff9700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fea800b2910 0x7fea800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.581+0000 7fea977fe700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fea800b2910 0x7fea800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.581+0000 7fea94ff9700 1 -- 172.21.15.69:0/411047391 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fea8c0e24c0 con 0x7fea9806c630 2024-05-14T16:48:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.581+0000 7fea94ff9700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] conn(0x7fea800bc4d0 0x7fea800be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.581+0000 7fea94ff9700 1 -- 172.21.15.69:0/411047391 --> [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea8c012ec0 con 0x7fea800bc4d0 2024-05-14T16:48:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.581+0000 7fea94ff9700 1 -- 172.21.15.69:0/411047391 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fea8c0e28f0 con 0x7fea9806c630 2024-05-14T16:48:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.582+0000 7fea977fe700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fea800b2910 0x7fea800b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea88006010 tx=0x7fea8800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.582+0000 7fea97fff700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] conn(0x7fea800bc4d0 0x7fea800be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.582+0000 7fea97fff700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] conn(0x7fea800bc4d0 0x7fea800be970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.583+0000 7fea94ff9700 1 -- 172.21.15.69:0/411047391 <== osd.1 v2:172.21.15.92:6800/1169355341 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea8c012ec0 con 0x7fea800bc4d0 2024-05-14T16:48:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.601+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 --> [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea98064150 con 0x7fea800bc4d0 2024-05-14T16:48:01.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.601+0000 7fea94ff9700 1 -- 172.21.15.69:0/411047391 <== osd.1 v2:172.21.15.92:6800/1169355341 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fea98064150 con 0x7fea800bc4d0 2024-05-14T16:48:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 >> [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] conn(0x7fea800bc4d0 msgr2=0x7fea800be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea9e16e700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] conn(0x7fea800bc4d0 0x7fea800be970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fea800b2910 msgr2=0x7fea800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea9e16e700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fea800b2910 0x7fea800b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea88006010 tx=0x7fea8800b540 comp rx=0 tx=0).stop 2024-05-14T16:48:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea9806c630 msgr2=0x7fea9819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea9e16e700 1 --2- 172.21.15.69:0/411047391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea9806c630 0x7fea9819fdf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00c920 tx=0x7fea8c00cce0 comp rx=0 tx=0).stop 2024-05-14T16:48:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea97fff700 1 -- 172.21.15.69:0/411047391 reap_dead start 2024-05-14T16:48:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 shutdown_connections 2024-05-14T16:48:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 >> 172.21.15.69:0/411047391 conn(0x7fea980793f0 msgr2=0x7fea981011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 shutdown_connections 2024-05-14T16:48:01.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:01.602+0000 7fea9e16e700 1 -- 172.21.15.69:0/411047391 wait complete. 2024-05-14T16:48:02.048 INFO:teuthology.orchestra.run.smithi069.stdout:77309411403 2024-05-14T16:48:02.049 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-14T16:48:02.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.491+0000 7fca356a4700 1 -- 172.21.15.69:0/1609421054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca30108520 msgr2=0x7fca30108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.491+0000 7fca356a4700 1 --2- 172.21.15.69:0/1609421054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca30108520 0x7fca30108900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fca20009990 tx=0x7fca20009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.491+0000 7fca356a4700 1 -- 172.21.15.69:0/1609421054 shutdown_connections 2024-05-14T16:48:02.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.492+0000 7fca356a4700 1 --2- 172.21.15.69:0/1609421054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca30108520 0x7fca30108900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.492+0000 7fca356a4700 1 --2- 172.21.15.69:0/1609421054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca30102e00 0x7fca3010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.492+0000 7fca356a4700 1 --2- 172.21.15.69:0/1609421054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca30102440 0x7fca301028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.492+0000 7fca356a4700 1 -- 172.21.15.69:0/1609421054 >> 172.21.15.69:0/1609421054 conn(0x7fca300793f0 msgr2=0x7fca30079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.492+0000 7fca356a4700 1 -- 172.21.15.69:0/1609421054 shutdown_connections 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.492+0000 7fca356a4700 1 -- 172.21.15.69:0/1609421054 wait complete. 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.492+0000 7fca356a4700 1 Processor -- start 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.492+0000 7fca356a4700 1 -- start start 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca356a4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca30102440 0x7fca3019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca356a4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca30102e00 0x7fca3019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca356a4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca30108520 0x7fca301a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca356a4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca301125d0 con 0x7fca30108520 2024-05-14T16:48:02.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca356a4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fca30112450 con 0x7fca30102440 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca356a4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fca30112750 con 0x7fca30102e00 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca30108520 0x7fca301a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca30108520 0x7fca301a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34918/0 (socket says 172.21.15.69:34918) 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2f7fe700 1 -- 172.21.15.69:0/3102876041 learned_addr learned my addr 172.21.15.69:0/3102876041 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2e7fc700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca30102e00 0x7fca3019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2effd700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca30102440 0x7fca3019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2f7fe700 1 -- 172.21.15.69:0/3102876041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca30102e00 msgr2=0x7fca3019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2f7fe700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca30102e00 0x7fca3019afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2f7fe700 1 -- 172.21.15.69:0/3102876041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca30102440 msgr2=0x7fca3019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2f7fe700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca30102440 0x7fca3019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2f7fe700 1 -- 172.21.15.69:0/3102876041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca301a6030 con 0x7fca30108520 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca2f7fe700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca30108520 0x7fca301a5af0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fca28007ee0 tx=0x7fca2800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca17fff700 1 -- 172.21.15.69:0/3102876041 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28004bb0 con 0x7fca30108520 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca301a6260 con 0x7fca30108520 2024-05-14T16:48:02.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca17fff700 1 -- 172.21.15.69:0/3102876041 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca28004d50 con 0x7fca30108520 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.493+0000 7fca17fff700 1 -- 172.21.15.69:0/3102876041 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28005920 con 0x7fca30108520 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.494+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca301a6770 con 0x7fca30108520 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.494+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca30063c50 con 0x7fca30108520 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.496+0000 7fca17fff700 1 -- 172.21.15.69:0/3102876041 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fca2801a020 con 0x7fca30108520 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.496+0000 7fca17fff700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fca180b2910 0x7fca180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.497+0000 7fca2effd700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fca180b2910 0x7fca180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.497+0000 7fca17fff700 1 -- 172.21.15.69:0/3102876041 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fca280e20a0 con 0x7fca30108520 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.497+0000 7fca17fff700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] conn(0x7fca180bc4d0 0x7fca180be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.497+0000 7fca17fff700 1 -- 172.21.15.69:0/3102876041 --> [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca180bf040 con 0x7fca180bc4d0 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.497+0000 7fca17fff700 1 -- 172.21.15.69:0/3102876041 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fca280e24d0 con 0x7fca30108520 2024-05-14T16:48:02.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.497+0000 7fca2e7fc700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] conn(0x7fca180bc4d0 0x7fca180be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:02.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.497+0000 7fca2effd700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fca180b2910 0x7fca180b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca200098e0 tx=0x7fca20005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:02.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.497+0000 7fca2e7fc700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] conn(0x7fca180bc4d0 0x7fca180be970 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:02.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.498+0000 7fca17fff700 1 -- 172.21.15.69:0/3102876041 <== osd.0 v2:172.21.15.69:6802/3097069954 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca180bf040 con 0x7fca180bc4d0 2024-05-14T16:48:02.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.514+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 --> [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fca30064050 con 0x7fca180bc4d0 2024-05-14T16:48:02.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.514+0000 7fca17fff700 1 -- 172.21.15.69:0/3102876041 <== osd.0 v2:172.21.15.69:6802/3097069954 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fca30064050 con 0x7fca180bc4d0 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.514+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] conn(0x7fca180bc4d0 msgr2=0x7fca180be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.514+0000 7fca356a4700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] conn(0x7fca180bc4d0 0x7fca180be970 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.515+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fca180b2910 msgr2=0x7fca180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.515+0000 7fca356a4700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fca180b2910 0x7fca180b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca200098e0 tx=0x7fca20005960 comp rx=0 tx=0).stop 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.515+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca30108520 msgr2=0x7fca301a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.515+0000 7fca356a4700 1 --2- 172.21.15.69:0/3102876041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca30108520 0x7fca301a5af0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fca28007ee0 tx=0x7fca2800d3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.515+0000 7fca2f7fe700 1 -- 172.21.15.69:0/3102876041 reap_dead start 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.515+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 shutdown_connections 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.515+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 >> 172.21.15.69:0/3102876041 conn(0x7fca300793f0 msgr2=0x7fca301012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.515+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 shutdown_connections 2024-05-14T16:48:02.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.515+0000 7fca356a4700 1 -- 172.21.15.69:0/3102876041 wait complete. 2024-05-14T16:48:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.928+0000 7f1912fc7700 1 -- 172.21.15.69:0/1619044641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f190c06c670 msgr2=0x7f190c110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.928+0000 7f1912fc7700 1 --2- 172.21.15.69:0/1619044641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f190c06c670 0x7f190c110420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f190400b0a0 tx=0x7f190400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.928+0000 7f1912fc7700 1 -- 172.21.15.69:0/1619044641 shutdown_connections 2024-05-14T16:48:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.928+0000 7f1912fc7700 1 --2- 172.21.15.69:0/1619044641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f190c06c670 0x7f190c110420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.928+0000 7f1912fc7700 1 --2- 172.21.15.69:0/1619044641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f190c1095e0 0x7f190c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.928+0000 7f1912fc7700 1 --2- 172.21.15.69:0/1619044641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c06bcb0 0x7f190c06c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.928+0000 7f1912fc7700 1 -- 172.21.15.69:0/1619044641 >> 172.21.15.69:0/1619044641 conn(0x7f190c0784b0 msgr2=0x7f190c0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.928+0000 7f1912fc7700 1 -- 172.21.15.69:0/1619044641 shutdown_connections 2024-05-14T16:48:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.928+0000 7f1912fc7700 1 -- 172.21.15.69:0/1619044641 wait complete. 2024-05-14T16:48:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1912fc7700 1 Processor -- start 2024-05-14T16:48:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1912fc7700 1 -- start start 2024-05-14T16:48:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1912fc7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c06bcb0 0x7f190c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1912fc7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f190c06c670 0x7f190c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1912fc7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f190c1095e0 0x7f190c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1912fc7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f190c113760 con 0x7f190c1095e0 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1912fc7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1135e0 con 0x7f190c06c670 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1912fc7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1138e0 con 0x7f190c06bcb0 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1911564700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f190c1095e0 0x7f190c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1911564700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f190c1095e0 0x7f190c1999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34944/0 (socket says 172.21.15.69:34944) 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1911564700 1 -- 172.21.15.69:0/3946306477 learned_addr learned my addr 172.21.15.69:0/3946306477 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1910d63700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c06bcb0 0x7f190c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f190bfff700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f190c06c670 0x7f190c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1911564700 1 -- 172.21.15.69:0/3946306477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c06bcb0 msgr2=0x7f190c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1911564700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f190c06bcb0 0x7f190c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1911564700 1 -- 172.21.15.69:0/3946306477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f190c06c670 msgr2=0x7f190c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1911564700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f190c06c670 0x7f190c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1911564700 1 -- 172.21.15.69:0/3946306477 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c19a100 con 0x7f190c1095e0 2024-05-14T16:48:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.929+0000 7f1911564700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f190c1095e0 0x7f190c1999a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f190400b830 tx=0x7f1904004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:02.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.930+0000 7f1909ffb700 1 -- 172.21.15.69:0/3946306477 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904021070 con 0x7f190c1095e0 2024-05-14T16:48:02.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.930+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c19a390 con 0x7f190c1095e0 2024-05-14T16:48:02.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.930+0000 7f1909ffb700 1 -- 172.21.15.69:0/3946306477 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1904007e30 con 0x7f190c1095e0 2024-05-14T16:48:02.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.930+0000 7f1909ffb700 1 -- 172.21.15.69:0/3946306477 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040047d0 con 0x7f190c1095e0 2024-05-14T16:48:02.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.930+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c1a66b0 con 0x7f190c1095e0 2024-05-14T16:48:02.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.931+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f190c063cd0 con 0x7f190c1095e0 2024-05-14T16:48:02.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.933+0000 7f1909ffb700 1 -- 172.21.15.69:0/3946306477 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1904007980 con 0x7f190c1095e0 2024-05-14T16:48:02.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.934+0000 7f1909ffb700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f18fc0b2910 0x7f18fc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:02.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.934+0000 7f1910d63700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f18fc0b2910 0x7f18fc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:02.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.934+0000 7f1909ffb700 1 -- 172.21.15.69:0/3946306477 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f190400f020 con 0x7f190c1095e0 2024-05-14T16:48:02.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.934+0000 7f1909ffb700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] conn(0x7f18fc0bc4d0 0x7f18fc0be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:02.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.934+0000 7f1909ffb700 1 -- 172.21.15.69:0/3946306477 --> [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19040020b0 con 0x7f18fc0bc4d0 2024-05-14T16:48:02.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.934+0000 7f1909ffb700 1 -- 172.21.15.69:0/3946306477 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f19040e3ca0 con 0x7f190c1095e0 2024-05-14T16:48:02.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.934+0000 7f190bfff700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] conn(0x7f18fc0bc4d0 0x7f18fc0be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:02.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.934+0000 7f1910d63700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f18fc0b2910 0x7f18fc0b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f190c19aae0 tx=0x7f190000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:02.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.935+0000 7f190bfff700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] conn(0x7f18fc0bc4d0 0x7f18fc0be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:02.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.935+0000 7f1909ffb700 1 -- 172.21.15.69:0/3946306477 <== osd.3 v2:172.21.15.69:6810/1656710465 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19040020b0 con 0x7f18fc0bc4d0 2024-05-14T16:48:02.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.952+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 --> [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f190c0640d0 con 0x7f18fc0bc4d0 2024-05-14T16:48:02.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.952+0000 7f1909ffb700 1 -- 172.21.15.69:0/3946306477 <== osd.3 v2:172.21.15.69:6810/1656710465 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f190c0640d0 con 0x7f18fc0bc4d0 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.952+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] conn(0x7f18fc0bc4d0 msgr2=0x7f18fc0be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.952+0000 7f1912fc7700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] conn(0x7f18fc0bc4d0 0x7f18fc0be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.953+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f18fc0b2910 msgr2=0x7f18fc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.953+0000 7f1912fc7700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f18fc0b2910 0x7f18fc0b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f190c19aae0 tx=0x7f190000b410 comp rx=0 tx=0).stop 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.953+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f190c1095e0 msgr2=0x7f190c1999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.953+0000 7f1912fc7700 1 --2- 172.21.15.69:0/3946306477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f190c1095e0 0x7f190c1999a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f190400b830 tx=0x7f1904004ce0 comp rx=0 tx=0).stop 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.953+0000 7f1911564700 1 -- 172.21.15.69:0/3946306477 reap_dead start 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.953+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 shutdown_connections 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.953+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 >> 172.21.15.69:0/3946306477 conn(0x7f190c0784b0 msgr2=0x7f190c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.953+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 shutdown_connections 2024-05-14T16:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.953+0000 7f1912fc7700 1 -- 172.21.15.69:0/3946306477 wait complete. 2024-05-14T16:48:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.709+0000 7f24d46b0700 1 -- 172.21.15.69:0/2215673174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24cc102e00 msgr2=0x7f24cc10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.709+0000 7f24d46b0700 1 --2- 172.21.15.69:0/2215673174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24cc102e00 0x7f24cc10f2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f24c400b0a0 tx=0x7f24c400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 -- 172.21.15.69:0/2215673174 shutdown_connections 2024-05-14T16:48:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 --2- 172.21.15.69:0/2215673174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24cc102e00 0x7f24cc10f2f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 --2- 172.21.15.69:0/2215673174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24cc108520 0x7f24cc108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 --2- 172.21.15.69:0/2215673174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24cc102440 0x7f24cc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 -- 172.21.15.69:0/2215673174 >> 172.21.15.69:0/2215673174 conn(0x7f24cc0793f0 msgr2=0x7f24cc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 -- 172.21.15.69:0/2215673174 shutdown_connections 2024-05-14T16:48:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 -- 172.21.15.69:0/2215673174 wait complete. 2024-05-14T16:48:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 Processor -- start 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 -- start start 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24cc102440 0x7f24cc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24cc102e00 0x7f24cc19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24cc108520 0x7f24cc1a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1125f0 con 0x7f24cc102440 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc112470 con 0x7f24cc108520 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.710+0000 7f24d46b0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc112770 con 0x7f24cc102e00 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d244c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24cc102440 0x7f24cc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d244c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24cc102440 0x7f24cc19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34928/0 (socket says 172.21.15.69:34928) 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d244c700 1 -- 172.21.15.69:0/2068260181 learned_addr learned my addr 172.21.15.69:0/2068260181 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d2c4d700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24cc108520 0x7f24cc1a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d1c4b700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24cc102e00 0x7f24cc19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d244c700 1 -- 172.21.15.69:0/2068260181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24cc102e00 msgr2=0x7f24cc19af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d244c700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24cc102e00 0x7f24cc19af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d244c700 1 -- 172.21.15.69:0/2068260181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24cc108520 msgr2=0x7f24cc1a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d244c700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24cc108520 0x7f24cc1a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d244c700 1 -- 172.21.15.69:0/2068260181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc1a6010 con 0x7f24cc102440 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d244c700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24cc102440 0x7f24cc19aa50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e9a0 tx=0x7f24c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24c37fe700 1 -- 172.21.15.69:0/2068260181 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c800ca10 con 0x7f24cc102440 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24cc1a61e0 con 0x7f24cc102440 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24c37fe700 1 -- 172.21.15.69:0/2068260181 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c800cbb0 con 0x7f24cc102440 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24c37fe700 1 -- 172.21.15.69:0/2068260181 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8018680 con 0x7f24cc102440 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.711+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24cc1a6720 con 0x7f24cc102440 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.712+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24cc063c10 con 0x7f24cc102440 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.713+0000 7f24c37fe700 1 -- 172.21.15.69:0/2068260181 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f24c8010430 con 0x7f24cc102440 2024-05-14T16:48:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.714+0000 7f24c37fe700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24b80b2910 0x7f24b80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.714+0000 7f24d1c4b700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24b80b2910 0x7f24b80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.714+0000 7f24c37fe700 1 -- 172.21.15.69:0/2068260181 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f24c80e2ab0 con 0x7f24cc102440 2024-05-14T16:48:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.714+0000 7f24c37fe700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] conn(0x7f24b80bc4d0 0x7f24b80be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.714+0000 7f24c37fe700 1 -- 172.21.15.69:0/2068260181 --> [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24c8024e70 con 0x7f24b80bc4d0 2024-05-14T16:48:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.714+0000 7f24c37fe700 1 -- 172.21.15.69:0/2068260181 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f24c80e2e60 con 0x7f24cc102440 2024-05-14T16:48:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.715+0000 7f24d1c4b700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24b80b2910 0x7f24b80b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24cc102270 tx=0x7f24bc00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.715+0000 7f24d2c4d700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] conn(0x7f24b80bc4d0 0x7f24b80be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.715+0000 7f24d2c4d700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] conn(0x7f24b80bc4d0 0x7f24b80be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.716+0000 7f24c37fe700 1 -- 172.21.15.69:0/2068260181 <== osd.10 v2:172.21.15.92:6824/1052364226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24c8024e70 con 0x7f24b80bc4d0 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.732+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 --> [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24cc064090 con 0x7f24b80bc4d0 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24c37fe700 1 -- 172.21.15.69:0/2068260181 <== osd.10 v2:172.21.15.92:6824/1052364226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f24cc064090 con 0x7f24b80bc4d0 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 >> [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] conn(0x7f24b80bc4d0 msgr2=0x7f24b80be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d46b0700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] conn(0x7f24b80bc4d0 0x7f24b80be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24b80b2910 msgr2=0x7f24b80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d46b0700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f24b80b2910 0x7f24b80b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24cc102270 tx=0x7f24bc00b3c0 comp rx=0 tx=0).stop 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24cc102440 msgr2=0x7f24cc19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d46b0700 1 --2- 172.21.15.69:0/2068260181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24cc102440 0x7f24cc19aa50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e9a0 tx=0x7f24c800ecb0 comp rx=0 tx=0).stop 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d2c4d700 1 -- 172.21.15.69:0/2068260181 reap_dead start 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 shutdown_connections 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 >> 172.21.15.69:0/2068260181 conn(0x7f24cc0793f0 msgr2=0x7f24cc100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 shutdown_connections 2024-05-14T16:48:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:02.733+0000 7f24d46b0700 1 -- 172.21.15.69:0/2068260181 wait complete. 2024-05-14T16:48:03.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:02 smithi069 ceph-mon[31507]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:03.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:02 smithi203 ceph-mon[35787]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:03.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:02 smithi092 ceph-mon[35763]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:05.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:04 smithi203 ceph-mon[35787]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:05.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:04 smithi092 ceph-mon[35763]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:05.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:04 smithi069 ceph-mon[31507]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:05.525 INFO:teuthology.orchestra.run.smithi069.stdout:227633266735 2024-05-14T16:48:05.525 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-14T16:48:05.957 INFO:teuthology.orchestra.run.smithi069.stdout:300647710751 2024-05-14T16:48:05.958 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-14T16:48:06.544 INFO:teuthology.orchestra.run.smithi069.stdout:274877906980 2024-05-14T16:48:06.545 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-14T16:48:07.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 -- 172.21.15.69:0/4150643554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a01084b0 msgr2=0x7ff9a0108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:07.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 --2- 172.21.15.69:0/4150643554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a01084b0 0x7ff9a0108890 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009990 tx=0x7ff99c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:07.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 -- 172.21.15.69:0/4150643554 shutdown_connections 2024-05-14T16:48:07.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 --2- 172.21.15.69:0/4150643554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a01084b0 0x7ff9a0108890 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:07.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 --2- 172.21.15.69:0/4150643554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a0102e30 0x7ff9a010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:07.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 --2- 172.21.15.69:0/4150643554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a0102470 0x7ff9a01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 -- 172.21.15.69:0/4150643554 >> 172.21.15.69:0/4150643554 conn(0x7ff9a00793f0 msgr2=0x7ff9a0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 -- 172.21.15.69:0/4150643554 shutdown_connections 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 -- 172.21.15.69:0/4150643554 wait complete. 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 Processor -- start 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 -- start start 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a0102470 0x7ff9a019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.149+0000 7ff9a75c3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a0102e30 0x7ff9a019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a75c3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a01084b0 0x7ff9a01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a75c3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0112690 con 0x7ff9a0102e30 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a75c3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0112510 con 0x7ff9a01084b0 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a75c3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0112810 con 0x7ff9a0102470 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a4b5e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a0102e30 0x7ff9a019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a4b5e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a0102e30 0x7ff9a019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43828/0 (socket says 172.21.15.69:43828) 2024-05-14T16:48:07.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a4b5e700 1 -- 172.21.15.69:0/1294563201 learned_addr learned my addr 172.21.15.69:0/1294563201 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a535f700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a0102470 0x7ff9a019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a5b60700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a01084b0 0x7ff9a01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a4b5e700 1 -- 172.21.15.69:0/1294563201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a0102470 msgr2=0x7ff9a019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a4b5e700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a0102470 0x7ff9a019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a4b5e700 1 -- 172.21.15.69:0/1294563201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a01084b0 msgr2=0x7ff9a01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a4b5e700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a01084b0 0x7ff9a01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a4b5e700 1 -- 172.21.15.69:0/1294563201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a0199150 con 0x7ff9a0102e30 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a5b60700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a01084b0 0x7ff9a01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a4b5e700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a0102e30 0x7ff9a019ed20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff99000b530 tx=0x7ff99000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9967fc700 1 -- 172.21.15.69:0/1294563201 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990015020 con 0x7ff9a0102e30 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9967fc700 1 -- 172.21.15.69:0/1294563201 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff990011ce0 con 0x7ff9a0102e30 2024-05-14T16:48:07.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9967fc700 1 -- 172.21.15.69:0/1294563201 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990010e40 con 0x7ff9a0102e30 2024-05-14T16:48:07.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a0199440 con 0x7ff9a0102e30 2024-05-14T16:48:07.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.150+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a01a56b0 con 0x7ff9a0102e30 2024-05-14T16:48:07.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.151+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9a0063cd0 con 0x7ff9a0102e30 2024-05-14T16:48:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.153+0000 7ff9967fc700 1 -- 172.21.15.69:0/1294563201 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff990018020 con 0x7ff9a0102e30 2024-05-14T16:48:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.153+0000 7ff9967fc700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff98c0b2910 0x7ff98c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.153+0000 7ff9a535f700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff98c0b2910 0x7ff98c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.153+0000 7ff9967fc700 1 -- 172.21.15.69:0/1294563201 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9900e2240 con 0x7ff9a0102e30 2024-05-14T16:48:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.154+0000 7ff9967fc700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] conn(0x7ff98c0bc4d0 0x7ff98c0be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.154+0000 7ff9967fc700 1 -- 172.21.15.69:0/1294563201 --> [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff99001be50 con 0x7ff98c0bc4d0 2024-05-14T16:48:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.154+0000 7ff9967fc700 1 -- 172.21.15.69:0/1294563201 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7ff9900e2670 con 0x7ff9a0102e30 2024-05-14T16:48:07.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.154+0000 7ff9a5b60700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] conn(0x7ff98c0bc4d0 0x7ff98c0be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:07.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.154+0000 7ff9a535f700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff98c0b2910 0x7ff98c0b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0102100 tx=0x7ff99c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:07.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.154+0000 7ff9a5b60700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] conn(0x7ff98c0bc4d0 0x7ff98c0be970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:07.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.155+0000 7ff9967fc700 1 -- 172.21.15.69:0/1294563201 <== osd.6 v2:172.21.15.69:6818/2234812220 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff99001be50 con 0x7ff98c0bc4d0 2024-05-14T16:48:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.175+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 --> [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9a00640d0 con 0x7ff98c0bc4d0 2024-05-14T16:48:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.175+0000 7ff9967fc700 1 -- 172.21.15.69:0/1294563201 <== osd.6 v2:172.21.15.69:6818/2234812220 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff9a00640d0 con 0x7ff98c0bc4d0 2024-05-14T16:48:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] conn(0x7ff98c0bc4d0 msgr2=0x7ff98c0be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a75c3700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] conn(0x7ff98c0bc4d0 0x7ff98c0be970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff98c0b2910 msgr2=0x7ff98c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a75c3700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff98c0b2910 0x7ff98c0b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0102100 tx=0x7ff99c00b540 comp rx=0 tx=0).stop 2024-05-14T16:48:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a0102e30 msgr2=0x7ff9a019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a75c3700 1 --2- 172.21.15.69:0/1294563201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a0102e30 0x7ff9a019ed20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff99000b530 tx=0x7ff99000b840 comp rx=0 tx=0).stop 2024-05-14T16:48:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a5b60700 1 -- 172.21.15.69:0/1294563201 reap_dead start 2024-05-14T16:48:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 shutdown_connections 2024-05-14T16:48:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 >> 172.21.15.69:0/1294563201 conn(0x7ff9a00793f0 msgr2=0x7ff9a0100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 shutdown_connections 2024-05-14T16:48:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:07.176+0000 7ff9a75c3700 1 -- 172.21.15.69:0/1294563201 wait complete. 2024-05-14T16:48:07.188 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:06 smithi069 ceph-mon[31507]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:07.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:06 smithi092 ceph-mon[35763]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:07.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:06 smithi203 ceph-mon[35787]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:07.800 INFO:teuthology.orchestra.run.smithi069.stdout:55834574930 2024-05-14T16:48:07.801 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-14T16:48:08.343 INFO:teuthology.orchestra.run.smithi069.stdout:107374182472 2024-05-14T16:48:08.343 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-14T16:48:08.962 INFO:teuthology.orchestra.run.smithi069.stdout:180388626491 2024-05-14T16:48:08.962 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-14T16:48:09.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:08 smithi069 ceph-mon[31507]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:09.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:08 smithi092 ceph-mon[35763]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:09.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:08 smithi203 ceph-mon[35787]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:09.466 INFO:teuthology.orchestra.run.smithi069.stdout:34359738455 2024-05-14T16:48:09.466 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-14T16:48:09.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.663+0000 7fa3b17eb700 1 -- 172.21.15.69:0/3099826918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3ac102af0 msgr2=0x7fa3ac10b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.663+0000 7fa3b17eb700 1 --2- 172.21.15.69:0/3099826918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3ac102af0 0x7fa3ac10b6a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa3940099c0 tx=0x7fa394009cd0 comp rx=0 tx=0).stop 2024-05-14T16:48:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.663+0000 7fa3b17eb700 1 -- 172.21.15.69:0/3099826918 shutdown_connections 2024-05-14T16:48:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.663+0000 7fa3b17eb700 1 --2- 172.21.15.69:0/3099826918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac10bd80 0x7fa3ac112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.663+0000 7fa3b17eb700 1 --2- 172.21.15.69:0/3099826918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3ac102140 0x7fa3ac102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.663+0000 7fa3b17eb700 1 --2- 172.21.15.69:0/3099826918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3ac102af0 0x7fa3ac10b6a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.663+0000 7fa3b17eb700 1 -- 172.21.15.69:0/3099826918 >> 172.21.15.69:0/3099826918 conn(0x7fa3ac0777e0 msgr2=0x7fa3ac077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.663+0000 7fa3b17eb700 1 -- 172.21.15.69:0/3099826918 shutdown_connections 2024-05-14T16:48:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.663+0000 7fa3b17eb700 1 -- 172.21.15.69:0/3099826918 wait complete. 2024-05-14T16:48:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.664+0000 7fa3b17eb700 1 Processor -- start 2024-05-14T16:48:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.664+0000 7fa3b17eb700 1 -- start start 2024-05-14T16:48:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.664+0000 7fa3b17eb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3ac102140 0x7fa3ac19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.664+0000 7fa3b17eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac102af0 0x7fa3ac19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.664+0000 7fa3b17eb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3ac10bd80 0x7fa3ac1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.664+0000 7fa3b17eb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac115990 con 0x7fa3ac10bd80 2024-05-14T16:48:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.664+0000 7fa3b17eb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac115810 con 0x7fa3ac102af0 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.664+0000 7fa3b17eb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac115b10 con 0x7fa3ac102140 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.664+0000 7fa3ab7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3ac10bd80 0x7fa3ac1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3ab7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3ac10bd80 0x7fa3ac1a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43848/0 (socket says 172.21.15.69:43848) 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3ab7fe700 1 -- 172.21.15.69:0/1793126501 learned_addr learned my addr 172.21.15.69:0/1793126501 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3aaffd700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3ac102140 0x7fa3ac19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3a3fff700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac102af0 0x7fa3ac19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3ab7fe700 1 -- 172.21.15.69:0/1793126501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3ac102140 msgr2=0x7fa3ac19be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3ab7fe700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3ac102140 0x7fa3ac19be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3ab7fe700 1 -- 172.21.15.69:0/1793126501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac102af0 msgr2=0x7fa3ac19c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3ab7fe700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac102af0 0x7fa3ac19c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3ab7fe700 1 -- 172.21.15.69:0/1793126501 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac1a0e10 con 0x7fa3ac10bd80 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3ab7fe700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3ac10bd80 0x7fa3ac1a06d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400ada0 tx=0x7fa3a400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3a8ff9700 1 -- 172.21.15.69:0/1793126501 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4017020 con 0x7fa3ac10bd80 2024-05-14T16:48:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3a8ff9700 1 -- 172.21.15.69:0/1793126501 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a400cd80 con 0x7fa3ac10bd80 2024-05-14T16:48:09.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3a8ff9700 1 -- 172.21.15.69:0/1793126501 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4005260 con 0x7fa3ac10bd80 2024-05-14T16:48:09.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac1a1100 con 0x7fa3ac10bd80 2024-05-14T16:48:09.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.665+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac1a8980 con 0x7fa3ac10bd80 2024-05-14T16:48:09.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.666+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3ac063cd0 con 0x7fa3ac10bd80 2024-05-14T16:48:09.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.668+0000 7fa3a8ff9700 1 -- 172.21.15.69:0/1793126501 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa3a400a080 con 0x7fa3ac10bd80 2024-05-14T16:48:09.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.669+0000 7fa3a8ff9700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa3980b2910 0x7fa3980b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:09.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.669+0000 7fa3aaffd700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa3980b2910 0x7fa3980b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:09.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.669+0000 7fa3a8ff9700 1 -- 172.21.15.69:0/1793126501 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fa3a40e2480 con 0x7fa3ac10bd80 2024-05-14T16:48:09.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.669+0000 7fa3a8ff9700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] conn(0x7fa3980bc4d0 0x7fa3980be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:09.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.669+0000 7fa3a8ff9700 1 -- 172.21.15.69:0/1793126501 --> [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3980bf040 con 0x7fa3980bc4d0 2024-05-14T16:48:09.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.669+0000 7fa3a8ff9700 1 -- 172.21.15.69:0/1793126501 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fa3a40e28b0 con 0x7fa3ac10bd80 2024-05-14T16:48:09.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.669+0000 7fa3a3fff700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] conn(0x7fa3980bc4d0 0x7fa3980be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:09.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.669+0000 7fa3aaffd700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa3980b2910 0x7fa3980b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa39c004630 tx=0x7fa39c00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:09.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.670+0000 7fa3a3fff700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] conn(0x7fa3980bc4d0 0x7fa3980be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:09.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.671+0000 7fa3a8ff9700 1 -- 172.21.15.69:0/1793126501 <== osd.9 v2:172.21.15.69:6826/1618024425 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3980bf040 con 0x7fa3980bc4d0 2024-05-14T16:48:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.688+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 --> [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3ac064150 con 0x7fa3980bc4d0 2024-05-14T16:48:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.689+0000 7fa3a8ff9700 1 -- 172.21.15.69:0/1793126501 <== osd.9 v2:172.21.15.69:6826/1618024425 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa3ac064150 con 0x7fa3980bc4d0 2024-05-14T16:48:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.689+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] conn(0x7fa3980bc4d0 msgr2=0x7fa3980be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.689+0000 7fa3b17eb700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] conn(0x7fa3980bc4d0 0x7fa3980be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.689+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa3980b2910 msgr2=0x7fa3980b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.689+0000 7fa3b17eb700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa3980b2910 0x7fa3980b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa39c004630 tx=0x7fa39c00b3c0 comp rx=0 tx=0).stop 2024-05-14T16:48:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.689+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3ac10bd80 msgr2=0x7fa3ac1a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.689+0000 7fa3b17eb700 1 --2- 172.21.15.69:0/1793126501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3ac10bd80 0x7fa3ac1a06d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400ada0 tx=0x7fa3a400c330 comp rx=0 tx=0).stop 2024-05-14T16:48:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.689+0000 7fa3ab7fe700 1 -- 172.21.15.69:0/1793126501 reap_dead start 2024-05-14T16:48:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.689+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 shutdown_connections 2024-05-14T16:48:09.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.689+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 >> 172.21.15.69:0/1793126501 conn(0x7fa3ac0777e0 msgr2=0x7fa3ac1098f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:09.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.690+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 shutdown_connections 2024-05-14T16:48:09.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:09.690+0000 7fa3b17eb700 1 -- 172.21.15.69:0/1793126501 wait complete. 2024-05-14T16:48:10.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.735+0000 7f3fcc043700 1 -- 172.21.15.69:0/3795942627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4108520 msgr2=0x7f3fc4108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:10.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.735+0000 7f3fcc043700 1 --2- 172.21.15.69:0/3795942627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4108520 0x7f3fc4108900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400b0a0 tx=0x7f3fb400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.735+0000 7f3fcc043700 1 -- 172.21.15.69:0/3795942627 shutdown_connections 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.735+0000 7f3fcc043700 1 --2- 172.21.15.69:0/3795942627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4108520 0x7f3fc4108900 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.735+0000 7f3fcc043700 1 --2- 172.21.15.69:0/3795942627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc4102e00 0x7f3fc410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.735+0000 7f3fcc043700 1 --2- 172.21.15.69:0/3795942627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fc4102440 0x7f3fc41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.735+0000 7f3fcc043700 1 -- 172.21.15.69:0/3795942627 >> 172.21.15.69:0/3795942627 conn(0x7f3fc40793f0 msgr2=0x7f3fc4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.736+0000 7f3fcc043700 1 -- 172.21.15.69:0/3795942627 shutdown_connections 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.736+0000 7f3fcc043700 1 -- 172.21.15.69:0/3795942627 wait complete. 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.736+0000 7f3fcc043700 1 Processor -- start 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.736+0000 7f3fcc043700 1 -- start start 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.736+0000 7f3fcc043700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fc4102440 0x7f3fc4111670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.736+0000 7f3fcc043700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc4102e00 0x7f3fc4111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.736+0000 7f3fcc043700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4108520 0x7f3fc41120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.736+0000 7f3fcc043700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41137b0 con 0x7f3fc4108520 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.736+0000 7f3fcc043700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc406c150 con 0x7f3fc4102440 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.736+0000 7f3fcc043700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4113930 con 0x7f3fc4102e00 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fca5e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4108520 0x7f3fc41120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fca5e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4108520 0x7f3fc41120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43862/0 (socket says 172.21.15.69:43862) 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fca5e0700 1 -- 172.21.15.69:0/899706707 learned_addr learned my addr 172.21.15.69:0/899706707 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fc9ddf700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fc4102440 0x7f3fc4111670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fc95de700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc4102e00 0x7f3fc4111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fca5e0700 1 -- 172.21.15.69:0/899706707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc4102e00 msgr2=0x7f3fc4111bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fca5e0700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc4102e00 0x7f3fc4111bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fca5e0700 1 -- 172.21.15.69:0/899706707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fc4102440 msgr2=0x7f3fc4111670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fca5e0700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fc4102440 0x7f3fc4111670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fca5e0700 1 -- 172.21.15.69:0/899706707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc41a8270 con 0x7f3fc4108520 2024-05-14T16:48:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fc95de700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc4102e00 0x7f3fc4111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:48:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fca5e0700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4108520 0x7f3fc41120f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00b550 tx=0x7f3fbc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fc2ffd700 1 -- 172.21.15.69:0/899706707 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc015020 con 0x7f3fc4108520 2024-05-14T16:48:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fc2ffd700 1 -- 172.21.15.69:0/899706707 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fbc011ce0 con 0x7f3fc4108520 2024-05-14T16:48:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc41a8560 con 0x7f3fc4108520 2024-05-14T16:48:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fc2ffd700 1 -- 172.21.15.69:0/899706707 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc010d40 con 0x7f3fc4108520 2024-05-14T16:48:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.737+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc41a8aa0 con 0x7f3fc4108520 2024-05-14T16:48:10.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.738+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3fc4063c50 con 0x7f3fc4108520 2024-05-14T16:48:10.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.739+0000 7f3fc2ffd700 1 -- 172.21.15.69:0/899706707 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3fbc018020 con 0x7f3fc4108520 2024-05-14T16:48:10.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.740+0000 7f3fc2ffd700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f3fac0b2990 0x7f3fac0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:10.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.740+0000 7f3fc9ddf700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f3fac0b2990 0x7f3fac0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:10.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.740+0000 7f3fc2ffd700 1 -- 172.21.15.69:0/899706707 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f3fbc0e20f0 con 0x7f3fc4108520 2024-05-14T16:48:10.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.740+0000 7f3fc2ffd700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] conn(0x7f3fac0bc550 0x7f3fac0be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:10.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.740+0000 7f3fc2ffd700 1 -- 172.21.15.69:0/899706707 --> [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3fbc01be50 con 0x7f3fac0bc550 2024-05-14T16:48:10.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.740+0000 7f3fc2ffd700 1 -- 172.21.15.69:0/899706707 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f3fbc0e2520 con 0x7f3fc4108520 2024-05-14T16:48:10.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.741+0000 7f3fc9ddf700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f3fac0b2990 0x7f3fac0b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4006130 tx=0x7f3fb40060a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:10.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.741+0000 7f3fc95de700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] conn(0x7f3fac0bc550 0x7f3fac0be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:10.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.741+0000 7f3fc95de700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] conn(0x7f3fac0bc550 0x7f3fac0be9f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:10.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.742+0000 7f3fc2ffd700 1 -- 172.21.15.69:0/899706707 <== osd.4 v2:172.21.15.92:6808/301070380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3fbc01be50 con 0x7f3fac0bc550 2024-05-14T16:48:10.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.758+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 --> [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3fc4064050 con 0x7f3fac0bc550 2024-05-14T16:48:10.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fc2ffd700 1 -- 172.21.15.69:0/899706707 <== osd.4 v2:172.21.15.92:6808/301070380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3fc4064050 con 0x7f3fac0bc550 2024-05-14T16:48:10.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 >> [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] conn(0x7f3fac0bc550 msgr2=0x7f3fac0be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:10.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fcc043700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] conn(0x7f3fac0bc550 0x7f3fac0be9f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:10.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f3fac0b2990 msgr2=0x7f3fac0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:10.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fcc043700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f3fac0b2990 0x7f3fac0b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4006130 tx=0x7f3fb40060a0 comp rx=0 tx=0).stop 2024-05-14T16:48:10.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4108520 msgr2=0x7f3fc41120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:10.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fcc043700 1 --2- 172.21.15.69:0/899706707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc4108520 0x7f3fc41120f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00b550 tx=0x7f3fbc00b910 comp rx=0 tx=0).stop 2024-05-14T16:48:10.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fca5e0700 1 -- 172.21.15.69:0/899706707 reap_dead start 2024-05-14T16:48:10.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 shutdown_connections 2024-05-14T16:48:10.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 >> 172.21.15.69:0/899706707 conn(0x7f3fc40793f0 msgr2=0x7f3fc4101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:10.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 shutdown_connections 2024-05-14T16:48:10.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:10.759+0000 7f3fcc043700 1 -- 172.21.15.69:0/899706707 wait complete. 2024-05-14T16:48:11.171 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:10 smithi069 ceph-mon[31507]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:11.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:10 smithi092 ceph-mon[35763]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.276+0000 7f1d10b56700 1 -- 172.21.15.69:0/4266415436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d0c101b30 msgr2=0x7f1d0c101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.276+0000 7f1d10b56700 1 --2- 172.21.15.69:0/4266415436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d0c101b30 0x7f1d0c101fb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40099c0 tx=0x7f1cf4009cd0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.276+0000 7f1d10b56700 1 -- 172.21.15.69:0/4266415436 shutdown_connections 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.276+0000 7f1d10b56700 1 --2- 172.21.15.69:0/4266415436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d0c1024f0 0x7f1d0c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.276+0000 7f1d10b56700 1 --2- 172.21.15.69:0/4266415436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d0c109520 0x7f1d0c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.276+0000 7f1d10b56700 1 --2- 172.21.15.69:0/4266415436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d0c101b30 0x7f1d0c101fb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.276+0000 7f1d10b56700 1 -- 172.21.15.69:0/4266415436 >> 172.21.15.69:0/4266415436 conn(0x7f1d0c078ab0 msgr2=0x7f1d0c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.276+0000 7f1d10b56700 1 -- 172.21.15.69:0/4266415436 shutdown_connections 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.276+0000 7f1d10b56700 1 -- 172.21.15.69:0/4266415436 wait complete. 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d10b56700 1 Processor -- start 2024-05-14T16:48:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d10b56700 1 -- start start 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d10b56700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d0c101b30 0x7f1d0c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d10b56700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d0c1024f0 0x7f1d0c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d10b56700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d0c109520 0x7f1d0c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d10b56700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c113760 con 0x7f1d0c101b30 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d10b56700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c1135e0 con 0x7f1d0c1024f0 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d10b56700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c1138e0 con 0x7f1d0c109520 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d0a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d0c101b30 0x7f1d0c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d0a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d0c101b30 0x7f1d0c19f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43890/0 (socket says 172.21.15.69:43890) 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d0a59c700 1 -- 172.21.15.69:0/3919124628 learned_addr learned my addr 172.21.15.69:0/3919124628 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d09d9b700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d0c1024f0 0x7f1d0c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d0ad9d700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d0c109520 0x7f1d0c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d0a59c700 1 -- 172.21.15.69:0/3919124628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d0c109520 msgr2=0x7f1d0c199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d0a59c700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d0c109520 0x7f1d0c199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d0a59c700 1 -- 172.21.15.69:0/3919124628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d0c1024f0 msgr2=0x7f1d0c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:11.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d0a59c700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d0c1024f0 0x7f1d0c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.277+0000 7f1d0a59c700 1 -- 172.21.15.69:0/3919124628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c19a0e0 con 0x7f1d0c101b30 2024-05-14T16:48:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.278+0000 7f1d0a59c700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d0c101b30 0x7f1d0c19f7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1d040060b0 tx=0x7f1d0400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.278+0000 7f1d037fe700 1 -- 172.21.15.69:0/3919124628 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0401a070 con 0x7f1d0c101b30 2024-05-14T16:48:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.278+0000 7f1d037fe700 1 -- 172.21.15.69:0/3919124628 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0400ddd0 con 0x7f1d0c101b30 2024-05-14T16:48:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.278+0000 7f1d10b56700 1 -- 172.21.15.69:0/3919124628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0c19a3d0 con 0x7f1d0c101b30 2024-05-14T16:48:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.278+0000 7f1d037fe700 1 -- 172.21.15.69:0/3919124628 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04014720 con 0x7f1d0c101b30 2024-05-14T16:48:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.278+0000 7f1d10b56700 1 -- 172.21.15.69:0/3919124628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0c1a6690 con 0x7f1d0c101b30 2024-05-14T16:48:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.279+0000 7f1d10b56700 1 -- 172.21.15.69:0/3919124628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d0c104e50 con 0x7f1d0c101b30 2024-05-14T16:48:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.280+0000 7f1d037fe700 1 -- 172.21.15.69:0/3919124628 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1d0401d070 con 0x7f1d0c101b30 2024-05-14T16:48:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.281+0000 7f1d037fe700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1cf80b2910 0x7f1cf80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.281+0000 7f1d09d9b700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1cf80b2910 0x7f1cf80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.281+0000 7f1d037fe700 1 -- 172.21.15.69:0/3919124628 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1d040e1bb0 con 0x7f1d0c101b30 2024-05-14T16:48:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.281+0000 7f1d09d9b700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1cf80b2910 0x7f1cf80b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009990 tx=0x7f1cf400b790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.286+0000 7f1d037fe700 1 -- 172.21.15.69:0/3919124628 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d04010070 con 0x7f1d0c101b30 2024-05-14T16:48:11.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:10 smithi203 ceph-mon[35787]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:11.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.498+0000 7f1d10b56700 1 -- 172.21.15.69:0/3919124628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1d0c02e560 con 0x7f1d0c101b30 2024-05-14T16:48:11.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.499+0000 7f1d037fe700 1 -- 172.21.15.69:0/3919124628 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1d040a9770 con 0x7f1d0c101b30 2024-05-14T16:48:11.499 INFO:teuthology.orchestra.run.smithi069.stdout:154618822720 2024-05-14T16:48:11.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 -- 172.21.15.69:0/3919124628 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1cf80b2910 msgr2=0x7f1cf80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:11.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1cf80b2910 0x7f1cf80b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009990 tx=0x7f1cf400b790 comp rx=0 tx=0).stop 2024-05-14T16:48:11.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 -- 172.21.15.69:0/3919124628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d0c101b30 msgr2=0x7f1d0c19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:11.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d0c101b30 0x7f1d0c19f7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1d040060b0 tx=0x7f1d0400d3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 -- 172.21.15.69:0/3919124628 shutdown_connections 2024-05-14T16:48:11.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1cf80b2910 0x7f1cf80b4dd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d0c101b30 0x7f1d0c19f7c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d0c109520 0x7f1d0c199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 --2- 172.21.15.69:0/3919124628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d0c1024f0 0x7f1d0c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:11.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 -- 172.21.15.69:0/3919124628 >> 172.21.15.69:0/3919124628 conn(0x7f1d0c078ab0 msgr2=0x7f1d0c100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:11.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 -- 172.21.15.69:0/3919124628 shutdown_connections 2024-05-14T16:48:11.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:11.501+0000 7f1d10b56700 1 -- 172.21.15.69:0/3919124628 wait complete. 2024-05-14T16:48:11.701 INFO:teuthology.orchestra.run.smithi069.stdout:253403070508 2024-05-14T16:48:11.701 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-14T16:48:12.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:11 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3919124628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T16:48:12.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:11 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3919124628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T16:48:12.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:11 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3919124628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T16:48:12.557 INFO:teuthology.orchestra.run.smithi069.stdout:133143986245 2024-05-14T16:48:12.557 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-14T16:48:12.981 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:12.986 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822716 got 154618822720 for osd.5 2024-05-14T16:48:12.986 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:12.991 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:13.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:12 smithi069 ceph-mon[31507]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:13.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:12 smithi092 ceph-mon[35763]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:13.360 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:12 smithi203 ceph-mon[35787]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.584+0000 7f6aa91a2700 1 -- 172.21.15.69:0/1814882776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa4076180 msgr2=0x7f6aa410f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.584+0000 7f6aa91a2700 1 --2- 172.21.15.69:0/1814882776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa4076180 0x7f6aa410f7d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009990 tx=0x7f6a8c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.584+0000 7f6aa91a2700 1 -- 172.21.15.69:0/1814882776 shutdown_connections 2024-05-14T16:48:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.584+0000 7f6aa91a2700 1 --2- 172.21.15.69:0/1814882776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa410fd10 0x7f6aa4112100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.584+0000 7f6aa91a2700 1 --2- 172.21.15.69:0/1814882776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6aa4075860 0x7f6aa4075c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.584+0000 7f6aa91a2700 1 --2- 172.21.15.69:0/1814882776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa4076180 0x7f6aa410f7d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.584+0000 7f6aa91a2700 1 -- 172.21.15.69:0/1814882776 >> 172.21.15.69:0/1814882776 conn(0x7f6aa40fef60 msgr2=0x7f6aa4101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.584+0000 7f6aa91a2700 1 -- 172.21.15.69:0/1814882776 shutdown_connections 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.584+0000 7f6aa91a2700 1 -- 172.21.15.69:0/1814882776 wait complete. 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa91a2700 1 Processor -- start 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa91a2700 1 -- start start 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa91a2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6aa4075860 0x7f6aa419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa91a2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa4076180 0x7f6aa419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa91a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa410fd10 0x7f6aa41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa91a2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41158f0 con 0x7f6aa4076180 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa91a2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa4115770 con 0x7f6aa410fd10 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa91a2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa4115a70 con 0x7f6aa4075860 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6a9bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa4076180 0x7f6aa419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6a9bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa4076180 0x7f6aa419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43904/0 (socket says 172.21.15.69:43904) 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6a9bfff700 1 -- 172.21.15.69:0/417654814 learned_addr learned my addr 172.21.15.69:0/417654814 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6a9bfff700 1 -- 172.21.15.69:0/417654814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6aa4075860 msgr2=0x7f6aa419bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa3fff700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6aa4075860 0x7f6aa419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6a9bfff700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6aa4075860 0x7f6aa419bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6a9bfff700 1 -- 172.21.15.69:0/417654814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa410fd10 msgr2=0x7f6aa41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa89a1700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa410fd10 0x7f6aa41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6a9bfff700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa410fd10 0x7f6aa41a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6a9bfff700 1 -- 172.21.15.69:0/417654814 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa41a0ea0 con 0x7f6aa4076180 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6aa89a1700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa410fd10 0x7f6aa41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.585+0000 7f6a9bfff700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa4076180 0x7f6aa419c420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c000c00 tx=0x7f6a8c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.586+0000 7f6aa1ffb700 1 -- 172.21.15.69:0/417654814 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c027070 con 0x7f6aa4076180 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.586+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa41a1130 con 0x7f6aa4076180 2024-05-14T16:48:13.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.586+0000 7f6aa1ffb700 1 -- 172.21.15.69:0/417654814 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c0041d0 con 0x7f6aa4076180 2024-05-14T16:48:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.586+0000 7f6aa1ffb700 1 -- 172.21.15.69:0/417654814 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00f460 con 0x7f6aa4076180 2024-05-14T16:48:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.586+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa41a8a10 con 0x7f6aa4076180 2024-05-14T16:48:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.586+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6aa4063c50 con 0x7f6aa4076180 2024-05-14T16:48:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.588+0000 7f6aa1ffb700 1 -- 172.21.15.69:0/417654814 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6a8c02a070 con 0x7f6aa4076180 2024-05-14T16:48:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.589+0000 7f6aa1ffb700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6a900b2910 0x7f6a900b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.589+0000 7f6aa3fff700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6a900b2910 0x7f6a900b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.589+0000 7f6aa1ffb700 1 -- 172.21.15.69:0/417654814 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6a8c0e3ae0 con 0x7f6aa4076180 2024-05-14T16:48:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.589+0000 7f6aa1ffb700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] conn(0x7f6a900bc4d0 0x7f6a900be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.589+0000 7f6aa1ffb700 1 -- 172.21.15.69:0/417654814 --> [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a8c0043d0 con 0x7f6a900bc4d0 2024-05-14T16:48:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.589+0000 7f6aa1ffb700 1 -- 172.21.15.69:0/417654814 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f6a8c0e3f10 con 0x7f6aa4076180 2024-05-14T16:48:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.589+0000 7f6aa89a1700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] conn(0x7f6a900bc4d0 0x7f6a900be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.589+0000 7f6aa3fff700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6a900b2910 0x7f6a900b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6a94006fd0 tx=0x7f6a94008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.590+0000 7f6aa89a1700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] conn(0x7f6a900bc4d0 0x7f6a900be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.591+0000 7f6aa1ffb700 1 -- 172.21.15.69:0/417654814 <== osd.7 v2:172.21.15.92:6816/144400151 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a8c0043d0 con 0x7f6a900bc4d0 2024-05-14T16:48:13.595 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:13.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.608+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 --> [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6aa4064050 con 0x7f6a900bc4d0 2024-05-14T16:48:13.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.609+0000 7f6aa1ffb700 1 -- 172.21.15.69:0/417654814 <== osd.7 v2:172.21.15.92:6816/144400151 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6aa4064050 con 0x7f6a900bc4d0 2024-05-14T16:48:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.609+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 >> [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] conn(0x7f6a900bc4d0 msgr2=0x7f6a900be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.609+0000 7f6aa91a2700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] conn(0x7f6a900bc4d0 0x7f6a900be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.609+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6a900b2910 msgr2=0x7f6a900b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.609+0000 7f6aa91a2700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6a900b2910 0x7f6a900b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6a94006fd0 tx=0x7f6a94008040 comp rx=0 tx=0).stop 2024-05-14T16:48:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.609+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa4076180 msgr2=0x7f6aa419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.609+0000 7f6aa91a2700 1 --2- 172.21.15.69:0/417654814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa4076180 0x7f6aa419c420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c000c00 tx=0x7f6a8c0047b0 comp rx=0 tx=0).stop 2024-05-14T16:48:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.610+0000 7f6aa89a1700 1 -- 172.21.15.69:0/417654814 reap_dead start 2024-05-14T16:48:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.610+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 shutdown_connections 2024-05-14T16:48:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.610+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 >> 172.21.15.69:0/417654814 conn(0x7f6aa40fef60 msgr2=0x7f6aa40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.610+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 shutdown_connections 2024-05-14T16:48:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:13.610+0000 7f6aa91a2700 1 -- 172.21.15.69:0/417654814 wait complete. 2024-05-14T16:48:14.296 INFO:teuthology.orchestra.run.smithi069.stdout:201863462966 2024-05-14T16:48:14.297 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-14T16:48:14.967 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:14.968 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:14.968 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:15.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:14 smithi092 ceph-mon[35763]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:48:15.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:14 smithi069 ceph-mon[31507]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:48:15.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:14 smithi203 ceph-mon[35787]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:48:17.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:16 smithi069 ceph-mon[31507]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:17.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:16 smithi092 ceph-mon[35763]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:16 smithi203 ceph-mon[35787]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:19.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:18 smithi069 ceph-mon[31507]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:19.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:18 smithi092 ceph-mon[35763]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:19.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:18 smithi203 ceph-mon[35787]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:20.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2088098638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c06bc70 msgr2=0x7f1f9c06c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:20.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 --2- 172.21.15.69:0/2088098638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c06bc70 0x7f1f9c06c0f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009990 tx=0x7f1f8c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:20.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2088098638 shutdown_connections 2024-05-14T16:48:20.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 --2- 172.21.15.69:0/2088098638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c06c630 0x7f1f9c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:20.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 --2- 172.21.15.69:0/2088098638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f9c1095b0 0x7f1f9c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:20.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 --2- 172.21.15.69:0/2088098638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c06bc70 0x7f1f9c06c0f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:20.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2088098638 >> 172.21.15.69:0/2088098638 conn(0x7f1f9c0793f0 msgr2=0x7f1f9c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2088098638 shutdown_connections 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2088098638 wait complete. 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 Processor -- start 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 -- start start 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f9c06bc70 0x7f1f9c102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c06c630 0x7f1f9c102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.669+0000 7f1fa36e4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1095b0 0x7f1f9c102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa36e4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1137c0 con 0x7f1f9c1095b0 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa36e4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c113640 con 0x7f1f9c06c630 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa36e4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c113940 con 0x7f1f9c06bc70 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa1c81700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1095b0 0x7f1f9c102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa1c81700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1095b0 0x7f1f9c102fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56936/0 (socket says 172.21.15.69:56936) 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa1c81700 1 -- 172.21.15.69:0/2323582755 learned_addr learned my addr 172.21.15.69:0/2323582755 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:20.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa1480700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f9c06bc70 0x7f1f9c102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa1c81700 1 -- 172.21.15.69:0/2323582755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f9c06bc70 msgr2=0x7f1f9c102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa1c81700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f9c06bc70 0x7f1f9c102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa1c81700 1 -- 172.21.15.69:0/2323582755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c06c630 msgr2=0x7f1f9c102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa1c81700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c06c630 0x7f1f9c102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa1c81700 1 -- 172.21.15.69:0/2323582755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c1a82a0 con 0x7f1f9c1095b0 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa1c81700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1095b0 0x7f1f9c102fc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400b970 tx=0x7f1f9400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1f927fc700 1 -- 172.21.15.69:0/2323582755 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94019070 con 0x7f1f9c1095b0 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1f927fc700 1 -- 172.21.15.69:0/2323582755 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f94014440 con 0x7f1f9c1095b0 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1f927fc700 1 -- 172.21.15.69:0/2323582755 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400ce60 con 0x7f1f9c1095b0 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2323582755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c1a8590 con 0x7f1f9c1095b0 2024-05-14T16:48:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.670+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2323582755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c1a8b20 con 0x7f1f9c1095b0 2024-05-14T16:48:20.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.671+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2323582755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9c10ba90 con 0x7f1f9c1095b0 2024-05-14T16:48:20.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.673+0000 7f1f927fc700 1 -- 172.21.15.69:0/2323582755 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1f9401c070 con 0x7f1f9c1095b0 2024-05-14T16:48:20.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.673+0000 7f1f927fc700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f880b2990 0x7f1f880b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:20.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.673+0000 7f1fa1480700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f880b2990 0x7f1f880b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:20.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.673+0000 7f1f927fc700 1 -- 172.21.15.69:0/2323582755 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1f940e1b10 con 0x7f1f9c1095b0 2024-05-14T16:48:20.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.674+0000 7f1fa1480700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f880b2990 0x7f1f880b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1f98006fd0 tx=0x7f1f98008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:20.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.677+0000 7f1f927fc700 1 -- 172.21.15.69:0/2323582755 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f940a97e0 con 0x7f1f9c1095b0 2024-05-14T16:48:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.896+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2323582755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1f9c10b470 con 0x7f1f9c1095b0 2024-05-14T16:48:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.897+0000 7f1f927fc700 1 -- 172.21.15.69:0/2323582755 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1f940a90e0 con 0x7f1f9c1095b0 2024-05-14T16:48:20.896 INFO:teuthology.orchestra.run.smithi069.stdout:107374182476 2024-05-14T16:48:20.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2323582755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f880b2990 msgr2=0x7f1f880b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:20.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f880b2990 0x7f1f880b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1f98006fd0 tx=0x7f1f98008040 comp rx=0 tx=0).stop 2024-05-14T16:48:20.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2323582755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1095b0 msgr2=0x7f1f9c102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:20.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1095b0 0x7f1f9c102fc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400b970 tx=0x7f1f9400bd30 comp rx=0 tx=0).stop 2024-05-14T16:48:20.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2323582755 shutdown_connections 2024-05-14T16:48:20.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f880b2990 0x7f1f880b4e50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:20.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f9c1095b0 0x7f1f9c102fc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:20.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f9c06bc70 0x7f1f9c102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:20.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 --2- 172.21.15.69:0/2323582755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f9c06c630 0x7f1f9c102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:20.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2323582755 >> 172.21.15.69:0/2323582755 conn(0x7f1f9c0793f0 msgr2=0x7f1f9c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:20.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2323582755 shutdown_connections 2024-05-14T16:48:20.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:20.899+0000 7f1fa36e4700 1 -- 172.21.15.69:0/2323582755 wait complete. 2024-05-14T16:48:21.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:20 smithi069 ceph-mon[31507]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:21.071 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:20 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2323582755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T16:48:21.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:20 smithi092 ceph-mon[35763]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:21.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:20 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2323582755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T16:48:21.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:20 smithi203 ceph-mon[35787]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:21.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:20 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2323582755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T16:48:23.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:22 smithi092 ceph-mon[35763]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:23.251 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:22 smithi069 ceph-mon[31507]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:23.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:22 smithi203 ceph-mon[35787]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.208+0000 7fce7c3f3700 1 -- 172.21.15.69:0/3867929832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108520 msgr2=0x7fce74108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.208+0000 7fce7c3f3700 1 --2- 172.21.15.69:0/3867929832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108520 0x7fce74108900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00b0a0 tx=0x7fce6c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.208+0000 7fce7c3f3700 1 -- 172.21.15.69:0/3867929832 shutdown_connections 2024-05-14T16:48:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.208+0000 7fce7c3f3700 1 --2- 172.21.15.69:0/3867929832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108520 0x7fce74108900 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.208+0000 7fce7c3f3700 1 --2- 172.21.15.69:0/3867929832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce74102e00 0x7fce7410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.208+0000 7fce7c3f3700 1 --2- 172.21.15.69:0/3867929832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce74102440 0x7fce741028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.208+0000 7fce7c3f3700 1 -- 172.21.15.69:0/3867929832 >> 172.21.15.69:0/3867929832 conn(0x7fce740793f0 msgr2=0x7fce74079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.208+0000 7fce7c3f3700 1 -- 172.21.15.69:0/3867929832 shutdown_connections 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.208+0000 7fce7c3f3700 1 -- 172.21.15.69:0/3867929832 wait complete. 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7c3f3700 1 Processor -- start 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7c3f3700 1 -- start start 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7c3f3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74102440 0x7fce74104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7c3f3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce74102e00 0x7fce741045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7c3f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce74108520 0x7fce74104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7c3f3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fce741125f0 con 0x7fce74102440 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7c3f3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce74112470 con 0x7fce74108520 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7c3f3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fce74112770 con 0x7fce74102e00 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7a18f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74102440 0x7fce74104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7a18f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74102440 0x7fce74104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56946/0 (socket says 172.21.15.69:56946) 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7a18f700 1 -- 172.21.15.69:0/2963715396 learned_addr learned my addr 172.21.15.69:0/2963715396 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:24.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.209+0000 7fce7998e700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce74102e00 0x7fce741045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce7a990700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce74108520 0x7fce74104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce7a18f700 1 -- 172.21.15.69:0/2963715396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce74102e00 msgr2=0x7fce741045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce7a18f700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce74102e00 0x7fce741045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce7a18f700 1 -- 172.21.15.69:0/2963715396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce74108520 msgr2=0x7fce74104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce7a18f700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce74108520 0x7fce74104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce7a18f700 1 -- 172.21.15.69:0/2963715396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce741a8070 con 0x7fce74102440 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce7a18f700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74102440 0x7fce74104080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00b070 tx=0x7fce6c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce737fe700 1 -- 172.21.15.69:0/2963715396 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c021070 con 0x7fce74102440 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce737fe700 1 -- 172.21.15.69:0/2963715396 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce6c007e30 con 0x7fce74102440 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce737fe700 1 -- 172.21.15.69:0/2963715396 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c004760 con 0x7fce74102440 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce7c3f3700 1 -- 172.21.15.69:0/2963715396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce741a8300 con 0x7fce74102440 2024-05-14T16:48:24.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.210+0000 7fce7c3f3700 1 -- 172.21.15.69:0/2963715396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce741a8760 con 0x7fce74102440 2024-05-14T16:48:24.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.211+0000 7fce7c3f3700 1 -- 172.21.15.69:0/2963715396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce7406a230 con 0x7fce74102440 2024-05-14T16:48:24.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.212+0000 7fce737fe700 1 -- 172.21.15.69:0/2963715396 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fce6c003de0 con 0x7fce74102440 2024-05-14T16:48:24.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.213+0000 7fce737fe700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fce5c0b2910 0x7fce5c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:24.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.213+0000 7fce7998e700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fce5c0b2910 0x7fce5c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:24.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.213+0000 7fce737fe700 1 -- 172.21.15.69:0/2963715396 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fce6c00f020 con 0x7fce74102440 2024-05-14T16:48:24.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.214+0000 7fce7998e700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fce5c0b2910 0x7fce5c0b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fce74079960 tx=0x7fce6800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:24.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.217+0000 7fce737fe700 1 -- 172.21.15.69:0/2963715396 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce6c02a020 con 0x7fce74102440 2024-05-14T16:48:24.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.433+0000 7fce7c3f3700 1 -- 172.21.15.69:0/2963715396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fce741a8a30 con 0x7fce74102440 2024-05-14T16:48:24.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.433+0000 7fce737fe700 1 -- 172.21.15.69:0/2963715396 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fce6c02e3e0 con 0x7fce74102440 2024-05-14T16:48:24.434 INFO:teuthology.orchestra.run.smithi069.stdout:300647710756 2024-05-14T16:48:24.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.435+0000 7fce7c3f3700 1 -- 172.21.15.69:0/2963715396 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fce5c0b2910 msgr2=0x7fce5c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:24.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.435+0000 7fce7c3f3700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fce5c0b2910 0x7fce5c0b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fce74079960 tx=0x7fce6800a380 comp rx=0 tx=0).stop 2024-05-14T16:48:24.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.435+0000 7fce7c3f3700 1 -- 172.21.15.69:0/2963715396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74102440 msgr2=0x7fce74104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:24.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.435+0000 7fce7c3f3700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74102440 0x7fce74104080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00b070 tx=0x7fce6c004d00 comp rx=0 tx=0).stop 2024-05-14T16:48:24.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.435+0000 7fce7c3f3700 1 -- 172.21.15.69:0/2963715396 shutdown_connections 2024-05-14T16:48:24.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.435+0000 7fce7c3f3700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fce5c0b2910 0x7fce5c0b4dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:24.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.436+0000 7fce7c3f3700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74102440 0x7fce74104080 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:24.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.436+0000 7fce7c3f3700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce74108520 0x7fce74104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:24.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.436+0000 7fce7c3f3700 1 --2- 172.21.15.69:0/2963715396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce74102e00 0x7fce741045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:24.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.436+0000 7fce7c3f3700 1 -- 172.21.15.69:0/2963715396 >> 172.21.15.69:0/2963715396 conn(0x7fce740793f0 msgr2=0x7fce74101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:24.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.436+0000 7fce7c3f3700 1 -- 172.21.15.69:0/2963715396 shutdown_connections 2024-05-14T16:48:24.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:24.436+0000 7fce7c3f3700 1 -- 172.21.15.69:0/2963715396 wait complete. 2024-05-14T16:48:25.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:24 smithi069 ceph-mon[31507]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:25.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:24 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2963715396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T16:48:25.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:24 smithi092 ceph-mon[35763]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:25.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:24 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2963715396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T16:48:25.258 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182472 got 107374182476 for osd.3 2024-05-14T16:48:25.258 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:25.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:24 smithi203 ceph-mon[35787]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:25.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:24 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2963715396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T16:48:25.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.852+0000 7fc1bebd7700 1 -- 172.21.15.69:0/1356912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8107a00 msgr2=0x7fc1b810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:25.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.852+0000 7fc1bebd7700 1 --2- 172.21.15.69:0/1356912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8107a00 0x7fc1b810f310 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000b0a0 tx=0x7fc1b000b3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.853+0000 7fc1bebd7700 1 -- 172.21.15.69:0/1356912 shutdown_connections 2024-05-14T16:48:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.853+0000 7fc1bebd7700 1 --2- 172.21.15.69:0/1356912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8107a00 0x7fc1b810f310 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.853+0000 7fc1bebd7700 1 --2- 172.21.15.69:0/1356912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1b81029e0 0x7fc1b8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.853+0000 7fc1bebd7700 1 --2- 172.21.15.69:0/1356912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1b8107060 0x7fc1b81074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.853+0000 7fc1bebd7700 1 -- 172.21.15.69:0/1356912 >> 172.21.15.69:0/1356912 conn(0x7fc1b8079330 msgr2=0x7fc1b8079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.853+0000 7fc1bebd7700 1 -- 172.21.15.69:0/1356912 shutdown_connections 2024-05-14T16:48:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.853+0000 7fc1bebd7700 1 -- 172.21.15.69:0/1356912 wait complete. 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.853+0000 7fc1bebd7700 1 Processor -- start 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.853+0000 7fc1bebd7700 1 -- start start 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1bebd7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1b81029e0 0x7fc1b8104100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1bebd7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8107060 0x7fc1b8104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1bebd7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1b8107a00 0x7fc1b8104b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1bebd7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b8112650 con 0x7fc1b8107060 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1bebd7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b81124d0 con 0x7fc1b81029e0 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1bebd7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b81127d0 con 0x7fc1b8107a00 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8107060 0x7fc1b8104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8107060 0x7fc1b8104640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40736/0 (socket says 172.21.15.69:40736) 2024-05-14T16:48:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b7fff700 1 -- 172.21.15.69:0/2375291125 learned_addr learned my addr 172.21.15.69:0/2375291125 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1bc973700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1b81029e0 0x7fc1b8104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1bd174700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1b8107a00 0x7fc1b8104b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b7fff700 1 -- 172.21.15.69:0/2375291125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1b8107a00 msgr2=0x7fc1b8104b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b7fff700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1b8107a00 0x7fc1b8104b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b7fff700 1 -- 172.21.15.69:0/2375291125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1b81029e0 msgr2=0x7fc1b8104100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b7fff700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1b81029e0 0x7fc1b8104100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b7fff700 1 -- 172.21.15.69:0/2375291125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b81a80d0 con 0x7fc1b8107060 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b7fff700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8107060 0x7fc1b8104640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b530 tx=0x7fc1a400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b5ffb700 1 -- 172.21.15.69:0/2375291125 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4015020 con 0x7fc1b8107060 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b5ffb700 1 -- 172.21.15.69:0/2375291125 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a4011ce0 con 0x7fc1b8107060 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1b5ffb700 1 -- 172.21.15.69:0/2375291125 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4010e40 con 0x7fc1b8107060 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.854+0000 7fc1bebd7700 1 -- 172.21.15.69:0/2375291125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b81a83c0 con 0x7fc1b8107060 2024-05-14T16:48:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.855+0000 7fc1bebd7700 1 -- 172.21.15.69:0/2375291125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b81a88b0 con 0x7fc1b8107060 2024-05-14T16:48:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.856+0000 7fc1bebd7700 1 -- 172.21.15.69:0/2375291125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b8103bb0 con 0x7fc1b8107060 2024-05-14T16:48:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.857+0000 7fc1b5ffb700 1 -- 172.21.15.69:0/2375291125 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc1a4018020 con 0x7fc1b8107060 2024-05-14T16:48:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.858+0000 7fc1b5ffb700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc1a80b2910 0x7fc1a80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.858+0000 7fc1bc973700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc1a80b2910 0x7fc1a80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.858+0000 7fc1b5ffb700 1 -- 172.21.15.69:0/2375291125 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc1a40e20d0 con 0x7fc1b8107060 2024-05-14T16:48:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.858+0000 7fc1bc973700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc1a80b2910 0x7fc1a80b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac005950 tx=0x7fc1ac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:25.861+0000 7fc1b5ffb700 1 -- 172.21.15.69:0/2375291125 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1a401f020 con 0x7fc1b8107060 2024-05-14T16:48:26.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.074+0000 7fc1bebd7700 1 -- 172.21.15.69:0/2375291125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc1b802e560 con 0x7fc1b8107060 2024-05-14T16:48:26.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.074+0000 7fc1b5ffb700 1 -- 172.21.15.69:0/2375291125 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc1a4010510 con 0x7fc1b8107060 2024-05-14T16:48:26.074 INFO:teuthology.orchestra.run.smithi069.stdout:133143986248 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.076+0000 7fc1bebd7700 1 -- 172.21.15.69:0/2375291125 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc1a80b2910 msgr2=0x7fc1a80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc1a80b2910 0x7fc1a80b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac005950 tx=0x7fc1ac00b410 comp rx=0 tx=0).stop 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 -- 172.21.15.69:0/2375291125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8107060 msgr2=0x7fc1b8104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8107060 0x7fc1b8104640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b530 tx=0x7fc1a400b840 comp rx=0 tx=0).stop 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 -- 172.21.15.69:0/2375291125 shutdown_connections 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1b8107a00 0x7fc1b8104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1b81029e0 0x7fc1b8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc1a80b2910 0x7fc1a80b4dd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 --2- 172.21.15.69:0/2375291125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8107060 0x7fc1b8104640 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 -- 172.21.15.69:0/2375291125 >> 172.21.15.69:0/2375291125 conn(0x7fc1b8079330 msgr2=0x7fc1b806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 -- 172.21.15.69:0/2375291125 shutdown_connections 2024-05-14T16:48:26.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.077+0000 7fc1bebd7700 1 -- 172.21.15.69:0/2375291125 wait complete. 2024-05-14T16:48:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:26 smithi092 ceph-mon[35763]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:26 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2375291125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T16:48:27.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:26 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2942659209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T16:48:27.255 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:26 smithi069 ceph-mon[31507]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:27.255 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:26 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2375291125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T16:48:27.256 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:26 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2942659209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T16:48:27.438 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:27.438 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:27.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:26 smithi203 ceph-mon[35787]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:27.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:26 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2375291125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T16:48:27.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:26 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2942659209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T16:48:27.945 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986245 got 133143986248 for osd.4 2024-05-14T16:48:27.945 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:28.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.086+0000 7fc8008f0700 1 -- 172.21.15.69:0/4283235660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc06bc10 msgr2=0x7fc7fc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:28.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.086+0000 7fc8008f0700 1 --2- 172.21.15.69:0/4283235660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc06bc10 0x7fc7fc06c090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009990 tx=0x7fc7ec009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.086+0000 7fc8008f0700 1 -- 172.21.15.69:0/4283235660 shutdown_connections 2024-05-14T16:48:28.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.086+0000 7fc8008f0700 1 --2- 172.21.15.69:0/4283235660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7fc06c5d0 0x7fc7fc1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.086+0000 7fc8008f0700 1 --2- 172.21.15.69:0/4283235660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7fc1095d0 0x7fc7fc1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.086+0000 7fc8008f0700 1 --2- 172.21.15.69:0/4283235660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc06bc10 0x7fc7fc06c090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.086+0000 7fc8008f0700 1 -- 172.21.15.69:0/4283235660 >> 172.21.15.69:0/4283235660 conn(0x7fc7fc0793f0 msgr2=0x7fc7fc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:28.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.086+0000 7fc8008f0700 1 -- 172.21.15.69:0/4283235660 shutdown_connections 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.086+0000 7fc8008f0700 1 -- 172.21.15.69:0/4283235660 wait complete. 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.087+0000 7fc8008f0700 1 Processor -- start 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.087+0000 7fc8008f0700 1 -- start start 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc8008f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7fc06bc10 0x7fc7fc19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc8008f0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc06c5d0 0x7fc7fc19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc8008f0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7fc1095d0 0x7fc7fc199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc8008f0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc113740 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc8008f0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1135c0 con 0x7fc7fc06bc10 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc8008f0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1138c0 con 0x7fc7fc1095d0 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc7f9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc06c5d0 0x7fc7fc19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc7f9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc06c5d0 0x7fc7fc19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40740/0 (socket says 172.21.15.69:40740) 2024-05-14T16:48:28.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc7f9d9b700 1 -- 172.21.15.69:0/2942659209 learned_addr learned my addr 172.21.15.69:0/2942659209 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:28.607 INFO:teuthology.orchestra.run.smithi069.stdout:77309411409 2024-05-14T16:48:28.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc7fad9d700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7fc1095d0 0x7fc7fc199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:28.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc7fa59c700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7fc06bc10 0x7fc7fc19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:28.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc7f9d9b700 1 -- 172.21.15.69:0/2942659209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7fc1095d0 msgr2=0x7fc7fc199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:28.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc7f9d9b700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7fc1095d0 0x7fc7fc199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc7f9d9b700 1 -- 172.21.15.69:0/2942659209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7fc06bc10 msgr2=0x7fc7fc19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:28.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc7f9d9b700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7fc06bc10 0x7fc7fc19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.088+0000 7fc7f9d9b700 1 -- 172.21.15.69:0/2942659209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7fc19a180 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.089+0000 7fc7f9d9b700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc06c5d0 0x7fc7fc19fda0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005910 tx=0x7fc7ec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:28.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.089+0000 7fc7eb7fe700 1 -- 172.21.15.69:0/2942659209 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec027070 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.089+0000 7fc8008f0700 1 -- 172.21.15.69:0/2942659209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7fc19a410 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.089+0000 7fc7eb7fe700 1 -- 172.21.15.69:0/2942659209 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7ec003f20 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.089+0000 7fc7eb7fe700 1 -- 172.21.15.69:0/2942659209 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec022c60 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.089+0000 7fc8008f0700 1 -- 172.21.15.69:0/2942659209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7fc1a6730 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.090+0000 7fc8008f0700 1 -- 172.21.15.69:0/2942659209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7fc06b4c0 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.091+0000 7fc7eb7fe700 1 -- 172.21.15.69:0/2942659209 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc7ec02a070 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.092+0000 7fc7eb7fe700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc7e40b2910 0x7fc7e40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.092+0000 7fc7fa59c700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc7e40b2910 0x7fc7e40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.092+0000 7fc7eb7fe700 1 -- 172.21.15.69:0/2942659209 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc7ec022e00 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.092+0000 7fc7fa59c700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc7e40b2910 0x7fc7e40b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc19ab60 tx=0x7fc7f000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.095+0000 7fc7eb7fe700 1 -- 172.21.15.69:0/2942659209 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc7ec031070 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.308+0000 7fc8008f0700 1 -- 172.21.15.69:0/2942659209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc7fc02e560 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.308+0000 7fc7eb7fe700 1 -- 172.21.15.69:0/2942659209 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc7ec013070 con 0x7fc7fc06c5d0 2024-05-14T16:48:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.310+0000 7fc8008f0700 1 -- 172.21.15.69:0/2942659209 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc7e40b2910 msgr2=0x7fc7e40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.310+0000 7fc8008f0700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc7e40b2910 0x7fc7e40b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc19ab60 tx=0x7fc7f000a380 comp rx=0 tx=0).stop 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.310+0000 7fc8008f0700 1 -- 172.21.15.69:0/2942659209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc06c5d0 msgr2=0x7fc7fc19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.310+0000 7fc8008f0700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc06c5d0 0x7fc7fc19fda0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005910 tx=0x7fc7ec0047b0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.311+0000 7fc8008f0700 1 -- 172.21.15.69:0/2942659209 shutdown_connections 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.311+0000 7fc8008f0700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7fc1095d0 0x7fc7fc199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.311+0000 7fc8008f0700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7fc06bc10 0x7fc7fc19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.311+0000 7fc8008f0700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc7e40b2910 0x7fc7e40b4dd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.311+0000 7fc8008f0700 1 --2- 172.21.15.69:0/2942659209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc06c5d0 0x7fc7fc19fda0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.311+0000 7fc8008f0700 1 -- 172.21.15.69:0/2942659209 >> 172.21.15.69:0/2942659209 conn(0x7fc7fc0793f0 msgr2=0x7fc7fc101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.311+0000 7fc8008f0700 1 -- 172.21.15.69:0/2942659209 shutdown_connections 2024-05-14T16:48:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:26.311+0000 7fc8008f0700 1 -- 172.21.15.69:0/2942659209 wait complete. 2024-05-14T16:48:29.242 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411403 got 77309411409 for osd.2 2024-05-14T16:48:29.242 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:29.637 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710751 got 300647710756 for osd.11 2024-05-14T16:48:29.637 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:29.864 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:29 smithi069 ceph-mon[31507]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:29.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.881+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2223039059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfac1084c0 msgr2=0x7fbfac1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:29.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.881+0000 7fbfb0fe0700 1 --2- 172.21.15.69:0/2223039059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfac1084c0 0x7fbfac1088a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009990 tx=0x7fbf9c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:29.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.881+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2223039059 shutdown_connections 2024-05-14T16:48:29.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.881+0000 7fbfb0fe0700 1 --2- 172.21.15.69:0/2223039059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfac1084c0 0x7fbfac1088a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:29.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.881+0000 7fbfb0fe0700 1 --2- 172.21.15.69:0/2223039059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfac103600 0x7fbfac10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:29.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.881+0000 7fbfb0fe0700 1 --2- 172.21.15.69:0/2223039059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac102c40 0x7fbfac1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:29.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.881+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2223039059 >> 172.21.15.69:0/2223039059 conn(0x7fbfac078ab0 msgr2=0x7fbfac0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:29.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.881+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2223039059 shutdown_connections 2024-05-14T16:48:29.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.881+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2223039059 wait complete. 2024-05-14T16:48:29.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfb0fe0700 1 Processor -- start 2024-05-14T16:48:29.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfb0fe0700 1 -- start start 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfb0fe0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac102c40 0x7fbfac19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfb0fe0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfac103600 0x7fbfac19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfb0fe0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfac1084c0 0x7fbfac199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfb0fe0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac112670 con 0x7fbfac103600 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfb0fe0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1124f0 con 0x7fbfac102c40 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfb0fe0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1127f0 con 0x7fbfac1084c0 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfa9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfac103600 0x7fbfac19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfaa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac102c40 0x7fbfac19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfa9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfac103600 0x7fbfac19fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40756/0 (socket says 172.21.15.69:40756) 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.882+0000 7fbfa9d9b700 1 -- 172.21.15.69:0/2429383188 learned_addr learned my addr 172.21.15.69:0/2429383188 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbfaad9d700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfac1084c0 0x7fbfac199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbfa9d9b700 1 -- 172.21.15.69:0/2429383188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfac1084c0 msgr2=0x7fbfac199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbfa9d9b700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfac1084c0 0x7fbfac199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:29.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbfa9d9b700 1 -- 172.21.15.69:0/2429383188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac102c40 msgr2=0x7fbfac19f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:29.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbfa9d9b700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac102c40 0x7fbfac19f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:29.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbfa9d9b700 1 -- 172.21.15.69:0/2429383188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac19a1b0 con 0x7fbfac103600 2024-05-14T16:48:29.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbfa9d9b700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfac103600 0x7fbfac19fdd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000b530 tx=0x7fbfa000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:29.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbf9b7fe700 1 -- 172.21.15.69:0/2429383188 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0015020 con 0x7fbfac103600 2024-05-14T16:48:29.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbf9b7fe700 1 -- 172.21.15.69:0/2429383188 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa0011ce0 con 0x7fbfac103600 2024-05-14T16:48:29.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2429383188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac19a4a0 con 0x7fbfac103600 2024-05-14T16:48:29.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbf9b7fe700 1 -- 172.21.15.69:0/2429383188 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0010e40 con 0x7fbfac103600 2024-05-14T16:48:29.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.883+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2429383188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac1a6760 con 0x7fbfac103600 2024-05-14T16:48:29.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.884+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2429383188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfac103c50 con 0x7fbfac103600 2024-05-14T16:48:29.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.886+0000 7fbf9b7fe700 1 -- 172.21.15.69:0/2429383188 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbfa0018020 con 0x7fbfac103600 2024-05-14T16:48:29.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.887+0000 7fbf9b7fe700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbf940b2910 0x7fbf940b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:29.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.887+0000 7fbfaa59c700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbf940b2910 0x7fbf940b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.887+0000 7fbf9b7fe700 1 -- 172.21.15.69:0/2429383188 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbfa00e1fc0 con 0x7fbfac103600 2024-05-14T16:48:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.887+0000 7fbfaa59c700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbf940b2910 0x7fbf940b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c006010 tx=0x7fbf9c00b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:29.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:29.890+0000 7fbf9b7fe700 1 -- 172.21.15.69:0/2429383188 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbfa001f020 con 0x7fbfac103600 2024-05-14T16:48:29.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:29 smithi203 ceph-mon[35787]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:30.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:29 smithi092 ceph-mon[35763]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:30.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.096+0000 7f944951c700 1 -- 172.21.15.69:0/1397265593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444103f90 msgr2=0x7f9444110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:30.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.096+0000 7f944951c700 1 --2- 172.21.15.69:0/1397265593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444103f90 0x7f9444110440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f943c00b0d0 tx=0x7f943c00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.096+0000 7f944951c700 1 -- 172.21.15.69:0/1397265593 shutdown_connections 2024-05-14T16:48:30.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.096+0000 7f944951c700 1 --2- 172.21.15.69:0/1397265593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444103f90 0x7f9444110440 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.096+0000 7f944951c700 1 --2- 172.21.15.69:0/1397265593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9444109670 0x7f9444109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.096+0000 7f944951c700 1 --2- 172.21.15.69:0/1397265593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94441035d0 0x7f9444103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.096+0000 7f944951c700 1 -- 172.21.15.69:0/1397265593 >> 172.21.15.69:0/1397265593 conn(0x7f944407a730 msgr2=0x7f944407ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:30.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.096+0000 7f944951c700 1 -- 172.21.15.69:0/1397265593 shutdown_connections 2024-05-14T16:48:30.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.096+0000 7f944951c700 1 -- 172.21.15.69:0/1397265593 wait complete. 2024-05-14T16:48:30.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f944951c700 1 Processor -- start 2024-05-14T16:48:30.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f944951c700 1 -- start start 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f944951c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94441035d0 0x7f9444075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f944951c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9444103f90 0x7f9444075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f944951c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444109670 0x7f94440762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f944951c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9444113770 con 0x7f9444109670 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f944951c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f94441135f0 con 0x7f9444103f90 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f944951c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94441138f0 con 0x7f94441035d0 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f94437fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444109670 0x7f94440762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f94437fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444109670 0x7f94440762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40762/0 (socket says 172.21.15.69:40762) 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.097+0000 7f94437fe700 1 -- 172.21.15.69:0/3796600723 learned_addr learned my addr 172.21.15.69:0/3796600723 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f94427fc700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9444103f90 0x7f9444075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f9442ffd700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94441035d0 0x7f9444075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f94437fe700 1 -- 172.21.15.69:0/3796600723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94441035d0 msgr2=0x7f9444075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:30.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f94437fe700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94441035d0 0x7f9444075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f94437fe700 1 -- 172.21.15.69:0/3796600723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9444103f90 msgr2=0x7f9444075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f94437fe700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9444103f90 0x7f9444075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f94437fe700 1 -- 172.21.15.69:0/3796600723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94441a80e0 con 0x7f9444109670 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f9442ffd700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94441035d0 0x7f9444075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f94437fe700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444109670 0x7f94440762c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f943c009fd0 tx=0x7f943c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f942bfff700 1 -- 172.21.15.69:0/3796600723 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c01f070 con 0x7f9444109670 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f944951c700 1 -- 172.21.15.69:0/3796600723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94441a8370 con 0x7f9444109670 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f942bfff700 1 -- 172.21.15.69:0/3796600723 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c003c00 con 0x7f9444109670 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f942bfff700 1 -- 172.21.15.69:0/3796600723 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c01b7b0 con 0x7f9444109670 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.098+0000 7f944951c700 1 -- 172.21.15.69:0/3796600723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94441a87d0 con 0x7f9444109670 2024-05-14T16:48:30.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.099+0000 7f944951c700 1 -- 172.21.15.69:0/3796600723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944406a230 con 0x7f9444109670 2024-05-14T16:48:30.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.101+0000 7f942bfff700 1 -- 172.21.15.69:0/3796600723 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f943c022070 con 0x7f9444109670 2024-05-14T16:48:30.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.101+0000 7f942bfff700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f942c0b2910 0x7f942c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:30.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.101+0000 7f9442ffd700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f942c0b2910 0x7f942c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:30.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.101+0000 7f942bfff700 1 -- 172.21.15.69:0/3796600723 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f943c00f020 con 0x7f9444109670 2024-05-14T16:48:30.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.102+0000 7f9442ffd700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f942c0b2910 0x7f942c0b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f943400baa0 tx=0x7f9434005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:30.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.109+0000 7f942bfff700 1 -- 172.21.15.69:0/3796600723 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f943c029020 con 0x7f9444109670 2024-05-14T16:48:30.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.110+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2429383188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fbfac06a230 con 0x7fbfac103600 2024-05-14T16:48:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.110+0000 7fbf9b7fe700 1 -- 172.21.15.69:0/2429383188 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbfac06a230 con 0x7fbfac103600 2024-05-14T16:48:30.110 INFO:teuthology.orchestra.run.smithi069.stdout:227633266740 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2429383188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbf940b2910 msgr2=0x7fbf940b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbf940b2910 0x7fbf940b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c006010 tx=0x7fbf9c00b560 comp rx=0 tx=0).stop 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2429383188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfac103600 msgr2=0x7fbfac19fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfac103600 0x7fbfac19fdd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000b530 tx=0x7fbfa000b840 comp rx=0 tx=0).stop 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2429383188 shutdown_connections 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfac1084c0 0x7fbfac199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfac102c40 0x7fbfac19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbf940b2910 0x7fbf940b4dd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 --2- 172.21.15.69:0/2429383188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfac103600 0x7fbfac19fdd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2429383188 >> 172.21.15.69:0/2429383188 conn(0x7fbfac078ab0 msgr2=0x7fbfac1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:30.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2429383188 shutdown_connections 2024-05-14T16:48:30.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.113+0000 7fbfb0fe0700 1 -- 172.21.15.69:0/2429383188 wait complete. 2024-05-14T16:48:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.322+0000 7f944951c700 1 -- 172.21.15.69:0/3796600723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f944402e560 con 0x7f9444109670 2024-05-14T16:48:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.322+0000 7f942bfff700 1 -- 172.21.15.69:0/3796600723 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f943c0ab670 con 0x7f9444109670 2024-05-14T16:48:30.322 INFO:teuthology.orchestra.run.smithi069.stdout:55834574935 2024-05-14T16:48:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.324+0000 7f944951c700 1 -- 172.21.15.69:0/3796600723 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f942c0b2910 msgr2=0x7f942c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.324+0000 7f944951c700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f942c0b2910 0x7f942c0b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f943400baa0 tx=0x7f9434005db0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.324+0000 7f944951c700 1 -- 172.21.15.69:0/3796600723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444109670 msgr2=0x7f94440762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.324+0000 7f944951c700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444109670 0x7f94440762c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f943c009fd0 tx=0x7f943c007a20 comp rx=0 tx=0).stop 2024-05-14T16:48:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.325+0000 7f944951c700 1 -- 172.21.15.69:0/3796600723 shutdown_connections 2024-05-14T16:48:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.325+0000 7f944951c700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f942c0b2910 0x7f942c0b4dd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.325+0000 7f944951c700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444109670 0x7f94440762c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.325+0000 7f944951c700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94441035d0 0x7f9444075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.325+0000 7f944951c700 1 --2- 172.21.15.69:0/3796600723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9444103f90 0x7f9444075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.325+0000 7f944951c700 1 -- 172.21.15.69:0/3796600723 >> 172.21.15.69:0/3796600723 conn(0x7f944407a730 msgr2=0x7f9444102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.325+0000 7f944951c700 1 -- 172.21.15.69:0/3796600723 shutdown_connections 2024-05-14T16:48:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:30.325+0000 7f944951c700 1 -- 172.21.15.69:0/3796600723 wait complete. 2024-05-14T16:48:30.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:30 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2429383188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T16:48:30.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:30 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3796600723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T16:48:30.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:30 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2429383188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T16:48:30.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:30 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3796600723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T16:48:31.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:30 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2429383188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T16:48:31.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:30 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3796600723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T16:48:31.295 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266735 got 227633266740 for osd.8 2024-05-14T16:48:31.296 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:31.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:31 smithi069 ceph-mon[31507]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:31 smithi203 ceph-mon[35787]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:32.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:31 smithi092 ceph-mon[35763]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:33.321 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574930 got 55834574935 for osd.1 2024-05-14T16:48:33.321 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:33.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.489+0000 7f77b391f700 1 -- 172.21.15.69:0/1242275795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac109520 msgr2=0x7f77ac109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:33.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.489+0000 7f77b391f700 1 --2- 172.21.15.69:0/1242275795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac109520 0x7f77ac109900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f779c0099c0 tx=0x7f779c009cd0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.489+0000 7f77b391f700 1 -- 172.21.15.69:0/1242275795 shutdown_connections 2024-05-14T16:48:33.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.489+0000 7f77b391f700 1 --2- 172.21.15.69:0/1242275795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac109520 0x7f77ac109900 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.489+0000 7f77b391f700 1 --2- 172.21.15.69:0/1242275795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ac1024f0 0x7f77ac1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.489+0000 7f77b391f700 1 --2- 172.21.15.69:0/1242275795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77ac101b30 0x7f77ac101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.489+0000 7f77b391f700 1 -- 172.21.15.69:0/1242275795 >> 172.21.15.69:0/1242275795 conn(0x7f77ac078ab0 msgr2=0x7f77ac0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.489+0000 7f77b391f700 1 -- 172.21.15.69:0/1242275795 shutdown_connections 2024-05-14T16:48:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.489+0000 7f77b391f700 1 -- 172.21.15.69:0/1242275795 wait complete. 2024-05-14T16:48:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b391f700 1 Processor -- start 2024-05-14T16:48:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b391f700 1 -- start start 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b391f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac101b30 0x7f77ac19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b391f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ac1024f0 0x7f77ac19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b391f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77ac109520 0x7f77ac199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b391f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac113760 con 0x7f77ac101b30 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b391f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1135e0 con 0x7f77ac109520 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b391f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1138e0 con 0x7f77ac1024f0 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b16bb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac101b30 0x7f77ac19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b16bb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac101b30 0x7f77ac19f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40778/0 (socket says 172.21.15.69:40778) 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b16bb700 1 -- 172.21.15.69:0/148713422 learned_addr learned my addr 172.21.15.69:0/148713422 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b0eba700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ac1024f0 0x7f77ac19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b16bb700 1 -- 172.21.15.69:0/148713422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ac1024f0 msgr2=0x7f77ac19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b16bb700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ac1024f0 0x7f77ac19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b1ebc700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77ac109520 0x7f77ac199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b16bb700 1 -- 172.21.15.69:0/148713422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77ac109520 msgr2=0x7f77ac199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b16bb700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77ac109520 0x7f77ac199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b16bb700 1 -- 172.21.15.69:0/148713422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac19a1e0 con 0x7f77ac101b30 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.490+0000 7f77b1ebc700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77ac109520 0x7f77ac199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.491+0000 7f77b16bb700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac101b30 0x7f77ac19f8c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f779c00b580 tx=0x7f779c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.491+0000 7f77a27fc700 1 -- 172.21.15.69:0/148713422 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c027070 con 0x7f77ac101b30 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.491+0000 7f77b391f700 1 -- 172.21.15.69:0/148713422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ac19a470 con 0x7f77ac101b30 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.491+0000 7f77a27fc700 1 -- 172.21.15.69:0/148713422 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f779c003f20 con 0x7f77ac101b30 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.491+0000 7f77a27fc700 1 -- 172.21.15.69:0/148713422 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c022d70 con 0x7f77ac101b30 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.491+0000 7f77b391f700 1 -- 172.21.15.69:0/148713422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ac1a6790 con 0x7f77ac101b30 2024-05-14T16:48:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.492+0000 7f77b391f700 1 -- 172.21.15.69:0/148713422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ac104e50 con 0x7f77ac101b30 2024-05-14T16:48:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.493+0000 7f77a27fc700 1 -- 172.21.15.69:0/148713422 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f779c022500 con 0x7f77ac101b30 2024-05-14T16:48:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.494+0000 7f77a27fc700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f77980b2880 0x7f77980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.494+0000 7f77b0eba700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f77980b2880 0x7f77980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.494+0000 7f77a27fc700 1 -- 172.21.15.69:0/148713422 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f779c0e3300 con 0x7f77ac101b30 2024-05-14T16:48:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.494+0000 7f77b0eba700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f77980b2880 0x7f77980b4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f77ac108c40 tx=0x7f77a800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.498+0000 7f77a27fc700 1 -- 172.21.15.69:0/148713422 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f779c013070 con 0x7f77ac101b30 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.668+0000 7f5c2af66700 1 -- 172.21.15.69:0/2039152474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c24102df0 msgr2=0x7f5c2410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.668+0000 7f5c2af66700 1 --2- 172.21.15.69:0/2039152474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c24102df0 0x7f5c2410f2e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00b0a0 tx=0x7f5c1c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.668+0000 7f5c2af66700 1 -- 172.21.15.69:0/2039152474 shutdown_connections 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.668+0000 7f5c2af66700 1 --2- 172.21.15.69:0/2039152474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c24102df0 0x7f5c2410f2e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.668+0000 7f5c2af66700 1 --2- 172.21.15.69:0/2039152474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c24108510 0x7f5c241088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.668+0000 7f5c2af66700 1 --2- 172.21.15.69:0/2039152474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102430 0x7f5c241028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.668+0000 7f5c2af66700 1 -- 172.21.15.69:0/2039152474 >> 172.21.15.69:0/2039152474 conn(0x7f5c240793b0 msgr2=0x7f5c240797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.669+0000 7f5c2af66700 1 -- 172.21.15.69:0/2039152474 shutdown_connections 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.669+0000 7f5c2af66700 1 -- 172.21.15.69:0/2039152474 wait complete. 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.669+0000 7f5c2af66700 1 Processor -- start 2024-05-14T16:48:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.669+0000 7f5c2af66700 1 -- start start 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.669+0000 7f5c2af66700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102430 0x7f5c2419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.669+0000 7f5c2af66700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c24102df0 0x7f5c2419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.669+0000 7f5c2af66700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c24108510 0x7f5c241999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.669+0000 7f5c2af66700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24112610 con 0x7f5c24102df0 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.669+0000 7f5c2af66700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24112490 con 0x7f5c24108510 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.669+0000 7f5c2af66700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24112790 con 0x7f5c24102430 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c23fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c24102df0 0x7f5c2419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c23fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c24102df0 0x7f5c2419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40788/0 (socket says 172.21.15.69:40788) 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c23fff700 1 -- 172.21.15.69:0/1739269714 learned_addr learned my addr 172.21.15.69:0/1739269714 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c29503700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c24108510 0x7f5c241999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c28d02700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102430 0x7f5c2419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c23fff700 1 -- 172.21.15.69:0/1739269714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102430 msgr2=0x7f5c2419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c23fff700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102430 0x7f5c2419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c23fff700 1 -- 172.21.15.69:0/1739269714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c24108510 msgr2=0x7f5c241999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c23fff700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c24108510 0x7f5c241999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c23fff700 1 -- 172.21.15.69:0/1739269714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2419a140 con 0x7f5c24102df0 2024-05-14T16:48:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c23fff700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c24102df0 0x7f5c2419fd60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000d950 tx=0x7f5c1000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c21ffb700 1 -- 172.21.15.69:0/1739269714 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c10016070 con 0x7f5c24102df0 2024-05-14T16:48:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c2af66700 1 -- 172.21.15.69:0/1739269714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2419a430 con 0x7f5c24102df0 2024-05-14T16:48:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c21ffb700 1 -- 172.21.15.69:0/1739269714 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c1000fd40 con 0x7f5c24102df0 2024-05-14T16:48:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.670+0000 7f5c21ffb700 1 -- 172.21.15.69:0/1739269714 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1000e3b0 con 0x7f5c24102df0 2024-05-14T16:48:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.671+0000 7f5c2af66700 1 -- 172.21.15.69:0/1739269714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c241a66f0 con 0x7f5c24102df0 2024-05-14T16:48:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.671+0000 7f5c2af66700 1 -- 172.21.15.69:0/1739269714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2406a230 con 0x7f5c24102df0 2024-05-14T16:48:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.674+0000 7f5c21ffb700 1 -- 172.21.15.69:0/1739269714 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5c10019070 con 0x7f5c24102df0 2024-05-14T16:48:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.674+0000 7f5c21ffb700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c140b2900 0x7f5c140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.674+0000 7f5c28d02700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c140b2900 0x7f5c140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.674+0000 7f5c21ffb700 1 -- 172.21.15.69:0/1739269714 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f5c100e1be0 con 0x7f5c24102df0 2024-05-14T16:48:33.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.675+0000 7f5c28d02700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c140b2900 0x7f5c140b4dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005950 tx=0x7f5c1800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:33.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.678+0000 7f5c21ffb700 1 -- 172.21.15.69:0/1739269714 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5c100a9900 con 0x7f5c24102df0 2024-05-14T16:48:33.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.720+0000 7f77b391f700 1 -- 172.21.15.69:0/148713422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f77ac02e560 con 0x7f77ac101b30 2024-05-14T16:48:33.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.720+0000 7f77a27fc700 1 -- 172.21.15.69:0/148713422 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f779c0ab0b0 con 0x7f77ac101b30 2024-05-14T16:48:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:34359738460 2024-05-14T16:48:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 -- 172.21.15.69:0/148713422 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f77980b2880 msgr2=0x7f77980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f77980b2880 0x7f77980b4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f77ac108c40 tx=0x7f77a800b410 comp rx=0 tx=0).stop 2024-05-14T16:48:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 -- 172.21.15.69:0/148713422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac101b30 msgr2=0x7f77ac19f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac101b30 0x7f77ac19f8c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f779c00b580 tx=0x7f779c0047b0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 -- 172.21.15.69:0/148713422 shutdown_connections 2024-05-14T16:48:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f77980b2880 0x7f77980b4d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac101b30 0x7f77ac19f8c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77ac109520 0x7f77ac199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 --2- 172.21.15.69:0/148713422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77ac1024f0 0x7f77ac19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 -- 172.21.15.69:0/148713422 >> 172.21.15.69:0/148713422 conn(0x7f77ac078ab0 msgr2=0x7f77ac100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:33.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 -- 172.21.15.69:0/148713422 shutdown_connections 2024-05-14T16:48:33.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.723+0000 7f77b391f700 1 -- 172.21.15.69:0/148713422 wait complete. 2024-05-14T16:48:33.739 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:33 smithi069 ceph-mon[31507]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:33.739 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:33 smithi069 ceph-mon[31507]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-14T16:48:33.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.898+0000 7f5c2af66700 1 -- 172.21.15.69:0/1739269714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f5c2402e560 con 0x7f5c24102df0 2024-05-14T16:48:33.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.898+0000 7f5c21ffb700 1 -- 172.21.15.69:0/1739269714 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5c100a9200 con 0x7f5c24102df0 2024-05-14T16:48:33.899 INFO:teuthology.orchestra.run.smithi069.stdout:274877906986 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 -- 172.21.15.69:0/1739269714 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c140b2900 msgr2=0x7f5c140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c140b2900 0x7f5c140b4dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005950 tx=0x7f5c1800a300 comp rx=0 tx=0).stop 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 -- 172.21.15.69:0/1739269714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c24102df0 msgr2=0x7f5c2419fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c24102df0 0x7f5c2419fd60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000d950 tx=0x7f5c1000dc60 comp rx=0 tx=0).stop 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 -- 172.21.15.69:0/1739269714 shutdown_connections 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c24108510 0x7f5c241999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102430 0x7f5c2419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f5c140b2900 0x7f5c140b4dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 --2- 172.21.15.69:0/1739269714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c24102df0 0x7f5c2419fd60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 -- 172.21.15.69:0/1739269714 >> 172.21.15.69:0/1739269714 conn(0x7f5c240793b0 msgr2=0x7f5c2406bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 -- 172.21.15.69:0/1739269714 shutdown_connections 2024-05-14T16:48:33.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:33.901+0000 7f5c2af66700 1 -- 172.21.15.69:0/1739269714 wait complete. 2024-05-14T16:48:33.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:33 smithi203 ceph-mon[35787]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:33.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:33 smithi203 ceph-mon[35787]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-14T16:48:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:33 smithi092 ceph-mon[35763]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:33 smithi092 ceph-mon[35763]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-14T16:48:34.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.329+0000 7f4881a1c700 1 -- 172.21.15.69:0/1930541219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f487c1095c0 msgr2=0x7f487c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:34.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.329+0000 7f4881a1c700 1 --2- 172.21.15.69:0/1930541219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f487c1095c0 0x7f487c1099a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f486c009990 tx=0x7f486c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.329+0000 7f4881a1c700 1 -- 172.21.15.69:0/1930541219 shutdown_connections 2024-05-14T16:48:34.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.329+0000 7f4881a1c700 1 --2- 172.21.15.69:0/1930541219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f487c1095c0 0x7f487c1099a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.329+0000 7f4881a1c700 1 --2- 172.21.15.69:0/1930541219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c06c5d0 0x7f487c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.329+0000 7f4881a1c700 1 --2- 172.21.15.69:0/1930541219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f487c06bc10 0x7f487c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.329+0000 7f4881a1c700 1 -- 172.21.15.69:0/1930541219 >> 172.21.15.69:0/1930541219 conn(0x7f487c0793f0 msgr2=0x7f487c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:34.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.329+0000 7f4881a1c700 1 -- 172.21.15.69:0/1930541219 shutdown_connections 2024-05-14T16:48:34.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.329+0000 7f4881a1c700 1 -- 172.21.15.69:0/1930541219 wait complete. 2024-05-14T16:48:34.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f4881a1c700 1 Processor -- start 2024-05-14T16:48:34.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f4881a1c700 1 -- start start 2024-05-14T16:48:34.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f4881a1c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f487c06bc10 0x7f487c19f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f4881a1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c06c5d0 0x7f487c19fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f4881a1c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f487c1095c0 0x7f487c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f4881a1c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f487c113750 con 0x7f487c1095c0 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f4881a1c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1135d0 con 0x7f487c06c5d0 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f4881a1c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1138d0 con 0x7f487c06bc10 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f487b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f487c1095c0 0x7f487c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f487b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f487c1095c0 0x7f487c199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40816/0 (socket says 172.21.15.69:40816) 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f487b7fe700 1 -- 172.21.15.69:0/4279589817 learned_addr learned my addr 172.21.15.69:0/4279589817 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f4873fff700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c06c5d0 0x7f487c19fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f487b7fe700 1 -- 172.21.15.69:0/4279589817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f487c06bc10 msgr2=0x7f487c19f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f487b7fe700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f487c06bc10 0x7f487c19f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f487b7fe700 1 -- 172.21.15.69:0/4279589817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c06c5d0 msgr2=0x7f487c19fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f487b7fe700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c06c5d0 0x7f487c19fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.330+0000 7f487b7fe700 1 -- 172.21.15.69:0/4279589817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c19a250 con 0x7f487c1095c0 2024-05-14T16:48:34.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.331+0000 7f487b7fe700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f487c1095c0 0x7f487c199af0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4874007ee0 tx=0x7f487400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:34.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.331+0000 7f4880a1a700 1 -- 172.21.15.69:0/4279589817 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487400dbd0 con 0x7f487c1095c0 2024-05-14T16:48:34.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.331+0000 7f4880a1a700 1 -- 172.21.15.69:0/4279589817 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f487400dd70 con 0x7f487c1095c0 2024-05-14T16:48:34.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.331+0000 7f4881a1c700 1 -- 172.21.15.69:0/4279589817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c19a540 con 0x7f487c1095c0 2024-05-14T16:48:34.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.331+0000 7f4880a1a700 1 -- 172.21.15.69:0/4279589817 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874005990 con 0x7f487c1095c0 2024-05-14T16:48:34.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.331+0000 7f4881a1c700 1 -- 172.21.15.69:0/4279589817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c1a6800 con 0x7f487c1095c0 2024-05-14T16:48:34.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.332+0000 7f4881a1c700 1 -- 172.21.15.69:0/4279589817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487c104c90 con 0x7f487c1095c0 2024-05-14T16:48:34.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.334+0000 7f4880a1a700 1 -- 172.21.15.69:0/4279589817 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f487401a020 con 0x7f487c1095c0 2024-05-14T16:48:34.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.335+0000 7f4880a1a700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48680b2900 0x7f48680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:34.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.335+0000 7f487affd700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48680b2900 0x7f48680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:34.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.335+0000 7f4880a1a700 1 -- 172.21.15.69:0/4279589817 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f48740e20e0 con 0x7f487c1095c0 2024-05-14T16:48:34.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.335+0000 7f487affd700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48680b2900 0x7f48680b4dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f486c0098e0 tx=0x7f486c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:34.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.338+0000 7f4880a1a700 1 -- 172.21.15.69:0/4279589817 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4874010070 con 0x7f487c1095c0 2024-05-14T16:48:34.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.545+0000 7f4881a1c700 1 -- 172.21.15.69:0/4279589817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f487c02e560 con 0x7f487c1095c0 2024-05-14T16:48:34.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.546+0000 7f4880a1a700 1 -- 172.21.15.69:0/4279589817 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f48740a3e20 con 0x7f487c1095c0 2024-05-14T16:48:34.547 INFO:teuthology.orchestra.run.smithi069.stdout:180388626496 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 -- 172.21.15.69:0/4279589817 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48680b2900 msgr2=0x7f48680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48680b2900 0x7f48680b4dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f486c0098e0 tx=0x7f486c005fb0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 -- 172.21.15.69:0/4279589817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f487c1095c0 msgr2=0x7f487c199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f487c1095c0 0x7f487c199af0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4874007ee0 tx=0x7f487400d3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 -- 172.21.15.69:0/4279589817 shutdown_connections 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48680b2900 0x7f48680b4dc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f487c1095c0 0x7f487c199af0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f487c06bc10 0x7f487c19f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 --2- 172.21.15.69:0/4279589817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c06c5d0 0x7f487c19fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 -- 172.21.15.69:0/4279589817 >> 172.21.15.69:0/4279589817 conn(0x7f487c0793f0 msgr2=0x7f487c101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 -- 172.21.15.69:0/4279589817 shutdown_connections 2024-05-14T16:48:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:34.548+0000 7f4881a1c700 1 -- 172.21.15.69:0/4279589817 wait complete. 2024-05-14T16:48:34.832 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:34 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/148713422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T16:48:34.832 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:34 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1739269714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T16:48:34.832 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:34 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/4279589817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T16:48:34.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:34 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/148713422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T16:48:34.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:34 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1739269714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T16:48:34.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:34 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/4279589817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T16:48:35.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:34 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/148713422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T16:48:35.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:34 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1739269714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T16:48:35.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:34 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/4279589817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T16:48:35.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.354+0000 7ff234493700 1 -- 172.21.15.69:0/4205304705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c101ee0 msgr2=0x7ff22c102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:35.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.354+0000 7ff234493700 1 --2- 172.21.15.69:0/4205304705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c101ee0 0x7ff22c102360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff228009990 tx=0x7ff228009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.354+0000 7ff234493700 1 -- 172.21.15.69:0/4205304705 shutdown_connections 2024-05-14T16:48:35.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.354+0000 7ff234493700 1 --2- 172.21.15.69:0/4205304705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c1028a0 0x7ff22c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.354+0000 7ff234493700 1 --2- 172.21.15.69:0/4205304705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c108600 0x7ff22c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.354+0000 7ff234493700 1 --2- 172.21.15.69:0/4205304705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c101ee0 0x7ff22c102360 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.354+0000 7ff234493700 1 -- 172.21.15.69:0/4205304705 >> 172.21.15.69:0/4205304705 conn(0x7ff22c077760 msgr2=0x7ff22c077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.354+0000 7ff234493700 1 -- 172.21.15.69:0/4205304705 shutdown_connections 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.354+0000 7ff234493700 1 -- 172.21.15.69:0/4205304705 wait complete. 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff234493700 1 Processor -- start 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff234493700 1 -- start start 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff234493700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c101ee0 0x7ff22c19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff234493700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c1028a0 0x7ff22c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff234493700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c108600 0x7ff22c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff234493700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c113790 con 0x7ff22c1028a0 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff234493700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c113610 con 0x7ff22c101ee0 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff234493700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c113910 con 0x7ff22c108600 2024-05-14T16:48:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff231a2e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c1028a0 0x7ff22c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff231a2e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c1028a0 0x7ff22c19c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47196/0 (socket says 172.21.15.69:47196) 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff231a2e700 1 -- 172.21.15.69:0/3183752514 learned_addr learned my addr 172.21.15.69:0/3183752514 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff23222f700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c101ee0 0x7ff22c19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.355+0000 7ff232a30700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c108600 0x7ff22c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff231a2e700 1 -- 172.21.15.69:0/3183752514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c108600 msgr2=0x7ff22c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff231a2e700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c108600 0x7ff22c1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff231a2e700 1 -- 172.21.15.69:0/3183752514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c101ee0 msgr2=0x7ff22c19be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff231a2e700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c101ee0 0x7ff22c19be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff231a2e700 1 -- 172.21.15.69:0/3183752514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c1a0e90 con 0x7ff22c1028a0 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff23222f700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c101ee0 0x7ff22c19be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff232a30700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c108600 0x7ff22c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff231a2e700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c1028a0 0x7ff22c19c3c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff228000c00 tx=0x7ff2280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff2237fe700 1 -- 172.21.15.69:0/3183752514 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228027070 con 0x7ff22c1028a0 2024-05-14T16:48:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff234493700 1 -- 172.21.15.69:0/3183752514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c1a1120 con 0x7ff22c1028a0 2024-05-14T16:48:35.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff2237fe700 1 -- 172.21.15.69:0/3183752514 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2280041d0 con 0x7ff22c1028a0 2024-05-14T16:48:35.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff2237fe700 1 -- 172.21.15.69:0/3183752514 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22800f460 con 0x7ff22c1028a0 2024-05-14T16:48:35.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.356+0000 7ff234493700 1 -- 172.21.15.69:0/3183752514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c1a8a00 con 0x7ff22c1028a0 2024-05-14T16:48:35.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.357+0000 7ff234493700 1 -- 172.21.15.69:0/3183752514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22c06a230 con 0x7ff22c1028a0 2024-05-14T16:48:35.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.358+0000 7ff2237fe700 1 -- 172.21.15.69:0/3183752514 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff22802a070 con 0x7ff22c1028a0 2024-05-14T16:48:35.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.359+0000 7ff2237fe700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2180b2880 0x7ff2180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:35.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.359+0000 7ff23222f700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2180b2880 0x7ff2180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:35.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.359+0000 7ff2237fe700 1 -- 172.21.15.69:0/3183752514 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff2280e3650 con 0x7ff22c1028a0 2024-05-14T16:48:35.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.360+0000 7ff23222f700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2180b2880 0x7ff2180b4d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff21c005950 tx=0x7ff21c005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:35.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.363+0000 7ff2237fe700 1 -- 172.21.15.69:0/3183752514 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff228031070 con 0x7ff22c1028a0 2024-05-14T16:48:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.577+0000 7ff234493700 1 -- 172.21.15.69:0/3183752514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff22c1a8bd0 con 0x7ff22c1028a0 2024-05-14T16:48:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.577+0000 7ff2237fe700 1 -- 172.21.15.69:0/3183752514 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff228013070 con 0x7ff22c1028a0 2024-05-14T16:48:35.889 INFO:teuthology.orchestra.run.smithi069.stdout:253403070513 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 -- 172.21.15.69:0/3183752514 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2180b2880 msgr2=0x7ff2180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2180b2880 0x7ff2180b4d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff21c005950 tx=0x7ff21c005f50 comp rx=0 tx=0).stop 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 -- 172.21.15.69:0/3183752514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c1028a0 msgr2=0x7ff22c19c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c1028a0 0x7ff22c19c3c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff228000c00 tx=0x7ff2280047b0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 -- 172.21.15.69:0/3183752514 shutdown_connections 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c108600 0x7ff22c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c101ee0 0x7ff22c19be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff2180b2880 0x7ff2180b4d40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 --2- 172.21.15.69:0/3183752514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c1028a0 0x7ff22c19c3c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 -- 172.21.15.69:0/3183752514 >> 172.21.15.69:0/3183752514 conn(0x7ff22c077760 msgr2=0x7ff22c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 -- 172.21.15.69:0/3183752514 shutdown_connections 2024-05-14T16:48:35.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:35.580+0000 7ff234493700 1 -- 172.21.15.69:0/3183752514 wait complete. 2024-05-14T16:48:35.891 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:35 smithi069 ceph-mon[31507]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T16:48:35.891 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:35 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3183752514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T16:48:35.891 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738455 got 34359738460 for osd.0 2024-05-14T16:48:35.892 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:35.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:35 smithi203 ceph-mon[35787]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T16:48:35.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:35 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3183752514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T16:48:36.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:35 smithi092 ceph-mon[35763]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T16:48:36.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:35 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3183752514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T16:48:36.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626491 got 180388626496 for osd.6 2024-05-14T16:48:36.246 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:37.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:37 smithi069 ceph-mon[31507]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T16:48:37.718 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906980 got 274877906986 for osd.10 2024-05-14T16:48:37.718 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:37.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:37 smithi203 ceph-mon[35787]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T16:48:38.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:37 smithi092 ceph-mon[35763]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T16:48:38.300 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070508 got 253403070513 for osd.9 2024-05-14T16:48:38.301 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.699+0000 7fc6c5747700 1 -- 172.21.15.69:0/3705519530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0101e20 msgr2=0x7fc6c0102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.699+0000 7fc6c5747700 1 --2- 172.21.15.69:0/3705519530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0101e20 0x7fc6c0102200 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00099c0 tx=0x7fc6b0009cd0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.699+0000 7fc6c5747700 1 -- 172.21.15.69:0/3705519530 shutdown_connections 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.699+0000 7fc6c5747700 1 --2- 172.21.15.69:0/3705519530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0101e20 0x7fc6c0102200 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.699+0000 7fc6c5747700 1 --2- 172.21.15.69:0/3705519530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0106f00 0x7fc6c010ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.699+0000 7fc6c5747700 1 --2- 172.21.15.69:0/3705519530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6c01027d0 0x7fc6c0106820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.699+0000 7fc6c5747700 1 -- 172.21.15.69:0/3705519530 >> 172.21.15.69:0/3705519530 conn(0x7fc6c0078a40 msgr2=0x7fc6c00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.699+0000 7fc6c5747700 1 -- 172.21.15.69:0/3705519530 shutdown_connections 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.699+0000 7fc6c5747700 1 -- 172.21.15.69:0/3705519530 wait complete. 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.700+0000 7fc6c5747700 1 Processor -- start 2024-05-14T16:48:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.700+0000 7fc6c5747700 1 -- start start 2024-05-14T16:48:38.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.700+0000 7fc6c5747700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6c0101e20 0x7fc6c019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:38.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.700+0000 7fc6c5747700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c01027d0 0x7fc6c019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:38.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.700+0000 7fc6c5747700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0106f00 0x7fc6c01a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:38.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.700+0000 7fc6c5747700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0112630 con 0x7fc6c01027d0 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.700+0000 7fc6c5747700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c01124b0 con 0x7fc6c0106f00 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.700+0000 7fc6c5747700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c01127b0 con 0x7fc6c0101e20 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6be7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c01027d0 0x7fc6c019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6be7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c01027d0 0x7fc6c019af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47212/0 (socket says 172.21.15.69:47212) 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6be7fc700 1 -- 172.21.15.69:0/737052475 learned_addr learned my addr 172.21.15.69:0/737052475 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6beffd700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6c0101e20 0x7fc6c019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6bf7fe700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0106f00 0x7fc6c01a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6be7fc700 1 -- 172.21.15.69:0/737052475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6c0101e20 msgr2=0x7fc6c019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6be7fc700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6c0101e20 0x7fc6c019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6be7fc700 1 -- 172.21.15.69:0/737052475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0106f00 msgr2=0x7fc6c01a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6be7fc700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0106f00 0x7fc6c01a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6be7fc700 1 -- 172.21.15.69:0/737052475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c01a5ff0 con 0x7fc6c01027d0 2024-05-14T16:48:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6beffd700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6c0101e20 0x7fc6c019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:48:38.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6bf7fe700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0106f00 0x7fc6c01a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:48:38.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6be7fc700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c01027d0 0x7fc6c019af70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400c9e0 tx=0x7fc6b400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:38.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6a7fff700 1 -- 172.21.15.69:0/737052475 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4007aa0 con 0x7fc6c01027d0 2024-05-14T16:48:38.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6c5747700 1 -- 172.21.15.69:0/737052475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c01a6280 con 0x7fc6c01027d0 2024-05-14T16:48:38.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6a7fff700 1 -- 172.21.15.69:0/737052475 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6b4007c40 con 0x7fc6c01027d0 2024-05-14T16:48:38.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6a7fff700 1 -- 172.21.15.69:0/737052475 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4018660 con 0x7fc6c01027d0 2024-05-14T16:48:38.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.701+0000 7fc6c5747700 1 -- 172.21.15.69:0/737052475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c01a6790 con 0x7fc6c01027d0 2024-05-14T16:48:38.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.702+0000 7fc6c5747700 1 -- 172.21.15.69:0/737052475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c006a230 con 0x7fc6c01027d0 2024-05-14T16:48:38.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.704+0000 7fc6a7fff700 1 -- 172.21.15.69:0/737052475 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc6b4018800 con 0x7fc6c01027d0 2024-05-14T16:48:38.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.704+0000 7fc6a7fff700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc6a80b2880 0x7fc6a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:38.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.704+0000 7fc6beffd700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc6a80b2880 0x7fc6a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:38.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.705+0000 7fc6a7fff700 1 -- 172.21.15.69:0/737052475 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc6b40e23e0 con 0x7fc6c01027d0 2024-05-14T16:48:38.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.705+0000 7fc6beffd700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc6a80b2880 0x7fc6a80b4d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0005b40 tx=0x7fc6b001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.708+0000 7fc6a7fff700 1 -- 172.21.15.69:0/737052475 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6b4054df0 con 0x7fc6c01027d0 2024-05-14T16:48:38.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.911+0000 7fc6c5747700 1 -- 172.21.15.69:0/737052475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc6c002e560 con 0x7fc6c01027d0 2024-05-14T16:48:38.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.912+0000 7fc6a7fff700 1 -- 172.21.15.69:0/737052475 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc6b4014030 con 0x7fc6c01027d0 2024-05-14T16:48:38.912 INFO:teuthology.orchestra.run.smithi069.stdout:201863462971 2024-05-14T16:48:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.914+0000 7fc6c5747700 1 -- 172.21.15.69:0/737052475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc6a80b2880 msgr2=0x7fc6a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.914+0000 7fc6c5747700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc6a80b2880 0x7fc6a80b4d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0005b40 tx=0x7fc6b001a040 comp rx=0 tx=0).stop 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.914+0000 7fc6c5747700 1 -- 172.21.15.69:0/737052475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c01027d0 msgr2=0x7fc6c019af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.915+0000 7fc6c5747700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c01027d0 0x7fc6c019af70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400c9e0 tx=0x7fc6b400cda0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.915+0000 7fc6c5747700 1 -- 172.21.15.69:0/737052475 shutdown_connections 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.915+0000 7fc6c5747700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6c0106f00 0x7fc6c01a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.915+0000 7fc6c5747700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6c0101e20 0x7fc6c019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.915+0000 7fc6c5747700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc6a80b2880 0x7fc6a80b4d40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.915+0000 7fc6c5747700 1 --2- 172.21.15.69:0/737052475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c01027d0 0x7fc6c019af70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.915+0000 7fc6c5747700 1 -- 172.21.15.69:0/737052475 >> 172.21.15.69:0/737052475 conn(0x7fc6c0078a40 msgr2=0x7fc6c00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.915+0000 7fc6c5747700 1 -- 172.21.15.69:0/737052475 shutdown_connections 2024-05-14T16:48:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:38.915+0000 7fc6c5747700 1 -- 172.21.15.69:0/737052475 wait complete. 2024-05-14T16:48:39.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462966 got 201863462971 for osd.7 2024-05-14T16:48:39.592 DEBUG:teuthology.parallel:result is None 2024-05-14T16:48:39.592 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T16:48:39.593 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:48:39.849 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:39.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:39 smithi069 ceph-mon[31507]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T16:48:39.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:39 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/737052475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T16:48:39.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:39 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:39.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:39 smithi203 ceph-mon[35787]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T16:48:39.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:39 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/737052475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T16:48:39.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:39 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:39 smithi092 ceph-mon[35763]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T16:48:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:39 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/737052475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T16:48:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:39 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:41.724 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:41 smithi069 ceph-mon[31507]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T16:48:41.724 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:41.724 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:41.724 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:41.724 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:41 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:41 smithi203 ceph-mon[35787]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T16:48:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:41.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:41.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:41 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:41 smithi092 ceph-mon[35763]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T16:48:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:41 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:42.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.143+0000 7fa4f74d9700 1 -- 172.21.15.69:0/230790786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4f0102e90 msgr2=0x7fa4f010f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:42.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.143+0000 7fa4f74d9700 1 --2- 172.21.15.69:0/230790786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4f0102e90 0x7fa4f010f310 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800b0a0 tx=0x7fa4e800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.143+0000 7fa4f74d9700 1 -- 172.21.15.69:0/230790786 shutdown_connections 2024-05-14T16:48:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.143+0000 7fa4f74d9700 1 --2- 172.21.15.69:0/230790786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4f0102e90 0x7fa4f010f310 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.143+0000 7fa4f74d9700 1 --2- 172.21.15.69:0/230790786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4f01084d0 0x7fa4f01088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.143+0000 7fa4f74d9700 1 --2- 172.21.15.69:0/230790786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f01024d0 0x7fa4f0102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.143+0000 7fa4f74d9700 1 -- 172.21.15.69:0/230790786 >> 172.21.15.69:0/230790786 conn(0x7fa4f0078490 msgr2=0x7fa4f00788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:42.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.144+0000 7fa4f74d9700 1 -- 172.21.15.69:0/230790786 shutdown_connections 2024-05-14T16:48:42.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.144+0000 7fa4f74d9700 1 -- 172.21.15.69:0/230790786 wait complete. 2024-05-14T16:48:42.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.144+0000 7fa4f74d9700 1 Processor -- start 2024-05-14T16:48:42.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.144+0000 7fa4f74d9700 1 -- start start 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.144+0000 7fa4f74d9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4f01024d0 0x7fa4f019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.144+0000 7fa4f74d9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4f0102e90 0x7fa4f019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.144+0000 7fa4f74d9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f01084d0 0x7fa4f0199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.144+0000 7fa4f74d9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0113780 con 0x7fa4f01024d0 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.144+0000 7fa4f74d9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0113600 con 0x7fa4f0102e90 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f74d9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0113900 con 0x7fa4f01084d0 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f5275700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4f01024d0 0x7fa4f019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f5275700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4f01024d0 0x7fa4f019f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47226/0 (socket says 172.21.15.69:47226) 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f5275700 1 -- 172.21.15.69:0/3038103238 learned_addr learned my addr 172.21.15.69:0/3038103238 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f5a76700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f01084d0 0x7fa4f0199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f4a74700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4f0102e90 0x7fa4f019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f5275700 1 -- 172.21.15.69:0/3038103238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f01084d0 msgr2=0x7fa4f0199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f5275700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f01084d0 0x7fa4f0199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f5275700 1 -- 172.21.15.69:0/3038103238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4f0102e90 msgr2=0x7fa4f019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:42.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f5275700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4f0102e90 0x7fa4f019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f5275700 1 -- 172.21.15.69:0/3038103238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f019a1c0 con 0x7fa4f01024d0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f5275700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4f01024d0 0x7fa4f019f8a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00b530 tx=0x7fa4ec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4e27fc700 1 -- 172.21.15.69:0/3038103238 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec015020 con 0x7fa4f01024d0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4e27fc700 1 -- 172.21.15.69:0/3038103238 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ec011ce0 con 0x7fa4f01024d0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f74d9700 1 -- 172.21.15.69:0/3038103238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f019a4b0 con 0x7fa4f01024d0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4e27fc700 1 -- 172.21.15.69:0/3038103238 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec010dd0 con 0x7fa4f01024d0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.145+0000 7fa4f74d9700 1 -- 172.21.15.69:0/3038103238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f01a6770 con 0x7fa4f01024d0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.146+0000 7fa4f74d9700 1 -- 172.21.15.69:0/3038103238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f006a230 con 0x7fa4f01024d0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.148+0000 7fa4e27fc700 1 -- 172.21.15.69:0/3038103238 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa4ec018020 con 0x7fa4f01024d0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.148+0000 7fa4e27fc700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa4dc0b2880 0x7fa4dc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.148+0000 7fa4f4a74700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa4dc0b2880 0x7fa4dc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.148+0000 7fa4e27fc700 1 -- 172.21.15.69:0/3038103238 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fa4ec0e1e60 con 0x7fa4f01024d0 2024-05-14T16:48:42.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.149+0000 7fa4f4a74700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa4dc0b2880 0x7fa4dc0b4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0078a90 tx=0x7fa4e400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:42.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.152+0000 7fa4e27fc700 1 -- 172.21.15.69:0/3038103238 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa4ec01f020 con 0x7fa4f01024d0 2024-05-14T16:48:42.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.389+0000 7fa4f74d9700 1 -- 172.21.15.69:0/3038103238 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4f019af90 con 0x7fa4dc0b2880 2024-05-14T16:48:42.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.397+0000 7fa4e27fc700 1 -- 172.21.15.69:0/3038103238 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259059 (secure 0 0 0) 0x7fa4f019af90 con 0x7fa4dc0b2880 2024-05-14T16:48:42.401 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:48:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 -- 172.21.15.69:0/3038103238 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa4dc0b2880 msgr2=0x7fa4dc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa4dc0b2880 0x7fa4dc0b4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0078a90 tx=0x7fa4e400b3c0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 -- 172.21.15.69:0/3038103238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4f01024d0 msgr2=0x7fa4f019f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4f01024d0 0x7fa4f019f8a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00b530 tx=0x7fa4ec00b840 comp rx=0 tx=0).stop 2024-05-14T16:48:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 -- 172.21.15.69:0/3038103238 shutdown_connections 2024-05-14T16:48:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa4dc0b2880 0x7fa4dc0b4d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4f01024d0 0x7fa4f019f8a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4f01084d0 0x7fa4f0199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 --2- 172.21.15.69:0/3038103238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4f0102e90 0x7fa4f019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 -- 172.21.15.69:0/3038103238 >> 172.21.15.69:0/3038103238 conn(0x7fa4f0078490 msgr2=0x7fa4f0100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 -- 172.21.15.69:0/3038103238 shutdown_connections 2024-05-14T16:48:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:42.400+0000 7fa4f74d9700 1 -- 172.21.15.69:0/3038103238 wait complete. 2024-05-14T16:48:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-05-14T16:48:43.052 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-14T16:48:41.886984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656211,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":692,"num_read_kb":445,"num_write":294,"num_write_kb":2266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952568,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":737,"ondisk_log_size":737,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351488,"kb_used_data":10656,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590336,"statfs":{"total":1151940427776,"available":1151580504064,"internally_reserved":0,"allocated":10911744,"data_stored":7064253,"data_compressed":176766,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":36,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002880"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.890772+0000","last_change":"2024-05-14T16:46:43.572738+0000","last_active":"2024-05-14T16:48:31.890772+0000","last_peered":"2024-05-14T16:48:31.890772+0000","last_clean":"2024-05-14T16:48:31.890772+0000","last_became_active":"2024-05-14T16:46:43.572400+0000","last_became_peered":"2024-05-14T16:46:43.572400+0000","last_unstale":"2024-05-14T16:48:31.890772+0000","last_undegraded":"2024-05-14T16:48:31.890772+0000","last_fullsized":"2024-05-14T16:48:31.890772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:44.332862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.888529+0000","last_change":"2024-05-14T16:46:43.566193+0000","last_active":"2024-05-14T16:48:31.888529+0000","last_peered":"2024-05-14T16:48:31.888529+0000","last_clean":"2024-05-14T16:48:31.888529+0000","last_became_active":"2024-05-14T16:46:43.565853+0000","last_became_peered":"2024-05-14T16:46:43.565853+0000","last_unstale":"2024-05-14T16:48:31.888529+0000","last_undegraded":"2024-05-14T16:48:31.888529+0000","last_fullsized":"2024-05-14T16:48:31.888529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:19:34.481225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.890380+0000","last_change":"2024-05-14T16:46:43.574937+0000","last_active":"2024-05-14T16:48:31.890380+0000","last_peered":"2024-05-14T16:48:31.890380+0000","last_clean":"2024-05-14T16:48:31.890380+0000","last_became_active":"2024-05-14T16:46:43.574317+0000","last_became_peered":"2024-05-14T16:46:43.574317+0000","last_unstale":"2024-05-14T16:48:31.890380+0000","last_undegraded":"2024-05-14T16:48:31.890380+0000","last_fullsized":"2024-05-14T16:48:31.890380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:35:49.933658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.885735+0000","last_change":"2024-05-14T16:47:33.711809+0000","last_active":"2024-05-14T16:48:31.885735+0000","last_peered":"2024-05-14T16:48:31.885735+0000","last_clean":"2024-05-14T16:48:31.885735+0000","last_became_active":"2024-05-14T16:47:33.711422+0000","last_became_peered":"2024-05-14T16:47:33.711422+0000","last_unstale":"2024-05-14T16:48:31.885735+0000","last_undegraded":"2024-05-14T16:48:31.885735+0000","last_fullsized":"2024-05-14T16:48:31.885735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:44:26.005189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.875757+0000","last_change":"2024-05-14T16:46:43.567104+0000","last_active":"2024-05-14T16:48:31.875757+0000","last_peered":"2024-05-14T16:48:31.875757+0000","last_clean":"2024-05-14T16:48:31.875757+0000","last_became_active":"2024-05-14T16:46:43.566762+0000","last_became_peered":"2024-05-14T16:46:43.566762+0000","last_unstale":"2024-05-14T16:48:31.875757+0000","last_undegraded":"2024-05-14T16:48:31.875757+0000","last_fullsized":"2024-05-14T16:48:31.875757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:08.819856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.875768+0000","last_change":"2024-05-14T16:46:43.565055+0000","last_active":"2024-05-14T16:48:31.875768+0000","last_peered":"2024-05-14T16:48:31.875768+0000","last_clean":"2024-05-14T16:48:31.875768+0000","last_became_active":"2024-05-14T16:46:43.564699+0000","last_became_peered":"2024-05-14T16:46:43.564699+0000","last_unstale":"2024-05-14T16:48:31.875768+0000","last_undegraded":"2024-05-14T16:48:31.875768+0000","last_fullsized":"2024-05-14T16:48:31.875768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:01.903025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.871869+0000","last_change":"2024-05-14T16:47:33.711866+0000","last_active":"2024-05-14T16:48:31.871869+0000","last_peered":"2024-05-14T16:48:31.871869+0000","last_clean":"2024-05-14T16:48:31.871869+0000","last_became_active":"2024-05-14T16:47:33.711481+0000","last_became_peered":"2024-05-14T16:47:33.711481+0000","last_unstale":"2024-05-14T16:48:31.871869+0000","last_undegraded":"2024-05-14T16:48:31.871869+0000","last_fullsized":"2024-05-14T16:48:31.871869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:01:22.724678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.887357+0000","last_change":"2024-05-14T16:46:43.564470+0000","last_active":"2024-05-14T16:48:31.887357+0000","last_peered":"2024-05-14T16:48:31.887357+0000","last_clean":"2024-05-14T16:48:31.887357+0000","last_became_active":"2024-05-14T16:46:43.564169+0000","last_became_peered":"2024-05-14T16:46:43.564169+0000","last_unstale":"2024-05-14T16:48:31.887357+0000","last_undegraded":"2024-05-14T16:48:31.887357+0000","last_fullsized":"2024-05-14T16:48:31.887357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:38.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892107+0000","last_change":"2024-05-14T16:46:43.580988+0000","last_active":"2024-05-14T16:48:31.892107+0000","last_peered":"2024-05-14T16:48:31.892107+0000","last_clean":"2024-05-14T16:48:31.892107+0000","last_became_active":"2024-05-14T16:46:43.580657+0000","last_became_peered":"2024-05-14T16:46:43.580657+0000","last_unstale":"2024-05-14T16:48:31.892107+0000","last_undegraded":"2024-05-14T16:48:31.892107+0000","last_fullsized":"2024-05-14T16:48:31.892107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:39:01.568822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.884437+0000","last_change":"2024-05-14T16:46:43.565742+0000","last_active":"2024-05-14T16:48:31.884437+0000","last_peered":"2024-05-14T16:48:31.884437+0000","last_clean":"2024-05-14T16:48:31.884437+0000","last_became_active":"2024-05-14T16:46:43.565260+0000","last_became_peered":"2024-05-14T16:46:43.565260+0000","last_unstale":"2024-05-14T16:48:31.884437+0000","last_undegraded":"2024-05-14T16:48:31.884437+0000","last_fullsized":"2024-05-14T16:48:31.884437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:25.718174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.891385+0000","last_change":"2024-05-14T16:46:43.576268+0000","last_active":"2024-05-14T16:48:31.891385+0000","last_peered":"2024-05-14T16:48:31.891385+0000","last_clean":"2024-05-14T16:48:31.891385+0000","last_became_active":"2024-05-14T16:46:43.575856+0000","last_became_peered":"2024-05-14T16:46:43.575856+0000","last_unstale":"2024-05-14T16:48:31.891385+0000","last_undegraded":"2024-05-14T16:48:31.891385+0000","last_fullsized":"2024-05-14T16:48:31.891385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:12:16.549127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.882004+0000","last_change":"2024-05-14T16:46:43.573315+0000","last_active":"2024-05-14T16:48:31.882004+0000","last_peered":"2024-05-14T16:48:31.882004+0000","last_clean":"2024-05-14T16:48:31.882004+0000","last_became_active":"2024-05-14T16:46:43.572981+0000","last_became_peered":"2024-05-14T16:46:43.572981+0000","last_unstale":"2024-05-14T16:48:31.882004+0000","last_undegraded":"2024-05-14T16:48:31.882004+0000","last_fullsized":"2024-05-14T16:48:31.882004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:07.757539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.885637+0000","last_change":"2024-05-14T16:46:43.569388+0000","last_active":"2024-05-14T16:48:31.885637+0000","last_peered":"2024-05-14T16:48:31.885637+0000","last_clean":"2024-05-14T16:48:31.885637+0000","last_became_active":"2024-05-14T16:46:43.568992+0000","last_became_peered":"2024-05-14T16:46:43.568992+0000","last_unstale":"2024-05-14T16:48:31.885637+0000","last_undegraded":"2024-05-14T16:48:31.885637+0000","last_fullsized":"2024-05-14T16:48:31.885637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:08:06.236797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.865859+0000","last_change":"2024-05-14T16:46:43.564137+0000","last_active":"2024-05-14T16:48:31.865859+0000","last_peered":"2024-05-14T16:48:31.865859+0000","last_clean":"2024-05-14T16:48:31.865859+0000","last_became_active":"2024-05-14T16:46:43.563831+0000","last_became_peered":"2024-05-14T16:46:43.563831+0000","last_unstale":"2024-05-14T16:48:31.865859+0000","last_undegraded":"2024-05-14T16:48:31.865859+0000","last_fullsized":"2024-05-14T16:48:31.865859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:58.473863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.879484+0000","last_change":"2024-05-14T16:46:43.571789+0000","last_active":"2024-05-14T16:48:31.879484+0000","last_peered":"2024-05-14T16:48:31.879484+0000","last_clean":"2024-05-14T16:48:31.879484+0000","last_became_active":"2024-05-14T16:46:43.571521+0000","last_became_peered":"2024-05-14T16:46:43.571521+0000","last_unstale":"2024-05-14T16:48:31.879484+0000","last_undegraded":"2024-05-14T16:48:31.879484+0000","last_fullsized":"2024-05-14T16:48:31.879484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:23.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892635+0000","last_change":"2024-05-14T16:46:43.569532+0000","last_active":"2024-05-14T16:48:31.892635+0000","last_peered":"2024-05-14T16:48:31.892635+0000","last_clean":"2024-05-14T16:48:31.892635+0000","last_became_active":"2024-05-14T16:46:43.569132+0000","last_became_peered":"2024-05-14T16:46:43.569132+0000","last_unstale":"2024-05-14T16:48:31.892635+0000","last_undegraded":"2024-05-14T16:48:31.892635+0000","last_fullsized":"2024-05-14T16:48:31.892635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:09.328241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892354+0000","last_change":"2024-05-14T16:46:43.574451+0000","last_active":"2024-05-14T16:48:31.892354+0000","last_peered":"2024-05-14T16:48:31.892354+0000","last_clean":"2024-05-14T16:48:31.892354+0000","last_became_active":"2024-05-14T16:46:43.574143+0000","last_became_peered":"2024-05-14T16:46:43.574143+0000","last_unstale":"2024-05-14T16:48:31.892354+0000","last_undegraded":"2024-05-14T16:48:31.892354+0000","last_fullsized":"2024-05-14T16:48:31.892354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:01:42.831262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.890687+0000","last_change":"2024-05-14T16:46:43.568799+0000","last_active":"2024-05-14T16:48:31.890687+0000","last_peered":"2024-05-14T16:48:31.890687+0000","last_clean":"2024-05-14T16:48:31.890687+0000","last_became_active":"2024-05-14T16:46:43.568467+0000","last_became_peered":"2024-05-14T16:46:43.568467+0000","last_unstale":"2024-05-14T16:48:31.890687+0000","last_undegraded":"2024-05-14T16:48:31.890687+0000","last_fullsized":"2024-05-14T16:48:31.890687+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:59.036372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.878469+0000","last_change":"2024-05-14T16:46:43.566879+0000","last_active":"2024-05-14T16:48:31.878469+0000","last_peered":"2024-05-14T16:48:31.878469+0000","last_clean":"2024-05-14T16:48:31.878469+0000","last_became_active":"2024-05-14T16:46:43.566346+0000","last_became_peered":"2024-05-14T16:46:43.566346+0000","last_unstale":"2024-05-14T16:48:31.878469+0000","last_undegraded":"2024-05-14T16:48:31.878469+0000","last_fullsized":"2024-05-14T16:48:31.878469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:11.027173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.705064+0000","last_change":"2024-05-14T16:46:40.546990+0000","last_active":"2024-05-14T16:47:33.705064+0000","last_peered":"2024-05-14T16:47:33.705064+0000","last_clean":"2024-05-14T16:47:33.705064+0000","last_became_active":"2024-05-14T16:46:40.546646+0000","last_became_peered":"2024-05-14T16:46:40.546646+0000","last_unstale":"2024-05-14T16:47:33.705064+0000","last_undegraded":"2024-05-14T16:47:33.705064+0000","last_fullsized":"2024-05-14T16:47:33.705064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:47.001760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.876111+0000","last_change":"2024-05-14T16:46:43.570352+0000","last_active":"2024-05-14T16:48:31.876111+0000","last_peered":"2024-05-14T16:48:31.876111+0000","last_clean":"2024-05-14T16:48:31.876111+0000","last_became_active":"2024-05-14T16:46:43.569748+0000","last_became_peered":"2024-05-14T16:46:43.569748+0000","last_unstale":"2024-05-14T16:48:31.876111+0000","last_undegraded":"2024-05-14T16:48:31.876111+0000","last_fullsized":"2024-05-14T16:48:31.876111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:24.707586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.702199+0000","last_change":"2024-05-14T16:46:40.855034+0000","last_active":"2024-05-14T16:47:33.702199+0000","last_peered":"2024-05-14T16:47:33.702199+0000","last_clean":"2024-05-14T16:47:33.702199+0000","last_became_active":"2024-05-14T16:46:40.854686+0000","last_became_peered":"2024-05-14T16:46:40.854686+0000","last_unstale":"2024-05-14T16:47:33.702199+0000","last_undegraded":"2024-05-14T16:47:33.702199+0000","last_fullsized":"2024-05-14T16:47:33.702199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:56.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.890398+0000","last_change":"2024-05-14T16:46:43.574682+0000","last_active":"2024-05-14T16:48:31.890398+0000","last_peered":"2024-05-14T16:48:31.890398+0000","last_clean":"2024-05-14T16:48:31.890398+0000","last_became_active":"2024-05-14T16:46:43.574322+0000","last_became_peered":"2024-05-14T16:46:43.574322+0000","last_unstale":"2024-05-14T16:48:31.890398+0000","last_undegraded":"2024-05-14T16:48:31.890398+0000","last_fullsized":"2024-05-14T16:48:31.890398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:20.169428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.710333+0000","last_change":"2024-05-14T16:46:40.551307+0000","last_active":"2024-05-14T16:47:33.710333+0000","last_peered":"2024-05-14T16:47:33.710333+0000","last_clean":"2024-05-14T16:47:33.710333+0000","last_became_active":"2024-05-14T16:46:40.550987+0000","last_became_peered":"2024-05-14T16:46:40.550987+0000","last_unstale":"2024-05-14T16:47:33.710333+0000","last_undegraded":"2024-05-14T16:47:33.710333+0000","last_fullsized":"2024-05-14T16:47:33.710333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:27.819457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.920360+0000","last_change":"2024-05-14T16:46:43.575200+0000","last_active":"2024-05-14T16:48:31.920360+0000","last_peered":"2024-05-14T16:48:31.920360+0000","last_clean":"2024-05-14T16:48:31.920360+0000","last_became_active":"2024-05-14T16:46:43.574864+0000","last_became_peered":"2024-05-14T16:46:43.574864+0000","last_unstale":"2024-05-14T16:48:31.920360+0000","last_undegraded":"2024-05-14T16:48:31.920360+0000","last_fullsized":"2024-05-14T16:48:31.920360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:48.630101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.710590+0000","last_change":"2024-05-14T16:46:40.548397+0000","last_active":"2024-05-14T16:47:33.710590+0000","last_peered":"2024-05-14T16:47:33.710590+0000","last_clean":"2024-05-14T16:47:33.710590+0000","last_became_active":"2024-05-14T16:46:40.547803+0000","last_became_peered":"2024-05-14T16:46:40.547803+0000","last_unstale":"2024-05-14T16:47:33.710590+0000","last_undegraded":"2024-05-14T16:47:33.710590+0000","last_fullsized":"2024-05-14T16:47:33.710590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:21:19.812413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892954+0000","last_change":"2024-05-14T16:46:43.578301+0000","last_active":"2024-05-14T16:48:31.892954+0000","last_peered":"2024-05-14T16:48:31.892954+0000","last_clean":"2024-05-14T16:48:31.892954+0000","last_became_active":"2024-05-14T16:46:43.577796+0000","last_became_peered":"2024-05-14T16:46:43.577796+0000","last_unstale":"2024-05-14T16:48:31.892954+0000","last_undegraded":"2024-05-14T16:48:31.892954+0000","last_fullsized":"2024-05-14T16:48:31.892954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:13.159570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":128,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:35.757812+0000","last_change":"2024-05-14T16:46:40.547818+0000","last_active":"2024-05-14T16:48:35.757812+0000","last_peered":"2024-05-14T16:48:35.757812+0000","last_clean":"2024-05-14T16:48:35.757812+0000","last_became_active":"2024-05-14T16:46:40.547454+0000","last_became_peered":"2024-05-14T16:46:40.547454+0000","last_unstale":"2024-05-14T16:48:35.757812+0000","last_undegraded":"2024-05-14T16:48:35.757812+0000","last_fullsized":"2024-05-14T16:48:35.757812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:16.362415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.879530+0000","last_change":"2024-05-14T16:46:43.570743+0000","last_active":"2024-05-14T16:48:31.879530+0000","last_peered":"2024-05-14T16:48:31.879530+0000","last_clean":"2024-05-14T16:48:31.879530+0000","last_became_active":"2024-05-14T16:46:43.570427+0000","last_became_peered":"2024-05-14T16:46:43.570427+0000","last_unstale":"2024-05-14T16:48:31.879530+0000","last_undegraded":"2024-05-14T16:48:31.879530+0000","last_fullsized":"2024-05-14T16:48:31.879530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:04:11.091285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.708297+0000","last_change":"2024-05-14T16:46:40.546675+0000","last_active":"2024-05-14T16:47:33.708297+0000","last_peered":"2024-05-14T16:47:33.708297+0000","last_clean":"2024-05-14T16:47:33.708297+0000","last_became_active":"2024-05-14T16:46:40.546250+0000","last_became_peered":"2024-05-14T16:46:40.546250+0000","last_unstale":"2024-05-14T16:47:33.708297+0000","last_undegraded":"2024-05-14T16:47:33.708297+0000","last_fullsized":"2024-05-14T16:47:33.708297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:42:52.367719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881313+0000","last_change":"2024-05-14T16:46:43.579119+0000","last_active":"2024-05-14T16:48:31.881313+0000","last_peered":"2024-05-14T16:48:31.881313+0000","last_clean":"2024-05-14T16:48:31.881313+0000","last_became_active":"2024-05-14T16:46:43.578808+0000","last_became_peered":"2024-05-14T16:46:43.578808+0000","last_unstale":"2024-05-14T16:48:31.881313+0000","last_undegraded":"2024-05-14T16:48:31.881313+0000","last_fullsized":"2024-05-14T16:48:31.881313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:57.852982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.716259+0000","last_change":"2024-05-14T16:46:40.538827+0000","last_active":"2024-05-14T16:47:33.716259+0000","last_peered":"2024-05-14T16:47:33.716259+0000","last_clean":"2024-05-14T16:47:33.716259+0000","last_became_active":"2024-05-14T16:46:40.538369+0000","last_became_peered":"2024-05-14T16:46:40.538369+0000","last_unstale":"2024-05-14T16:47:33.716259+0000","last_undegraded":"2024-05-14T16:47:33.716259+0000","last_fullsized":"2024-05-14T16:47:33.716259+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:48:03.599457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.878104+0000","last_change":"2024-05-14T16:46:43.558411+0000","last_active":"2024-05-14T16:48:31.878104+0000","last_peered":"2024-05-14T16:48:31.878104+0000","last_clean":"2024-05-14T16:48:31.878104+0000","last_became_active":"2024-05-14T16:46:43.558101+0000","last_became_peered":"2024-05-14T16:46:43.558101+0000","last_unstale":"2024-05-14T16:48:31.878104+0000","last_undegraded":"2024-05-14T16:48:31.878104+0000","last_fullsized":"2024-05-14T16:48:31.878104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:10.595833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.702005+0000","last_change":"2024-05-14T16:46:40.544858+0000","last_active":"2024-05-14T16:47:33.702005+0000","last_peered":"2024-05-14T16:47:33.702005+0000","last_clean":"2024-05-14T16:47:33.702005+0000","last_became_active":"2024-05-14T16:46:40.544197+0000","last_became_peered":"2024-05-14T16:46:40.544197+0000","last_unstale":"2024-05-14T16:47:33.702005+0000","last_undegraded":"2024-05-14T16:47:33.702005+0000","last_fullsized":"2024-05-14T16:47:33.702005+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:16:45.249626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.889028+0000","last_change":"2024-05-14T16:46:43.567019+0000","last_active":"2024-05-14T16:48:31.889028+0000","last_peered":"2024-05-14T16:48:31.889028+0000","last_clean":"2024-05-14T16:48:31.889028+0000","last_became_active":"2024-05-14T16:46:43.566700+0000","last_became_peered":"2024-05-14T16:46:43.566700+0000","last_unstale":"2024-05-14T16:48:31.889028+0000","last_undegraded":"2024-05-14T16:48:31.889028+0000","last_fullsized":"2024-05-14T16:48:31.889028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:43.373467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":126,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:36.758705+0000","last_change":"2024-05-14T16:46:40.541876+0000","last_active":"2024-05-14T16:48:36.758705+0000","last_peered":"2024-05-14T16:48:36.758705+0000","last_clean":"2024-05-14T16:48:36.758705+0000","last_became_active":"2024-05-14T16:46:40.541460+0000","last_became_peered":"2024-05-14T16:46:40.541460+0000","last_unstale":"2024-05-14T16:48:36.758705+0000","last_undegraded":"2024-05-14T16:48:36.758705+0000","last_fullsized":"2024-05-14T16:48:36.758705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:07:22.536429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892334+0000","last_change":"2024-05-14T16:46:43.577015+0000","last_active":"2024-05-14T16:48:31.892334+0000","last_peered":"2024-05-14T16:48:31.892334+0000","last_clean":"2024-05-14T16:48:31.892334+0000","last_became_active":"2024-05-14T16:46:43.576657+0000","last_became_peered":"2024-05-14T16:46:43.576657+0000","last_unstale":"2024-05-14T16:48:31.892334+0000","last_undegraded":"2024-05-14T16:48:31.892334+0000","last_fullsized":"2024-05-14T16:48:31.892334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:29.242445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.705613+0000","last_change":"2024-05-14T16:46:40.550727+0000","last_active":"2024-05-14T16:47:33.705613+0000","last_peered":"2024-05-14T16:47:33.705613+0000","last_clean":"2024-05-14T16:47:33.705613+0000","last_became_active":"2024-05-14T16:46:40.550310+0000","last_became_peered":"2024-05-14T16:46:40.550310+0000","last_unstale":"2024-05-14T16:47:33.705613+0000","last_undegraded":"2024-05-14T16:47:33.705613+0000","last_fullsized":"2024-05-14T16:47:33.705613+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:04.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881970+0000","last_change":"2024-05-14T16:47:33.711766+0000","last_active":"2024-05-14T16:48:31.881970+0000","last_peered":"2024-05-14T16:48:31.881970+0000","last_clean":"2024-05-14T16:48:31.881970+0000","last_became_active":"2024-05-14T16:47:33.711352+0000","last_became_peered":"2024-05-14T16:47:33.711352+0000","last_unstale":"2024-05-14T16:48:31.881970+0000","last_undegraded":"2024-05-14T16:48:31.881970+0000","last_fullsized":"2024-05-14T16:48:31.881970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:47.924117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.702407+0000","last_change":"2024-05-14T16:46:40.855680+0000","last_active":"2024-05-14T16:47:33.702407+0000","last_peered":"2024-05-14T16:47:33.702407+0000","last_clean":"2024-05-14T16:47:33.702407+0000","last_became_active":"2024-05-14T16:46:40.855265+0000","last_became_peered":"2024-05-14T16:46:40.855265+0000","last_unstale":"2024-05-14T16:47:33.702407+0000","last_undegraded":"2024-05-14T16:47:33.702407+0000","last_fullsized":"2024-05-14T16:47:33.702407+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:26.065298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.890619+0000","last_change":"2024-05-14T16:46:43.579423+0000","last_active":"2024-05-14T16:48:31.890619+0000","last_peered":"2024-05-14T16:48:31.890619+0000","last_clean":"2024-05-14T16:48:31.890619+0000","last_became_active":"2024-05-14T16:46:43.579023+0000","last_became_peered":"2024-05-14T16:46:43.579023+0000","last_unstale":"2024-05-14T16:48:31.890619+0000","last_undegraded":"2024-05-14T16:48:31.890619+0000","last_fullsized":"2024-05-14T16:48:31.890619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:21.311018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'33","reported_seq":263,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:37.760258+0000","last_change":"2024-05-14T16:46:40.550871+0000","last_active":"2024-05-14T16:48:37.760258+0000","last_peered":"2024-05-14T16:48:37.760258+0000","last_clean":"2024-05-14T16:48:37.760258+0000","last_became_active":"2024-05-14T16:46:40.550419+0000","last_became_peered":"2024-05-14T16:46:40.550419+0000","last_unstale":"2024-05-14T16:48:37.760258+0000","last_undegraded":"2024-05-14T16:48:37.760258+0000","last_fullsized":"2024-05-14T16:48:37.760258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:53:09.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61999,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":204,"num_read_kb":162,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.880092+0000","last_change":"2024-05-14T16:46:43.569996+0000","last_active":"2024-05-14T16:48:31.880092+0000","last_peered":"2024-05-14T16:48:31.880092+0000","last_clean":"2024-05-14T16:48:31.880092+0000","last_became_active":"2024-05-14T16:46:43.569501+0000","last_became_peered":"2024-05-14T16:46:43.569501+0000","last_unstale":"2024-05-14T16:48:31.880092+0000","last_undegraded":"2024-05-14T16:48:31.880092+0000","last_fullsized":"2024-05-14T16:48:31.880092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:55.135103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.733718+0000","last_change":"2024-05-14T16:47:33.733316+0000","last_active":"2024-05-14T16:47:33.733718+0000","last_peered":"2024-05-14T16:47:33.733718+0000","last_clean":"2024-05-14T16:47:33.733718+0000","last_became_active":"2024-05-14T16:47:33.709886+0000","last_became_peered":"2024-05-14T16:47:33.709886+0000","last_unstale":"2024-05-14T16:47:33.733718+0000","last_undegraded":"2024-05-14T16:47:33.733718+0000","last_fullsized":"2024-05-14T16:47:33.733718+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:00:47.464686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.878040+0000","last_change":"2024-05-14T16:46:43.562773+0000","last_active":"2024-05-14T16:48:31.878040+0000","last_peered":"2024-05-14T16:48:31.878040+0000","last_clean":"2024-05-14T16:48:31.878040+0000","last_became_active":"2024-05-14T16:46:43.562449+0000","last_became_peered":"2024-05-14T16:46:43.562449+0000","last_unstale":"2024-05-14T16:48:31.878040+0000","last_undegraded":"2024-05-14T16:48:31.878040+0000","last_fullsized":"2024-05-14T16:48:31.878040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:45.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.702480+0000","last_change":"2024-05-14T16:46:40.545882+0000","last_active":"2024-05-14T16:47:33.702480+0000","last_peered":"2024-05-14T16:47:33.702480+0000","last_clean":"2024-05-14T16:47:33.702480+0000","last_became_active":"2024-05-14T16:46:40.545439+0000","last_became_peered":"2024-05-14T16:46:40.545439+0000","last_unstale":"2024-05-14T16:47:33.702480+0000","last_undegraded":"2024-05-14T16:47:33.702480+0000","last_fullsized":"2024-05-14T16:47:33.702480+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:07.983329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.879226+0000","last_change":"2024-05-14T16:46:43.568109+0000","last_active":"2024-05-14T16:48:31.879226+0000","last_peered":"2024-05-14T16:48:31.879226+0000","last_clean":"2024-05-14T16:48:31.879226+0000","last_became_active":"2024-05-14T16:46:43.567106+0000","last_became_peered":"2024-05-14T16:46:43.567106+0000","last_unstale":"2024-05-14T16:48:31.879226+0000","last_undegraded":"2024-05-14T16:48:31.879226+0000","last_fullsized":"2024-05-14T16:48:31.879226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:29.705387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":128,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:36.758841+0000","last_change":"2024-05-14T16:46:40.542204+0000","last_active":"2024-05-14T16:48:36.758841+0000","last_peered":"2024-05-14T16:48:36.758841+0000","last_clean":"2024-05-14T16:48:36.758841+0000","last_became_active":"2024-05-14T16:46:40.541807+0000","last_became_peered":"2024-05-14T16:46:40.541807+0000","last_unstale":"2024-05-14T16:48:36.758841+0000","last_undegraded":"2024-05-14T16:48:36.758841+0000","last_fullsized":"2024-05-14T16:48:36.758841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:30:45.115647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.883770+0000","last_change":"2024-05-14T16:46:43.573546+0000","last_active":"2024-05-14T16:48:31.883770+0000","last_peered":"2024-05-14T16:48:31.883770+0000","last_clean":"2024-05-14T16:48:31.883770+0000","last_became_active":"2024-05-14T16:46:43.573183+0000","last_became_peered":"2024-05-14T16:46:43.573183+0000","last_unstale":"2024-05-14T16:48:31.883770+0000","last_undegraded":"2024-05-14T16:48:31.883770+0000","last_fullsized":"2024-05-14T16:48:31.883770+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:49.873097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.727138+0000","last_change":"2024-05-14T16:47:33.726700+0000","last_active":"2024-05-14T16:47:33.727138+0000","last_peered":"2024-05-14T16:47:33.727138+0000","last_clean":"2024-05-14T16:47:33.727138+0000","last_became_active":"2024-05-14T16:47:33.709297+0000","last_became_peered":"2024-05-14T16:47:33.709297+0000","last_unstale":"2024-05-14T16:47:33.727138+0000","last_undegraded":"2024-05-14T16:47:33.727138+0000","last_fullsized":"2024-05-14T16:47:33.727138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:35.028197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892554+0000","last_change":"2024-05-14T16:46:43.579059+0000","last_active":"2024-05-14T16:48:31.892554+0000","last_peered":"2024-05-14T16:48:31.892554+0000","last_clean":"2024-05-14T16:48:31.892554+0000","last_became_active":"2024-05-14T16:46:43.578748+0000","last_became_peered":"2024-05-14T16:46:43.578748+0000","last_unstale":"2024-05-14T16:48:31.892554+0000","last_undegraded":"2024-05-14T16:48:31.892554+0000","last_fullsized":"2024-05-14T16:48:31.892554+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:42.446422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.710532+0000","last_change":"2024-05-14T16:46:40.549047+0000","last_active":"2024-05-14T16:47:33.710532+0000","last_peered":"2024-05-14T16:47:33.710532+0000","last_clean":"2024-05-14T16:47:33.710532+0000","last_became_active":"2024-05-14T16:46:40.548648+0000","last_became_peered":"2024-05-14T16:46:40.548648+0000","last_unstale":"2024-05-14T16:47:33.710532+0000","last_undegraded":"2024-05-14T16:47:33.710532+0000","last_fullsized":"2024-05-14T16:47:33.710532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:52:56.370871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.882588+0000","last_change":"2024-05-14T16:46:43.569624+0000","last_active":"2024-05-14T16:48:31.882588+0000","last_peered":"2024-05-14T16:48:31.882588+0000","last_clean":"2024-05-14T16:48:31.882588+0000","last_became_active":"2024-05-14T16:46:43.569077+0000","last_became_peered":"2024-05-14T16:46:43.569077+0000","last_unstale":"2024-05-14T16:48:31.882588+0000","last_undegraded":"2024-05-14T16:48:31.882588+0000","last_fullsized":"2024-05-14T16:48:31.882588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:45:50.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.705048+0000","last_change":"2024-05-14T16:46:40.856302+0000","last_active":"2024-05-14T16:47:33.705048+0000","last_peered":"2024-05-14T16:47:33.705048+0000","last_clean":"2024-05-14T16:47:33.705048+0000","last_became_active":"2024-05-14T16:46:40.855964+0000","last_became_peered":"2024-05-14T16:46:40.855964+0000","last_unstale":"2024-05-14T16:47:33.705048+0000","last_undegraded":"2024-05-14T16:47:33.705048+0000","last_fullsized":"2024-05-14T16:47:33.705048+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:38.501418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881404+0000","last_change":"2024-05-14T16:46:43.562847+0000","last_active":"2024-05-14T16:48:31.881404+0000","last_peered":"2024-05-14T16:48:31.881404+0000","last_clean":"2024-05-14T16:48:31.881404+0000","last_became_active":"2024-05-14T16:46:43.560978+0000","last_became_peered":"2024-05-14T16:46:43.560978+0000","last_unstale":"2024-05-14T16:48:31.881404+0000","last_undegraded":"2024-05-14T16:48:31.881404+0000","last_fullsized":"2024-05-14T16:48:31.881404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:12:33.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.704946+0000","last_change":"2024-05-14T16:46:40.544017+0000","last_active":"2024-05-14T16:47:33.704946+0000","last_peered":"2024-05-14T16:47:33.704946+0000","last_clean":"2024-05-14T16:47:33.704946+0000","last_became_active":"2024-05-14T16:46:40.543176+0000","last_became_peered":"2024-05-14T16:46:40.543176+0000","last_unstale":"2024-05-14T16:47:33.704946+0000","last_undegraded":"2024-05-14T16:47:33.704946+0000","last_fullsized":"2024-05-14T16:47:33.704946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:09.817891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.882527+0000","last_change":"2024-05-14T16:46:43.564395+0000","last_active":"2024-05-14T16:48:31.882527+0000","last_peered":"2024-05-14T16:48:31.882527+0000","last_clean":"2024-05-14T16:48:31.882527+0000","last_became_active":"2024-05-14T16:46:43.564065+0000","last_became_peered":"2024-05-14T16:46:43.564065+0000","last_unstale":"2024-05-14T16:48:31.882527+0000","last_undegraded":"2024-05-14T16:48:31.882527+0000","last_fullsized":"2024-05-14T16:48:31.882527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:52.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.880631+0000","last_change":"2024-05-14T16:46:43.577067+0000","last_active":"2024-05-14T16:48:31.880631+0000","last_peered":"2024-05-14T16:48:31.880631+0000","last_clean":"2024-05-14T16:48:31.880631+0000","last_became_active":"2024-05-14T16:46:43.576750+0000","last_became_peered":"2024-05-14T16:46:43.576750+0000","last_unstale":"2024-05-14T16:48:31.880631+0000","last_undegraded":"2024-05-14T16:48:31.880631+0000","last_fullsized":"2024-05-14T16:48:31.880631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:25.961124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:40.763703+0000","last_change":"2024-05-14T16:46:40.855073+0000","last_active":"2024-05-14T16:48:40.763703+0000","last_peered":"2024-05-14T16:48:40.763703+0000","last_clean":"2024-05-14T16:48:40.763703+0000","last_became_active":"2024-05-14T16:46:40.854641+0000","last_became_peered":"2024-05-14T16:46:40.854641+0000","last_unstale":"2024-05-14T16:48:40.763703+0000","last_undegraded":"2024-05-14T16:48:40.763703+0000","last_fullsized":"2024-05-14T16:48:40.763703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:58:11.670926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892276+0000","last_change":"2024-05-14T16:46:43.569030+0000","last_active":"2024-05-14T16:48:31.892276+0000","last_peered":"2024-05-14T16:48:31.892276+0000","last_clean":"2024-05-14T16:48:31.892276+0000","last_became_active":"2024-05-14T16:46:43.568646+0000","last_became_peered":"2024-05-14T16:46:43.568646+0000","last_unstale":"2024-05-14T16:48:31.892276+0000","last_undegraded":"2024-05-14T16:48:31.892276+0000","last_fullsized":"2024-05-14T16:48:31.892276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:38.086818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.706973+0000","last_change":"2024-05-14T16:46:40.542624+0000","last_active":"2024-05-14T16:47:33.706973+0000","last_peered":"2024-05-14T16:47:33.706973+0000","last_clean":"2024-05-14T16:47:33.706973+0000","last_became_active":"2024-05-14T16:46:40.542230+0000","last_became_peered":"2024-05-14T16:46:40.542230+0000","last_unstale":"2024-05-14T16:47:33.706973+0000","last_undegraded":"2024-05-14T16:47:33.706973+0000","last_fullsized":"2024-05-14T16:47:33.706973+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:54:50.083362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.704708+0000","last_change":"2024-05-14T16:45:15.724994+0000","last_active":"2024-05-14T16:47:33.704708+0000","last_peered":"2024-05-14T16:47:33.704708+0000","last_clean":"2024-05-14T16:47:33.704708+0000","last_became_active":"2024-05-14T16:45:15.720080+0000","last_became_peered":"2024-05-14T16:45:15.720080+0000","last_unstale":"2024-05-14T16:47:33.704708+0000","last_undegraded":"2024-05-14T16:47:33.704708+0000","last_fullsized":"2024-05-14T16:47:33.704708+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_clean_scrub_stamp":"2024-05-14T16:41:54.615675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:32:43.569611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.874957+0000","last_change":"2024-05-14T16:46:43.567143+0000","last_active":"2024-05-14T16:48:31.874957+0000","last_peered":"2024-05-14T16:48:31.874957+0000","last_clean":"2024-05-14T16:48:31.874957+0000","last_became_active":"2024-05-14T16:46:43.564717+0000","last_became_peered":"2024-05-14T16:46:43.564717+0000","last_unstale":"2024-05-14T16:48:31.874957+0000","last_undegraded":"2024-05-14T16:48:31.874957+0000","last_fullsized":"2024-05-14T16:48:31.874957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:59.025279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.704764+0000","last_change":"2024-05-14T16:46:40.546709+0000","last_active":"2024-05-14T16:47:33.704764+0000","last_peered":"2024-05-14T16:47:33.704764+0000","last_clean":"2024-05-14T16:47:33.704764+0000","last_became_active":"2024-05-14T16:46:40.546295+0000","last_became_peered":"2024-05-14T16:46:40.546295+0000","last_unstale":"2024-05-14T16:47:33.704764+0000","last_undegraded":"2024-05-14T16:47:33.704764+0000","last_fullsized":"2024-05-14T16:47:33.704764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:42:50.255289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.710122+0000","last_change":"2024-05-14T16:46:40.856822+0000","last_active":"2024-05-14T16:47:33.710122+0000","last_peered":"2024-05-14T16:47:33.710122+0000","last_clean":"2024-05-14T16:47:33.710122+0000","last_became_active":"2024-05-14T16:46:40.856434+0000","last_became_peered":"2024-05-14T16:46:40.856434+0000","last_unstale":"2024-05-14T16:47:33.710122+0000","last_undegraded":"2024-05-14T16:47:33.710122+0000","last_fullsized":"2024-05-14T16:47:33.710122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:38.997679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.874412+0000","last_change":"2024-05-14T16:46:43.563918+0000","last_active":"2024-05-14T16:48:31.874412+0000","last_peered":"2024-05-14T16:48:31.874412+0000","last_clean":"2024-05-14T16:48:31.874412+0000","last_became_active":"2024-05-14T16:46:43.563527+0000","last_became_peered":"2024-05-14T16:46:43.563527+0000","last_unstale":"2024-05-14T16:48:31.874412+0000","last_undegraded":"2024-05-14T16:48:31.874412+0000","last_fullsized":"2024-05-14T16:48:31.874412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:58:52.984695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'12","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.705196+0000","last_change":"2024-05-14T16:46:40.547480+0000","last_active":"2024-05-14T16:47:33.705196+0000","last_peered":"2024-05-14T16:47:33.705196+0000","last_clean":"2024-05-14T16:47:33.705196+0000","last_became_active":"2024-05-14T16:46:40.547062+0000","last_became_peered":"2024-05-14T16:46:40.547062+0000","last_unstale":"2024-05-14T16:47:33.705196+0000","last_undegraded":"2024-05-14T16:47:33.705196+0000","last_fullsized":"2024-05-14T16:47:33.705196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:30.211421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881743+0000","last_change":"2024-05-14T16:46:43.563870+0000","last_active":"2024-05-14T16:48:31.881743+0000","last_peered":"2024-05-14T16:48:31.881743+0000","last_clean":"2024-05-14T16:48:31.881743+0000","last_became_active":"2024-05-14T16:46:43.563542+0000","last_became_peered":"2024-05-14T16:46:43.563542+0000","last_unstale":"2024-05-14T16:48:31.881743+0000","last_undegraded":"2024-05-14T16:48:31.881743+0000","last_fullsized":"2024-05-14T16:48:31.881743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:50.699601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.706652+0000","last_change":"2024-05-14T16:46:40.665192+0000","last_active":"2024-05-14T16:47:33.706652+0000","last_peered":"2024-05-14T16:47:33.706652+0000","last_clean":"2024-05-14T16:47:33.706652+0000","last_became_active":"2024-05-14T16:46:40.664827+0000","last_became_peered":"2024-05-14T16:46:40.664827+0000","last_unstale":"2024-05-14T16:47:33.706652+0000","last_undegraded":"2024-05-14T16:47:33.706652+0000","last_fullsized":"2024-05-14T16:47:33.706652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:41:47.179677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.882985+0000","last_change":"2024-05-14T16:47:33.717718+0000","last_active":"2024-05-14T16:48:31.882985+0000","last_peered":"2024-05-14T16:48:31.882985+0000","last_clean":"2024-05-14T16:48:31.882985+0000","last_became_active":"2024-05-14T16:47:33.717368+0000","last_became_peered":"2024-05-14T16:47:33.717368+0000","last_unstale":"2024-05-14T16:48:31.882985+0000","last_undegraded":"2024-05-14T16:48:31.882985+0000","last_fullsized":"2024-05-14T16:48:31.882985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:07:43.990388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.704714+0000","last_change":"2024-05-14T16:46:40.855458+0000","last_active":"2024-05-14T16:47:33.704714+0000","last_peered":"2024-05-14T16:47:33.704714+0000","last_clean":"2024-05-14T16:47:33.704714+0000","last_became_active":"2024-05-14T16:46:40.855122+0000","last_became_peered":"2024-05-14T16:46:40.855122+0000","last_unstale":"2024-05-14T16:47:33.704714+0000","last_undegraded":"2024-05-14T16:47:33.704714+0000","last_fullsized":"2024-05-14T16:47:33.704714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:20:43.435860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.854946+0000","last_change":"2024-05-14T16:46:43.566482+0000","last_active":"2024-05-14T16:48:31.854946+0000","last_peered":"2024-05-14T16:48:31.854946+0000","last_clean":"2024-05-14T16:48:31.854946+0000","last_became_active":"2024-05-14T16:46:43.566145+0000","last_became_peered":"2024-05-14T16:46:43.566145+0000","last_unstale":"2024-05-14T16:48:31.854946+0000","last_undegraded":"2024-05-14T16:48:31.854946+0000","last_fullsized":"2024-05-14T16:48:31.854946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:14:22.731629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.701729+0000","last_change":"2024-05-14T16:46:40.541655+0000","last_active":"2024-05-14T16:47:33.701729+0000","last_peered":"2024-05-14T16:47:33.701729+0000","last_clean":"2024-05-14T16:47:33.701729+0000","last_became_active":"2024-05-14T16:46:40.541331+0000","last_became_peered":"2024-05-14T16:46:40.541331+0000","last_unstale":"2024-05-14T16:47:33.701729+0000","last_undegraded":"2024-05-14T16:47:33.701729+0000","last_fullsized":"2024-05-14T16:47:33.701729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:47:27.950532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.889088+0000","last_change":"2024-05-14T16:46:43.564216+0000","last_active":"2024-05-14T16:48:31.889088+0000","last_peered":"2024-05-14T16:48:31.889088+0000","last_clean":"2024-05-14T16:48:31.889088+0000","last_became_active":"2024-05-14T16:46:43.563894+0000","last_became_peered":"2024-05-14T16:46:43.563894+0000","last_unstale":"2024-05-14T16:48:31.889088+0000","last_undegraded":"2024-05-14T16:48:31.889088+0000","last_fullsized":"2024-05-14T16:48:31.889088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:16:09.256909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.705831+0000","last_change":"2024-05-14T16:46:40.551034+0000","last_active":"2024-05-14T16:47:33.705831+0000","last_peered":"2024-05-14T16:47:33.705831+0000","last_clean":"2024-05-14T16:47:33.705831+0000","last_became_active":"2024-05-14T16:46:40.550211+0000","last_became_peered":"2024-05-14T16:46:40.550211+0000","last_unstale":"2024-05-14T16:47:33.705831+0000","last_undegraded":"2024-05-14T16:47:33.705831+0000","last_fullsized":"2024-05-14T16:47:33.705831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:15.720942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.887108+0000","last_change":"2024-05-14T16:46:43.558264+0000","last_active":"2024-05-14T16:48:31.887108+0000","last_peered":"2024-05-14T16:48:31.887108+0000","last_clean":"2024-05-14T16:48:31.887108+0000","last_became_active":"2024-05-14T16:46:43.557907+0000","last_became_peered":"2024-05-14T16:46:43.557907+0000","last_unstale":"2024-05-14T16:48:31.887108+0000","last_undegraded":"2024-05-14T16:48:31.887108+0000","last_fullsized":"2024-05-14T16:48:31.887108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:20:52.752482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.709461+0000","last_change":"2024-05-14T16:46:40.551013+0000","last_active":"2024-05-14T16:47:33.709461+0000","last_peered":"2024-05-14T16:47:33.709461+0000","last_clean":"2024-05-14T16:47:33.709461+0000","last_became_active":"2024-05-14T16:46:40.550658+0000","last_became_peered":"2024-05-14T16:46:40.550658+0000","last_unstale":"2024-05-14T16:47:33.709461+0000","last_undegraded":"2024-05-14T16:47:33.709461+0000","last_fullsized":"2024-05-14T16:47:33.709461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:36.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.891455+0000","last_change":"2024-05-14T16:46:43.572270+0000","last_active":"2024-05-14T16:48:31.891455+0000","last_peered":"2024-05-14T16:48:31.891455+0000","last_clean":"2024-05-14T16:48:31.891455+0000","last_became_active":"2024-05-14T16:46:43.571886+0000","last_became_peered":"2024-05-14T16:46:43.571886+0000","last_unstale":"2024-05-14T16:48:31.891455+0000","last_undegraded":"2024-05-14T16:48:31.891455+0000","last_fullsized":"2024-05-14T16:48:31.891455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:11.890881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.715246+0000","last_change":"2024-05-14T16:47:33.715246+0000","last_active":"2024-05-14T16:47:33.715246+0000","last_peered":"2024-05-14T16:47:33.715246+0000","last_clean":"2024-05-14T16:47:33.715246+0000","last_became_active":"2024-05-14T16:47:33.708725+0000","last_became_peered":"2024-05-14T16:47:33.708725+0000","last_unstale":"2024-05-14T16:47:33.715246+0000","last_undegraded":"2024-05-14T16:47:33.715246+0000","last_fullsized":"2024-05-14T16:47:33.715246+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:19.118454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.891367+0000","last_change":"2024-05-14T16:46:43.572455+0000","last_active":"2024-05-14T16:48:31.891367+0000","last_peered":"2024-05-14T16:48:31.891367+0000","last_clean":"2024-05-14T16:48:31.891367+0000","last_became_active":"2024-05-14T16:46:43.571930+0000","last_became_peered":"2024-05-14T16:46:43.571930+0000","last_unstale":"2024-05-14T16:48:31.891367+0000","last_undegraded":"2024-05-14T16:48:31.891367+0000","last_fullsized":"2024-05-14T16:48:31.891367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:35:39.643239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.702270+0000","last_change":"2024-05-14T16:46:40.541708+0000","last_active":"2024-05-14T16:47:33.702270+0000","last_peered":"2024-05-14T16:47:33.702270+0000","last_clean":"2024-05-14T16:47:33.702270+0000","last_became_active":"2024-05-14T16:46:40.541202+0000","last_became_peered":"2024-05-14T16:46:40.541202+0000","last_unstale":"2024-05-14T16:47:33.702270+0000","last_undegraded":"2024-05-14T16:47:33.702270+0000","last_fullsized":"2024-05-14T16:47:33.702270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:50:01.821013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.883443+0000","last_change":"2024-05-14T16:46:43.574039+0000","last_active":"2024-05-14T16:48:31.883443+0000","last_peered":"2024-05-14T16:48:31.883443+0000","last_clean":"2024-05-14T16:48:31.883443+0000","last_became_active":"2024-05-14T16:46:43.573237+0000","last_became_peered":"2024-05-14T16:46:43.573237+0000","last_unstale":"2024-05-14T16:48:31.883443+0000","last_undegraded":"2024-05-14T16:48:31.883443+0000","last_fullsized":"2024-05-14T16:48:31.883443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:13.234495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.716565+0000","last_change":"2024-05-14T16:46:40.549519+0000","last_active":"2024-05-14T16:47:33.716565+0000","last_peered":"2024-05-14T16:47:33.716565+0000","last_clean":"2024-05-14T16:47:33.716565+0000","last_became_active":"2024-05-14T16:46:40.549169+0000","last_became_peered":"2024-05-14T16:46:40.549169+0000","last_unstale":"2024-05-14T16:47:33.716565+0000","last_undegraded":"2024-05-14T16:47:33.716565+0000","last_fullsized":"2024-05-14T16:47:33.716565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:22.559724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.880573+0000","last_change":"2024-05-14T16:46:43.567674+0000","last_active":"2024-05-14T16:48:31.880573+0000","last_peered":"2024-05-14T16:48:31.880573+0000","last_clean":"2024-05-14T16:48:31.880573+0000","last_became_active":"2024-05-14T16:46:43.567337+0000","last_became_peered":"2024-05-14T16:46:43.567337+0000","last_unstale":"2024-05-14T16:48:31.880573+0000","last_undegraded":"2024-05-14T16:48:31.880573+0000","last_fullsized":"2024-05-14T16:48:31.880573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:02.321955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.883562+0000","last_change":"2024-05-14T16:46:43.567627+0000","last_active":"2024-05-14T16:48:31.883562+0000","last_peered":"2024-05-14T16:48:31.883562+0000","last_clean":"2024-05-14T16:48:31.883562+0000","last_became_active":"2024-05-14T16:46:43.567313+0000","last_became_peered":"2024-05-14T16:46:43.567313+0000","last_unstale":"2024-05-14T16:48:31.883562+0000","last_undegraded":"2024-05-14T16:48:31.883562+0000","last_fullsized":"2024-05-14T16:48:31.883562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:47.396923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.884887+0000","last_change":"2024-05-14T16:47:33.708354+0000","last_active":"2024-05-14T16:48:31.884887+0000","last_peered":"2024-05-14T16:48:31.884887+0000","last_clean":"2024-05-14T16:48:31.884887+0000","last_became_active":"2024-05-14T16:47:33.707842+0000","last_became_peered":"2024-05-14T16:47:33.707842+0000","last_unstale":"2024-05-14T16:48:31.884887+0000","last_undegraded":"2024-05-14T16:48:31.884887+0000","last_fullsized":"2024-05-14T16:48:31.884887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:56:26.845782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.884217+0000","last_change":"2024-05-14T16:47:33.708332+0000","last_active":"2024-05-14T16:48:31.884217+0000","last_peered":"2024-05-14T16:48:31.884217+0000","last_clean":"2024-05-14T16:48:31.884217+0000","last_became_active":"2024-05-14T16:47:33.707774+0000","last_became_peered":"2024-05-14T16:47:33.707774+0000","last_unstale":"2024-05-14T16:48:31.884217+0000","last_undegraded":"2024-05-14T16:48:31.884217+0000","last_fullsized":"2024-05-14T16:48:31.884217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:44:47.699303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.883799+0000","last_change":"2024-05-14T16:46:43.572168+0000","last_active":"2024-05-14T16:48:31.883799+0000","last_peered":"2024-05-14T16:48:31.883799+0000","last_clean":"2024-05-14T16:48:31.883799+0000","last_became_active":"2024-05-14T16:46:43.571822+0000","last_became_peered":"2024-05-14T16:46:43.571822+0000","last_unstale":"2024-05-14T16:48:31.883799+0000","last_undegraded":"2024-05-14T16:48:31.883799+0000","last_fullsized":"2024-05-14T16:48:31.883799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:57.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.884631+0000","last_change":"2024-05-14T16:46:43.570935+0000","last_active":"2024-05-14T16:48:31.884631+0000","last_peered":"2024-05-14T16:48:31.884631+0000","last_clean":"2024-05-14T16:48:31.884631+0000","last_became_active":"2024-05-14T16:46:43.570605+0000","last_became_peered":"2024-05-14T16:46:43.570605+0000","last_unstale":"2024-05-14T16:48:31.884631+0000","last_undegraded":"2024-05-14T16:48:31.884631+0000","last_fullsized":"2024-05-14T16:48:31.884631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:17:55.161195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.878325+0000","last_change":"2024-05-14T16:46:43.568442+0000","last_active":"2024-05-14T16:48:31.878325+0000","last_peered":"2024-05-14T16:48:31.878325+0000","last_clean":"2024-05-14T16:48:31.878325+0000","last_became_active":"2024-05-14T16:46:43.568118+0000","last_became_peered":"2024-05-14T16:46:43.568118+0000","last_unstale":"2024-05-14T16:48:31.878325+0000","last_undegraded":"2024-05-14T16:48:31.878325+0000","last_fullsized":"2024-05-14T16:48:31.878325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:52.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.891386+0000","last_change":"2024-05-14T16:46:43.576174+0000","last_active":"2024-05-14T16:48:31.891386+0000","last_peered":"2024-05-14T16:48:31.891386+0000","last_clean":"2024-05-14T16:48:31.891386+0000","last_became_active":"2024-05-14T16:46:43.575737+0000","last_became_peered":"2024-05-14T16:46:43.575737+0000","last_unstale":"2024-05-14T16:48:31.891386+0000","last_undegraded":"2024-05-14T16:48:31.891386+0000","last_fullsized":"2024-05-14T16:48:31.891386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:08.453070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.878931+0000","last_change":"2024-05-14T16:46:43.566256+0000","last_active":"2024-05-14T16:48:31.878931+0000","last_peered":"2024-05-14T16:48:31.878931+0000","last_clean":"2024-05-14T16:48:31.878931+0000","last_became_active":"2024-05-14T16:46:43.565886+0000","last_became_peered":"2024-05-14T16:46:43.565886+0000","last_unstale":"2024-05-14T16:48:31.878931+0000","last_undegraded":"2024-05-14T16:48:31.878931+0000","last_fullsized":"2024-05-14T16:48:31.878931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:50.021286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881134+0000","last_change":"2024-05-14T16:46:43.569819+0000","last_active":"2024-05-14T16:48:31.881134+0000","last_peered":"2024-05-14T16:48:31.881134+0000","last_clean":"2024-05-14T16:48:31.881134+0000","last_became_active":"2024-05-14T16:46:43.569461+0000","last_became_peered":"2024-05-14T16:46:43.569461+0000","last_unstale":"2024-05-14T16:48:31.881134+0000","last_undegraded":"2024-05-14T16:48:31.881134+0000","last_fullsized":"2024-05-14T16:48:31.881134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:52.013604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.877968+0000","last_change":"2024-05-14T16:46:43.567569+0000","last_active":"2024-05-14T16:48:31.877968+0000","last_peered":"2024-05-14T16:48:31.877968+0000","last_clean":"2024-05-14T16:48:31.877968+0000","last_became_active":"2024-05-14T16:46:43.564808+0000","last_became_peered":"2024-05-14T16:46:43.564808+0000","last_unstale":"2024-05-14T16:48:31.877968+0000","last_undegraded":"2024-05-14T16:48:31.877968+0000","last_fullsized":"2024-05-14T16:48:31.877968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:13.744730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892610+0000","last_change":"2024-05-14T16:46:43.578417+0000","last_active":"2024-05-14T16:48:31.892610+0000","last_peered":"2024-05-14T16:48:31.892610+0000","last_clean":"2024-05-14T16:48:31.892610+0000","last_became_active":"2024-05-14T16:46:43.577687+0000","last_became_peered":"2024-05-14T16:46:43.577687+0000","last_unstale":"2024-05-14T16:48:31.892610+0000","last_undegraded":"2024-05-14T16:48:31.892610+0000","last_fullsized":"2024-05-14T16:48:31.892610+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:32.060934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.884899+0000","last_change":"2024-05-14T16:46:43.564114+0000","last_active":"2024-05-14T16:48:31.884899+0000","last_peered":"2024-05-14T16:48:31.884899+0000","last_clean":"2024-05-14T16:48:31.884899+0000","last_became_active":"2024-05-14T16:46:43.563734+0000","last_became_peered":"2024-05-14T16:46:43.563734+0000","last_unstale":"2024-05-14T16:48:31.884899+0000","last_undegraded":"2024-05-14T16:48:31.884899+0000","last_fullsized":"2024-05-14T16:48:31.884899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:19.016741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.883035+0000","last_change":"2024-05-14T16:46:43.566490+0000","last_active":"2024-05-14T16:48:31.883035+0000","last_peered":"2024-05-14T16:48:31.883035+0000","last_clean":"2024-05-14T16:48:31.883035+0000","last_became_active":"2024-05-14T16:46:43.566086+0000","last_became_peered":"2024-05-14T16:46:43.566086+0000","last_unstale":"2024-05-14T16:48:31.883035+0000","last_undegraded":"2024-05-14T16:48:31.883035+0000","last_fullsized":"2024-05-14T16:48:31.883035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:39:09.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245266,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65609,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":611,"num_read_kb":376,"num_write":169,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":728,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710760,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":425363,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906988,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1078697,"data_compressed":41531,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070514,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":425453,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266743,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":425261,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":1016912,"data_compressed":41531,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":424842,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":486634,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":426113,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":487368,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":425153,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":425861,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822726,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":1016596,"data_compressed":41531,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:48:43.057 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:48:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:43.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:43 smithi069 ceph-mon[31507]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T16:48:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:43 smithi069 ceph-mon[31507]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:48:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:43.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:43 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:43.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:43 smithi203 ceph-mon[35787]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T16:48:43.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:43 smithi203 ceph-mon[35787]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:48:43.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:43.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:43 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:44.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:43 smithi092 ceph-mon[35763]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T16:48:44.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:43 smithi092 ceph-mon[35763]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:48:44.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:44.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:43 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:44.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:44 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:44.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:44 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:44.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:44 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.937+0000 7f1a8652f700 1 -- 172.21.15.69:0/38664798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a80102e30 msgr2=0x7f1a8010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.937+0000 7f1a8652f700 1 --2- 172.21.15.69:0/38664798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a80102e30 0x7f1a8010f2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800b0a0 tx=0x7f1a7800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.937+0000 7f1a8652f700 1 -- 172.21.15.69:0/38664798 shutdown_connections 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.937+0000 7f1a8652f700 1 --2- 172.21.15.69:0/38664798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a80102e30 0x7f1a8010f2f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.937+0000 7f1a8652f700 1 --2- 172.21.15.69:0/38664798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a801084b0 0x7f1a80108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.937+0000 7f1a8652f700 1 --2- 172.21.15.69:0/38664798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80102470 0x7f1a801028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.937+0000 7f1a8652f700 1 -- 172.21.15.69:0/38664798 >> 172.21.15.69:0/38664798 conn(0x7f1a800784b0 msgr2=0x7f1a800788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.937+0000 7f1a8652f700 1 -- 172.21.15.69:0/38664798 shutdown_connections 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.937+0000 7f1a8652f700 1 -- 172.21.15.69:0/38664798 wait complete. 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.938+0000 7f1a8652f700 1 Processor -- start 2024-05-14T16:48:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.938+0000 7f1a8652f700 1 -- start start 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.938+0000 7f1a8652f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80102470 0x7f1a80104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.938+0000 7f1a8652f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a80102e30 0x7f1a801045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.938+0000 7f1a8652f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a801084b0 0x7f1a80104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.938+0000 7f1a8652f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80112640 con 0x7f1a801084b0 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.938+0000 7f1a8652f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801124c0 con 0x7f1a80102470 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.938+0000 7f1a8652f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801127c0 con 0x7f1a80102e30 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.938+0000 7f1a84acc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a801084b0 0x7f1a80104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a84acc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a801084b0 0x7f1a80104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56862/0 (socket says 172.21.15.69:56862) 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a84acc700 1 -- 172.21.15.69:0/847806260 learned_addr learned my addr 172.21.15.69:0/847806260 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a7f7fe700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a80102e30 0x7f1a801045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a7ffff700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80102470 0x7f1a80104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a84acc700 1 -- 172.21.15.69:0/847806260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a80102e30 msgr2=0x7f1a801045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a84acc700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a80102e30 0x7f1a801045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a84acc700 1 -- 172.21.15.69:0/847806260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80102470 msgr2=0x7f1a80104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a84acc700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80102470 0x7f1a80104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a84acc700 1 -- 172.21.15.69:0/847806260 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a801a80b0 con 0x7f1a801084b0 2024-05-14T16:48:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a84acc700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a801084b0 0x7f1a80104b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1a780062a0 tx=0x7f1a78004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a7d7fa700 1 -- 172.21.15.69:0/847806260 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78021070 con 0x7f1a801084b0 2024-05-14T16:48:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a8652f700 1 -- 172.21.15.69:0/847806260 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a801a8340 con 0x7f1a801084b0 2024-05-14T16:48:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a7d7fa700 1 -- 172.21.15.69:0/847806260 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a78007c80 con 0x7f1a801084b0 2024-05-14T16:48:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a7d7fa700 1 -- 172.21.15.69:0/847806260 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7801d450 con 0x7f1a801084b0 2024-05-14T16:48:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.939+0000 7f1a8652f700 1 -- 172.21.15.69:0/847806260 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a801a87f0 con 0x7f1a801084b0 2024-05-14T16:48:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.940+0000 7f1a8652f700 1 -- 172.21.15.69:0/847806260 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a80103b60 con 0x7f1a801084b0 2024-05-14T16:48:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.942+0000 7f1a7d7fa700 1 -- 172.21.15.69:0/847806260 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1a78004000 con 0x7f1a801084b0 2024-05-14T16:48:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.942+0000 7f1a7d7fa700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1a680b2880 0x7f1a680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.942+0000 7f1a7ffff700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1a680b2880 0x7f1a680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.942+0000 7f1a7d7fa700 1 -- 172.21.15.69:0/847806260 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1a7800f020 con 0x7f1a801084b0 2024-05-14T16:48:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.943+0000 7f1a7ffff700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1a680b2880 0x7f1a680b4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a70005ea0 tx=0x7f1a70005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:44.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:44.946+0000 7f1a7d7fa700 1 -- 172.21.15.69:0/847806260 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a7802a020 con 0x7f1a801084b0 2024-05-14T16:48:44.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:44 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:44.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:44 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:44.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:44 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:45.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:44 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:45.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:44 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:45.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:44 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:48:45.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.145+0000 7f1a8652f700 1 -- 172.21.15.69:0/847806260 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a80063980 con 0x7f1a680b2880 2024-05-14T16:48:45.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.152+0000 7f1a7d7fa700 1 -- 172.21.15.69:0/847806260 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259062 (secure 0 0 0) 0x7f1a80063980 con 0x7f1a680b2880 2024-05-14T16:48:45.156 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:48:45.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 -- 172.21.15.69:0/847806260 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1a680b2880 msgr2=0x7f1a680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1a680b2880 0x7f1a680b4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a70005ea0 tx=0x7f1a70005e30 comp rx=0 tx=0).stop 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 -- 172.21.15.69:0/847806260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a801084b0 msgr2=0x7f1a80104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a801084b0 0x7f1a80104b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1a780062a0 tx=0x7f1a78004e20 comp rx=0 tx=0).stop 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 -- 172.21.15.69:0/847806260 shutdown_connections 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1a680b2880 0x7f1a680b4d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a801084b0 0x7f1a80104b00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a80102470 0x7f1a80104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 --2- 172.21.15.69:0/847806260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a80102e30 0x7f1a801045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 -- 172.21.15.69:0/847806260 >> 172.21.15.69:0/847806260 conn(0x7f1a800784b0 msgr2=0x7f1a8006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 -- 172.21.15.69:0/847806260 shutdown_connections 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:45.156+0000 7f1a8652f700 1 -- 172.21.15.69:0/847806260 wait complete. 2024-05-14T16:48:45.158 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-05-14T16:48:45.842 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-05-14T16:48:43.887589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656211,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":717,"num_read_kb":455,"num_write":294,"num_write_kb":2266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952568,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":737,"ondisk_log_size":737,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351488,"kb_used_data":10656,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590336,"statfs":{"total":1151940427776,"available":1151580504064,"internally_reserved":0,"allocated":10911744,"data_stored":7064253,"data_compressed":176766,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":41,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003007"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.890772+0000","last_change":"2024-05-14T16:46:43.572738+0000","last_active":"2024-05-14T16:48:31.890772+0000","last_peered":"2024-05-14T16:48:31.890772+0000","last_clean":"2024-05-14T16:48:31.890772+0000","last_became_active":"2024-05-14T16:46:43.572400+0000","last_became_peered":"2024-05-14T16:46:43.572400+0000","last_unstale":"2024-05-14T16:48:31.890772+0000","last_undegraded":"2024-05-14T16:48:31.890772+0000","last_fullsized":"2024-05-14T16:48:31.890772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:44.332862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.888529+0000","last_change":"2024-05-14T16:46:43.566193+0000","last_active":"2024-05-14T16:48:31.888529+0000","last_peered":"2024-05-14T16:48:31.888529+0000","last_clean":"2024-05-14T16:48:31.888529+0000","last_became_active":"2024-05-14T16:46:43.565853+0000","last_became_peered":"2024-05-14T16:46:43.565853+0000","last_unstale":"2024-05-14T16:48:31.888529+0000","last_undegraded":"2024-05-14T16:48:31.888529+0000","last_fullsized":"2024-05-14T16:48:31.888529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:19:34.481225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.890380+0000","last_change":"2024-05-14T16:46:43.574937+0000","last_active":"2024-05-14T16:48:31.890380+0000","last_peered":"2024-05-14T16:48:31.890380+0000","last_clean":"2024-05-14T16:48:31.890380+0000","last_became_active":"2024-05-14T16:46:43.574317+0000","last_became_peered":"2024-05-14T16:46:43.574317+0000","last_unstale":"2024-05-14T16:48:31.890380+0000","last_undegraded":"2024-05-14T16:48:31.890380+0000","last_fullsized":"2024-05-14T16:48:31.890380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:35:49.933658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.885735+0000","last_change":"2024-05-14T16:47:33.711809+0000","last_active":"2024-05-14T16:48:31.885735+0000","last_peered":"2024-05-14T16:48:31.885735+0000","last_clean":"2024-05-14T16:48:31.885735+0000","last_became_active":"2024-05-14T16:47:33.711422+0000","last_became_peered":"2024-05-14T16:47:33.711422+0000","last_unstale":"2024-05-14T16:48:31.885735+0000","last_undegraded":"2024-05-14T16:48:31.885735+0000","last_fullsized":"2024-05-14T16:48:31.885735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:44:26.005189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.875757+0000","last_change":"2024-05-14T16:46:43.567104+0000","last_active":"2024-05-14T16:48:31.875757+0000","last_peered":"2024-05-14T16:48:31.875757+0000","last_clean":"2024-05-14T16:48:31.875757+0000","last_became_active":"2024-05-14T16:46:43.566762+0000","last_became_peered":"2024-05-14T16:46:43.566762+0000","last_unstale":"2024-05-14T16:48:31.875757+0000","last_undegraded":"2024-05-14T16:48:31.875757+0000","last_fullsized":"2024-05-14T16:48:31.875757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:08.819856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.875768+0000","last_change":"2024-05-14T16:46:43.565055+0000","last_active":"2024-05-14T16:48:31.875768+0000","last_peered":"2024-05-14T16:48:31.875768+0000","last_clean":"2024-05-14T16:48:31.875768+0000","last_became_active":"2024-05-14T16:46:43.564699+0000","last_became_peered":"2024-05-14T16:46:43.564699+0000","last_unstale":"2024-05-14T16:48:31.875768+0000","last_undegraded":"2024-05-14T16:48:31.875768+0000","last_fullsized":"2024-05-14T16:48:31.875768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:01.903025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.871869+0000","last_change":"2024-05-14T16:47:33.711866+0000","last_active":"2024-05-14T16:48:31.871869+0000","last_peered":"2024-05-14T16:48:31.871869+0000","last_clean":"2024-05-14T16:48:31.871869+0000","last_became_active":"2024-05-14T16:47:33.711481+0000","last_became_peered":"2024-05-14T16:47:33.711481+0000","last_unstale":"2024-05-14T16:48:31.871869+0000","last_undegraded":"2024-05-14T16:48:31.871869+0000","last_fullsized":"2024-05-14T16:48:31.871869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:01:22.724678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.887357+0000","last_change":"2024-05-14T16:46:43.564470+0000","last_active":"2024-05-14T16:48:31.887357+0000","last_peered":"2024-05-14T16:48:31.887357+0000","last_clean":"2024-05-14T16:48:31.887357+0000","last_became_active":"2024-05-14T16:46:43.564169+0000","last_became_peered":"2024-05-14T16:46:43.564169+0000","last_unstale":"2024-05-14T16:48:31.887357+0000","last_undegraded":"2024-05-14T16:48:31.887357+0000","last_fullsized":"2024-05-14T16:48:31.887357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:38.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892107+0000","last_change":"2024-05-14T16:46:43.580988+0000","last_active":"2024-05-14T16:48:31.892107+0000","last_peered":"2024-05-14T16:48:31.892107+0000","last_clean":"2024-05-14T16:48:31.892107+0000","last_became_active":"2024-05-14T16:46:43.580657+0000","last_became_peered":"2024-05-14T16:46:43.580657+0000","last_unstale":"2024-05-14T16:48:31.892107+0000","last_undegraded":"2024-05-14T16:48:31.892107+0000","last_fullsized":"2024-05-14T16:48:31.892107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:39:01.568822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.884437+0000","last_change":"2024-05-14T16:46:43.565742+0000","last_active":"2024-05-14T16:48:31.884437+0000","last_peered":"2024-05-14T16:48:31.884437+0000","last_clean":"2024-05-14T16:48:31.884437+0000","last_became_active":"2024-05-14T16:46:43.565260+0000","last_became_peered":"2024-05-14T16:46:43.565260+0000","last_unstale":"2024-05-14T16:48:31.884437+0000","last_undegraded":"2024-05-14T16:48:31.884437+0000","last_fullsized":"2024-05-14T16:48:31.884437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:25.718174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.891385+0000","last_change":"2024-05-14T16:46:43.576268+0000","last_active":"2024-05-14T16:48:31.891385+0000","last_peered":"2024-05-14T16:48:31.891385+0000","last_clean":"2024-05-14T16:48:31.891385+0000","last_became_active":"2024-05-14T16:46:43.575856+0000","last_became_peered":"2024-05-14T16:46:43.575856+0000","last_unstale":"2024-05-14T16:48:31.891385+0000","last_undegraded":"2024-05-14T16:48:31.891385+0000","last_fullsized":"2024-05-14T16:48:31.891385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:12:16.549127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.882004+0000","last_change":"2024-05-14T16:46:43.573315+0000","last_active":"2024-05-14T16:48:31.882004+0000","last_peered":"2024-05-14T16:48:31.882004+0000","last_clean":"2024-05-14T16:48:31.882004+0000","last_became_active":"2024-05-14T16:46:43.572981+0000","last_became_peered":"2024-05-14T16:46:43.572981+0000","last_unstale":"2024-05-14T16:48:31.882004+0000","last_undegraded":"2024-05-14T16:48:31.882004+0000","last_fullsized":"2024-05-14T16:48:31.882004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:07.757539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.885637+0000","last_change":"2024-05-14T16:46:43.569388+0000","last_active":"2024-05-14T16:48:31.885637+0000","last_peered":"2024-05-14T16:48:31.885637+0000","last_clean":"2024-05-14T16:48:31.885637+0000","last_became_active":"2024-05-14T16:46:43.568992+0000","last_became_peered":"2024-05-14T16:46:43.568992+0000","last_unstale":"2024-05-14T16:48:31.885637+0000","last_undegraded":"2024-05-14T16:48:31.885637+0000","last_fullsized":"2024-05-14T16:48:31.885637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:08:06.236797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.865859+0000","last_change":"2024-05-14T16:46:43.564137+0000","last_active":"2024-05-14T16:48:31.865859+0000","last_peered":"2024-05-14T16:48:31.865859+0000","last_clean":"2024-05-14T16:48:31.865859+0000","last_became_active":"2024-05-14T16:46:43.563831+0000","last_became_peered":"2024-05-14T16:46:43.563831+0000","last_unstale":"2024-05-14T16:48:31.865859+0000","last_undegraded":"2024-05-14T16:48:31.865859+0000","last_fullsized":"2024-05-14T16:48:31.865859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:58.473863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.879484+0000","last_change":"2024-05-14T16:46:43.571789+0000","last_active":"2024-05-14T16:48:31.879484+0000","last_peered":"2024-05-14T16:48:31.879484+0000","last_clean":"2024-05-14T16:48:31.879484+0000","last_became_active":"2024-05-14T16:46:43.571521+0000","last_became_peered":"2024-05-14T16:46:43.571521+0000","last_unstale":"2024-05-14T16:48:31.879484+0000","last_undegraded":"2024-05-14T16:48:31.879484+0000","last_fullsized":"2024-05-14T16:48:31.879484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:23.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892635+0000","last_change":"2024-05-14T16:46:43.569532+0000","last_active":"2024-05-14T16:48:31.892635+0000","last_peered":"2024-05-14T16:48:31.892635+0000","last_clean":"2024-05-14T16:48:31.892635+0000","last_became_active":"2024-05-14T16:46:43.569132+0000","last_became_peered":"2024-05-14T16:46:43.569132+0000","last_unstale":"2024-05-14T16:48:31.892635+0000","last_undegraded":"2024-05-14T16:48:31.892635+0000","last_fullsized":"2024-05-14T16:48:31.892635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:09.328241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892354+0000","last_change":"2024-05-14T16:46:43.574451+0000","last_active":"2024-05-14T16:48:31.892354+0000","last_peered":"2024-05-14T16:48:31.892354+0000","last_clean":"2024-05-14T16:48:31.892354+0000","last_became_active":"2024-05-14T16:46:43.574143+0000","last_became_peered":"2024-05-14T16:46:43.574143+0000","last_unstale":"2024-05-14T16:48:31.892354+0000","last_undegraded":"2024-05-14T16:48:31.892354+0000","last_fullsized":"2024-05-14T16:48:31.892354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:01:42.831262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.890687+0000","last_change":"2024-05-14T16:46:43.568799+0000","last_active":"2024-05-14T16:48:31.890687+0000","last_peered":"2024-05-14T16:48:31.890687+0000","last_clean":"2024-05-14T16:48:31.890687+0000","last_became_active":"2024-05-14T16:46:43.568467+0000","last_became_peered":"2024-05-14T16:46:43.568467+0000","last_unstale":"2024-05-14T16:48:31.890687+0000","last_undegraded":"2024-05-14T16:48:31.890687+0000","last_fullsized":"2024-05-14T16:48:31.890687+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:59.036372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.878469+0000","last_change":"2024-05-14T16:46:43.566879+0000","last_active":"2024-05-14T16:48:31.878469+0000","last_peered":"2024-05-14T16:48:31.878469+0000","last_clean":"2024-05-14T16:48:31.878469+0000","last_became_active":"2024-05-14T16:46:43.566346+0000","last_became_peered":"2024-05-14T16:46:43.566346+0000","last_unstale":"2024-05-14T16:48:31.878469+0000","last_undegraded":"2024-05-14T16:48:31.878469+0000","last_fullsized":"2024-05-14T16:48:31.878469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:11.027173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.705064+0000","last_change":"2024-05-14T16:46:40.546990+0000","last_active":"2024-05-14T16:47:33.705064+0000","last_peered":"2024-05-14T16:47:33.705064+0000","last_clean":"2024-05-14T16:47:33.705064+0000","last_became_active":"2024-05-14T16:46:40.546646+0000","last_became_peered":"2024-05-14T16:46:40.546646+0000","last_unstale":"2024-05-14T16:47:33.705064+0000","last_undegraded":"2024-05-14T16:47:33.705064+0000","last_fullsized":"2024-05-14T16:47:33.705064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:47.001760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.876111+0000","last_change":"2024-05-14T16:46:43.570352+0000","last_active":"2024-05-14T16:48:31.876111+0000","last_peered":"2024-05-14T16:48:31.876111+0000","last_clean":"2024-05-14T16:48:31.876111+0000","last_became_active":"2024-05-14T16:46:43.569748+0000","last_became_peered":"2024-05-14T16:46:43.569748+0000","last_unstale":"2024-05-14T16:48:31.876111+0000","last_undegraded":"2024-05-14T16:48:31.876111+0000","last_fullsized":"2024-05-14T16:48:31.876111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:24.707586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.702199+0000","last_change":"2024-05-14T16:46:40.855034+0000","last_active":"2024-05-14T16:47:33.702199+0000","last_peered":"2024-05-14T16:47:33.702199+0000","last_clean":"2024-05-14T16:47:33.702199+0000","last_became_active":"2024-05-14T16:46:40.854686+0000","last_became_peered":"2024-05-14T16:46:40.854686+0000","last_unstale":"2024-05-14T16:47:33.702199+0000","last_undegraded":"2024-05-14T16:47:33.702199+0000","last_fullsized":"2024-05-14T16:47:33.702199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:56.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.890398+0000","last_change":"2024-05-14T16:46:43.574682+0000","last_active":"2024-05-14T16:48:31.890398+0000","last_peered":"2024-05-14T16:48:31.890398+0000","last_clean":"2024-05-14T16:48:31.890398+0000","last_became_active":"2024-05-14T16:46:43.574322+0000","last_became_peered":"2024-05-14T16:46:43.574322+0000","last_unstale":"2024-05-14T16:48:31.890398+0000","last_undegraded":"2024-05-14T16:48:31.890398+0000","last_fullsized":"2024-05-14T16:48:31.890398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:20.169428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.710333+0000","last_change":"2024-05-14T16:46:40.551307+0000","last_active":"2024-05-14T16:47:33.710333+0000","last_peered":"2024-05-14T16:47:33.710333+0000","last_clean":"2024-05-14T16:47:33.710333+0000","last_became_active":"2024-05-14T16:46:40.550987+0000","last_became_peered":"2024-05-14T16:46:40.550987+0000","last_unstale":"2024-05-14T16:47:33.710333+0000","last_undegraded":"2024-05-14T16:47:33.710333+0000","last_fullsized":"2024-05-14T16:47:33.710333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:27.819457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.920360+0000","last_change":"2024-05-14T16:46:43.575200+0000","last_active":"2024-05-14T16:48:31.920360+0000","last_peered":"2024-05-14T16:48:31.920360+0000","last_clean":"2024-05-14T16:48:31.920360+0000","last_became_active":"2024-05-14T16:46:43.574864+0000","last_became_peered":"2024-05-14T16:46:43.574864+0000","last_unstale":"2024-05-14T16:48:31.920360+0000","last_undegraded":"2024-05-14T16:48:31.920360+0000","last_fullsized":"2024-05-14T16:48:31.920360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:48.630101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.710590+0000","last_change":"2024-05-14T16:46:40.548397+0000","last_active":"2024-05-14T16:47:33.710590+0000","last_peered":"2024-05-14T16:47:33.710590+0000","last_clean":"2024-05-14T16:47:33.710590+0000","last_became_active":"2024-05-14T16:46:40.547803+0000","last_became_peered":"2024-05-14T16:46:40.547803+0000","last_unstale":"2024-05-14T16:47:33.710590+0000","last_undegraded":"2024-05-14T16:47:33.710590+0000","last_fullsized":"2024-05-14T16:47:33.710590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:21:19.812413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892954+0000","last_change":"2024-05-14T16:46:43.578301+0000","last_active":"2024-05-14T16:48:31.892954+0000","last_peered":"2024-05-14T16:48:31.892954+0000","last_clean":"2024-05-14T16:48:31.892954+0000","last_became_active":"2024-05-14T16:46:43.577796+0000","last_became_peered":"2024-05-14T16:46:43.577796+0000","last_unstale":"2024-05-14T16:48:31.892954+0000","last_undegraded":"2024-05-14T16:48:31.892954+0000","last_fullsized":"2024-05-14T16:48:31.892954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:13.159570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":133,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:40.764583+0000","last_change":"2024-05-14T16:46:40.547818+0000","last_active":"2024-05-14T16:48:40.764583+0000","last_peered":"2024-05-14T16:48:40.764583+0000","last_clean":"2024-05-14T16:48:40.764583+0000","last_became_active":"2024-05-14T16:46:40.547454+0000","last_became_peered":"2024-05-14T16:46:40.547454+0000","last_unstale":"2024-05-14T16:48:40.764583+0000","last_undegraded":"2024-05-14T16:48:40.764583+0000","last_fullsized":"2024-05-14T16:48:40.764583+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:16.362415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.879530+0000","last_change":"2024-05-14T16:46:43.570743+0000","last_active":"2024-05-14T16:48:31.879530+0000","last_peered":"2024-05-14T16:48:31.879530+0000","last_clean":"2024-05-14T16:48:31.879530+0000","last_became_active":"2024-05-14T16:46:43.570427+0000","last_became_peered":"2024-05-14T16:46:43.570427+0000","last_unstale":"2024-05-14T16:48:31.879530+0000","last_undegraded":"2024-05-14T16:48:31.879530+0000","last_fullsized":"2024-05-14T16:48:31.879530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:04:11.091285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.708297+0000","last_change":"2024-05-14T16:46:40.546675+0000","last_active":"2024-05-14T16:47:33.708297+0000","last_peered":"2024-05-14T16:47:33.708297+0000","last_clean":"2024-05-14T16:47:33.708297+0000","last_became_active":"2024-05-14T16:46:40.546250+0000","last_became_peered":"2024-05-14T16:46:40.546250+0000","last_unstale":"2024-05-14T16:47:33.708297+0000","last_undegraded":"2024-05-14T16:47:33.708297+0000","last_fullsized":"2024-05-14T16:47:33.708297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:42:52.367719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881313+0000","last_change":"2024-05-14T16:46:43.579119+0000","last_active":"2024-05-14T16:48:31.881313+0000","last_peered":"2024-05-14T16:48:31.881313+0000","last_clean":"2024-05-14T16:48:31.881313+0000","last_became_active":"2024-05-14T16:46:43.578808+0000","last_became_peered":"2024-05-14T16:46:43.578808+0000","last_unstale":"2024-05-14T16:48:31.881313+0000","last_undegraded":"2024-05-14T16:48:31.881313+0000","last_fullsized":"2024-05-14T16:48:31.881313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:57.852982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.716259+0000","last_change":"2024-05-14T16:46:40.538827+0000","last_active":"2024-05-14T16:47:33.716259+0000","last_peered":"2024-05-14T16:47:33.716259+0000","last_clean":"2024-05-14T16:47:33.716259+0000","last_became_active":"2024-05-14T16:46:40.538369+0000","last_became_peered":"2024-05-14T16:46:40.538369+0000","last_unstale":"2024-05-14T16:47:33.716259+0000","last_undegraded":"2024-05-14T16:47:33.716259+0000","last_fullsized":"2024-05-14T16:47:33.716259+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:48:03.599457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.878104+0000","last_change":"2024-05-14T16:46:43.558411+0000","last_active":"2024-05-14T16:48:31.878104+0000","last_peered":"2024-05-14T16:48:31.878104+0000","last_clean":"2024-05-14T16:48:31.878104+0000","last_became_active":"2024-05-14T16:46:43.558101+0000","last_became_peered":"2024-05-14T16:46:43.558101+0000","last_unstale":"2024-05-14T16:48:31.878104+0000","last_undegraded":"2024-05-14T16:48:31.878104+0000","last_fullsized":"2024-05-14T16:48:31.878104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:10.595833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.702005+0000","last_change":"2024-05-14T16:46:40.544858+0000","last_active":"2024-05-14T16:47:33.702005+0000","last_peered":"2024-05-14T16:47:33.702005+0000","last_clean":"2024-05-14T16:47:33.702005+0000","last_became_active":"2024-05-14T16:46:40.544197+0000","last_became_peered":"2024-05-14T16:46:40.544197+0000","last_unstale":"2024-05-14T16:47:33.702005+0000","last_undegraded":"2024-05-14T16:47:33.702005+0000","last_fullsized":"2024-05-14T16:47:33.702005+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:16:45.249626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.889028+0000","last_change":"2024-05-14T16:46:43.567019+0000","last_active":"2024-05-14T16:48:31.889028+0000","last_peered":"2024-05-14T16:48:31.889028+0000","last_clean":"2024-05-14T16:48:31.889028+0000","last_became_active":"2024-05-14T16:46:43.566700+0000","last_became_peered":"2024-05-14T16:46:43.566700+0000","last_unstale":"2024-05-14T16:48:31.889028+0000","last_undegraded":"2024-05-14T16:48:31.889028+0000","last_fullsized":"2024-05-14T16:48:31.889028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:43.373467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":131,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:41.765586+0000","last_change":"2024-05-14T16:46:40.541876+0000","last_active":"2024-05-14T16:48:41.765586+0000","last_peered":"2024-05-14T16:48:41.765586+0000","last_clean":"2024-05-14T16:48:41.765586+0000","last_became_active":"2024-05-14T16:46:40.541460+0000","last_became_peered":"2024-05-14T16:46:40.541460+0000","last_unstale":"2024-05-14T16:48:41.765586+0000","last_undegraded":"2024-05-14T16:48:41.765586+0000","last_fullsized":"2024-05-14T16:48:41.765586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:07:22.536429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892334+0000","last_change":"2024-05-14T16:46:43.577015+0000","last_active":"2024-05-14T16:48:31.892334+0000","last_peered":"2024-05-14T16:48:31.892334+0000","last_clean":"2024-05-14T16:48:31.892334+0000","last_became_active":"2024-05-14T16:46:43.576657+0000","last_became_peered":"2024-05-14T16:46:43.576657+0000","last_unstale":"2024-05-14T16:48:31.892334+0000","last_undegraded":"2024-05-14T16:48:31.892334+0000","last_fullsized":"2024-05-14T16:48:31.892334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:29.242445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.705613+0000","last_change":"2024-05-14T16:46:40.550727+0000","last_active":"2024-05-14T16:47:33.705613+0000","last_peered":"2024-05-14T16:47:33.705613+0000","last_clean":"2024-05-14T16:47:33.705613+0000","last_became_active":"2024-05-14T16:46:40.550310+0000","last_became_peered":"2024-05-14T16:46:40.550310+0000","last_unstale":"2024-05-14T16:47:33.705613+0000","last_undegraded":"2024-05-14T16:47:33.705613+0000","last_fullsized":"2024-05-14T16:47:33.705613+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:04.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881970+0000","last_change":"2024-05-14T16:47:33.711766+0000","last_active":"2024-05-14T16:48:31.881970+0000","last_peered":"2024-05-14T16:48:31.881970+0000","last_clean":"2024-05-14T16:48:31.881970+0000","last_became_active":"2024-05-14T16:47:33.711352+0000","last_became_peered":"2024-05-14T16:47:33.711352+0000","last_unstale":"2024-05-14T16:48:31.881970+0000","last_undegraded":"2024-05-14T16:48:31.881970+0000","last_fullsized":"2024-05-14T16:48:31.881970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:47.924117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.702407+0000","last_change":"2024-05-14T16:46:40.855680+0000","last_active":"2024-05-14T16:47:33.702407+0000","last_peered":"2024-05-14T16:47:33.702407+0000","last_clean":"2024-05-14T16:47:33.702407+0000","last_became_active":"2024-05-14T16:46:40.855265+0000","last_became_peered":"2024-05-14T16:46:40.855265+0000","last_unstale":"2024-05-14T16:47:33.702407+0000","last_undegraded":"2024-05-14T16:47:33.702407+0000","last_fullsized":"2024-05-14T16:47:33.702407+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:26.065298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.890619+0000","last_change":"2024-05-14T16:46:43.579423+0000","last_active":"2024-05-14T16:48:31.890619+0000","last_peered":"2024-05-14T16:48:31.890619+0000","last_clean":"2024-05-14T16:48:31.890619+0000","last_became_active":"2024-05-14T16:46:43.579023+0000","last_became_peered":"2024-05-14T16:46:43.579023+0000","last_unstale":"2024-05-14T16:48:31.890619+0000","last_undegraded":"2024-05-14T16:48:31.890619+0000","last_fullsized":"2024-05-14T16:48:31.890619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:21.311018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'33","reported_seq":273,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:42.766879+0000","last_change":"2024-05-14T16:46:40.550871+0000","last_active":"2024-05-14T16:48:42.766879+0000","last_peered":"2024-05-14T16:48:42.766879+0000","last_clean":"2024-05-14T16:48:42.766879+0000","last_became_active":"2024-05-14T16:46:40.550419+0000","last_became_peered":"2024-05-14T16:46:40.550419+0000","last_unstale":"2024-05-14T16:48:42.766879+0000","last_undegraded":"2024-05-14T16:48:42.766879+0000","last_fullsized":"2024-05-14T16:48:42.766879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:53:09.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61999,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":214,"num_read_kb":167,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.880092+0000","last_change":"2024-05-14T16:46:43.569996+0000","last_active":"2024-05-14T16:48:31.880092+0000","last_peered":"2024-05-14T16:48:31.880092+0000","last_clean":"2024-05-14T16:48:31.880092+0000","last_became_active":"2024-05-14T16:46:43.569501+0000","last_became_peered":"2024-05-14T16:46:43.569501+0000","last_unstale":"2024-05-14T16:48:31.880092+0000","last_undegraded":"2024-05-14T16:48:31.880092+0000","last_fullsized":"2024-05-14T16:48:31.880092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:55.135103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.733718+0000","last_change":"2024-05-14T16:47:33.733316+0000","last_active":"2024-05-14T16:47:33.733718+0000","last_peered":"2024-05-14T16:47:33.733718+0000","last_clean":"2024-05-14T16:47:33.733718+0000","last_became_active":"2024-05-14T16:47:33.709886+0000","last_became_peered":"2024-05-14T16:47:33.709886+0000","last_unstale":"2024-05-14T16:47:33.733718+0000","last_undegraded":"2024-05-14T16:47:33.733718+0000","last_fullsized":"2024-05-14T16:47:33.733718+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:00:47.464686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.878040+0000","last_change":"2024-05-14T16:46:43.562773+0000","last_active":"2024-05-14T16:48:31.878040+0000","last_peered":"2024-05-14T16:48:31.878040+0000","last_clean":"2024-05-14T16:48:31.878040+0000","last_became_active":"2024-05-14T16:46:43.562449+0000","last_became_peered":"2024-05-14T16:46:43.562449+0000","last_unstale":"2024-05-14T16:48:31.878040+0000","last_undegraded":"2024-05-14T16:48:31.878040+0000","last_fullsized":"2024-05-14T16:48:31.878040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:45.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.702480+0000","last_change":"2024-05-14T16:46:40.545882+0000","last_active":"2024-05-14T16:47:33.702480+0000","last_peered":"2024-05-14T16:47:33.702480+0000","last_clean":"2024-05-14T16:47:33.702480+0000","last_became_active":"2024-05-14T16:46:40.545439+0000","last_became_peered":"2024-05-14T16:46:40.545439+0000","last_unstale":"2024-05-14T16:47:33.702480+0000","last_undegraded":"2024-05-14T16:47:33.702480+0000","last_fullsized":"2024-05-14T16:47:33.702480+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:07.983329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.879226+0000","last_change":"2024-05-14T16:46:43.568109+0000","last_active":"2024-05-14T16:48:31.879226+0000","last_peered":"2024-05-14T16:48:31.879226+0000","last_clean":"2024-05-14T16:48:31.879226+0000","last_became_active":"2024-05-14T16:46:43.567106+0000","last_became_peered":"2024-05-14T16:46:43.567106+0000","last_unstale":"2024-05-14T16:48:31.879226+0000","last_undegraded":"2024-05-14T16:48:31.879226+0000","last_fullsized":"2024-05-14T16:48:31.879226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:29.705387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":133,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:41.765557+0000","last_change":"2024-05-14T16:46:40.542204+0000","last_active":"2024-05-14T16:48:41.765557+0000","last_peered":"2024-05-14T16:48:41.765557+0000","last_clean":"2024-05-14T16:48:41.765557+0000","last_became_active":"2024-05-14T16:46:40.541807+0000","last_became_peered":"2024-05-14T16:46:40.541807+0000","last_unstale":"2024-05-14T16:48:41.765557+0000","last_undegraded":"2024-05-14T16:48:41.765557+0000","last_fullsized":"2024-05-14T16:48:41.765557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:30:45.115647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.883770+0000","last_change":"2024-05-14T16:46:43.573546+0000","last_active":"2024-05-14T16:48:31.883770+0000","last_peered":"2024-05-14T16:48:31.883770+0000","last_clean":"2024-05-14T16:48:31.883770+0000","last_became_active":"2024-05-14T16:46:43.573183+0000","last_became_peered":"2024-05-14T16:46:43.573183+0000","last_unstale":"2024-05-14T16:48:31.883770+0000","last_undegraded":"2024-05-14T16:48:31.883770+0000","last_fullsized":"2024-05-14T16:48:31.883770+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:49.873097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.727138+0000","last_change":"2024-05-14T16:47:33.726700+0000","last_active":"2024-05-14T16:47:33.727138+0000","last_peered":"2024-05-14T16:47:33.727138+0000","last_clean":"2024-05-14T16:47:33.727138+0000","last_became_active":"2024-05-14T16:47:33.709297+0000","last_became_peered":"2024-05-14T16:47:33.709297+0000","last_unstale":"2024-05-14T16:47:33.727138+0000","last_undegraded":"2024-05-14T16:47:33.727138+0000","last_fullsized":"2024-05-14T16:47:33.727138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:35.028197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892554+0000","last_change":"2024-05-14T16:46:43.579059+0000","last_active":"2024-05-14T16:48:31.892554+0000","last_peered":"2024-05-14T16:48:31.892554+0000","last_clean":"2024-05-14T16:48:31.892554+0000","last_became_active":"2024-05-14T16:46:43.578748+0000","last_became_peered":"2024-05-14T16:46:43.578748+0000","last_unstale":"2024-05-14T16:48:31.892554+0000","last_undegraded":"2024-05-14T16:48:31.892554+0000","last_fullsized":"2024-05-14T16:48:31.892554+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:42.446422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.710532+0000","last_change":"2024-05-14T16:46:40.549047+0000","last_active":"2024-05-14T16:47:33.710532+0000","last_peered":"2024-05-14T16:47:33.710532+0000","last_clean":"2024-05-14T16:47:33.710532+0000","last_became_active":"2024-05-14T16:46:40.548648+0000","last_became_peered":"2024-05-14T16:46:40.548648+0000","last_unstale":"2024-05-14T16:47:33.710532+0000","last_undegraded":"2024-05-14T16:47:33.710532+0000","last_fullsized":"2024-05-14T16:47:33.710532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:52:56.370871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.882588+0000","last_change":"2024-05-14T16:46:43.569624+0000","last_active":"2024-05-14T16:48:31.882588+0000","last_peered":"2024-05-14T16:48:31.882588+0000","last_clean":"2024-05-14T16:48:31.882588+0000","last_became_active":"2024-05-14T16:46:43.569077+0000","last_became_peered":"2024-05-14T16:46:43.569077+0000","last_unstale":"2024-05-14T16:48:31.882588+0000","last_undegraded":"2024-05-14T16:48:31.882588+0000","last_fullsized":"2024-05-14T16:48:31.882588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:45:50.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.705048+0000","last_change":"2024-05-14T16:46:40.856302+0000","last_active":"2024-05-14T16:47:33.705048+0000","last_peered":"2024-05-14T16:47:33.705048+0000","last_clean":"2024-05-14T16:47:33.705048+0000","last_became_active":"2024-05-14T16:46:40.855964+0000","last_became_peered":"2024-05-14T16:46:40.855964+0000","last_unstale":"2024-05-14T16:47:33.705048+0000","last_undegraded":"2024-05-14T16:47:33.705048+0000","last_fullsized":"2024-05-14T16:47:33.705048+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:38.501418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881404+0000","last_change":"2024-05-14T16:46:43.562847+0000","last_active":"2024-05-14T16:48:31.881404+0000","last_peered":"2024-05-14T16:48:31.881404+0000","last_clean":"2024-05-14T16:48:31.881404+0000","last_became_active":"2024-05-14T16:46:43.560978+0000","last_became_peered":"2024-05-14T16:46:43.560978+0000","last_unstale":"2024-05-14T16:48:31.881404+0000","last_undegraded":"2024-05-14T16:48:31.881404+0000","last_fullsized":"2024-05-14T16:48:31.881404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:12:33.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.704946+0000","last_change":"2024-05-14T16:46:40.544017+0000","last_active":"2024-05-14T16:47:33.704946+0000","last_peered":"2024-05-14T16:47:33.704946+0000","last_clean":"2024-05-14T16:47:33.704946+0000","last_became_active":"2024-05-14T16:46:40.543176+0000","last_became_peered":"2024-05-14T16:46:40.543176+0000","last_unstale":"2024-05-14T16:47:33.704946+0000","last_undegraded":"2024-05-14T16:47:33.704946+0000","last_fullsized":"2024-05-14T16:47:33.704946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:09.817891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.882527+0000","last_change":"2024-05-14T16:46:43.564395+0000","last_active":"2024-05-14T16:48:31.882527+0000","last_peered":"2024-05-14T16:48:31.882527+0000","last_clean":"2024-05-14T16:48:31.882527+0000","last_became_active":"2024-05-14T16:46:43.564065+0000","last_became_peered":"2024-05-14T16:46:43.564065+0000","last_unstale":"2024-05-14T16:48:31.882527+0000","last_undegraded":"2024-05-14T16:48:31.882527+0000","last_fullsized":"2024-05-14T16:48:31.882527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:52.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881248+0000","last_change":"2024-05-14T16:46:43.577067+0000","last_active":"2024-05-14T16:48:31.881248+0000","last_peered":"2024-05-14T16:48:31.881248+0000","last_clean":"2024-05-14T16:48:31.881248+0000","last_became_active":"2024-05-14T16:46:43.576750+0000","last_became_peered":"2024-05-14T16:46:43.576750+0000","last_unstale":"2024-05-14T16:48:31.881248+0000","last_undegraded":"2024-05-14T16:48:31.881248+0000","last_fullsized":"2024-05-14T16:48:31.881248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:25.961124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:40.763703+0000","last_change":"2024-05-14T16:46:40.855073+0000","last_active":"2024-05-14T16:48:40.763703+0000","last_peered":"2024-05-14T16:48:40.763703+0000","last_clean":"2024-05-14T16:48:40.763703+0000","last_became_active":"2024-05-14T16:46:40.854641+0000","last_became_peered":"2024-05-14T16:46:40.854641+0000","last_unstale":"2024-05-14T16:48:40.763703+0000","last_undegraded":"2024-05-14T16:48:40.763703+0000","last_fullsized":"2024-05-14T16:48:40.763703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:58:11.670926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892276+0000","last_change":"2024-05-14T16:46:43.569030+0000","last_active":"2024-05-14T16:48:31.892276+0000","last_peered":"2024-05-14T16:48:31.892276+0000","last_clean":"2024-05-14T16:48:31.892276+0000","last_became_active":"2024-05-14T16:46:43.568646+0000","last_became_peered":"2024-05-14T16:46:43.568646+0000","last_unstale":"2024-05-14T16:48:31.892276+0000","last_undegraded":"2024-05-14T16:48:31.892276+0000","last_fullsized":"2024-05-14T16:48:31.892276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:38.086818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.706973+0000","last_change":"2024-05-14T16:46:40.542624+0000","last_active":"2024-05-14T16:47:33.706973+0000","last_peered":"2024-05-14T16:47:33.706973+0000","last_clean":"2024-05-14T16:47:33.706973+0000","last_became_active":"2024-05-14T16:46:40.542230+0000","last_became_peered":"2024-05-14T16:46:40.542230+0000","last_unstale":"2024-05-14T16:47:33.706973+0000","last_undegraded":"2024-05-14T16:47:33.706973+0000","last_fullsized":"2024-05-14T16:47:33.706973+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:54:50.083362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.704708+0000","last_change":"2024-05-14T16:45:15.724994+0000","last_active":"2024-05-14T16:47:33.704708+0000","last_peered":"2024-05-14T16:47:33.704708+0000","last_clean":"2024-05-14T16:47:33.704708+0000","last_became_active":"2024-05-14T16:45:15.720080+0000","last_became_peered":"2024-05-14T16:45:15.720080+0000","last_unstale":"2024-05-14T16:47:33.704708+0000","last_undegraded":"2024-05-14T16:47:33.704708+0000","last_fullsized":"2024-05-14T16:47:33.704708+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_clean_scrub_stamp":"2024-05-14T16:41:54.615675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:32:43.569611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.874957+0000","last_change":"2024-05-14T16:46:43.567143+0000","last_active":"2024-05-14T16:48:31.874957+0000","last_peered":"2024-05-14T16:48:31.874957+0000","last_clean":"2024-05-14T16:48:31.874957+0000","last_became_active":"2024-05-14T16:46:43.564717+0000","last_became_peered":"2024-05-14T16:46:43.564717+0000","last_unstale":"2024-05-14T16:48:31.874957+0000","last_undegraded":"2024-05-14T16:48:31.874957+0000","last_fullsized":"2024-05-14T16:48:31.874957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:59.025279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.704815+0000","last_change":"2024-05-14T16:46:40.546709+0000","last_active":"2024-05-14T16:47:33.704815+0000","last_peered":"2024-05-14T16:47:33.704815+0000","last_clean":"2024-05-14T16:47:33.704815+0000","last_became_active":"2024-05-14T16:46:40.546295+0000","last_became_peered":"2024-05-14T16:46:40.546295+0000","last_unstale":"2024-05-14T16:47:33.704815+0000","last_undegraded":"2024-05-14T16:47:33.704815+0000","last_fullsized":"2024-05-14T16:47:33.704815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:42:50.255289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.710122+0000","last_change":"2024-05-14T16:46:40.856822+0000","last_active":"2024-05-14T16:47:33.710122+0000","last_peered":"2024-05-14T16:47:33.710122+0000","last_clean":"2024-05-14T16:47:33.710122+0000","last_became_active":"2024-05-14T16:46:40.856434+0000","last_became_peered":"2024-05-14T16:46:40.856434+0000","last_unstale":"2024-05-14T16:47:33.710122+0000","last_undegraded":"2024-05-14T16:47:33.710122+0000","last_fullsized":"2024-05-14T16:47:33.710122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:38.997679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.874412+0000","last_change":"2024-05-14T16:46:43.563918+0000","last_active":"2024-05-14T16:48:31.874412+0000","last_peered":"2024-05-14T16:48:31.874412+0000","last_clean":"2024-05-14T16:48:31.874412+0000","last_became_active":"2024-05-14T16:46:43.563527+0000","last_became_peered":"2024-05-14T16:46:43.563527+0000","last_unstale":"2024-05-14T16:48:31.874412+0000","last_undegraded":"2024-05-14T16:48:31.874412+0000","last_fullsized":"2024-05-14T16:48:31.874412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:58:52.984695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'12","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.705196+0000","last_change":"2024-05-14T16:46:40.547480+0000","last_active":"2024-05-14T16:47:33.705196+0000","last_peered":"2024-05-14T16:47:33.705196+0000","last_clean":"2024-05-14T16:47:33.705196+0000","last_became_active":"2024-05-14T16:46:40.547062+0000","last_became_peered":"2024-05-14T16:46:40.547062+0000","last_unstale":"2024-05-14T16:47:33.705196+0000","last_undegraded":"2024-05-14T16:47:33.705196+0000","last_fullsized":"2024-05-14T16:47:33.705196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:30.211421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881743+0000","last_change":"2024-05-14T16:46:43.563870+0000","last_active":"2024-05-14T16:48:31.881743+0000","last_peered":"2024-05-14T16:48:31.881743+0000","last_clean":"2024-05-14T16:48:31.881743+0000","last_became_active":"2024-05-14T16:46:43.563542+0000","last_became_peered":"2024-05-14T16:46:43.563542+0000","last_unstale":"2024-05-14T16:48:31.881743+0000","last_undegraded":"2024-05-14T16:48:31.881743+0000","last_fullsized":"2024-05-14T16:48:31.881743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:50.699601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.706652+0000","last_change":"2024-05-14T16:46:40.665192+0000","last_active":"2024-05-14T16:47:33.706652+0000","last_peered":"2024-05-14T16:47:33.706652+0000","last_clean":"2024-05-14T16:47:33.706652+0000","last_became_active":"2024-05-14T16:46:40.664827+0000","last_became_peered":"2024-05-14T16:46:40.664827+0000","last_unstale":"2024-05-14T16:47:33.706652+0000","last_undegraded":"2024-05-14T16:47:33.706652+0000","last_fullsized":"2024-05-14T16:47:33.706652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:41:47.179677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.882985+0000","last_change":"2024-05-14T16:47:33.717718+0000","last_active":"2024-05-14T16:48:31.882985+0000","last_peered":"2024-05-14T16:48:31.882985+0000","last_clean":"2024-05-14T16:48:31.882985+0000","last_became_active":"2024-05-14T16:47:33.717368+0000","last_became_peered":"2024-05-14T16:47:33.717368+0000","last_unstale":"2024-05-14T16:48:31.882985+0000","last_undegraded":"2024-05-14T16:48:31.882985+0000","last_fullsized":"2024-05-14T16:48:31.882985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:07:43.990388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.704714+0000","last_change":"2024-05-14T16:46:40.855458+0000","last_active":"2024-05-14T16:47:33.704714+0000","last_peered":"2024-05-14T16:47:33.704714+0000","last_clean":"2024-05-14T16:47:33.704714+0000","last_became_active":"2024-05-14T16:46:40.855122+0000","last_became_peered":"2024-05-14T16:46:40.855122+0000","last_unstale":"2024-05-14T16:47:33.704714+0000","last_undegraded":"2024-05-14T16:47:33.704714+0000","last_fullsized":"2024-05-14T16:47:33.704714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:20:43.435860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.854946+0000","last_change":"2024-05-14T16:46:43.566482+0000","last_active":"2024-05-14T16:48:31.854946+0000","last_peered":"2024-05-14T16:48:31.854946+0000","last_clean":"2024-05-14T16:48:31.854946+0000","last_became_active":"2024-05-14T16:46:43.566145+0000","last_became_peered":"2024-05-14T16:46:43.566145+0000","last_unstale":"2024-05-14T16:48:31.854946+0000","last_undegraded":"2024-05-14T16:48:31.854946+0000","last_fullsized":"2024-05-14T16:48:31.854946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:14:22.731629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.701729+0000","last_change":"2024-05-14T16:46:40.541655+0000","last_active":"2024-05-14T16:47:33.701729+0000","last_peered":"2024-05-14T16:47:33.701729+0000","last_clean":"2024-05-14T16:47:33.701729+0000","last_became_active":"2024-05-14T16:46:40.541331+0000","last_became_peered":"2024-05-14T16:46:40.541331+0000","last_unstale":"2024-05-14T16:47:33.701729+0000","last_undegraded":"2024-05-14T16:47:33.701729+0000","last_fullsized":"2024-05-14T16:47:33.701729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:47:27.950532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.889088+0000","last_change":"2024-05-14T16:46:43.564216+0000","last_active":"2024-05-14T16:48:31.889088+0000","last_peered":"2024-05-14T16:48:31.889088+0000","last_clean":"2024-05-14T16:48:31.889088+0000","last_became_active":"2024-05-14T16:46:43.563894+0000","last_became_peered":"2024-05-14T16:46:43.563894+0000","last_unstale":"2024-05-14T16:48:31.889088+0000","last_undegraded":"2024-05-14T16:48:31.889088+0000","last_fullsized":"2024-05-14T16:48:31.889088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:16:09.256909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.705831+0000","last_change":"2024-05-14T16:46:40.551034+0000","last_active":"2024-05-14T16:47:33.705831+0000","last_peered":"2024-05-14T16:47:33.705831+0000","last_clean":"2024-05-14T16:47:33.705831+0000","last_became_active":"2024-05-14T16:46:40.550211+0000","last_became_peered":"2024-05-14T16:46:40.550211+0000","last_unstale":"2024-05-14T16:47:33.705831+0000","last_undegraded":"2024-05-14T16:47:33.705831+0000","last_fullsized":"2024-05-14T16:47:33.705831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:15.720942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.887108+0000","last_change":"2024-05-14T16:46:43.558264+0000","last_active":"2024-05-14T16:48:31.887108+0000","last_peered":"2024-05-14T16:48:31.887108+0000","last_clean":"2024-05-14T16:48:31.887108+0000","last_became_active":"2024-05-14T16:46:43.557907+0000","last_became_peered":"2024-05-14T16:46:43.557907+0000","last_unstale":"2024-05-14T16:48:31.887108+0000","last_undegraded":"2024-05-14T16:48:31.887108+0000","last_fullsized":"2024-05-14T16:48:31.887108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:20:52.752482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.709461+0000","last_change":"2024-05-14T16:46:40.551013+0000","last_active":"2024-05-14T16:47:33.709461+0000","last_peered":"2024-05-14T16:47:33.709461+0000","last_clean":"2024-05-14T16:47:33.709461+0000","last_became_active":"2024-05-14T16:46:40.550658+0000","last_became_peered":"2024-05-14T16:46:40.550658+0000","last_unstale":"2024-05-14T16:47:33.709461+0000","last_undegraded":"2024-05-14T16:47:33.709461+0000","last_fullsized":"2024-05-14T16:47:33.709461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:36.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.891455+0000","last_change":"2024-05-14T16:46:43.572270+0000","last_active":"2024-05-14T16:48:31.891455+0000","last_peered":"2024-05-14T16:48:31.891455+0000","last_clean":"2024-05-14T16:48:31.891455+0000","last_became_active":"2024-05-14T16:46:43.571886+0000","last_became_peered":"2024-05-14T16:46:43.571886+0000","last_unstale":"2024-05-14T16:48:31.891455+0000","last_undegraded":"2024-05-14T16:48:31.891455+0000","last_fullsized":"2024-05-14T16:48:31.891455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:11.890881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.715246+0000","last_change":"2024-05-14T16:47:33.715246+0000","last_active":"2024-05-14T16:47:33.715246+0000","last_peered":"2024-05-14T16:47:33.715246+0000","last_clean":"2024-05-14T16:47:33.715246+0000","last_became_active":"2024-05-14T16:47:33.708725+0000","last_became_peered":"2024-05-14T16:47:33.708725+0000","last_unstale":"2024-05-14T16:47:33.715246+0000","last_undegraded":"2024-05-14T16:47:33.715246+0000","last_fullsized":"2024-05-14T16:47:33.715246+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:19.118454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.891367+0000","last_change":"2024-05-14T16:46:43.572455+0000","last_active":"2024-05-14T16:48:31.891367+0000","last_peered":"2024-05-14T16:48:31.891367+0000","last_clean":"2024-05-14T16:48:31.891367+0000","last_became_active":"2024-05-14T16:46:43.571930+0000","last_became_peered":"2024-05-14T16:46:43.571930+0000","last_unstale":"2024-05-14T16:48:31.891367+0000","last_undegraded":"2024-05-14T16:48:31.891367+0000","last_fullsized":"2024-05-14T16:48:31.891367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:35:39.643239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.702270+0000","last_change":"2024-05-14T16:46:40.541708+0000","last_active":"2024-05-14T16:47:33.702270+0000","last_peered":"2024-05-14T16:47:33.702270+0000","last_clean":"2024-05-14T16:47:33.702270+0000","last_became_active":"2024-05-14T16:46:40.541202+0000","last_became_peered":"2024-05-14T16:46:40.541202+0000","last_unstale":"2024-05-14T16:47:33.702270+0000","last_undegraded":"2024-05-14T16:47:33.702270+0000","last_fullsized":"2024-05-14T16:47:33.702270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:50:01.821013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.883443+0000","last_change":"2024-05-14T16:46:43.574039+0000","last_active":"2024-05-14T16:48:31.883443+0000","last_peered":"2024-05-14T16:48:31.883443+0000","last_clean":"2024-05-14T16:48:31.883443+0000","last_became_active":"2024-05-14T16:46:43.573237+0000","last_became_peered":"2024-05-14T16:46:43.573237+0000","last_unstale":"2024-05-14T16:48:31.883443+0000","last_undegraded":"2024-05-14T16:48:31.883443+0000","last_fullsized":"2024-05-14T16:48:31.883443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:13.234495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:47:33.716565+0000","last_change":"2024-05-14T16:46:40.549519+0000","last_active":"2024-05-14T16:47:33.716565+0000","last_peered":"2024-05-14T16:47:33.716565+0000","last_clean":"2024-05-14T16:47:33.716565+0000","last_became_active":"2024-05-14T16:46:40.549169+0000","last_became_peered":"2024-05-14T16:46:40.549169+0000","last_unstale":"2024-05-14T16:47:33.716565+0000","last_undegraded":"2024-05-14T16:47:33.716565+0000","last_fullsized":"2024-05-14T16:47:33.716565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:22.559724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.880573+0000","last_change":"2024-05-14T16:46:43.567674+0000","last_active":"2024-05-14T16:48:31.880573+0000","last_peered":"2024-05-14T16:48:31.880573+0000","last_clean":"2024-05-14T16:48:31.880573+0000","last_became_active":"2024-05-14T16:46:43.567337+0000","last_became_peered":"2024-05-14T16:46:43.567337+0000","last_unstale":"2024-05-14T16:48:31.880573+0000","last_undegraded":"2024-05-14T16:48:31.880573+0000","last_fullsized":"2024-05-14T16:48:31.880573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:02.321955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.883562+0000","last_change":"2024-05-14T16:46:43.567627+0000","last_active":"2024-05-14T16:48:31.883562+0000","last_peered":"2024-05-14T16:48:31.883562+0000","last_clean":"2024-05-14T16:48:31.883562+0000","last_became_active":"2024-05-14T16:46:43.567313+0000","last_became_peered":"2024-05-14T16:46:43.567313+0000","last_unstale":"2024-05-14T16:48:31.883562+0000","last_undegraded":"2024-05-14T16:48:31.883562+0000","last_fullsized":"2024-05-14T16:48:31.883562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:47.396923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.884887+0000","last_change":"2024-05-14T16:47:33.708354+0000","last_active":"2024-05-14T16:48:31.884887+0000","last_peered":"2024-05-14T16:48:31.884887+0000","last_clean":"2024-05-14T16:48:31.884887+0000","last_became_active":"2024-05-14T16:47:33.707842+0000","last_became_peered":"2024-05-14T16:47:33.707842+0000","last_unstale":"2024-05-14T16:48:31.884887+0000","last_undegraded":"2024-05-14T16:48:31.884887+0000","last_fullsized":"2024-05-14T16:48:31.884887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:56:26.845782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.884217+0000","last_change":"2024-05-14T16:47:33.708332+0000","last_active":"2024-05-14T16:48:31.884217+0000","last_peered":"2024-05-14T16:48:31.884217+0000","last_clean":"2024-05-14T16:48:31.884217+0000","last_became_active":"2024-05-14T16:47:33.707774+0000","last_became_peered":"2024-05-14T16:47:33.707774+0000","last_unstale":"2024-05-14T16:48:31.884217+0000","last_undegraded":"2024-05-14T16:48:31.884217+0000","last_fullsized":"2024-05-14T16:48:31.884217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:44:47.699303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.883799+0000","last_change":"2024-05-14T16:46:43.572168+0000","last_active":"2024-05-14T16:48:31.883799+0000","last_peered":"2024-05-14T16:48:31.883799+0000","last_clean":"2024-05-14T16:48:31.883799+0000","last_became_active":"2024-05-14T16:46:43.571822+0000","last_became_peered":"2024-05-14T16:46:43.571822+0000","last_unstale":"2024-05-14T16:48:31.883799+0000","last_undegraded":"2024-05-14T16:48:31.883799+0000","last_fullsized":"2024-05-14T16:48:31.883799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:57.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.884631+0000","last_change":"2024-05-14T16:46:43.570935+0000","last_active":"2024-05-14T16:48:31.884631+0000","last_peered":"2024-05-14T16:48:31.884631+0000","last_clean":"2024-05-14T16:48:31.884631+0000","last_became_active":"2024-05-14T16:46:43.570605+0000","last_became_peered":"2024-05-14T16:46:43.570605+0000","last_unstale":"2024-05-14T16:48:31.884631+0000","last_undegraded":"2024-05-14T16:48:31.884631+0000","last_fullsized":"2024-05-14T16:48:31.884631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:17:55.161195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.878325+0000","last_change":"2024-05-14T16:46:43.568442+0000","last_active":"2024-05-14T16:48:31.878325+0000","last_peered":"2024-05-14T16:48:31.878325+0000","last_clean":"2024-05-14T16:48:31.878325+0000","last_became_active":"2024-05-14T16:46:43.568118+0000","last_became_peered":"2024-05-14T16:46:43.568118+0000","last_unstale":"2024-05-14T16:48:31.878325+0000","last_undegraded":"2024-05-14T16:48:31.878325+0000","last_fullsized":"2024-05-14T16:48:31.878325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:52.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.891386+0000","last_change":"2024-05-14T16:46:43.576174+0000","last_active":"2024-05-14T16:48:31.891386+0000","last_peered":"2024-05-14T16:48:31.891386+0000","last_clean":"2024-05-14T16:48:31.891386+0000","last_became_active":"2024-05-14T16:46:43.575737+0000","last_became_peered":"2024-05-14T16:46:43.575737+0000","last_unstale":"2024-05-14T16:48:31.891386+0000","last_undegraded":"2024-05-14T16:48:31.891386+0000","last_fullsized":"2024-05-14T16:48:31.891386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:08.453070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.878931+0000","last_change":"2024-05-14T16:46:43.566256+0000","last_active":"2024-05-14T16:48:31.878931+0000","last_peered":"2024-05-14T16:48:31.878931+0000","last_clean":"2024-05-14T16:48:31.878931+0000","last_became_active":"2024-05-14T16:46:43.565886+0000","last_became_peered":"2024-05-14T16:46:43.565886+0000","last_unstale":"2024-05-14T16:48:31.878931+0000","last_undegraded":"2024-05-14T16:48:31.878931+0000","last_fullsized":"2024-05-14T16:48:31.878931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:50.021286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.881134+0000","last_change":"2024-05-14T16:46:43.569819+0000","last_active":"2024-05-14T16:48:31.881134+0000","last_peered":"2024-05-14T16:48:31.881134+0000","last_clean":"2024-05-14T16:48:31.881134+0000","last_became_active":"2024-05-14T16:46:43.569461+0000","last_became_peered":"2024-05-14T16:46:43.569461+0000","last_unstale":"2024-05-14T16:48:31.881134+0000","last_undegraded":"2024-05-14T16:48:31.881134+0000","last_fullsized":"2024-05-14T16:48:31.881134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:52.013604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.877968+0000","last_change":"2024-05-14T16:46:43.567569+0000","last_active":"2024-05-14T16:48:31.877968+0000","last_peered":"2024-05-14T16:48:31.877968+0000","last_clean":"2024-05-14T16:48:31.877968+0000","last_became_active":"2024-05-14T16:46:43.564808+0000","last_became_peered":"2024-05-14T16:46:43.564808+0000","last_unstale":"2024-05-14T16:48:31.877968+0000","last_undegraded":"2024-05-14T16:48:31.877968+0000","last_fullsized":"2024-05-14T16:48:31.877968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:13.744730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.892610+0000","last_change":"2024-05-14T16:46:43.578417+0000","last_active":"2024-05-14T16:48:31.892610+0000","last_peered":"2024-05-14T16:48:31.892610+0000","last_clean":"2024-05-14T16:48:31.892610+0000","last_became_active":"2024-05-14T16:46:43.577687+0000","last_became_peered":"2024-05-14T16:46:43.577687+0000","last_unstale":"2024-05-14T16:48:31.892610+0000","last_undegraded":"2024-05-14T16:48:31.892610+0000","last_fullsized":"2024-05-14T16:48:31.892610+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:32.060934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.884899+0000","last_change":"2024-05-14T16:46:43.564114+0000","last_active":"2024-05-14T16:48:31.884899+0000","last_peered":"2024-05-14T16:48:31.884899+0000","last_clean":"2024-05-14T16:48:31.884899+0000","last_became_active":"2024-05-14T16:46:43.563734+0000","last_became_peered":"2024-05-14T16:46:43.563734+0000","last_unstale":"2024-05-14T16:48:31.884899+0000","last_undegraded":"2024-05-14T16:48:31.884899+0000","last_fullsized":"2024-05-14T16:48:31.884899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:19.016741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:48:31.883035+0000","last_change":"2024-05-14T16:46:43.566490+0000","last_active":"2024-05-14T16:48:31.883035+0000","last_peered":"2024-05-14T16:48:31.883035+0000","last_clean":"2024-05-14T16:48:31.883035+0000","last_became_active":"2024-05-14T16:46:43.566086+0000","last_became_peered":"2024-05-14T16:46:43.566086+0000","last_unstale":"2024-05-14T16:48:31.883035+0000","last_undegraded":"2024-05-14T16:48:31.883035+0000","last_fullsized":"2024-05-14T16:48:31.883035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:39:09.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245266,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65609,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":636,"num_read_kb":386,"num_write":169,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":728,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710760,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":425363,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906989,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1078697,"data_compressed":41531,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070515,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":425453,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":425261,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":1016912,"data_compressed":41531,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":424842,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":486634,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":426113,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":487368,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":425153,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":425861,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822727,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":1016596,"data_compressed":41531,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34110,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:48:45.847 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T16:48:45.847 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-14T16:48:45.847 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-14T16:48:45.847 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-14T16:48:45.869 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:45 smithi069 ceph-mon[31507]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T16:48:45.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:45 smithi203 ceph-mon[35787]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T16:48:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:45 smithi092 ceph-mon[35763]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T16:48:46.086 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:46.898 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:46 smithi069 ceph-mon[31507]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:48:46.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:46 smithi203 ceph-mon[35787]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:48:47.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:46 smithi092 ceph-mon[35763]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:48:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 -- 172.21.15.69:0/4102492721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99cc103450 msgr2=0x7f99cc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 --2- 172.21.15.69:0/4102492721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99cc103450 0x7f99cc1038d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009990 tx=0x7f99bc009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 -- 172.21.15.69:0/4102492721 shutdown_connections 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 --2- 172.21.15.69:0/4102492721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99cc103e10 0x7f99cc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 --2- 172.21.15.69:0/4102492721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99cc1094e0 0x7f99cc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 --2- 172.21.15.69:0/4102492721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99cc103450 0x7f99cc1038d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 -- 172.21.15.69:0/4102492721 >> 172.21.15.69:0/4102492721 conn(0x7f99cc0fef60 msgr2=0x7f99cc101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 -- 172.21.15.69:0/4102492721 shutdown_connections 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 -- 172.21.15.69:0/4102492721 wait complete. 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 Processor -- start 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.753+0000 7f99d319d700 1 -- start start 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d319d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99cc103450 0x7f99cc19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d319d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99cc103e10 0x7f99cc19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d319d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99cc1094e0 0x7f99cc199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d319d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc1135f0 con 0x7f99cc103450 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d319d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc113470 con 0x7f99cc1094e0 2024-05-14T16:48:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d319d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc113770 con 0x7f99cc103e10 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d219b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99cc103450 0x7f99cc19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d219b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99cc103450 0x7f99cc19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56878/0 (socket says 172.21.15.69:56878) 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d219b700 1 -- 172.21.15.69:0/1450644640 learned_addr learned my addr 172.21.15.69:0/1450644640 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d199a700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99cc103e10 0x7f99cc19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d219b700 1 -- 172.21.15.69:0/1450644640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99cc103e10 msgr2=0x7f99cc19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d219b700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99cc103e10 0x7f99cc19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d219b700 1 -- 172.21.15.69:0/1450644640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99cc1094e0 msgr2=0x7f99cc199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d219b700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99cc1094e0 0x7f99cc199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d219b700 1 -- 172.21.15.69:0/1450644640 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc19a0b0 con 0x7f99cc103450 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d219b700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99cc103450 0x7f99cc19f790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c920 tx=0x7f99c800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99c37fe700 1 -- 172.21.15.69:0/1450644640 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8007b00 con 0x7f99cc103450 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99c37fe700 1 -- 172.21.15.69:0/1450644640 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99c8004ce0 con 0x7f99cc103450 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99c37fe700 1 -- 172.21.15.69:0/1450644640 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8005550 con 0x7f99cc103450 2024-05-14T16:48:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d319d700 1 -- 172.21.15.69:0/1450644640 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99cc19a3a0 con 0x7f99cc103450 2024-05-14T16:48:47.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.754+0000 7f99d319d700 1 -- 172.21.15.69:0/1450644640 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99cc1a6660 con 0x7f99cc103450 2024-05-14T16:48:47.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.755+0000 7f99d319d700 1 -- 172.21.15.69:0/1450644640 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99cc06a230 con 0x7f99cc103450 2024-05-14T16:48:47.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.757+0000 7f99c37fe700 1 -- 172.21.15.69:0/1450644640 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f99c8007ca0 con 0x7f99cc103450 2024-05-14T16:48:47.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.757+0000 7f99c37fe700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f99b80b2900 0x7f99b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:47.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.757+0000 7f99d199a700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f99b80b2900 0x7f99b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:47.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.757+0000 7f99c37fe700 1 -- 172.21.15.69:0/1450644640 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f99c80e21b0 con 0x7f99cc103450 2024-05-14T16:48:47.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.758+0000 7f99d199a700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f99b80b2900 0x7f99b80b4dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00b5c0 tx=0x7f99bc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:47.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:47.760+0000 7f99c37fe700 1 -- 172.21.15.69:0/1450644640 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f99c8014030 con 0x7f99cc103450 2024-05-14T16:48:47.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:47 smithi069 ceph-mon[31507]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:47.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:47 smithi203 ceph-mon[35787]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:48.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:47 smithi092 ceph-mon[35763]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:48.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.013+0000 7f99d319d700 1 -- 172.21.15.69:0/1450644640 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f99cc19ae80 con 0x7f99cc103450 2024-05-14T16:48:48.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.013+0000 7f99c37fe700 1 -- 172.21.15.69:0/1450644640 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f99c80a9f50 con 0x7f99cc103450 2024-05-14T16:48:48.013 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:48:48.013 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-14T16:48:48.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 -- 172.21.15.69:0/1450644640 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f99b80b2900 msgr2=0x7f99b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:48.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f99b80b2900 0x7f99b80b4dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00b5c0 tx=0x7f99bc005960 comp rx=0 tx=0).stop 2024-05-14T16:48:48.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 -- 172.21.15.69:0/1450644640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99cc103450 msgr2=0x7f99cc19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99cc103450 0x7f99cc19f790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c920 tx=0x7f99c800cc30 comp rx=0 tx=0).stop 2024-05-14T16:48:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 -- 172.21.15.69:0/1450644640 shutdown_connections 2024-05-14T16:48:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f99b80b2900 0x7f99b80b4dc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99cc103450 0x7f99cc19f790 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99cc1094e0 0x7f99cc199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 --2- 172.21.15.69:0/1450644640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99cc103e10 0x7f99cc19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 -- 172.21.15.69:0/1450644640 >> 172.21.15.69:0/1450644640 conn(0x7f99cc0fef60 msgr2=0x7f99cc0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 -- 172.21.15.69:0/1450644640 shutdown_connections 2024-05-14T16:48:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:48.016+0000 7f99d319d700 1 -- 172.21.15.69:0/1450644640 wait complete. 2024-05-14T16:48:48.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:48 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1450644640' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T16:48:48.711 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-14T16:48:48.711 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:48:48.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:48 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1450644640' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T16:48:48.994 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:49.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:48 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1450644640' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T16:48:49.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:49 smithi069 ceph-mon[31507]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:49.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:49 smithi203 ceph-mon[35787]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:50.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:49 smithi092 ceph-mon[35763]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:50.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.655+0000 7efc7b245700 1 -- 172.21.15.69:0/1182151114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc74102470 msgr2=0x7efc741028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:50.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.655+0000 7efc7b245700 1 --2- 172.21.15.69:0/1182151114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc74102470 0x7efc741028f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efc60009990 tx=0x7efc60009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:50.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.655+0000 7efc7b245700 1 -- 172.21.15.69:0/1182151114 shutdown_connections 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.655+0000 7efc7b245700 1 --2- 172.21.15.69:0/1182151114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc74102e30 0x7efc7410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.655+0000 7efc7b245700 1 --2- 172.21.15.69:0/1182151114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc741084b0 0x7efc74108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.655+0000 7efc7b245700 1 --2- 172.21.15.69:0/1182151114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc74102470 0x7efc741028f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.655+0000 7efc7b245700 1 -- 172.21.15.69:0/1182151114 >> 172.21.15.69:0/1182151114 conn(0x7efc74079410 msgr2=0x7efc74079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.655+0000 7efc7b245700 1 -- 172.21.15.69:0/1182151114 shutdown_connections 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.655+0000 7efc7b245700 1 -- 172.21.15.69:0/1182151114 wait complete. 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.655+0000 7efc7b245700 1 Processor -- start 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc7b245700 1 -- start start 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc7b245700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc74102470 0x7efc74104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc7b245700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc74102e30 0x7efc741045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc7b245700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc741084b0 0x7efc74104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc7b245700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efc741126c0 con 0x7efc741084b0 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc7b245700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efc74112540 con 0x7efc74102e30 2024-05-14T16:48:50.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc7b245700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efc74112840 con 0x7efc74102470 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc797e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc741084b0 0x7efc74104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc797e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc741084b0 0x7efc74104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56894/0 (socket says 172.21.15.69:56894) 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc797e2700 1 -- 172.21.15.69:0/1238538768 learned_addr learned my addr 172.21.15.69:0/1238538768 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc78fe1700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc74102470 0x7efc74104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc73fff700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc74102e30 0x7efc741045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc797e2700 1 -- 172.21.15.69:0/1238538768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc74102470 msgr2=0x7efc74104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc797e2700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc74102470 0x7efc74104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc797e2700 1 -- 172.21.15.69:0/1238538768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc74102e30 msgr2=0x7efc741045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc797e2700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc74102e30 0x7efc741045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc797e2700 1 -- 172.21.15.69:0/1238538768 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc741a8330 con 0x7efc741084b0 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.656+0000 7efc797e2700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc741084b0 0x7efc74104b00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00b970 tx=0x7efc6c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.657+0000 7efc71ffb700 1 -- 172.21.15.69:0/1238538768 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c019070 con 0x7efc741084b0 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.657+0000 7efc71ffb700 1 -- 172.21.15.69:0/1238538768 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc6c014440 con 0x7efc741084b0 2024-05-14T16:48:50.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.657+0000 7efc7b245700 1 -- 172.21.15.69:0/1238538768 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc741a8620 con 0x7efc741084b0 2024-05-14T16:48:50.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.657+0000 7efc71ffb700 1 -- 172.21.15.69:0/1238538768 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c00ce60 con 0x7efc741084b0 2024-05-14T16:48:50.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.657+0000 7efc7b245700 1 -- 172.21.15.69:0/1238538768 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc741a8bb0 con 0x7efc741084b0 2024-05-14T16:48:50.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.658+0000 7efc7b245700 1 -- 172.21.15.69:0/1238538768 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7406a230 con 0x7efc741084b0 2024-05-14T16:48:50.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.659+0000 7efc71ffb700 1 -- 172.21.15.69:0/1238538768 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efc6c01c070 con 0x7efc741084b0 2024-05-14T16:48:50.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.659+0000 7efc71ffb700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7efc640b2900 0x7efc640b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:50.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.660+0000 7efc78fe1700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7efc640b2900 0x7efc640b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:50.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.660+0000 7efc71ffb700 1 -- 172.21.15.69:0/1238538768 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7efc6c0e15c0 con 0x7efc741084b0 2024-05-14T16:48:50.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.660+0000 7efc78fe1700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7efc640b2900 0x7efc640b4dc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efc68006fd0 tx=0x7efc68008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:50.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.664+0000 7efc71ffb700 1 -- 172.21.15.69:0/1238538768 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc6c0a92e0 con 0x7efc741084b0 2024-05-14T16:48:50.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.915+0000 7efc7b245700 1 -- 172.21.15.69:0/1238538768 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc741055e0 con 0x7efc741084b0 2024-05-14T16:48:50.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.915+0000 7efc71ffb700 1 -- 172.21.15.69:0/1238538768 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7efc741055e0 con 0x7efc741084b0 2024-05-14T16:48:50.915 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:48:50.915 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:47:04.867362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24433,24446,24422]},"id":1}]} 2024-05-14T16:48:50.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 -- 172.21.15.69:0/1238538768 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7efc640b2900 msgr2=0x7efc640b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7efc640b2900 0x7efc640b4dc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efc68006fd0 tx=0x7efc68008040 comp rx=0 tx=0).stop 2024-05-14T16:48:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 -- 172.21.15.69:0/1238538768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc741084b0 msgr2=0x7efc74104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc741084b0 0x7efc74104b00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00b970 tx=0x7efc6c00bd30 comp rx=0 tx=0).stop 2024-05-14T16:48:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 -- 172.21.15.69:0/1238538768 shutdown_connections 2024-05-14T16:48:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7efc640b2900 0x7efc640b4dc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc741084b0 0x7efc74104b00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc74102470 0x7efc74104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 --2- 172.21.15.69:0/1238538768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc74102e30 0x7efc741045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 -- 172.21.15.69:0/1238538768 >> 172.21.15.69:0/1238538768 conn(0x7efc74079410 msgr2=0x7efc7406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 -- 172.21.15.69:0/1238538768 shutdown_connections 2024-05-14T16:48:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:50.918+0000 7efc7b245700 1 -- 172.21.15.69:0/1238538768 wait complete. 2024-05-14T16:48:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-05-14T16:48:51.590 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:48:51.836 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:51.875 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:51 smithi069 ceph-mon[31507]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:51.875 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:51 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1238538768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:48:51.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:51 smithi203 ceph-mon[35787]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:51.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:51 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1238538768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:48:52.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:51 smithi092 ceph-mon[35763]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:48:52.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:51 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1238538768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:48:53.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.480+0000 7f9a68670700 1 -- 172.21.15.69:0/1384233343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a60102e00 msgr2=0x7f9a6010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:53.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.480+0000 7f9a68670700 1 --2- 172.21.15.69:0/1384233343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a60102e00 0x7f9a6010f2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800b0a0 tx=0x7f9a5800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:53.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.480+0000 7f9a68670700 1 -- 172.21.15.69:0/1384233343 shutdown_connections 2024-05-14T16:48:53.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.480+0000 7f9a68670700 1 --2- 172.21.15.69:0/1384233343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a60102e00 0x7f9a6010f2f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:53.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.480+0000 7f9a68670700 1 --2- 172.21.15.69:0/1384233343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a60108520 0x7f9a60108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:53.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.480+0000 7f9a68670700 1 --2- 172.21.15.69:0/1384233343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a60102440 0x7f9a601028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.480+0000 7f9a68670700 1 -- 172.21.15.69:0/1384233343 >> 172.21.15.69:0/1384233343 conn(0x7f9a600793f0 msgr2=0x7f9a60079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.480+0000 7f9a68670700 1 -- 172.21.15.69:0/1384233343 shutdown_connections 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.480+0000 7f9a68670700 1 -- 172.21.15.69:0/1384233343 wait complete. 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.480+0000 7f9a68670700 1 Processor -- start 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a68670700 1 -- start start 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a68670700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a60102440 0x7f9a6019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a68670700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a60102e00 0x7f9a6019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a68670700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a60108520 0x7f9a60198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a68670700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601125d0 con 0x7f9a60102e00 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a68670700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60112450 con 0x7f9a60102440 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a68670700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60112750 con 0x7f9a60108520 2024-05-14T16:48:53.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a65c0b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a60102e00 0x7f9a6019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a65c0b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a60102e00 0x7f9a6019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56910/0 (socket says 172.21.15.69:56910) 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a65c0b700 1 -- 172.21.15.69:0/4002495013 learned_addr learned my addr 172.21.15.69:0/4002495013 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a66c0d700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a60108520 0x7f9a60198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a65c0b700 1 -- 172.21.15.69:0/4002495013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a60108520 msgr2=0x7f9a60198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a65c0b700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a60108520 0x7f9a60198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a65c0b700 1 -- 172.21.15.69:0/4002495013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a60102440 msgr2=0x7f9a6019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a65c0b700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a60102440 0x7f9a6019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a65c0b700 1 -- 172.21.15.69:0/4002495013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a60198f70 con 0x7f9a60102e00 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a65c0b700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a60102e00 0x7f9a6019eb50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400c9b0 tx=0x7f9a5400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a537fe700 1 -- 172.21.15.69:0/4002495013 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54007b00 con 0x7f9a60102e00 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a68670700 1 -- 172.21.15.69:0/4002495013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a60199260 con 0x7f9a60102e00 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a537fe700 1 -- 172.21.15.69:0/4002495013 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a54007ca0 con 0x7f9a60102e00 2024-05-14T16:48:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.481+0000 7f9a537fe700 1 -- 172.21.15.69:0/4002495013 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54018730 con 0x7f9a60102e00 2024-05-14T16:48:53.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.482+0000 7f9a68670700 1 -- 172.21.15.69:0/4002495013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a601a5470 con 0x7f9a60102e00 2024-05-14T16:48:53.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.482+0000 7f9a68670700 1 -- 172.21.15.69:0/4002495013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a60103b60 con 0x7f9a60102e00 2024-05-14T16:48:53.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.484+0000 7f9a537fe700 1 -- 172.21.15.69:0/4002495013 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9a5400f450 con 0x7f9a60102e00 2024-05-14T16:48:53.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.484+0000 7f9a537fe700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9a4c0b2880 0x7f9a4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:53.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.485+0000 7f9a6640c700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9a4c0b2880 0x7f9a4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:53.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.485+0000 7f9a537fe700 1 -- 172.21.15.69:0/4002495013 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9a540e25a0 con 0x7f9a60102e00 2024-05-14T16:48:53.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.485+0000 7f9a6640c700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9a4c0b2880 0x7f9a4c0b4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00a9b0 tx=0x7f9a5c005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:53.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.488+0000 7f9a537fe700 1 -- 172.21.15.69:0/4002495013 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a54014030 con 0x7f9a60102e00 2024-05-14T16:48:53.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.742+0000 7f9a68670700 1 -- 172.21.15.69:0/4002495013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a6002e560 con 0x7f9a60102e00 2024-05-14T16:48:53.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.743+0000 7f9a537fe700 1 -- 172.21.15.69:0/4002495013 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7f9a540aa350 con 0x7f9a60102e00 2024-05-14T16:48:53.743 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:48:53.743 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:47:04.867362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24433,24446,24422]},"id":1}]} 2024-05-14T16:48:53.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 -- 172.21.15.69:0/4002495013 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9a4c0b2880 msgr2=0x7f9a4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:53.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9a4c0b2880 0x7f9a4c0b4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00a9b0 tx=0x7f9a5c005c90 comp rx=0 tx=0).stop 2024-05-14T16:48:53.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 -- 172.21.15.69:0/4002495013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a60102e00 msgr2=0x7f9a6019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:53.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a60102e00 0x7f9a6019eb50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400c9b0 tx=0x7f9a5400cd70 comp rx=0 tx=0).stop 2024-05-14T16:48:53.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 -- 172.21.15.69:0/4002495013 shutdown_connections 2024-05-14T16:48:53.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a60108520 0x7f9a60198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:53.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a60102440 0x7f9a6019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:53.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f9a4c0b2880 0x7f9a4c0b4d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:53.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 --2- 172.21.15.69:0/4002495013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a60102e00 0x7f9a6019eb50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:53.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 -- 172.21.15.69:0/4002495013 >> 172.21.15.69:0/4002495013 conn(0x7f9a600793f0 msgr2=0x7f9a60100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:53.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 -- 172.21.15.69:0/4002495013 shutdown_connections 2024-05-14T16:48:53.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:53.745+0000 7f9a68670700 1 -- 172.21.15.69:0/4002495013 wait complete. 2024-05-14T16:48:53.746 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-05-14T16:48:53.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:53 smithi069 ceph-mon[31507]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:53.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:53 smithi203 ceph-mon[35787]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:54.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:53 smithi092 ceph-mon[35763]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:54.542 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:48:54.790 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:54.911 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:54 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/4002495013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:48:54.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:54 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/4002495013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:48:55.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:54 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/4002495013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:48:55.917 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:55 smithi069 ceph-mon[31507]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:48:55.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:55 smithi203 ceph-mon[35787]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:48:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:55 smithi092 ceph-mon[35763]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:48:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.423+0000 7ff58359e700 1 -- 172.21.15.69:0/1972327326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff58410da60 msgr2=0x7ff58410ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.423+0000 7ff58359e700 1 --2- 172.21.15.69:0/1972327326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff58410da60 0x7ff58410ff20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009990 tx=0x7ff57c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.423+0000 7ff58359e700 1 -- 172.21.15.69:0/1972327326 shutdown_connections 2024-05-14T16:48:56.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.423+0000 7ff58359e700 1 --2- 172.21.15.69:0/1972327326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff58410da60 0x7ff58410ff20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.423+0000 7ff58359e700 1 --2- 172.21.15.69:0/1972327326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff584104630 0x7ff584104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.423+0000 7ff58359e700 1 --2- 172.21.15.69:0/1972327326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff584104f50 0x7ff58410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.423+0000 7ff58359e700 1 -- 172.21.15.69:0/1972327326 >> 172.21.15.69:0/1972327326 conn(0x7ff5840fde60 msgr2=0x7ff584100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:56.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.423+0000 7ff58359e700 1 -- 172.21.15.69:0/1972327326 shutdown_connections 2024-05-14T16:48:56.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.424+0000 7ff58359e700 1 -- 172.21.15.69:0/1972327326 wait complete. 2024-05-14T16:48:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.424+0000 7ff58359e700 1 Processor -- start 2024-05-14T16:48:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.424+0000 7ff58359e700 1 -- start start 2024-05-14T16:48:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.424+0000 7ff58359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff584104630 0x7ff58419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.424+0000 7ff58359e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff584104f50 0x7ff58419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.424+0000 7ff58359e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff58410da60 0x7ff584199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.424+0000 7ff58359e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff584113660 con 0x7ff584104f50 2024-05-14T16:48:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.424+0000 7ff58359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5841134e0 con 0x7ff584104630 2024-05-14T16:48:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.424+0000 7ff58359e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff5841137e0 con 0x7ff58410da60 2024-05-14T16:48:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff581d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff584104f50 0x7ff58419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff581d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff584104f50 0x7ff58419fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49490/0 (socket says 172.21.15.69:49490) 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff581d9b700 1 -- 172.21.15.69:0/2319808727 learned_addr learned my addr 172.21.15.69:0/2319808727 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff58259c700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff584104630 0x7ff58419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff582d9d700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff58410da60 0x7ff584199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff581d9b700 1 -- 172.21.15.69:0/2319808727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff58410da60 msgr2=0x7ff584199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff581d9b700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff58410da60 0x7ff584199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff581d9b700 1 -- 172.21.15.69:0/2319808727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff584104630 msgr2=0x7ff58419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff581d9b700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff584104630 0x7ff58419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff581d9b700 1 -- 172.21.15.69:0/2319808727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58419a000 con 0x7ff584104f50 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff581d9b700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff584104f50 0x7ff58419fc40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c970 tx=0x7ff56c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff57b7fe700 1 -- 172.21.15.69:0/2319808727 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00e880 con 0x7ff584104f50 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff57b7fe700 1 -- 172.21.15.69:0/2319808727 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff56c00f450 con 0x7ff584104f50 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff58359e700 1 -- 172.21.15.69:0/2319808727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58419a2f0 con 0x7ff584104f50 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff57b7fe700 1 -- 172.21.15.69:0/2319808727 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c018660 con 0x7ff584104f50 2024-05-14T16:48:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.425+0000 7ff58359e700 1 -- 172.21.15.69:0/2319808727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5841a6520 con 0x7ff584104f50 2024-05-14T16:48:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.426+0000 7ff58359e700 1 -- 172.21.15.69:0/2319808727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58406a230 con 0x7ff584104f50 2024-05-14T16:48:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.429+0000 7ff57b7fe700 1 -- 172.21.15.69:0/2319808727 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff56c00ea20 con 0x7ff584104f50 2024-05-14T16:48:56.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.429+0000 7ff57b7fe700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff5700b2880 0x7ff5700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:56.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.429+0000 7ff58259c700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff5700b2880 0x7ff5700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:56.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.430+0000 7ff57b7fe700 1 -- 172.21.15.69:0/2319808727 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff56c0e2220 con 0x7ff584104f50 2024-05-14T16:48:56.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.430+0000 7ff58259c700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff5700b2880 0x7ff5700b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff57400f440 tx=0x7ff5740079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:56.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.433+0000 7ff57b7fe700 1 -- 172.21.15.69:0/2319808727 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff56c014030 con 0x7ff584104f50 2024-05-14T16:48:56.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.645+0000 7ff58359e700 1 -- 172.21.15.69:0/2319808727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5841a6810 con 0x7ff584104f50 2024-05-14T16:48:56.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.647+0000 7ff57b7fe700 1 -- 172.21.15.69:0/2319808727 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22664 (secure 0 0 0) 0x7ff56c0a9fd0 con 0x7ff584104f50 2024-05-14T16:48:56.647 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:48:56.647 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T16:47:33.698176+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:46:38.843325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:46:42.395401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:48:56.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 -- 172.21.15.69:0/2319808727 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff5700b2880 msgr2=0x7ff5700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:56.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff5700b2880 0x7ff5700b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff57400f440 tx=0x7ff5740079e0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 -- 172.21.15.69:0/2319808727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff584104f50 msgr2=0x7ff58419fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:56.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff584104f50 0x7ff58419fc40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c970 tx=0x7ff56c00cc80 comp rx=0 tx=0).stop 2024-05-14T16:48:56.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 -- 172.21.15.69:0/2319808727 shutdown_connections 2024-05-14T16:48:56.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff58410da60 0x7ff584199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff584104630 0x7ff58419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff5700b2880 0x7ff5700b4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 --2- 172.21.15.69:0/2319808727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff584104f50 0x7ff58419fc40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:56.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 -- 172.21.15.69:0/2319808727 >> 172.21.15.69:0/2319808727 conn(0x7ff5840fde60 msgr2=0x7ff58410f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:56.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 -- 172.21.15.69:0/2319808727 shutdown_connections 2024-05-14T16:48:56.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:56.649+0000 7ff58359e700 1 -- 172.21.15.69:0/2319808727 wait complete. 2024-05-14T16:48:57.353 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:48:57.593 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:48:57.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:57 smithi069 ceph-mon[31507]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:57.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:57 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2319808727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:48:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:57 smithi203 ceph-mon[35787]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:57 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2319808727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:48:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:57 smithi092 ceph-mon[35763]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:48:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:57 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2319808727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:48:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.313+0000 7ff218ce1700 1 -- 172.21.15.69:0/4161228714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2141037f0 msgr2=0x7ff214103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.313+0000 7ff218ce1700 1 --2- 172.21.15.69:0/4161228714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2141037f0 0x7ff214103bd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff2080099c0 tx=0x7ff208009cd0 comp rx=0 tx=0).stop 2024-05-14T16:48:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.313+0000 7ff218ce1700 1 -- 172.21.15.69:0/4161228714 shutdown_connections 2024-05-14T16:48:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.313+0000 7ff218ce1700 1 --2- 172.21.15.69:0/4161228714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2141037f0 0x7ff214103bd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.313+0000 7ff218ce1700 1 --2- 172.21.15.69:0/4161228714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2141088f0 0x7ff214110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.313+0000 7ff218ce1700 1 --2- 172.21.15.69:0/4161228714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2141041a0 0x7ff2141081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.313+0000 7ff218ce1700 1 -- 172.21.15.69:0/4161228714 >> 172.21.15.69:0/4161228714 conn(0x7ff21407a6a0 msgr2=0x7ff21407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.314+0000 7ff218ce1700 1 -- 172.21.15.69:0/4161228714 shutdown_connections 2024-05-14T16:48:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.314+0000 7ff218ce1700 1 -- 172.21.15.69:0/4161228714 wait complete. 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.314+0000 7ff218ce1700 1 Processor -- start 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.314+0000 7ff218ce1700 1 -- start start 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.314+0000 7ff218ce1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2141037f0 0x7ff21419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff218ce1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2141041a0 0x7ff21419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff218ce1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2141088f0 0x7ff2141999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff218ce1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141136e0 con 0x7ff2141041a0 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff218ce1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff214113560 con 0x7ff2141037f0 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff218ce1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff214113860 con 0x7ff2141088f0 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff211d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2141041a0 0x7ff21419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff211d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2141041a0 0x7ff21419fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49502/0 (socket says 172.21.15.69:49502) 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff211d9b700 1 -- 172.21.15.69:0/1364858660 learned_addr learned my addr 172.21.15.69:0/1364858660 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff21259c700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2141037f0 0x7ff21419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff212d9d700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2141088f0 0x7ff2141999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff211d9b700 1 -- 172.21.15.69:0/1364858660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2141088f0 msgr2=0x7ff2141999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff211d9b700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2141088f0 0x7ff2141999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff211d9b700 1 -- 172.21.15.69:0/1364858660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2141037f0 msgr2=0x7ff21419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff211d9b700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2141037f0 0x7ff21419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff211d9b700 1 -- 172.21.15.69:0/1364858660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21419a120 con 0x7ff2141041a0 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff211d9b700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2141041a0 0x7ff21419fd40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff20400d950 tx=0x7ff20400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff2037fe700 1 -- 172.21.15.69:0/1364858660 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204016070 con 0x7ff2141041a0 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff2037fe700 1 -- 172.21.15.69:0/1364858660 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff204009df0 con 0x7ff2141041a0 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff218ce1700 1 -- 172.21.15.69:0/1364858660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21419a410 con 0x7ff2141041a0 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff2037fe700 1 -- 172.21.15.69:0/1364858660 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20400f3c0 con 0x7ff2141041a0 2024-05-14T16:48:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.315+0000 7ff218ce1700 1 -- 172.21.15.69:0/1364858660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2141a66d0 con 0x7ff2141041a0 2024-05-14T16:48:59.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.316+0000 7ff218ce1700 1 -- 172.21.15.69:0/1364858660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21410bb30 con 0x7ff2141041a0 2024-05-14T16:48:59.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.318+0000 7ff2037fe700 1 -- 172.21.15.69:0/1364858660 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff204019070 con 0x7ff2141041a0 2024-05-14T16:48:59.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.319+0000 7ff2037fe700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff1fc0b2880 0x7ff1fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.319+0000 7ff21259c700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff1fc0b2880 0x7ff1fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.319+0000 7ff2037fe700 1 -- 172.21.15.69:0/1364858660 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff2040e1de0 con 0x7ff2141041a0 2024-05-14T16:48:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.319+0000 7ff21259c700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff1fc0b2880 0x7ff1fc0b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff21419ab00 tx=0x7ff208005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:59.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.322+0000 7ff2037fe700 1 -- 172.21.15.69:0/1364858660 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2040a9b00 con 0x7ff2141041a0 2024-05-14T16:48:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.569+0000 7ff218ce1700 1 -- 172.21.15.69:0/1364858660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff21406a230 con 0x7ff2141041a0 2024-05-14T16:48:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.570+0000 7ff2037fe700 1 -- 172.21.15.69:0/1364858660 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7ff2040a9400 con 0x7ff2141041a0 2024-05-14T16:48:59.570 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:48:59.570 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:47:04.867362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24433,24446,24422]},"id":1}]} 2024-05-14T16:48:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 -- 172.21.15.69:0/1364858660 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff1fc0b2880 msgr2=0x7ff1fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff1fc0b2880 0x7ff1fc0b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff21419ab00 tx=0x7ff208005c40 comp rx=0 tx=0).stop 2024-05-14T16:48:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 -- 172.21.15.69:0/1364858660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2141041a0 msgr2=0x7ff21419fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2141041a0 0x7ff21419fd40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff20400d950 tx=0x7ff20400dc60 comp rx=0 tx=0).stop 2024-05-14T16:48:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 -- 172.21.15.69:0/1364858660 shutdown_connections 2024-05-14T16:48:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2141088f0 0x7ff2141999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2141037f0 0x7ff21419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff1fc0b2880 0x7ff1fc0b4d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 --2- 172.21.15.69:0/1364858660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2141041a0 0x7ff21419fd40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 -- 172.21.15.69:0/1364858660 >> 172.21.15.69:0/1364858660 conn(0x7ff21407a6a0 msgr2=0x7ff2141018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 -- 172.21.15.69:0/1364858660 shutdown_connections 2024-05-14T16:48:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:48:59.572+0000 7ff218ce1700 1 -- 172.21.15.69:0/1364858660 wait complete. 2024-05-14T16:48:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-05-14T16:48:59.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:59 smithi069 ceph-mon[31507]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:59.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:48:59 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1364858660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:48:59.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:59 smithi203 ceph-mon[35787]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:48:59.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:48:59 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1364858660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:49:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:59 smithi092 ceph-mon[35763]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:48:59 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1364858660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:49:00.262 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T16:46:45.333110+0000', 'modified': '2024-05-14T16:47:04.867362+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24446, 'mds_1': 24433, 'mds_2': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24395': {'gid': 24395, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6835/2453543352', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 2453543352}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 2453543352}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/3334129544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 3334129544}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 3334129544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6837/1824212740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 1824212740}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 1824212740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.92:6837/1194779586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 1194779586}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1194779586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.203:6839/398835794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 398835794}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 398835794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6839/2169416341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 2169416341}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 2169416341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24446, 'qdb_cluster': [24433, 24446, 24422]} 2024-05-14T16:49:00.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T16:49:00.263 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-14T16:49:00.501 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:01.933 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:01 smithi069 ceph-mon[31507]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:01.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:01 smithi203 ceph-mon[35787]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:01 smithi092 ceph-mon[35763]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:02.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.166+0000 7fb8136fc700 1 -- 172.21.15.69:0/200079108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80c102470 msgr2=0x7fb80c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:02.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.166+0000 7fb8136fc700 1 --2- 172.21.15.69:0/200079108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80c102470 0x7fb80c1028f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009990 tx=0x7fb7fc009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.166+0000 7fb8136fc700 1 -- 172.21.15.69:0/200079108 shutdown_connections 2024-05-14T16:49:02.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.166+0000 7fb8136fc700 1 --2- 172.21.15.69:0/200079108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb80c102e30 0x7fb80c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.166+0000 7fb8136fc700 1 --2- 172.21.15.69:0/200079108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb80c1084b0 0x7fb80c108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.166+0000 7fb8136fc700 1 --2- 172.21.15.69:0/200079108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80c102470 0x7fb80c1028f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.166+0000 7fb8136fc700 1 -- 172.21.15.69:0/200079108 >> 172.21.15.69:0/200079108 conn(0x7fb80c079410 msgr2=0x7fb80c079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.167+0000 7fb8136fc700 1 -- 172.21.15.69:0/200079108 shutdown_connections 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.167+0000 7fb8136fc700 1 -- 172.21.15.69:0/200079108 wait complete. 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.167+0000 7fb8136fc700 1 Processor -- start 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.167+0000 7fb8136fc700 1 -- start start 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.167+0000 7fb8136fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb80c102470 0x7fb80c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.167+0000 7fb8136fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb80c102e30 0x7fb80c19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.167+0000 7fb8136fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80c1084b0 0x7fb80c1a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.167+0000 7fb8136fc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1126c0 con 0x7fb80c1084b0 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.167+0000 7fb8136fc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c112540 con 0x7fb80c102470 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.167+0000 7fb8136fc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c112840 con 0x7fb80c102e30 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb811c99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80c1084b0 0x7fb80c1a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb811c99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80c1084b0 0x7fb80c1a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49530/0 (socket says 172.21.15.69:49530) 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb811c99700 1 -- 172.21.15.69:0/3060153849 learned_addr learned my addr 172.21.15.69:0/3060153849 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb811498700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb80c102470 0x7fb80c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb810c97700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb80c102e30 0x7fb80c19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb811c99700 1 -- 172.21.15.69:0/3060153849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb80c102e30 msgr2=0x7fb80c19b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb811c99700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb80c102e30 0x7fb80c19b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb811c99700 1 -- 172.21.15.69:0/3060153849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb80c102470 msgr2=0x7fb80c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb811c99700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb80c102470 0x7fb80c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb811c99700 1 -- 172.21.15.69:0/3060153849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c1a6260 con 0x7fb80c1084b0 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb811c99700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80c1084b0 0x7fb80c1a5d20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb80400b970 tx=0x7fb80400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb8027fc700 1 -- 172.21.15.69:0/3060153849 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804019070 con 0x7fb80c1084b0 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb8136fc700 1 -- 172.21.15.69:0/3060153849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c1a6490 con 0x7fb80c1084b0 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb8027fc700 1 -- 172.21.15.69:0/3060153849 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb80400cde0 con 0x7fb80c1084b0 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb8027fc700 1 -- 172.21.15.69:0/3060153849 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804007370 con 0x7fb80c1084b0 2024-05-14T16:49:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.168+0000 7fb8136fc700 1 -- 172.21.15.69:0/3060153849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c1a69a0 con 0x7fb80c1084b0 2024-05-14T16:49:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.171+0000 7fb8027fc700 1 -- 172.21.15.69:0/3060153849 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb80401c070 con 0x7fb80c1084b0 2024-05-14T16:49:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.172+0000 7fb8027fc700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb7f80b2880 0x7fb7f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.172+0000 7fb811498700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb7f80b2880 0x7fb7f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.172+0000 7fb8027fc700 1 -- 172.21.15.69:0/3060153849 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb8040e1720 con 0x7fb80c1084b0 2024-05-14T16:49:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb811498700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb7f80b2880 0x7fb7f80b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb808009a10 tx=0x7fb808009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb81df92b80 1 Processor -- start 2024-05-14T16:49:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb81df92b80 1 -- start start 2024-05-14T16:49:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec218fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec218ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21900160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb81df92b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ec21855c40 con 0x55ec218ffd20 2024-05-14T16:49:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb81df92b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ec218457e0 con 0x55ec218fc020 2024-05-14T16:49:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb81df92b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ec21849820 con 0x55ec218fb700 2024-05-14T16:49:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb811c99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21900160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb811c99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21900160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49542/0 (socket says 172.21.15.69:49542) 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb811c99700 1 -- 172.21.15.69:0/2957743133 learned_addr learned my addr 172.21.15.69:0/2957743133 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb811498700 1 --2- 172.21.15.69:0/2957743133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec218fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.173+0000 7fb810c97700 1 --2- 172.21.15.69:0/2957743133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec218ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.174+0000 7fb811c99700 1 -- 172.21.15.69:0/2957743133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 msgr2=0x55ec218fbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.174+0000 7fb811c99700 1 --2- 172.21.15.69:0/2957743133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec218fbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.174+0000 7fb811c99700 1 -- 172.21.15.69:0/2957743133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 msgr2=0x55ec218ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.174+0000 7fb811c99700 1 --2- 172.21.15.69:0/2957743133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec218ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.174+0000 7fb811c99700 1 -- 172.21.15.69:0/2957743133 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec21795880 con 0x55ec218ffd20 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.174+0000 7fb810c97700 1 --2- 172.21.15.69:0/2957743133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec218ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.174+0000 7fb811c99700 1 --2- 172.21.15.69:0/2957743133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21900160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb8040a4dc0 tx=0x7fb804004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14f1f713df9f4f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.174+0000 7fb7f77fe700 1 -- 172.21.15.69:0/2957743133 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8040ad0f0 con 0x55ec218ffd20 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.174+0000 7fb7f77fe700 1 -- 172.21.15.69:0/2957743133 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8040b1ba0 con 0x55ec218ffd20 2024-05-14T16:49:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.174+0000 7fb7f77fe700 1 -- 172.21.15.69:0/2957743133 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8040b1970 con 0x55ec218ffd20 2024-05-14T16:49:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 -- 172.21.15.69:0/2957743133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 msgr2=0x55ec21900160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 --2- 172.21.15.69:0/2957743133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21900160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb8040a4dc0 tx=0x7fb804004ad0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 -- 172.21.15.69:0/2957743133 shutdown_connections 2024-05-14T16:49:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 --2- 172.21.15.69:0/2957743133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21900160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 --2- 172.21.15.69:0/2957743133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec218fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 --2- 172.21.15.69:0/2957743133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec218ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 -- 172.21.15.69:0/2957743133 >> 172.21.15.69:0/2957743133 conn(0x55ec21822200 msgr2=0x55ec218f1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 -- 172.21.15.69:0/2957743133 shutdown_connections 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 -- 172.21.15.69:0/2957743133 wait complete. 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 -- start start 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.175+0000 7fb81df92b80 1 Processor -- start 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec21907240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec219045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21904b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb81df92b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ec2174bf50 con 0x55ec218ffd20 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb81df92b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ec217e7770 con 0x55ec218fc020 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb81df92b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ec21907f80 con 0x55ec218fb700 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb81df92b80 10 client.0 _ll_register_callbacks cb 0x55ec217dc530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb811c99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21904b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb811c99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21904b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49548/0 (socket says 172.21.15.69:49548) 2024-05-14T16:49:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb811c99700 1 -- 172.21.15.69:0/1484716003 learned_addr learned my addr 172.21.15.69:0/1484716003 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb811498700 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec21907240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb810c97700 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec219045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb811c99700 1 -- 172.21.15.69:0/1484716003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 msgr2=0x55ec21907240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb811c99700 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec21907240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb811c99700 1 -- 172.21.15.69:0/1484716003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 msgr2=0x55ec219045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb811c99700 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec219045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb811c99700 1 -- 172.21.15.69:0/1484716003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec217e4640 con 0x55ec218ffd20 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb811c99700 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21904b20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb8040a8a40 tx=0x7fb8040c33f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e05a947537fcd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb7f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1484716003 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8040b1710 con 0x55ec218ffd20 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1484716003 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb8040b18b0 con 0x55ec218ffd20 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb81df92b80 1 -- 172.21.15.69:0/1484716003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8040b1ba0 con 0x55ec218ffd20 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1484716003 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb80400c930 con 0x55ec218ffd20 2024-05-14T16:49:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb81df92b80 10 client.14976 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.176+0000 7fb81df92b80 1 -- 172.21.15.69:0/1484716003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ec21912650 con 0x55ec218ffd20 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.177+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1484716003 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8040b1ba0 con 0x55ec218ffd20 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.177+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1484716003 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb8040a7a10 con 0x55ec218ffd20 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.177+0000 7fb81df92b80 10 client.14976 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.177+0000 7fb81df92b80 10 client.14976 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.177+0000 7fb81df92b80 20 client.14976 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.177+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55ec219139a0 0x55ec21915b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.177+0000 7fb81df92b80 4 client.14976 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.177+0000 7fb81df92b80 1 -- 172.21.15.69:0/1484716003 --> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec218c3310 con 0x55ec219139a0 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.177+0000 7fb811498700 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55ec219139a0 0x55ec21915b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.178+0000 7fb811498700 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55ec219139a0 0x55ec21915b70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1f4b0cd2737cc94 server_cookie=62122193a06ddbfa in_seq=0 out_seq=0 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.178+0000 7fb7f4ff9700 10 client.14976 ms_handle_connect on v2:172.21.15.92:6836/1194779586 2024-05-14T16:49:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.178+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1484716003 <== mds.2 v2:172.21.15.92:6836/1194779586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec218c3310 con 0x55ec219139a0 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.178+0000 7fb7f4ff9700 10 client.14976 handle_command_reply: tid=0 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 client.14976 shutdown 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 -- 172.21.15.69:0/1484716003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 msgr2=0x55ec21904b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21904b20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb8040a8a40 tx=0x7fb8040c33f0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 -- 172.21.15.69:0/1484716003 shutdown_connections 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55ec219139a0 0x55ec21915b70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21904b20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec21907240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1484716003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec219045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb7f4ff9700 0 client.14976 ms_handle_reset on v2:172.21.15.92:6836/1194779586 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 -- 172.21.15.69:0/1484716003 >> 172.21.15.69:0/1484716003 conn(0x55ec21822200 msgr2=0x55ec219011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 -- 172.21.15.69:0/1484716003 shutdown_connections 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 1 -- 172.21.15.69:0/1484716003 wait complete. 2024-05-14T16:49:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.179+0000 7fb81df92b80 20 client.14976 trim_cache size 0 max 16384 2024-05-14T16:49:02.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb81df92b80 1 Processor -- start 2024-05-14T16:49:02.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb81df92b80 1 -- start start 2024-05-14T16:49:02.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218fb700 0x55ec218ebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fc020 0x55ec21901110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218ffd20 0x55ec218f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb81df92b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ec21913eb0 con 0x55ec218fb700 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb81df92b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ec21914580 con 0x55ec218ffd20 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb81df92b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ec21914c50 con 0x55ec218fc020 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb811498700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218fb700 0x55ec218ebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb811498700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218fb700 0x55ec218ebdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49552/0 (socket says 172.21.15.69:49552) 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb811498700 1 -- 172.21.15.69:0/507799165 learned_addr learned my addr 172.21.15.69:0/507799165 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb810c97700 1 --2- 172.21.15.69:0/507799165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fc020 0x55ec21901110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb811498700 1 -- 172.21.15.69:0/507799165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fc020 msgr2=0x55ec21901110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb811498700 1 --2- 172.21.15.69:0/507799165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fc020 0x55ec21901110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb811498700 1 -- 172.21.15.69:0/507799165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218ffd20 msgr2=0x55ec218f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb811498700 1 --2- 172.21.15.69:0/507799165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218ffd20 0x55ec218f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb811498700 1 -- 172.21.15.69:0/507799165 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec21916ad0 con 0x55ec218fb700 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.200+0000 7fb811498700 1 --2- 172.21.15.69:0/507799165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218fb700 0x55ec218ebdc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb808006ea0 tx=0x7fb808015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6e1290775584d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.201+0000 7fb7f6ffd700 1 -- 172.21.15.69:0/507799165 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80801b040 con 0x55ec218fb700 2024-05-14T16:49:02.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.201+0000 7fb7f6ffd700 1 -- 172.21.15.69:0/507799165 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb80802f070 con 0x55ec218fb700 2024-05-14T16:49:02.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.201+0000 7fb7f6ffd700 1 -- 172.21.15.69:0/507799165 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808015640 con 0x55ec218fb700 2024-05-14T16:49:02.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.201+0000 7fb81df92b80 1 -- 172.21.15.69:0/507799165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218fb700 msgr2=0x55ec218ebdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:02.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.201+0000 7fb81df92b80 1 --2- 172.21.15.69:0/507799165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218fb700 0x55ec218ebdc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb808006ea0 tx=0x7fb808015000 comp rx=0 tx=0).stop 2024-05-14T16:49:02.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.201+0000 7fb81df92b80 1 -- 172.21.15.69:0/507799165 shutdown_connections 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.201+0000 7fb81df92b80 1 --2- 172.21.15.69:0/507799165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218fb700 0x55ec218ebdc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.201+0000 7fb81df92b80 1 --2- 172.21.15.69:0/507799165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218ffd20 0x55ec218f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.201+0000 7fb81df92b80 1 --2- 172.21.15.69:0/507799165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fc020 0x55ec21901110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.201+0000 7fb81df92b80 1 -- 172.21.15.69:0/507799165 >> 172.21.15.69:0/507799165 conn(0x55ec21822200 msgr2=0x55ec218f6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 1 -- 172.21.15.69:0/507799165 shutdown_connections 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 1 -- 172.21.15.69:0/507799165 wait complete. 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 1 -- start start 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 1 Processor -- start 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec2191eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec2191f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21919fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ec219159f0 con 0x55ec218ffd20 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ec21915320 con 0x55ec218fc020 2024-05-14T16:49:02.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ec219160c0 con 0x55ec218fb700 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb81df92b80 10 client.0 _ll_register_callbacks cb 0x55ec217dc530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.202+0000 7fb811c99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21919fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb811c99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21919fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49568/0 (socket says 172.21.15.69:49568) 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb811c99700 1 -- 172.21.15.69:0/1889240026 learned_addr learned my addr 172.21.15.69:0/1889240026 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb811498700 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec2191eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb811c99700 1 -- 172.21.15.69:0/1889240026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 msgr2=0x55ec2191eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb811c99700 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec2191eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb811c99700 1 -- 172.21.15.69:0/1889240026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 msgr2=0x55ec2191f530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb811c99700 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec2191f530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb811c99700 1 -- 172.21.15.69:0/1889240026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec2191a6c0 con 0x55ec218ffd20 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb811498700 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec2191eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb811c99700 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21919fa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb8040a4dc0 tx=0x7fb8040a48e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225046527f3131d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb7f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1889240026 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8040a4c00 con 0x55ec218ffd20 2024-05-14T16:49:02.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1889240026 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb804014680 con 0x55ec218ffd20 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb81df92b80 1 -- 172.21.15.69:0/1889240026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb80802f070 con 0x55ec218ffd20 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1889240026 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb8040a9440 con 0x55ec218ffd20 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb81df92b80 10 client.14982 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.203+0000 7fb81df92b80 1 -- 172.21.15.69:0/1889240026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ec2191a6c0 con 0x55ec218ffd20 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.204+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1889240026 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb80802f070 con 0x55ec218ffd20 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.204+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1889240026 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb8040b17a0 con 0x55ec218ffd20 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.204+0000 7fb81df92b80 10 client.14982 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.204+0000 7fb81df92b80 10 client.14982 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.204+0000 7fb81df92b80 20 client.14982 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.204+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55ec21984ce0 0x55ec219839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.204+0000 7fb81df92b80 4 client.14982 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.204+0000 7fb81df92b80 1 -- 172.21.15.69:0/1889240026 --> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] -- command(tid 0: {"prefix": "status"}) -- 0x55ec21857b90 con 0x55ec21984ce0 2024-05-14T16:49:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.204+0000 7fb811498700 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55ec21984ce0 0x55ec219839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb7f4ff9700 10 client.14982 ms_handle_connect on v2:172.21.15.92:6836/1194779586 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb811498700 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55ec21984ce0 0x55ec219839f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2ac833ced09f1b1 server_cookie=dc4393aac01ca2db in_seq=0 out_seq=0 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb7f4ff9700 1 -- 172.21.15.69:0/1889240026 <== mds.2 v2:172.21.15.92:6836/1194779586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ec21857b90 con 0x55ec21984ce0 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb7f4ff9700 10 client.14982 handle_command_reply: tid=0 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 client.14982 shutdown 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 -- 172.21.15.69:0/1889240026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 msgr2=0x55ec21919fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21919fa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb8040a4dc0 tx=0x7fb8040a48e0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 -- 172.21.15.69:0/1889240026 shutdown_connections 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55ec21984ce0 0x55ec219839f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec218ffd20 0x55ec21919fa0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec218fb700 0x55ec2191eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 --2- 172.21.15.69:0/1889240026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec218fc020 0x55ec2191f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 -- 172.21.15.69:0/1889240026 >> 172.21.15.69:0/1889240026 conn(0x55ec21822200 msgr2=0x55ec218f6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb7f4ff9700 0 client.14982 ms_handle_reset on v2:172.21.15.92:6836/1194779586 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 -- 172.21.15.69:0/1889240026 shutdown_connections 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 1 -- 172.21.15.69:0/1889240026 wait complete. 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.205+0000 7fb81df92b80 20 client.14982 trim_cache size 0 max 16384 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24422, 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-05-14T16:49:02.210 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-05-14T16:49:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 122.67770214399999, 2024-05-14T16:49:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 24, 2024-05-14T16:49:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 80, 2024-05-14T16:49:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:49:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 166.47946140799999 2024-05-14T16:49:02.211 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 -- 172.21.15.69:0/3060153849 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb7f80b2880 msgr2=0x7fb7f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb7f80b2880 0x7fb7f80b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb808009a10 tx=0x7fb808009190 comp rx=0 tx=0).stop 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 -- 172.21.15.69:0/3060153849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80c1084b0 msgr2=0x7fb80c1a5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80c1084b0 0x7fb80c1a5d20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb80400b970 tx=0x7fb80400bc80 comp rx=0 tx=0).stop 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 -- 172.21.15.69:0/3060153849 shutdown_connections 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb7f80b2880 0x7fb7f80b4d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80c1084b0 0x7fb80c1a5d20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb80c102470 0x7fb80c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 --2- 172.21.15.69:0/3060153849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb80c102e30 0x7fb80c19b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 -- 172.21.15.69:0/3060153849 >> 172.21.15.69:0/3060153849 conn(0x7fb80c079410 msgr2=0x7fb80c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 -- 172.21.15.69:0/3060153849 shutdown_connections 2024-05-14T16:49:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:02.206+0000 7fb8136fc700 1 -- 172.21.15.69:0/3060153849 wait complete. 2024-05-14T16:49:02.884 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-14T16:49:03.126 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:03.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:03 smithi069 ceph-mon[31507]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T16:49:03.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:03 smithi203 ceph-mon[35787]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T16:49:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:03 smithi092 ceph-mon[35763]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T16:49:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 -- 172.21.15.69:0/3369068299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1c102e90 msgr2=0x7f1f1c10f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 --2- 172.21.15.69:0/3369068299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1c102e90 0x7f1f1c10f310 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009990 tx=0x7f1f14009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 -- 172.21.15.69:0/3369068299 shutdown_connections 2024-05-14T16:49:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 --2- 172.21.15.69:0/3369068299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1c102e90 0x7f1f1c10f310 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 --2- 172.21.15.69:0/3369068299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f1c1084d0 0x7f1f1c1088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 --2- 172.21.15.69:0/3369068299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f1c1024d0 0x7f1f1c102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 -- 172.21.15.69:0/3369068299 >> 172.21.15.69:0/3369068299 conn(0x7f1f1c078490 msgr2=0x7f1f1c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:04.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 -- 172.21.15.69:0/3369068299 shutdown_connections 2024-05-14T16:49:04.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 -- 172.21.15.69:0/3369068299 wait complete. 2024-05-14T16:49:04.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 Processor -- start 2024-05-14T16:49:04.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.895+0000 7f1f23744700 1 -- start start 2024-05-14T16:49:04.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f23744700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1c1024d0 0x7f1f1c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f23744700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f1c102e90 0x7f1f1c19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f23744700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f1c1084d0 0x7f1f1c1a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f23744700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1126e0 con 0x7f1f1c1024d0 2024-05-14T16:49:04.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f23744700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c112560 con 0x7f1f1c102e90 2024-05-14T16:49:04.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f23744700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c112860 con 0x7f1f1c1084d0 2024-05-14T16:49:04.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f214e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1c1024d0 0x7f1f1c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f214e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1c1024d0 0x7f1f1c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50954/0 (socket says 172.21.15.69:50954) 2024-05-14T16:49:04.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f214e0700 1 -- 172.21.15.69:0/635980666 learned_addr learned my addr 172.21.15.69:0/635980666 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:04.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f1c102e90 0x7f1f1c19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f21ce1700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f1c1084d0 0x7f1f1c1a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f214e0700 1 -- 172.21.15.69:0/635980666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f1c1084d0 msgr2=0x7f1f1c1a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:04.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f214e0700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f1c1084d0 0x7f1f1c1a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f214e0700 1 -- 172.21.15.69:0/635980666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f1c102e90 msgr2=0x7f1f1c19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:04.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f214e0700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f1c102e90 0x7f1f1c19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f214e0700 1 -- 172.21.15.69:0/635980666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c1a6100 con 0x7f1f1c1024d0 2024-05-14T16:49:04.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f21ce1700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f1c1084d0 0x7f1f1c1a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:04.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f214e0700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1c1024d0 0x7f1f1c19ab40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800ba50 tx=0x7f1f1800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:04.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.896+0000 7f1f127fc700 1 -- 172.21.15.69:0/635980666 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18019070 con 0x7f1f1c1024d0 2024-05-14T16:49:04.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.897+0000 7f1f23744700 1 -- 172.21.15.69:0/635980666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1c1a6330 con 0x7f1f1c1024d0 2024-05-14T16:49:04.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.897+0000 7f1f127fc700 1 -- 172.21.15.69:0/635980666 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f18014440 con 0x7f1f1c1024d0 2024-05-14T16:49:04.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.897+0000 7f1f127fc700 1 -- 172.21.15.69:0/635980666 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180075d0 con 0x7f1f1c1024d0 2024-05-14T16:49:04.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.897+0000 7f1f23744700 1 -- 172.21.15.69:0/635980666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1c1a6890 con 0x7f1f1c1024d0 2024-05-14T16:49:04.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.899+0000 7f1f127fc700 1 -- 172.21.15.69:0/635980666 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1f1801c070 con 0x7f1f1c1024d0 2024-05-14T16:49:04.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.899+0000 7f1f127fc700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f080b2880 0x7f1f080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f080b2880 0x7f1f080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f127fc700 1 -- 172.21.15.69:0/635980666 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1f180e16d0 con 0x7f1f1c1024d0 2024-05-14T16:49:04.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f080b2880 0x7f1f080b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c006fd0 tx=0x7f1f0c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:04.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f2dfdab80 1 Processor -- start 2024-05-14T16:49:04.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f2dfdab80 1 -- start start 2024-05-14T16:49:04.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c7202f0 0x55726c7206d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c7243d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c724910 0x55726c724d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55726c67ec40 con 0x55726c720c10 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55726c66e7e0 con 0x55726c724910 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.900+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55726c672820 con 0x55726c7202f0 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f20cdf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c7243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f20cdf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c7243d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50970/0 (socket says 172.21.15.69:50970) 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f20cdf700 1 -- 172.21.15.69:0/608931715 learned_addr learned my addr 172.21.15.69:0/608931715 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f21ce1700 1 --2- 172.21.15.69:0/608931715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c724910 0x55726c724d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f214e0700 1 --2- 172.21.15.69:0/608931715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c7202f0 0x55726c7206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f20cdf700 1 -- 172.21.15.69:0/608931715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c7202f0 msgr2=0x55726c7206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/608931715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c7202f0 0x55726c7206d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f20cdf700 1 -- 172.21.15.69:0/608931715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c724910 msgr2=0x55726c724d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/608931715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c724910 0x55726c724d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f20cdf700 1 -- 172.21.15.69:0/608931715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726c5be880 con 0x55726c720c10 2024-05-14T16:49:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f21ce1700 1 --2- 172.21.15.69:0/608931715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c724910 0x55726c724d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/608931715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c7243d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00d6a0 tx=0x7f1f0c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad01a5dfda98c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f077fe700 1 -- 172.21.15.69:0/608931715 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c024070 con 0x55726c720c10 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f077fe700 1 -- 172.21.15.69:0/608931715 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0c014af0 con 0x55726c720c10 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.901+0000 7f1f077fe700 1 -- 172.21.15.69:0/608931715 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c01fb50 con 0x55726c720c10 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.902+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/608931715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 msgr2=0x55726c7243d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.902+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/608931715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c7243d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00d6a0 tx=0x7f1f0c010000 comp rx=0 tx=0).stop 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.902+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/608931715 shutdown_connections 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.902+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/608931715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c724910 0x55726c724d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.902+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/608931715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c7202f0 0x55726c7206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.902+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/608931715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c7243d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.902+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/608931715 >> 172.21.15.69:0/608931715 conn(0x55726c64b200 msgr2=0x55726c71a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.902+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/608931715 shutdown_connections 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.902+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/608931715 wait complete. 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.902+0000 7f1f2dfdab80 1 -- start start 2024-05-14T16:49:04.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f2dfdab80 1 Processor -- start 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c72bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c720c10 0x55726c729150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c729690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55726c574f50 con 0x55726c7202f0 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55726c610770 con 0x55726c720c10 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55726c72cb70 con 0x55726c724910 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f214e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c72bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f214e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c72bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50974/0 (socket says 172.21.15.69:50974) 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f214e0700 1 -- 172.21.15.69:0/4143188608 learned_addr learned my addr 172.21.15.69:0/4143188608 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c720c10 0x55726c729150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f2dfdab80 10 client.0 _ll_register_callbacks cb 0x55726c605530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f21ce1700 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c729690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f214e0700 1 -- 172.21.15.69:0/4143188608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 msgr2=0x55726c729690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f214e0700 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c729690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f214e0700 1 -- 172.21.15.69:0/4143188608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c720c10 msgr2=0x55726c729150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.903+0000 7f1f214e0700 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c720c10 0x55726c729150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f214e0700 1 -- 172.21.15.69:0/4143188608 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726c60d640 con 0x55726c7202f0 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/4143188608 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f0c014af0 con 0x55726c7202f0 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f214e0700 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c72bd40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f180a89f0 tx=0x7f1f18014c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807e862a633c957c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f04ff9700 10 client.14994 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f04ff9700 1 -- 172.21.15.69:0/4143188608 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180c8050 con 0x55726c7202f0 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f04ff9700 1 -- 172.21.15.69:0/4143188608 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1f180b18d0 con 0x55726c7202f0 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f04ff9700 1 -- 172.21.15.69:0/4143188608 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1f180c33c0 con 0x55726c7202f0 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f2dfdab80 10 client.14994 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/4143188608 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=24}) -- 0x55726c7371e0 con 0x55726c7202f0 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f04ff9700 1 -- 172.21.15.69:0/4143188608 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1f180a7f50 con 0x55726c7202f0 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f2dfdab80 10 client.14994 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f2dfdab80 10 client.14994 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T16:49:04.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f2dfdab80 20 client.14994 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55726c7396d0 0x55726c73ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f2dfdab80 4 client.14994 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.904+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/4143188608 --> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55726c6ec310 con 0x55726c7396d0 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.905+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55726c7396d0 0x55726c73ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.905+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55726c7396d0 0x55726c73ab40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a04d4cf0e4ba00a3 server_cookie=ec544ac0f2a22357 in_seq=0 out_seq=0 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.905+0000 7f1f04ff9700 10 client.14994 ms_handle_connect on v2:172.21.15.203:6838/398835794 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f04ff9700 1 -- 172.21.15.69:0/4143188608 <== mds.1 v2:172.21.15.203:6838/398835794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55726c6ec310 con 0x55726c7396d0 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f04ff9700 10 client.14994 handle_command_reply: tid=0 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 client.14994 shutdown 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/4143188608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 msgr2=0x55726c72bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c72bd40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f180a89f0 tx=0x7f1f18014c10 comp rx=0 tx=0).stop 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/4143188608 shutdown_connections 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55726c7396d0 0x55726c73ab40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c72bd40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f04ff9700 0 client.14994 ms_handle_reset on v2:172.21.15.203:6838/398835794 2024-05-14T16:49:04.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c729690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/4143188608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c720c10 0x55726c729150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/4143188608 >> 172.21.15.69:0/4143188608 conn(0x55726c64b200 msgr2=0x55726c725de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:04.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/4143188608 shutdown_connections 2024-05-14T16:49:04.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/4143188608 wait complete. 2024-05-14T16:49:04.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.906+0000 7f1f2dfdab80 20 client.14994 trim_cache size 0 max 16384 2024-05-14T16:49:04.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f2dfdab80 1 Processor -- start 2024-05-14T16:49:04.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f2dfdab80 1 -- start start 2024-05-14T16:49:04.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c729700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c720c10 0x55726c7280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c71f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55726c738e80 con 0x55726c7202f0 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55726c739550 con 0x55726c720c10 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55726c739c20 con 0x55726c724910 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f214e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c729700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f214e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c729700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50984/0 (socket says 172.21.15.69:50984) 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f214e0700 1 -- 172.21.15.69:0/1612701557 learned_addr learned my addr 172.21.15.69:0/1612701557 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f21ce1700 1 --2- 172.21.15.69:0/1612701557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c71f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f214e0700 1 -- 172.21.15.69:0/1612701557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 msgr2=0x55726c71f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f214e0700 1 --2- 172.21.15.69:0/1612701557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c71f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f214e0700 1 -- 172.21.15.69:0/1612701557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c720c10 msgr2=0x55726c7280f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:04.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f214e0700 1 --2- 172.21.15.69:0/1612701557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c720c10 0x55726c7280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f214e0700 1 -- 172.21.15.69:0/1612701557 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726c715a30 con 0x55726c7202f0 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.927+0000 7f1f21ce1700 1 --2- 172.21.15.69:0/1612701557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c71f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f214e0700 1 --2- 172.21.15.69:0/1612701557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c729700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f180a89a0 tx=0x7f1f1801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572f292d55b58fae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f06ffd700 1 -- 172.21.15.69:0/1612701557 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180c8070 con 0x55726c7202f0 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f06ffd700 1 -- 172.21.15.69:0/1612701557 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0c014af0 con 0x55726c7202f0 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f06ffd700 1 -- 172.21.15.69:0/1612701557 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180a7f50 con 0x55726c7202f0 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1612701557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 msgr2=0x55726c729700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/1612701557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c729700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f180a89a0 tx=0x7f1f1801f000 comp rx=0 tx=0).stop 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1612701557 shutdown_connections 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/1612701557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c7202f0 0x55726c729700 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/1612701557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c71f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/1612701557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c720c10 0x55726c7280f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1612701557 >> 172.21.15.69:0/1612701557 conn(0x55726c64b200 msgr2=0x55726c71f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:04.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1612701557 shutdown_connections 2024-05-14T16:49:04.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.928+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1612701557 wait complete. 2024-05-14T16:49:04.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f2dfdab80 1 -- start start 2024-05-14T16:49:04.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f2dfdab80 1 Processor -- start 2024-05-14T16:49:04.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c7202f0 0x55726c722b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c721330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f2dfdab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c721870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55726c73a9c0 con 0x55726c720c10 2024-05-14T16:49:04.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55726c73a2f0 con 0x55726c7202f0 2024-05-14T16:49:04.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f2dfdab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55726c73b090 con 0x55726c724910 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f2dfdab80 10 client.0 _ll_register_callbacks cb 0x55726c605530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f20cdf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c721330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f20cdf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c721330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50992/0 (socket says 172.21.15.69:50992) 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f214e0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c7202f0 0x55726c722b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.929+0000 7f1f20cdf700 1 -- 172.21.15.69:0/1791973381 learned_addr learned my addr 172.21.15.69:0/1791973381 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f21ce1700 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c721870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f20cdf700 1 -- 172.21.15.69:0/1791973381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 msgr2=0x55726c721870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c721870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f20cdf700 1 -- 172.21.15.69:0/1791973381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c7202f0 msgr2=0x55726c722b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c7202f0 0x55726c722b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f20cdf700 1 -- 172.21.15.69:0/1791973381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726c7220b0 con 0x55726c720c10 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f20cdf700 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c721330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00d6a0 tx=0x7f1f0c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec0d26fcf6ffbd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f04ff9700 1 -- 172.21.15.69:0/1791973381 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c024040 con 0x55726c720c10 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f04ff9700 1 -- 172.21.15.69:0/1791973381 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1f0c00dd40 con 0x55726c720c10 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1791973381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f0c014af0 con 0x55726c720c10 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f04ff9700 1 -- 172.21.15.69:0/1791973381 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1f0c014760 con 0x55726c720c10 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f2dfdab80 10 client.15000 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1791973381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55726c743e80 con 0x55726c720c10 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.930+0000 7f1f04ff9700 1 -- 172.21.15.69:0/1791973381 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0c014af0 con 0x55726c720c10 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f04ff9700 1 -- 172.21.15.69:0/1791973381 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1f180145f0 con 0x55726c720c10 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f2dfdab80 10 client.15000 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f2dfdab80 10 client.15000 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f2dfdab80 20 client.15000 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55726c7aa730 0x55726c7a9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f2dfdab80 4 client.15000 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:49:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1791973381 --> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] -- command(tid 0: {"prefix": "status"}) -- 0x55726c680b90 con 0x55726c7aa730 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f214e0700 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55726c7aa730 0x55726c7a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f04ff9700 10 client.15000 ms_handle_connect on v2:172.21.15.203:6838/398835794 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f214e0700 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55726c7aa730 0x55726c7a9440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0eab973c0c3c0ca server_cookie=8c2897f94982b449 in_seq=0 out_seq=0 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f04ff9700 1 -- 172.21.15.69:0/1791973381 <== mds.1 v2:172.21.15.203:6838/398835794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55726c680b90 con 0x55726c7aa730 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.931+0000 7f1f04ff9700 10 client.15000 handle_command_reply: tid=0 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 client.15000 shutdown 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1791973381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 msgr2=0x55726c721330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c721330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00d6a0 tx=0x7f1f0c009750 comp rx=0 tx=0).stop 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1791973381 shutdown_connections 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55726c7aa730 0x55726c7a9440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55726c724910 0x55726c721870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f04ff9700 0 client.15000 ms_handle_reset on v2:172.21.15.203:6838/398835794 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55726c7202f0 0x55726c722b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 --2- 172.21.15.69:0/1791973381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55726c720c10 0x55726c721330 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1791973381 >> 172.21.15.69:0/1791973381 conn(0x55726c64b200 msgr2=0x55726c726cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:04.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1791973381 shutdown_connections 2024-05-14T16:49:04.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 1 -- 172.21.15.69:0/1791973381 wait complete. 2024-05-14T16:49:04.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f2dfdab80 20 client.15000 trim_cache size 0 max 16384 2024-05-14T16:49:04.938 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:49:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:49:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-05-14T16:49:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24433, 2024-05-14T16:49:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-05-14T16:49:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-05-14T16:49:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-05-14T16:49:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 126.410178062, 2024-05-14T16:49:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 24, 2024-05-14T16:49:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 80, 2024-05-14T16:49:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:49:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 166.23624589400001 2024-05-14T16:49:04.939 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:49:04.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f23744700 1 -- 172.21.15.69:0/635980666 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f080b2880 msgr2=0x7f1f080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:04.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f23744700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f080b2880 0x7f1f080b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c006fd0 tx=0x7f1f0c008040 comp rx=0 tx=0).stop 2024-05-14T16:49:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f23744700 1 -- 172.21.15.69:0/635980666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1c1024d0 msgr2=0x7f1f1c19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f23744700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1c1024d0 0x7f1f1c19ab40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800ba50 tx=0x7f1f1800be10 comp rx=0 tx=0).stop 2024-05-14T16:49:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f23744700 1 -- 172.21.15.69:0/635980666 shutdown_connections 2024-05-14T16:49:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f23744700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f1f080b2880 0x7f1f080b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f23744700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1c1024d0 0x7f1f1c19ab40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.932+0000 7f1f23744700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f1c1084d0 0x7f1f1c1a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.933+0000 7f1f23744700 1 --2- 172.21.15.69:0/635980666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f1c102e90 0x7f1f1c19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.933+0000 7f1f23744700 1 -- 172.21.15.69:0/635980666 >> 172.21.15.69:0/635980666 conn(0x7f1f1c078490 msgr2=0x7f1f1c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.933+0000 7f1f23744700 1 -- 172.21.15.69:0/635980666 shutdown_connections 2024-05-14T16:49:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:04.933+0000 7f1f23744700 1 -- 172.21.15.69:0/635980666 wait complete. 2024-05-14T16:49:05.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T16:49:05.918 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:05.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:05 smithi069 ceph-mon[31507]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T16:49:05.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:05 smithi203 ceph-mon[35787]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T16:49:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:05 smithi092 ceph-mon[35763]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T16:49:07.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.630+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3813589174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d81084f0 msgr2=0x7fc3d81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.630+0000 7fc3dfdd7700 1 --2- 172.21.15.69:0/3813589174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d81084f0 0x7fc3d81088d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009990 tx=0x7fc3cc009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3813589174 shutdown_connections 2024-05-14T16:49:07.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 --2- 172.21.15.69:0/3813589174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d81084f0 0x7fc3d81088d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 --2- 172.21.15.69:0/3813589174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d8102e20 0x7fc3d810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 --2- 172.21.15.69:0/3813589174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3d8102460 0x7fc3d81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3813589174 >> 172.21.15.69:0/3813589174 conn(0x7fc3d80794e0 msgr2=0x7fc3d80798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3813589174 shutdown_connections 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3813589174 wait complete. 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 Processor -- start 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 -- start start 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3d8102460 0x7fc3d80ff490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8102e20 0x7fc3d80ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d81084f0 0x7fc3d80fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8112600 con 0x7fc3d8102e20 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8112480 con 0x7fc3d8102460 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.631+0000 7fc3dfdd7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8112780 con 0x7fc3d81084f0 2024-05-14T16:49:07.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dd372700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8102e20 0x7fc3d80ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dd372700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8102e20 0x7fc3d80ff9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51016/0 (socket says 172.21.15.69:51016) 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dd372700 1 -- 172.21.15.69:0/3258461437 learned_addr learned my addr 172.21.15.69:0/3258461437 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3d8102460 0x7fc3d80ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3de374700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d81084f0 0x7fc3d80fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dd372700 1 -- 172.21.15.69:0/3258461437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d81084f0 msgr2=0x7fc3d80fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dd372700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d81084f0 0x7fc3d80fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dd372700 1 -- 172.21.15.69:0/3258461437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3d8102460 msgr2=0x7fc3d80ff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dd372700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3d8102460 0x7fc3d80ff490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dd372700 1 -- 172.21.15.69:0/3258461437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d81a8150 con 0x7fc3d8102e20 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3de374700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d81084f0 0x7fc3d80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3d8102460 0x7fc3d80ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dd372700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8102e20 0x7fc3d80ff9d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400d950 tx=0x7fc3d400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:07.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3caffd700 1 -- 172.21.15.69:0/3258461437 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4016070 con 0x7fc3d8102e20 2024-05-14T16:49:07.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3258461437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d81a8440 con 0x7fc3d8102e20 2024-05-14T16:49:07.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3caffd700 1 -- 172.21.15.69:0/3258461437 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d4010460 con 0x7fc3d8102e20 2024-05-14T16:49:07.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3caffd700 1 -- 172.21.15.69:0/3258461437 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d400f620 con 0x7fc3d8102e20 2024-05-14T16:49:07.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.632+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3258461437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d81a89d0 con 0x7fc3d8102e20 2024-05-14T16:49:07.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.635+0000 7fc3caffd700 1 -- 172.21.15.69:0/3258461437 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc3d4019070 con 0x7fc3d8102e20 2024-05-14T16:49:07.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.635+0000 7fc3caffd700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3c40b2880 0x7fc3c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.636+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3c40b2880 0x7fc3c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.636+0000 7fc3caffd700 1 -- 172.21.15.69:0/3258461437 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc3d40e1ed0 con 0x7fc3d8102e20 2024-05-14T16:49:07.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.636+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3c40b2880 0x7fc3c40b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc006010 tx=0x7fc3cc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.636+0000 7fc3ea66db80 1 Processor -- start 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.636+0000 7fc3ea66db80 1 -- start start 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 0x55b9d00f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d00fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00fd910 0x55b9d00fdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b9d0057c40 con 0x55b9d00f9c10 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9d00477e0 con 0x55b9d00fd910 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b9d004b820 con 0x55b9d00f92f0 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3dd372700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d00fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3dd372700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d00fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51018/0 (socket says 172.21.15.69:51018) 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3dd372700 1 -- 172.21.15.69:0/3823528828 learned_addr learned my addr 172.21.15.69:0/3823528828 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/3823528828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 0x55b9d00f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3de374700 1 --2- 172.21.15.69:0/3823528828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00fd910 0x55b9d00fdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3dd372700 1 -- 172.21.15.69:0/3823528828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 msgr2=0x55b9d00f96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:07.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3dd372700 1 --2- 172.21.15.69:0/3823528828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 0x55b9d00f96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3dd372700 1 -- 172.21.15.69:0/3823528828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00fd910 msgr2=0x55b9d00fdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:07.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3dd372700 1 --2- 172.21.15.69:0/3823528828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00fd910 0x55b9d00fdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3dd372700 1 -- 172.21.15.69:0/3823528828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9cff97880 con 0x55b9d00f9c10 2024-05-14T16:49:07.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/3823528828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 0x55b9d00f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:07.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3dd372700 1 --2- 172.21.15.69:0/3823528828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d00fd3d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40a91f0 tx=0x7fc3d40a90d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be378591e5692282 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:07.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3c3fff700 1 -- 172.21.15.69:0/3823528828 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4020070 con 0x55b9d00f9c10 2024-05-14T16:49:07.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3c3fff700 1 -- 172.21.15.69:0/3823528828 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d40ad8a0 con 0x55b9d00f9c10 2024-05-14T16:49:07.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.637+0000 7fc3c3fff700 1 -- 172.21.15.69:0/3823528828 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40b1ec0 con 0x55b9d00f9c10 2024-05-14T16:49:07.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.638+0000 7fc3ea66db80 1 -- 172.21.15.69:0/3823528828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 msgr2=0x55b9d00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.638+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/3823528828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d00fd3d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40a91f0 tx=0x7fc3d40a90d0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.638+0000 7fc3ea66db80 1 -- 172.21.15.69:0/3823528828 shutdown_connections 2024-05-14T16:49:07.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.638+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/3823528828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00fd910 0x55b9d00fdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.638+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/3823528828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 0x55b9d00f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.638+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/3823528828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d00fd3d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.638+0000 7fc3ea66db80 1 -- 172.21.15.69:0/3823528828 >> 172.21.15.69:0/3823528828 conn(0x55b9d0024200 msgr2=0x55b9d00f3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.638+0000 7fc3ea66db80 1 -- 172.21.15.69:0/3823528828 shutdown_connections 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.638+0000 7fc3ea66db80 1 -- 172.21.15.69:0/3823528828 wait complete. 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3ea66db80 1 -- start start 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3ea66db80 1 Processor -- start 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f92f0 0x55b9d0104dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d0102170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00fd910 0x55b9d01026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b9cff4df50 con 0x55b9d00f9c10 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9cffe9770 con 0x55b9d00f92f0 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b9d0105b10 con 0x55b9d00fd910 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3ea66db80 10 client.0 _ll_register_callbacks cb 0x55b9cffde530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3dd372700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d0102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3ddb73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f92f0 0x55b9d0104dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3dd372700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d0102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51022/0 (socket says 172.21.15.69:51022) 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3dd372700 1 -- 172.21.15.69:0/2105643151 learned_addr learned my addr 172.21.15.69:0/2105643151 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.639+0000 7fc3de374700 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00fd910 0x55b9d01026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3dd372700 1 -- 172.21.15.69:0/2105643151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00fd910 msgr2=0x55b9d01026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3dd372700 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00fd910 0x55b9d01026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3dd372700 1 -- 172.21.15.69:0/2105643151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f92f0 msgr2=0x55b9d0104dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3dd372700 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f92f0 0x55b9d0104dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3dd372700 1 -- 172.21.15.69:0/2105643151 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9cffe6640 con 0x55b9d00f9c10 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3de374700 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00fd910 0x55b9d01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3dd372700 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d0102170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400b2f0 tx=0x7fc3d4010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564c6e208e73d603 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2105643151 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40c2020 con 0x55b9d00f9c10 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2105643151 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc3d40a9750 con 0x55b9d00f9c10 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2105643151 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d40ad8a0 con 0x55b9d00f9c10 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2105643151 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc3d4020070 con 0x55b9d00f9c10 2024-05-14T16:49:07.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3ea66db80 10 client.15012 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2105643151 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b9d0110210 con 0x55b9d00f9c10 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2105643151 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d40ad8a0 con 0x55b9d00f9c10 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2105643151 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc3d40a4e60 con 0x55b9d00f9c10 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3ea66db80 10 client.15012 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3ea66db80 10 client.15012 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.640+0000 7fc3ea66db80 20 client.15012 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.641+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b9d0112340 0x55b9d01137b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.641+0000 7fc3ea66db80 4 client.15012 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.641+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2105643151 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9d00c5310 con 0x55b9d0112340 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.641+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b9d0112340 0x55b9d01137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.641+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b9d0112340 0x55b9d01137b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a052c004b0308071 server_cookie=5844eb064831a485 in_seq=0 out_seq=0 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.641+0000 7fc3c17fa700 10 client.15012 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2105643151 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b9d00c5310 con 0x55b9d0112340 2024-05-14T16:49:07.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3c17fa700 10 client.15012 handle_command_reply: tid=0 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 client.15012 shutdown 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2105643151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 msgr2=0x55b9d0102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d0102170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400b2f0 tx=0x7fc3d4010b60 comp rx=0 tx=0).stop 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2105643151 shutdown_connections 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b9d0112340 0x55b9d01137b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00fd910 0x55b9d01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f92f0 0x55b9d0104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3c17fa700 0 client.15012 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2105643151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00f9c10 0x55b9d0102170 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2105643151 >> 172.21.15.69:0/2105643151 conn(0x55b9d0024200 msgr2=0x55b9d00fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2105643151 shutdown_connections 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2105643151 wait complete. 2024-05-14T16:49:07.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.642+0000 7fc3ea66db80 20 client.15012 trim_cache size 0 max 16384 2024-05-14T16:49:07.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3ea66db80 1 Processor -- start 2024-05-14T16:49:07.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3ea66db80 1 -- start start 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 0x55b9d00edba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f9c10 0x55b9d00fb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d00fb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b9d0111af0 con 0x55b9d00fd910 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9d01121c0 con 0x55b9d00f9c10 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b9d0112890 con 0x55b9d00f92f0 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3de374700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d00fb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3de374700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d00fb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51036/0 (socket says 172.21.15.69:51036) 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3de374700 1 -- 172.21.15.69:0/440320410 learned_addr learned my addr 172.21.15.69:0/440320410 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/440320410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 0x55b9d00edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3de374700 1 -- 172.21.15.69:0/440320410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 msgr2=0x55b9d00edba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:07.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3de374700 1 --2- 172.21.15.69:0/440320410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 0x55b9d00edba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3de374700 1 -- 172.21.15.69:0/440320410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f9c10 msgr2=0x55b9d00fb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:07.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.664+0000 7fc3dd372700 1 --2- 172.21.15.69:0/440320410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f9c10 0x55b9d00fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.665+0000 7fc3de374700 1 --2- 172.21.15.69:0/440320410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f9c10 0x55b9d00fb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.665+0000 7fc3de374700 1 -- 172.21.15.69:0/440320410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d0111920 con 0x55b9d00fd910 2024-05-14T16:49:07.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.665+0000 7fc3dd372700 1 --2- 172.21.15.69:0/440320410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f9c10 0x55b9d00fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:07.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.665+0000 7fc3de374700 1 --2- 172.21.15.69:0/440320410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d00fb880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0011400 tx=0x7fc3d0011710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9779e20b28111ea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:07.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.665+0000 7fc3c37fe700 1 -- 172.21.15.69:0/440320410 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0019070 con 0x55b9d00fd910 2024-05-14T16:49:07.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.665+0000 7fc3c37fe700 1 -- 172.21.15.69:0/440320410 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d00158e0 con 0x55b9d00fd910 2024-05-14T16:49:07.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.665+0000 7fc3c37fe700 1 -- 172.21.15.69:0/440320410 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00149a0 con 0x55b9d00fd910 2024-05-14T16:49:07.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 -- 172.21.15.69:0/440320410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 msgr2=0x55b9d00fb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/440320410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d00fb880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0011400 tx=0x7fc3d0011710 comp rx=0 tx=0).stop 2024-05-14T16:49:07.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 -- 172.21.15.69:0/440320410 shutdown_connections 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/440320410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d00fb880 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/440320410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f92f0 0x55b9d00edba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/440320410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f9c10 0x55b9d00fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 -- 172.21.15.69:0/440320410 >> 172.21.15.69:0/440320410 conn(0x55b9d0024200 msgr2=0x55b9d00f8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 -- 172.21.15.69:0/440320410 shutdown_connections 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 -- 172.21.15.69:0/440320410 wait complete. 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 -- start start 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 Processor -- start 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f92f0 0x55b9d0119a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f9c10 0x55b9d0119fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d011a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b9d0113630 con 0x55b9d00fd910 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9d0112f60 con 0x55b9d00f92f0 2024-05-14T16:49:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.666+0000 7fc3ea66db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b9d0113d00 con 0x55b9d00f9c10 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3ea66db80 10 client.0 _ll_register_callbacks cb 0x55b9cffde530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3de374700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d011a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3de374700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d011a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51040/0 (socket says 172.21.15.69:51040) 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3de374700 1 -- 172.21.15.69:0/2891284401 learned_addr learned my addr 172.21.15.69:0/2891284401 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f92f0 0x55b9d0119a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3dd372700 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f9c10 0x55b9d0119fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3de374700 1 -- 172.21.15.69:0/2891284401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f9c10 msgr2=0x55b9d0119fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3de374700 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f9c10 0x55b9d0119fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3de374700 1 -- 172.21.15.69:0/2891284401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f92f0 msgr2=0x55b9d0119a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3de374700 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f92f0 0x55b9d0119a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3de374700 1 -- 172.21.15.69:0/2891284401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d00fd740 con 0x55b9d00fd910 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3dd372700 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f9c10 0x55b9d0119fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3de374700 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d011a4f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0015cc0 tx=0x7fc3d0015da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f04c31ae74a925 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2891284401 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0019030 con 0x55b9d00fd910 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2891284401 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc3d0014620 con 0x55b9d00fd910 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2891284401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d00158e0 con 0x55b9d00fd910 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2891284401 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc3d0007530 con 0x55b9d00fd910 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3ea66db80 10 client.15018 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.667+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2891284401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b9d017eaf0 con 0x55b9d00fd910 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2891284401 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d00158e0 con 0x55b9d00fd910 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2891284401 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc3d0026070 con 0x55b9d00fd910 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3ea66db80 10 client.15018 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3ea66db80 10 client.15018 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3ea66db80 20 client.15018 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b9d01832e0 0x55b9d0181ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3ea66db80 4 client.15018 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2891284401 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "status"}) -- 0x55b9d0059b90 con 0x55b9d01832e0 2024-05-14T16:49:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b9d01832e0 0x55b9d0181ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3c17fa700 10 client.15018 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.668+0000 7fc3ddb73700 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b9d01832e0 0x55b9d0181ff0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97af388282c5c78 server_cookie=161aca351937eaa5 in_seq=0 out_seq=0 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3c17fa700 1 -- 172.21.15.69:0/2891284401 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b9d0059b90 con 0x55b9d01832e0 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3c17fa700 10 client.15018 handle_command_reply: tid=0 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 client.15018 shutdown 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2891284401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 msgr2=0x55b9d011a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d011a4f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0015cc0 tx=0x7fc3d0015da0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2891284401 shutdown_connections 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b9d01832e0 0x55b9d0181ff0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9d00fd910 0x55b9d011a4f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9d00f92f0 0x55b9d0119a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3c17fa700 0 client.15018 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 --2- 172.21.15.69:0/2891284401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b9d00f9c10 0x55b9d0119fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2891284401 >> 172.21.15.69:0/2891284401 conn(0x55b9d0024200 msgr2=0x55b9d00f8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2891284401 shutdown_connections 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 1 -- 172.21.15.69:0/2891284401 wait complete. 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3ea66db80 20 client.15018 trim_cache size 0 max 16384 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24446, 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-05-14T16:49:07.675 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 142.106619274, 2024-05-14T16:49:07.676 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 24, 2024-05-14T16:49:07.676 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 80, 2024-05-14T16:49:07.676 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:49:07.676 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 159.07153916600001 2024-05-14T16:49:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:49:07.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3258461437 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3c40b2880 msgr2=0x7fc3c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3dfdd7700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3c40b2880 0x7fc3c40b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc006010 tx=0x7fc3cc01a040 comp rx=0 tx=0).stop 2024-05-14T16:49:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3258461437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8102e20 msgr2=0x7fc3d80ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.669+0000 7fc3dfdd7700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8102e20 0x7fc3d80ff9d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400d950 tx=0x7fc3d400dd10 comp rx=0 tx=0).stop 2024-05-14T16:49:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.670+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3258461437 shutdown_connections 2024-05-14T16:49:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.670+0000 7fc3dfdd7700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d81084f0 0x7fc3d80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.670+0000 7fc3dfdd7700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3d8102460 0x7fc3d80ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.670+0000 7fc3dfdd7700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc3c40b2880 0x7fc3c40b4d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.670+0000 7fc3dfdd7700 1 --2- 172.21.15.69:0/3258461437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8102e20 0x7fc3d80ff9d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.670+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3258461437 >> 172.21.15.69:0/3258461437 conn(0x7fc3d80794e0 msgr2=0x7fc3d806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.670+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3258461437 shutdown_connections 2024-05-14T16:49:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:07.670+0000 7fc3dfdd7700 1 -- 172.21.15.69:0/3258461437 wait complete. 2024-05-14T16:49:07.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:07 smithi069 ceph-mon[31507]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T16:49:07.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:07 smithi203 ceph-mon[35787]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T16:49:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:07 smithi092 ceph-mon[35763]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T16:49:08.382 INFO:tasks.cephadm:Setup complete, yielding 2024-05-14T16:49:08.383 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-14T16:49:08.392 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T16:49:08.392 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-14T16:49:08.631 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:09.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:09 smithi069 ceph-mon[31507]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T16:49:09.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:09 smithi203 ceph-mon[35787]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T16:49:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:09 smithi092 ceph-mon[35763]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T16:49:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.295+0000 7f6cd38c6700 1 -- 172.21.15.69:0/733189148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc1047a0 msgr2=0x7f6ccc104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.295+0000 7f6cd38c6700 1 --2- 172.21.15.69:0/733189148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc1047a0 0x7f6ccc104c00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80099c0 tx=0x7f6cc8009cd0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.295+0000 7f6cd38c6700 1 -- 172.21.15.69:0/733189148 shutdown_connections 2024-05-14T16:49:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.295+0000 7f6cd38c6700 1 --2- 172.21.15.69:0/733189148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc105140 0x7f6ccc10b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.295+0000 7f6cd38c6700 1 --2- 172.21.15.69:0/733189148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ccc1035a0 0x7f6ccc1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.295+0000 7f6cd38c6700 1 --2- 172.21.15.69:0/733189148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc1047a0 0x7f6ccc104c00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.295+0000 7f6cd38c6700 1 -- 172.21.15.69:0/733189148 >> 172.21.15.69:0/733189148 conn(0x7f6ccc0feb40 msgr2=0x7f6ccc100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.295+0000 7f6cd38c6700 1 -- 172.21.15.69:0/733189148 shutdown_connections 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.295+0000 7f6cd38c6700 1 -- 172.21.15.69:0/733189148 wait complete. 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd38c6700 1 Processor -- start 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd38c6700 1 -- start start 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd38c6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc1035a0 0x7f6ccc0772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd38c6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc1047a0 0x7f6ccc075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd38c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ccc105140 0x7f6ccc075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd38c6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc10f130 con 0x7f6ccc1035a0 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd38c6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc10efb0 con 0x7f6ccc105140 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd38c6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc10f2b0 con 0x7f6ccc1047a0 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd1662700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc1035a0 0x7f6ccc0772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd1662700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc1035a0 0x7f6ccc0772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51058/0 (socket says 172.21.15.69:51058) 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd1662700 1 -- 172.21.15.69:0/151101405 learned_addr learned my addr 172.21.15.69:0/151101405 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd1e63700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ccc105140 0x7f6ccc075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd0e61700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc1047a0 0x7f6ccc075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd1662700 1 -- 172.21.15.69:0/151101405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc1047a0 msgr2=0x7f6ccc075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd1662700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc1047a0 0x7f6ccc075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd1662700 1 -- 172.21.15.69:0/151101405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ccc105140 msgr2=0x7f6ccc075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd1662700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ccc105140 0x7f6ccc075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd1662700 1 -- 172.21.15.69:0/151101405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc076720 con 0x7f6ccc1035a0 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.296+0000 7f6cd0e61700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc1047a0 0x7f6ccc075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.297+0000 7f6cd1662700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc1035a0 0x7f6ccc0772e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00b500 tx=0x7f6cbc00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.297+0000 7f6cc27fc700 1 -- 172.21.15.69:0/151101405 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc015020 con 0x7f6ccc1035a0 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.297+0000 7f6cc27fc700 1 -- 172.21.15.69:0/151101405 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cbc011ce0 con 0x7f6ccc1035a0 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.297+0000 7f6cd38c6700 1 -- 172.21.15.69:0/151101405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc1a3940 con 0x7f6ccc1035a0 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.297+0000 7f6cc27fc700 1 -- 172.21.15.69:0/151101405 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc010d60 con 0x7f6ccc1035a0 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.297+0000 7f6cd38c6700 1 -- 172.21.15.69:0/151101405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc1a3f00 con 0x7f6ccc1035a0 2024-05-14T16:49:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.298+0000 7f6cd38c6700 1 -- 172.21.15.69:0/151101405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ccc06a230 con 0x7f6ccc1035a0 2024-05-14T16:49:10.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.299+0000 7f6cc27fc700 1 -- 172.21.15.69:0/151101405 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6cbc018020 con 0x7f6ccc1035a0 2024-05-14T16:49:10.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.300+0000 7f6cc27fc700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6cb80b2900 0x7f6cb80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:10.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.300+0000 7f6cd0e61700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6cb80b2900 0x7f6cb80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:10.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.300+0000 7f6cc27fc700 1 -- 172.21.15.69:0/151101405 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6cbc0e1f60 con 0x7f6ccc1035a0 2024-05-14T16:49:10.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.300+0000 7f6cd0e61700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6cb80b2900 0x7f6cb80b4dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8006010 tx=0x7f6cc8009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:10.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.304+0000 7f6cc27fc700 1 -- 172.21.15.69:0/151101405 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6cbc01f020 con 0x7f6ccc1035a0 2024-05-14T16:49:10.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.527+0000 7f6cd38c6700 1 -- 172.21.15.69:0/151101405 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f6ccc063980 con 0x7f6cb80b2900 2024-05-14T16:49:10.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.528+0000 7f6cc27fc700 1 -- 172.21.15.69:0/151101405 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f6ccc063980 con 0x7f6cb80b2900 2024-05-14T16:49:10.528 INFO:teuthology.orchestra.run.smithi069.stdout:Backend: cephadm 2024-05-14T16:49:10.528 INFO:teuthology.orchestra.run.smithi069.stdout:Available: Yes 2024-05-14T16:49:10.528 INFO:teuthology.orchestra.run.smithi069.stdout:Paused: No 2024-05-14T16:49:10.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 -- 172.21.15.69:0/151101405 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6cb80b2900 msgr2=0x7f6cb80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:10.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6cb80b2900 0x7f6cb80b4dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8006010 tx=0x7f6cc8009f10 comp rx=0 tx=0).stop 2024-05-14T16:49:10.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 -- 172.21.15.69:0/151101405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc1035a0 msgr2=0x7f6ccc0772e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:10.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc1035a0 0x7f6ccc0772e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00b500 tx=0x7f6cbc00b8c0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 -- 172.21.15.69:0/151101405 shutdown_connections 2024-05-14T16:49:10.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6cb80b2900 0x7f6cb80b4dc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc1035a0 0x7f6ccc0772e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ccc105140 0x7f6ccc075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 --2- 172.21.15.69:0/151101405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc1047a0 0x7f6ccc075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:10.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 -- 172.21.15.69:0/151101405 >> 172.21.15.69:0/151101405 conn(0x7f6ccc0feb40 msgr2=0x7f6ccc107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:10.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 -- 172.21.15.69:0/151101405 shutdown_connections 2024-05-14T16:49:10.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:10.530+0000 7f6cd38c6700 1 -- 172.21.15.69:0/151101405 wait complete. 2024-05-14T16:49:11.158 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-14T16:49:11.396 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:11.851 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:11 smithi069 ceph-mon[31507]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T16:49:11.852 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:11 smithi069 ceph-mon[31507]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:11.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:11 smithi203 ceph-mon[35787]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T16:49:11.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:11 smithi203 ceph-mon[35787]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:12.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:11 smithi092 ceph-mon[35763]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T16:49:12.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:11 smithi092 ceph-mon[35763]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:13.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.010+0000 7fbf05119700 1 -- 172.21.15.69:0/3356977478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf00101e10 msgr2=0x7fbf00104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:13.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.010+0000 7fbf05119700 1 --2- 172.21.15.69:0/3356977478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf00101e10 0x7fbf00104240 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009990 tx=0x7fbef0009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.010+0000 7fbf05119700 1 -- 172.21.15.69:0/3356977478 shutdown_connections 2024-05-14T16:49:13.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.010+0000 7fbf05119700 1 --2- 172.21.15.69:0/3356977478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf00101e10 0x7fbf00104240 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.010+0000 7fbf05119700 1 --2- 172.21.15.69:0/3356977478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf00107150 0x7fbf001075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.010+0000 7fbf05119700 1 --2- 172.21.15.69:0/3356977478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf00104780 0x7fbf00106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.010+0000 7fbf05119700 1 -- 172.21.15.69:0/3356977478 >> 172.21.15.69:0/3356977478 conn(0x7fbf000fda30 msgr2=0x7fbf000ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.010+0000 7fbf05119700 1 -- 172.21.15.69:0/3356977478 shutdown_connections 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.010+0000 7fbf05119700 1 -- 172.21.15.69:0/3356977478 wait complete. 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.011+0000 7fbf05119700 1 Processor -- start 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbf05119700 1 -- start start 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbf05119700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf00101e10 0x7fbf0019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbf05119700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf00104780 0x7fbf001a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbf05119700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf00107150 0x7fbf001a7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbf05119700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001155a0 con 0x7fbf00101e10 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbf05119700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00115420 con 0x7fbf00104780 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbf05119700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00115720 con 0x7fbf00107150 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbefed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf00101e10 0x7fbf0019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbefed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf00101e10 0x7fbf0019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51076/0 (socket says 172.21.15.69:51076) 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbefed9d700 1 -- 172.21.15.69:0/1141559867 learned_addr learned my addr 172.21.15.69:0/1141559867 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbeff59e700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf00107150 0x7fbf001a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbefe59c700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf00104780 0x7fbf001a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbefed9d700 1 -- 172.21.15.69:0/1141559867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf00107150 msgr2=0x7fbf001a7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbefed9d700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf00107150 0x7fbf001a7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbefed9d700 1 -- 172.21.15.69:0/1141559867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf00104780 msgr2=0x7fbf001a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbefed9d700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf00104780 0x7fbf001a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbefed9d700 1 -- 172.21.15.69:0/1141559867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf001a7900 con 0x7fbf00101e10 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.012+0000 7fbefed9d700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf00101e10 0x7fbf0019fc80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbef00036f0 tx=0x7fbef00039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.013+0000 7fbee7fff700 1 -- 172.21.15.69:0/1141559867 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0027070 con 0x7fbf00101e10 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.013+0000 7fbf05119700 1 -- 172.21.15.69:0/1141559867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf001a7b90 con 0x7fbf00101e10 2024-05-14T16:49:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.013+0000 7fbee7fff700 1 -- 172.21.15.69:0/1141559867 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef0022470 con 0x7fbf00101e10 2024-05-14T16:49:13.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.013+0000 7fbee7fff700 1 -- 172.21.15.69:0/1141559867 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef000f6c0 con 0x7fbf00101e10 2024-05-14T16:49:13.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.013+0000 7fbf05119700 1 -- 172.21.15.69:0/1141559867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf001a80a0 con 0x7fbf00101e10 2024-05-14T16:49:13.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.014+0000 7fbf05119700 1 -- 172.21.15.69:0/1141559867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0006a230 con 0x7fbf00101e10 2024-05-14T16:49:13.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.016+0000 7fbee7fff700 1 -- 172.21.15.69:0/1141559867 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbef002a070 con 0x7fbf00101e10 2024-05-14T16:49:13.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.016+0000 7fbee7fff700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbee80b2880 0x7fbee80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:13.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.016+0000 7fbefe59c700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbee80b2880 0x7fbee80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:13.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.016+0000 7fbee7fff700 1 -- 172.21.15.69:0/1141559867 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbef00e3650 con 0x7fbf00101e10 2024-05-14T16:49:13.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.017+0000 7fbefe59c700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbee80b2880 0x7fbee80b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbf001a1190 tx=0x7fbef4005eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:13.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.020+0000 7fbee7fff700 1 -- 172.21.15.69:0/1141559867 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbef0013070 con 0x7fbf00101e10 2024-05-14T16:49:13.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.229+0000 7fbf05119700 1 -- 172.21.15.69:0/1141559867 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbf00063980 con 0x7fbee80b2880 2024-05-14T16:49:13.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.243+0000 7fbee7fff700 1 -- 172.21.15.69:0/1141559867 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fbf00063980 con 0x7fbee80b2880 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:mds.a smithi069 running (2m) 30s ago 2m 17.0M - 19.0.0-3687-g9b638668 c922e207469e 96a8766521a5 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:mds.b smithi092 running (2m) 32s ago 2m 27.4M - 19.0.0-3687-g9b638668 c922e207469e 9aa688b58ad4 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:mds.c smithi203 running (3m) 32s ago 3m 22.0M - 19.0.0-3687-g9b638668 c922e207469e 37afd030daf7 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:mds.d smithi069 running (3m) 30s ago 3m 16.2M - 19.0.0-3687-g9b638668 c922e207469e e8b2847e0489 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:mds.e smithi092 running (3m) 32s ago 3m 19.5M - 19.0.0-3687-g9b638668 c922e207469e a61898b8a3a9 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:mds.f smithi203 running (3m) 32s ago 3m 19.3M - 19.0.0-3687-g9b638668 c922e207469e 46634c2ae77d 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:mds.g smithi069 running (2m) 30s ago 2m 16.1M - 19.0.0-3687-g9b638668 c922e207469e 5a6ffcf4a6db 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:mds.h smithi092 running (2m) 32s ago 2m 21.9M - 19.0.0-3687-g9b638668 c922e207469e 545cf6bb8536 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:mds.i smithi203 running (2m) 32s ago 2m 19.7M - 19.0.0-3687-g9b638668 c922e207469e 330076950df9 2024-05-14T16:49:13.243 INFO:teuthology.orchestra.run.smithi069.stdout:mds.j smithi069 running (3m) 30s ago 3m 16.3M - 19.0.0-3687-g9b638668 c922e207469e 3d50ce5a9d6e 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:mds.k smithi092 running (3m) 32s ago 3m 15.0M - 19.0.0-3687-g9b638668 c922e207469e 5ae9d798f763 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:mds.l smithi203 running (3m) 32s ago 3m 17.4M - 19.0.0-3687-g9b638668 c922e207469e 1986f28d5d06 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.x smithi069 *:9283,8765 running (10m) 30s ago 10m 530M - 19.0.0-3687-g9b638668 c922e207469e 24c9b3caf1d9 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.y smithi092 *:8443,8765 running (8m) 32s ago 8m 468M - 19.0.0-3687-g9b638668 c922e207469e d1f975a7b58b 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.z smithi203 *:8443,8765 running (8m) 32s ago 8m 470M - 19.0.0-3687-g9b638668 c922e207469e de0a6ecb1807 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:mon.a smithi069 running (10m) 30s ago 10m 51.1M 2048M 19.0.0-3687-g9b638668 c922e207469e 3fcc6b76fa60 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:mon.b smithi092 running (8m) 32s ago 8m 41.0M 2048M 19.0.0-3687-g9b638668 c922e207469e 9105d8cf405a 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:mon.c smithi203 running (8m) 32s ago 8m 40.9M 2048M 19.0.0-3687-g9b638668 c922e207469e 92ff3665e8b0 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 smithi069 running (8m) 30s ago 8m 45.3M 4272M 19.0.0-3687-g9b638668 c922e207469e 24665bcfc0b5 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 smithi092 running (7m) 32s ago 7m 46.5M 4272M 19.0.0-3687-g9b638668 c922e207469e 3c35fbb5141f 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 smithi203 running (7m) 32s ago 7m 47.5M 4272M 19.0.0-3687-g9b638668 c922e207469e f373bf8f1170 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 smithi069 running (7m) 30s ago 7m 47.5M 4272M 19.0.0-3687-g9b638668 c922e207469e e04c88b27f1e 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 smithi092 running (6m) 32s ago 6m 57.5M 4272M 19.0.0-3687-g9b638668 c922e207469e 6b3e9e8125c7 2024-05-14T16:49:13.244 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 smithi203 running (6m) 32s ago 6m 55.1M 4272M 19.0.0-3687-g9b638668 c922e207469e 3f0d1a8584f2 2024-05-14T16:49:13.245 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 smithi069 running (5m) 30s ago 5m 43.2M 4272M 19.0.0-3687-g9b638668 c922e207469e 2b8e94f60e67 2024-05-14T16:49:13.245 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 smithi092 running (5m) 32s ago 5m 43.8M 4272M 19.0.0-3687-g9b638668 c922e207469e 4d095482f907 2024-05-14T16:49:13.245 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 smithi203 running (4m) 32s ago 4m 43.1M 4272M 19.0.0-3687-g9b638668 c922e207469e c906dce75de9 2024-05-14T16:49:13.245 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 smithi069 running (4m) 30s ago 4m 53.7M 4272M 19.0.0-3687-g9b638668 c922e207469e 3dd1dc5b487b 2024-05-14T16:49:13.245 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 smithi092 running (4m) 32s ago 4m 55.6M 4272M 19.0.0-3687-g9b638668 c922e207469e 8ff65bc1ddcd 2024-05-14T16:49:13.245 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 smithi203 running (3m) 32s ago 3m 43.3M 4272M 19.0.0-3687-g9b638668 c922e207469e c7e4472d8467 2024-05-14T16:49:13.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.245+0000 7fbf05119700 1 -- 172.21.15.69:0/1141559867 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbee80b2880 msgr2=0x7fbee80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.245+0000 7fbf05119700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbee80b2880 0x7fbee80b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbf001a1190 tx=0x7fbef4005eb0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.245+0000 7fbf05119700 1 -- 172.21.15.69:0/1141559867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf00101e10 msgr2=0x7fbf0019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.245+0000 7fbf05119700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf00101e10 0x7fbf0019fc80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbef00036f0 tx=0x7fbef00039f0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.245+0000 7fbf05119700 1 -- 172.21.15.69:0/1141559867 shutdown_connections 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.245+0000 7fbf05119700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fbee80b2880 0x7fbee80b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.245+0000 7fbf05119700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf00101e10 0x7fbf0019fc80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.245+0000 7fbf05119700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf00107150 0x7fbf001a7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.245+0000 7fbf05119700 1 --2- 172.21.15.69:0/1141559867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf00104780 0x7fbf001a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.245+0000 7fbf05119700 1 -- 172.21.15.69:0/1141559867 >> 172.21.15.69:0/1141559867 conn(0x7fbf000fda30 msgr2=0x7fbf001051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.246+0000 7fbf05119700 1 -- 172.21.15.69:0/1141559867 shutdown_connections 2024-05-14T16:49:13.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:13.246+0000 7fbf05119700 1 -- 172.21.15.69:0/1141559867 wait complete. 2024-05-14T16:49:13.856 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:13 smithi069 ceph-mon[31507]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T16:49:13.883 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-14T16:49:13.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:13 smithi203 ceph-mon[35787]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T16:49:14.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:13 smithi092 ceph-mon[35763]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T16:49:14.124 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:14.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:14 smithi069 ceph-mon[31507]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:14.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:14 smithi203 ceph-mon[35787]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:15.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:14 smithi092 ceph-mon[35763]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.712+0000 7ff7c58a0700 1 -- 172.21.15.69:0/928550408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7c01037b0 msgr2=0x7ff7c010a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.712+0000 7ff7c58a0700 1 --2- 172.21.15.69:0/928550408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7c01037b0 0x7ff7c010a060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800c240 tx=0x7ff7b800c550 comp rx=0 tx=0).stop 2024-05-14T16:49:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.712+0000 7ff7c58a0700 1 -- 172.21.15.69:0/928550408 shutdown_connections 2024-05-14T16:49:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.712+0000 7ff7c58a0700 1 --2- 172.21.15.69:0/928550408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7c01037b0 0x7ff7c010a060 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.712+0000 7ff7c58a0700 1 --2- 172.21.15.69:0/928550408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7c0101c10 0x7ff7c0102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.712+0000 7ff7c58a0700 1 --2- 172.21.15.69:0/928550408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c0102e10 0x7ff7c0103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.712+0000 7ff7c58a0700 1 -- 172.21.15.69:0/928550408 >> 172.21.15.69:0/928550408 conn(0x7ff7c00fd1d0 msgr2=0x7ff7c00ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.712+0000 7ff7c58a0700 1 -- 172.21.15.69:0/928550408 shutdown_connections 2024-05-14T16:49:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.712+0000 7ff7c58a0700 1 -- 172.21.15.69:0/928550408 wait complete. 2024-05-14T16:49:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.712+0000 7ff7c58a0700 1 Processor -- start 2024-05-14T16:49:15.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7c58a0700 1 -- start start 2024-05-14T16:49:15.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7c58a0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7c0101c10 0x7ff7c0199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:15.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7c58a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c0102e10 0x7ff7c0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:15.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7c58a0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7c01037b0 0x7ff7c01a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7c58a0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c010ce20 con 0x7ff7c01037b0 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7c58a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c010cca0 con 0x7ff7c0102e10 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7c58a0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c010cfa0 con 0x7ff7c0101c10 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7bf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7c01037b0 0x7ff7c01a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7bf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7c01037b0 0x7ff7c01a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34174/0 (socket says 172.21.15.69:34174) 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7bf7fe700 1 -- 172.21.15.69:0/1406851630 learned_addr learned my addr 172.21.15.69:0/1406851630 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7beffd700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7c0101c10 0x7ff7c0199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7be7fc700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c0102e10 0x7ff7c0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7bf7fe700 1 -- 172.21.15.69:0/1406851630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7c0101c10 msgr2=0x7ff7c0199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7bf7fe700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7c0101c10 0x7ff7c0199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7bf7fe700 1 -- 172.21.15.69:0/1406851630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c0102e10 msgr2=0x7ff7c0199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7bf7fe700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c0102e10 0x7ff7c0199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7bf7fe700 1 -- 172.21.15.69:0/1406851630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c01a12b0 con 0x7ff7c01037b0 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.713+0000 7ff7bf7fe700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7c01037b0 0x7ff7c01a0c10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800a720 tx=0x7ff7b8004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:15.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.714+0000 7ff7c489e700 1 -- 172.21.15.69:0/1406851630 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8022070 con 0x7ff7c01037b0 2024-05-14T16:49:15.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.714+0000 7ff7c58a0700 1 -- 172.21.15.69:0/1406851630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c01a1540 con 0x7ff7c01037b0 2024-05-14T16:49:15.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.714+0000 7ff7c489e700 1 -- 172.21.15.69:0/1406851630 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7b8009d20 con 0x7ff7c01037b0 2024-05-14T16:49:15.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.714+0000 7ff7c489e700 1 -- 172.21.15.69:0/1406851630 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80052a0 con 0x7ff7c01037b0 2024-05-14T16:49:15.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.714+0000 7ff7c58a0700 1 -- 172.21.15.69:0/1406851630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c01a1a40 con 0x7ff7c01037b0 2024-05-14T16:49:15.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.715+0000 7ff7c58a0700 1 -- 172.21.15.69:0/1406851630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c006a230 con 0x7ff7c01037b0 2024-05-14T16:49:15.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.716+0000 7ff7c489e700 1 -- 172.21.15.69:0/1406851630 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff7b8025070 con 0x7ff7c01037b0 2024-05-14T16:49:15.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.717+0000 7ff7c489e700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff7a80b2880 0x7ff7a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:15.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.717+0000 7ff7beffd700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff7a80b2880 0x7ff7a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:15.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.717+0000 7ff7c489e700 1 -- 172.21.15.69:0/1406851630 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff7b8010020 con 0x7ff7c01037b0 2024-05-14T16:49:15.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.717+0000 7ff7beffd700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff7a80b2880 0x7ff7a80b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0005950 tx=0x7ff7b000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:15.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.721+0000 7ff7c489e700 1 -- 172.21.15.69:0/1406851630 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7b802c020 con 0x7ff7c01037b0 2024-05-14T16:49:15.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.933+0000 7ff7c58a0700 1 -- 172.21.15.69:0/1406851630 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff7c0063980 con 0x7ff7a80b2880 2024-05-14T16:49:15.933 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:15 smithi069 ceph-mon[31507]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:15.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.940+0000 7ff7c489e700 1 -- 172.21.15.69:0/1406851630 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff7c0063980 con 0x7ff7a80b2880 2024-05-14T16:49:15.939 INFO:teuthology.orchestra.run.smithi069.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-14T16:49:15.940 INFO:teuthology.orchestra.run.smithi069.stdout:mds.all 12/12 34s ago 3m smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-05-14T16:49:15.940 INFO:teuthology.orchestra.run.smithi069.stdout:mgr 3/3 34s ago 8m smithi069=x;smithi092=y;smithi203=z;count:3 2024-05-14T16:49:15.940 INFO:teuthology.orchestra.run.smithi069.stdout:mon 3/3 34s ago 9m smithi069:172.21.15.69=a;smithi092:172.21.15.92=b;smithi203:172.21.15.203=c;count:3 2024-05-14T16:49:15.940 INFO:teuthology.orchestra.run.smithi069.stdout:osd 12 34s ago - 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 -- 172.21.15.69:0/1406851630 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff7a80b2880 msgr2=0x7ff7a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff7a80b2880 0x7ff7a80b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0005950 tx=0x7ff7b000a300 comp rx=0 tx=0).stop 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 -- 172.21.15.69:0/1406851630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7c01037b0 msgr2=0x7ff7c01a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7c01037b0 0x7ff7c01a0c10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800a720 tx=0x7ff7b8004f00 comp rx=0 tx=0).stop 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 -- 172.21.15.69:0/1406851630 shutdown_connections 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ff7a80b2880 0x7ff7a80b4d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7c01037b0 0x7ff7c01a0c10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7c0101c10 0x7ff7c0199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 --2- 172.21.15.69:0/1406851630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7c0102e10 0x7ff7c0199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 -- 172.21.15.69:0/1406851630 >> 172.21.15.69:0/1406851630 conn(0x7ff7c00fd1d0 msgr2=0x7ff7c0106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:15.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 -- 172.21.15.69:0/1406851630 shutdown_connections 2024-05-14T16:49:15.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:15.942+0000 7ff7c58a0700 1 -- 172.21.15.69:0/1406851630 wait complete. 2024-05-14T16:49:15.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:15 smithi203 ceph-mon[35787]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:16.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:15 smithi092 ceph-mon[35763]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:16.579 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-14T16:49:16.822 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:17.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:17 smithi069 ceph-mon[31507]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:17.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:17 smithi069 ceph-mon[31507]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:17 smithi203 ceph-mon[35787]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:17 smithi203 ceph-mon[35787]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:17 smithi092 ceph-mon[35763]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:17 smithi092 ceph-mon[35763]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:18.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.464+0000 7fc53e4d9700 1 -- 172.21.15.69:0/145091525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc538102490 msgr2=0x7fc5381028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:18.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.464+0000 7fc53e4d9700 1 --2- 172.21.15.69:0/145091525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc538102490 0x7fc5381028b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc5280099c0 tx=0x7fc528009cd0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.464+0000 7fc53e4d9700 1 -- 172.21.15.69:0/145091525 shutdown_connections 2024-05-14T16:49:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.464+0000 7fc53e4d9700 1 --2- 172.21.15.69:0/145091525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc538102490 0x7fc5381028b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.464+0000 7fc53e4d9700 1 --2- 172.21.15.69:0/145091525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538104030 0x7fc53810a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.464+0000 7fc53e4d9700 1 --2- 172.21.15.69:0/145091525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc538103690 0x7fc538103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.464+0000 7fc53e4d9700 1 -- 172.21.15.69:0/145091525 >> 172.21.15.69:0/145091525 conn(0x7fc5380fda30 msgr2=0x7fc5380ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.464+0000 7fc53e4d9700 1 -- 172.21.15.69:0/145091525 shutdown_connections 2024-05-14T16:49:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.464+0000 7fc53e4d9700 1 -- 172.21.15.69:0/145091525 wait complete. 2024-05-14T16:49:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc53e4d9700 1 Processor -- start 2024-05-14T16:49:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc53e4d9700 1 -- start start 2024-05-14T16:49:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc53e4d9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538102490 0x7fc5381985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc53e4d9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc538103690 0x7fc538198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc53e4d9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc538104030 0x7fc53819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc53e4d9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810df30 con 0x7fc538103690 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc53e4d9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810ddb0 con 0x7fc538104030 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc53e4d9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810e0b0 con 0x7fc538102490 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc52ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc538103690 0x7fc538198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc52ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc538103690 0x7fc538198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34198/0 (socket says 172.21.15.69:34198) 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc52ffff700 1 -- 172.21.15.69:0/3688017795 learned_addr learned my addr 172.21.15.69:0/3688017795 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc537fff700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538102490 0x7fc5381985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.465+0000 7fc53ca76700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc538104030 0x7fc53819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc52ffff700 1 -- 172.21.15.69:0/3688017795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538102490 msgr2=0x7fc5381985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc52ffff700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538102490 0x7fc5381985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc52ffff700 1 -- 172.21.15.69:0/3688017795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc538104030 msgr2=0x7fc53819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc52ffff700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc538104030 0x7fc53819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc52ffff700 1 -- 172.21.15.69:0/3688017795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5381a0230 con 0x7fc538103690 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc52ffff700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc538103690 0x7fc538198b10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc520007940 tx=0x7fc520007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc535ffb700 1 -- 172.21.15.69:0/3688017795 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520004020 con 0x7fc538103690 2024-05-14T16:49:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc535ffb700 1 -- 172.21.15.69:0/3688017795 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc520004830 con 0x7fc538103690 2024-05-14T16:49:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc53e4d9700 1 -- 172.21.15.69:0/3688017795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53806bb10 con 0x7fc538103690 2024-05-14T16:49:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc535ffb700 1 -- 172.21.15.69:0/3688017795 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520005520 con 0x7fc538103690 2024-05-14T16:49:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.466+0000 7fc53e4d9700 1 -- 172.21.15.69:0/3688017795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53806c0a0 con 0x7fc538103690 2024-05-14T16:49:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.467+0000 7fc53e4d9700 1 -- 172.21.15.69:0/3688017795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53806a230 con 0x7fc538103690 2024-05-14T16:49:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.468+0000 7fc535ffb700 1 -- 172.21.15.69:0/3688017795 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc52001f020 con 0x7fc538103690 2024-05-14T16:49:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.469+0000 7fc535ffb700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc5240b2900 0x7fc5240b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.469+0000 7fc537fff700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc5240b2900 0x7fc5240b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.469+0000 7fc535ffb700 1 -- 172.21.15.69:0/3688017795 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc5200e39e0 con 0x7fc538103690 2024-05-14T16:49:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.470+0000 7fc537fff700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc5240b2900 0x7fc5240b4dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc52800b580 tx=0x7fc528005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.472+0000 7fc535ffb700 1 -- 172.21.15.69:0/3688017795 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc520026070 con 0x7fc538103690 2024-05-14T16:49:18.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.684+0000 7fc53e4d9700 1 -- 172.21.15.69:0/3688017795 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc53806c300 con 0x7fc5240b2900 2024-05-14T16:49:18.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.685+0000 7fc535ffb700 1 -- 172.21.15.69:0/3688017795 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc53806c300 con 0x7fc5240b2900 2024-05-14T16:49:18.686 INFO:teuthology.orchestra.run.smithi069.stdout:HOST ADDR LABELS STATUS 2024-05-14T16:49:18.686 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 172.21.15.69 2024-05-14T16:49:18.686 INFO:teuthology.orchestra.run.smithi069.stdout:smithi092 172.21.15.92 2024-05-14T16:49:18.686 INFO:teuthology.orchestra.run.smithi069.stdout:smithi203 172.21.15.203 2024-05-14T16:49:18.686 INFO:teuthology.orchestra.run.smithi069.stdout:3 hosts in cluster 2024-05-14T16:49:18.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.687+0000 7fc53e4d9700 1 -- 172.21.15.69:0/3688017795 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc5240b2900 msgr2=0x7fc5240b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:18.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.687+0000 7fc53e4d9700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc5240b2900 0x7fc5240b4dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc52800b580 tx=0x7fc528005fb0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.687+0000 7fc53e4d9700 1 -- 172.21.15.69:0/3688017795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc538103690 msgr2=0x7fc538198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:18.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.687+0000 7fc53e4d9700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc538103690 0x7fc538198b10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc520007940 tx=0x7fc520007c50 comp rx=0 tx=0).stop 2024-05-14T16:49:18.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.688+0000 7fc53e4d9700 1 -- 172.21.15.69:0/3688017795 shutdown_connections 2024-05-14T16:49:18.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.688+0000 7fc53e4d9700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc538104030 0x7fc53819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.688+0000 7fc53e4d9700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538102490 0x7fc5381985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.688+0000 7fc53e4d9700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fc5240b2900 0x7fc5240b4dc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.688+0000 7fc53e4d9700 1 --2- 172.21.15.69:0/3688017795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc538103690 0x7fc538198b10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:18.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.688+0000 7fc53e4d9700 1 -- 172.21.15.69:0/3688017795 >> 172.21.15.69:0/3688017795 conn(0x7fc5380fda30 msgr2=0x7fc538106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:18.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.688+0000 7fc53e4d9700 1 -- 172.21.15.69:0/3688017795 shutdown_connections 2024-05-14T16:49:18.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:18.688+0000 7fc53e4d9700 1 -- 172.21.15.69:0/3688017795 wait complete. 2024-05-14T16:49:19.384 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-14T16:49:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:19.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:19 smithi069 ceph-mon[31507]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:19.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:19 smithi069 ceph-mon[31507]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:19.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:19 smithi203 ceph-mon[35787]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:19.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:19 smithi203 ceph-mon[35787]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:19 smithi092 ceph-mon[35763]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:19 smithi092 ceph-mon[35763]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:21.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.459+0000 7f2872736700 1 -- 172.21.15.69:0/4087659562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f286c104870 msgr2=0x7f286c104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:21.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.459+0000 7f2872736700 1 --2- 172.21.15.69:0/4087659562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f286c104870 0x7f286c104cd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2860009990 tx=0x7f2860009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.459+0000 7f2872736700 1 -- 172.21.15.69:0/4087659562 shutdown_connections 2024-05-14T16:49:21.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.459+0000 7f2872736700 1 --2- 172.21.15.69:0/4087659562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f286c105210 0x7f286c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.459+0000 7f2872736700 1 --2- 172.21.15.69:0/4087659562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f286c101890 0x7f286c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.459+0000 7f2872736700 1 --2- 172.21.15.69:0/4087659562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f286c104870 0x7f286c104cd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.459+0000 7f2872736700 1 -- 172.21.15.69:0/4087659562 >> 172.21.15.69:0/4087659562 conn(0x7f286c0fd1f0 msgr2=0x7f286c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:21.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.459+0000 7f2872736700 1 -- 172.21.15.69:0/4087659562 shutdown_connections 2024-05-14T16:49:21.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.459+0000 7f2872736700 1 -- 172.21.15.69:0/4087659562 wait complete. 2024-05-14T16:49:21.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.460+0000 7f2872736700 1 Processor -- start 2024-05-14T16:49:21.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.460+0000 7f2872736700 1 -- start start 2024-05-14T16:49:21.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.460+0000 7f2872736700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f286c101890 0x7f286c1041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:21.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.460+0000 7f2872736700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f286c104870 0x7f286c102820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.460+0000 7f2872736700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f286c105210 0x7f286c102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.460+0000 7f2872736700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10f050 con 0x7f286c104870 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.460+0000 7f2872736700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10eed0 con 0x7f286c101890 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.460+0000 7f2872736700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10f1d0 con 0x7f286c105210 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f286b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f286c104870 0x7f286c102820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f286b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f286c104870 0x7f286c102820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34212/0 (socket says 172.21.15.69:34212) 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f286b7fe700 1 -- 172.21.15.69:0/2794637582 learned_addr learned my addr 172.21.15.69:0/2794637582 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f2870cd3700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f286c105210 0x7f286c102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f286bfff700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f286c101890 0x7f286c1041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f286b7fe700 1 -- 172.21.15.69:0/2794637582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f286c105210 msgr2=0x7f286c102d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f286b7fe700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f286c105210 0x7f286c102d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f286b7fe700 1 -- 172.21.15.69:0/2794637582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f286c101890 msgr2=0x7f286c1041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f286b7fe700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f286c101890 0x7f286c1041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f286b7fe700 1 -- 172.21.15.69:0/2794637582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c103610 con 0x7f286c104870 2024-05-14T16:49:21.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f286b7fe700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f286c104870 0x7f286c102820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2860006010 tx=0x7f28600047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:21.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f28697fa700 1 -- 172.21.15.69:0/2794637582 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860027070 con 0x7f286c104870 2024-05-14T16:49:21.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f28697fa700 1 -- 172.21.15.69:0/2794637582 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28600040c0 con 0x7f286c104870 2024-05-14T16:49:21.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f2872736700 1 -- 172.21.15.69:0/2794637582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c1a37c0 con 0x7f286c104870 2024-05-14T16:49:21.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f28697fa700 1 -- 172.21.15.69:0/2794637582 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860022d50 con 0x7f286c104870 2024-05-14T16:49:21.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.461+0000 7f2872736700 1 -- 172.21.15.69:0/2794637582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c1a3c10 con 0x7f286c104870 2024-05-14T16:49:21.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.463+0000 7f2872736700 1 -- 172.21.15.69:0/2794637582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286c06a230 con 0x7f286c104870 2024-05-14T16:49:21.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.464+0000 7f28697fa700 1 -- 172.21.15.69:0/2794637582 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f286002a070 con 0x7f286c104870 2024-05-14T16:49:21.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.465+0000 7f28697fa700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28540b2900 0x7f28540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:21.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.465+0000 7f286bfff700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28540b2900 0x7f28540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:21.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.465+0000 7f28697fa700 1 -- 172.21.15.69:0/2794637582 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f28600e35f0 con 0x7f286c104870 2024-05-14T16:49:21.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.465+0000 7f286bfff700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28540b2900 0x7f28540b4dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f286c06bab0 tx=0x7f285c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:21.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.468+0000 7f28697fa700 1 -- 172.21.15.69:0/2794637582 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2860031070 con 0x7f286c104870 2024-05-14T16:49:21.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.681+0000 7f2872736700 1 -- 172.21.15.69:0/2794637582 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f286c063a40 con 0x7f28540b2900 2024-05-14T16:49:21.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.683+0000 7f28697fa700 1 -- 172.21.15.69:0/2794637582 <== mgr.14150 v2:172.21.15.69:6800/4012740750 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f286c063a40 con 0x7f28540b2900 2024-05-14T16:49:21.684 INFO:teuthology.orchestra.run.smithi069.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-14T16:49:21.684 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-14T16:49:21.684 INFO:teuthology.orchestra.run.smithi069.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-14T16:49:21.684 INFO:teuthology.orchestra.run.smithi069.stdout:smithi203 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-14T16:49:21.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.685+0000 7f2872736700 1 -- 172.21.15.69:0/2794637582 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28540b2900 msgr2=0x7f28540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:21.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.685+0000 7f2872736700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28540b2900 0x7f28540b4dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f286c06bab0 tx=0x7f285c00b410 comp rx=0 tx=0).stop 2024-05-14T16:49:21.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.685+0000 7f2872736700 1 -- 172.21.15.69:0/2794637582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f286c104870 msgr2=0x7f286c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:21.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.685+0000 7f2872736700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f286c104870 0x7f286c102820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2860006010 tx=0x7f28600047b0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.686+0000 7f2872736700 1 -- 172.21.15.69:0/2794637582 shutdown_connections 2024-05-14T16:49:21.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.686+0000 7f2872736700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f286c105210 0x7f286c102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.686+0000 7f2872736700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f286c101890 0x7f286c1041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.686+0000 7f2872736700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28540b2900 0x7f28540b4dc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.686+0000 7f2872736700 1 --2- 172.21.15.69:0/2794637582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f286c104870 0x7f286c102820 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:21.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.686+0000 7f2872736700 1 -- 172.21.15.69:0/2794637582 >> 172.21.15.69:0/2794637582 conn(0x7f286c0fd1f0 msgr2=0x7f286c107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:21.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.686+0000 7f2872736700 1 -- 172.21.15.69:0/2794637582 shutdown_connections 2024-05-14T16:49:21.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:21.686+0000 7f2872736700 1 -- 172.21.15.69:0/2794637582 wait complete. 2024-05-14T16:49:21.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:21 smithi069 ceph-mon[31507]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:21.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:21 smithi203 ceph-mon[35787]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:22.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:21 smithi092 ceph-mon[35763]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:22.409 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-14T16:49:22.663 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T16:49:22.663 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-14T16:49:22.920 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:22.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:22 smithi069 ceph-mon[31507]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:22.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:22 smithi203 ceph-mon[35787]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:23.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:22 smithi092 ceph-mon[35763]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:23.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:23 smithi069 ceph-mon[31507]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:23.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:23 smithi203 ceph-mon[35787]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:24.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:23 smithi092 ceph-mon[35763]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 -- 172.21.15.69:0/2005090648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0360104900 msgr2=0x7f0360104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 --2- 172.21.15.69:0/2005090648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0360104900 0x7f0360104d60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f034c009990 tx=0x7f034c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 -- 172.21.15.69:0/2005090648 shutdown_connections 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 --2- 172.21.15.69:0/2005090648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03601052a0 0x7f036010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 --2- 172.21.15.69:0/2005090648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0360101920 0x7f0360101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 --2- 172.21.15.69:0/2005090648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0360104900 0x7f0360104d60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 -- 172.21.15.69:0/2005090648 >> 172.21.15.69:0/2005090648 conn(0x7f03600fd280 msgr2=0x7f03600ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 -- 172.21.15.69:0/2005090648 shutdown_connections 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 -- 172.21.15.69:0/2005090648 wait complete. 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 Processor -- start 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.730+0000 7f0367041700 1 -- start start 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f0367041700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0360101920 0x7f0360104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f0367041700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0360104900 0x7f03601028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f0367041700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03601052a0 0x7f0360102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f0367041700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f036010f1e0 con 0x7f03601052a0 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f0367041700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f036010f060 con 0x7f0360104900 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f0367041700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f036010f360 con 0x7f0360101920 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f03655de700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03601052a0 0x7f0360102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f03655de700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03601052a0 0x7f0360102df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59906/0 (socket says 172.21.15.69:59906) 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f03655de700 1 -- 172.21.15.69:0/3866822224 learned_addr learned my addr 172.21.15.69:0/3866822224 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f0364ddd700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0360101920 0x7f0360104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f035ffff700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0360104900 0x7f03601028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f03655de700 1 -- 172.21.15.69:0/3866822224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0360101920 msgr2=0x7f0360104260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f03655de700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0360101920 0x7f0360104260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f03655de700 1 -- 172.21.15.69:0/3866822224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0360104900 msgr2=0x7f03601028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f03655de700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0360104900 0x7f03601028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f03655de700 1 -- 172.21.15.69:0/3866822224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03601036a0 con 0x7f03601052a0 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f03655de700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03601052a0 0x7f0360102df0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f035800b970 tx=0x7f035800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f035dffb700 1 -- 172.21.15.69:0/3866822224 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358019070 con 0x7f03601052a0 2024-05-14T16:49:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f0367041700 1 -- 172.21.15.69:0/3866822224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03601a39d0 con 0x7f03601052a0 2024-05-14T16:49:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f035dffb700 1 -- 172.21.15.69:0/3866822224 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0358014440 con 0x7f03601052a0 2024-05-14T16:49:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.731+0000 7f035dffb700 1 -- 172.21.15.69:0/3866822224 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035800ce60 con 0x7f03601052a0 2024-05-14T16:49:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.732+0000 7f0367041700 1 -- 172.21.15.69:0/3866822224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03601a3f10 con 0x7f03601052a0 2024-05-14T16:49:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.733+0000 7f0367041700 1 -- 172.21.15.69:0/3866822224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f036006a230 con 0x7f03601052a0 2024-05-14T16:49:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.735+0000 7f035dffb700 1 -- 172.21.15.69:0/3866822224 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f035801c070 con 0x7f03601052a0 2024-05-14T16:49:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.735+0000 7f035dffb700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f03500b2880 0x7f03500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.736+0000 7f0364ddd700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f03500b2880 0x7f03500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.736+0000 7f035dffb700 1 -- 172.21.15.69:0/3866822224 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f03580e1ae0 con 0x7f03601052a0 2024-05-14T16:49:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.736+0000 7f0364ddd700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f03500b2880 0x7f03500b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0354006fd0 tx=0x7f0354008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.739+0000 7f035dffb700 1 -- 172.21.15.69:0/3866822224 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f03580a9800 con 0x7f03601052a0 2024-05-14T16:49:24.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.984+0000 7f0367041700 1 -- 172.21.15.69:0/3866822224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f036002e560 con 0x7f03601052a0 2024-05-14T16:49:24.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.984+0000 7f035dffb700 1 -- 172.21.15.69:0/3866822224 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3042 (secure 0 0 0) 0x7f03580a9100 con 0x7f03601052a0 2024-05-14T16:49:24.984 INFO:teuthology.orchestra.run.smithi069.stdout:e24 2024-05-14T16:49:24.984 INFO:teuthology.orchestra.run.smithi069.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:legacy client fscid: 1 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:Filesystem 'cephfs' (1) 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:fs_name cephfs 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 24 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-05-14T16:46:45.333110+0000 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2024-05-14T16:47:04.867362+0000 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:tableserver 0 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:root 0 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:session_timeout 300 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:session_autoclose 300 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:max_file_size 1099511627776 2024-05-14T16:49:24.985 INFO:teuthology.orchestra.run.smithi069.stdout:max_xattr_size 65536 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:required_client_features {} 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure 0 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure_osd_epoch 0 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:max_mds 3 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:in 0,1,2 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:up {0=24446,1=24433,2=24422} 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:failed 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:damaged 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:stopped 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:data_pools [3] 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:metadata_pool 2 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:inline_data disabled 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:balancer 2024-05-14T16:49:24.986 INFO:teuthology.orchestra.run.smithi069.stdout:bal_rank_mask -1 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:standby_count_wanted 0 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:qdb_cluster leader: 24446 members: 24433,24446,24422 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.b{0:24446} state up:active seq 6 addr [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.c{0:24415} state up:standby-replay seq 1 addr [v2:172.21.15.203:6836/1824212740,v1:172.21.15.203:6837/1824212740] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.i{1:24433} state up:active seq 11 addr [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.f{1:24409} state up:standby-replay seq 1 addr [v2:172.21.15.203:6834/3334129544,v1:172.21.15.203:6835/3334129544] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.h{2:24422} state up:active seq 12 addr [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.e{2:24395} state up:standby-replay seq 1 addr [v2:172.21.15.92:6834/2453543352,v1:172.21.15.92:6835/2453543352] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:Standby daemons: 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.69:6834/894337492,v1:172.21.15.69:6835/894337492] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.69:6836/1744189392,v1:172.21.15.69:6837/1744189392] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.987 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.g{-1:14610} state up:standby seq 1 addr [v2:172.21.15.69:6838/3026437817,v1:172.21.15.69:6839/3026437817] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.988 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.a{-1:14616} state up:standby seq 1 addr [v2:172.21.15.69:6840/3552362754,v1:172.21.15.69:6841/3552362754] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.988 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.k{-1:24380} state up:standby seq 1 addr [v2:172.21.15.92:6832/786223395,v1:172.21.15.92:6833/786223395] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.988 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.l{-1:24385} state up:standby seq 1 addr [v2:172.21.15.203:6832/85237132,v1:172.21.15.203:6833/85237132] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 -- 172.21.15.69:0/3866822224 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f03500b2880 msgr2=0x7f03500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f03500b2880 0x7f03500b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0354006fd0 tx=0x7f0354008040 comp rx=0 tx=0).stop 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 -- 172.21.15.69:0/3866822224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03601052a0 msgr2=0x7f0360102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03601052a0 0x7f0360102df0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f035800b970 tx=0x7f035800bd30 comp rx=0 tx=0).stop 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 -- 172.21.15.69:0/3866822224 shutdown_connections 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f03500b2880 0x7f03500b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03601052a0 0x7f0360102df0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0360101920 0x7f0360104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 --2- 172.21.15.69:0/3866822224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0360104900 0x7f03601028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 -- 172.21.15.69:0/3866822224 >> 172.21.15.69:0/3866822224 conn(0x7f03600fd280 msgr2=0x7f0360107ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:24.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 -- 172.21.15.69:0/3866822224 shutdown_connections 2024-05-14T16:49:24.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:24.987+0000 7f0367041700 1 -- 172.21.15.69:0/3866822224 wait complete. 2024-05-14T16:49:24.990 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-05-14T16:49:25.669 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-14T16:49:25.912 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:25.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:25 smithi069 ceph-mon[31507]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:25.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:25 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3866822224' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T16:49:25.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:25 smithi203 ceph-mon[35787]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:25.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:25 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3866822224' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T16:49:26.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:25 smithi092 ceph-mon[35763]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:26.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:25 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3866822224' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T16:49:27.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.691+0000 7fa116845700 1 -- 172.21.15.69:0/3956534295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1101036a0 msgr2=0x7fa110103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:27.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.691+0000 7fa116845700 1 --2- 172.21.15.69:0/3956534295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1101036a0 0x7fa110103b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa1040099c0 tx=0x7fa104009cd0 comp rx=0 tx=0).stop 2024-05-14T16:49:27.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.691+0000 7fa116845700 1 -- 172.21.15.69:0/3956534295 shutdown_connections 2024-05-14T16:49:27.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.691+0000 7fa116845700 1 --2- 172.21.15.69:0/3956534295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa110104040 0x7fa11010a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:27.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.691+0000 7fa116845700 1 --2- 172.21.15.69:0/3956534295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1101024a0 0x7fa1101028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.691+0000 7fa116845700 1 --2- 172.21.15.69:0/3956534295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1101036a0 0x7fa110103b00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.691+0000 7fa116845700 1 -- 172.21.15.69:0/3956534295 >> 172.21.15.69:0/3956534295 conn(0x7fa1100fda40 msgr2=0x7fa1100ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.691+0000 7fa116845700 1 -- 172.21.15.69:0/3956534295 shutdown_connections 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.691+0000 7fa116845700 1 -- 172.21.15.69:0/3956534295 wait complete. 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa116845700 1 Processor -- start 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa116845700 1 -- start start 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa116845700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1101024a0 0x7fa11006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa116845700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1101036a0 0x7fa11019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa116845700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa110104040 0x7fa1101a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa116845700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010dfc0 con 0x7fa110104040 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa116845700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010de40 con 0x7fa1101024a0 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa116845700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010e140 con 0x7fa1101036a0 2024-05-14T16:49:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa114de2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa110104040 0x7fa1101a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa114de2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa110104040 0x7fa1101a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59912/0 (socket says 172.21.15.69:59912) 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa114de2700 1 -- 172.21.15.69:0/1464390099 learned_addr learned my addr 172.21.15.69:0/1464390099 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.692+0000 7fa10ffff700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1101024a0 0x7fa11006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa114de2700 1 -- 172.21.15.69:0/1464390099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1101036a0 msgr2=0x7fa11019e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa114de2700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1101036a0 0x7fa11019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa114de2700 1 -- 172.21.15.69:0/1464390099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1101024a0 msgr2=0x7fa11006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa114de2700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1101024a0 0x7fa11006bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa114de2700 1 -- 172.21.15.69:0/1464390099 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1101a13f0 con 0x7fa110104040 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa10ffff700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1101024a0 0x7fa11006bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa114de2700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa110104040 0x7fa1101a0b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa10800ada0 tx=0x7fa10800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa10d7fa700 1 -- 172.21.15.69:0/1464390099 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108017020 con 0x7fa110104040 2024-05-14T16:49:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa116845700 1 -- 172.21.15.69:0/1464390099 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1101a16e0 con 0x7fa110104040 2024-05-14T16:49:27.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa10d7fa700 1 -- 172.21.15.69:0/1464390099 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10800ce30 con 0x7fa110104040 2024-05-14T16:49:27.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa10d7fa700 1 -- 172.21.15.69:0/1464390099 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1080054a0 con 0x7fa110104040 2024-05-14T16:49:27.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.693+0000 7fa116845700 1 -- 172.21.15.69:0/1464390099 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1101a1c20 con 0x7fa110104040 2024-05-14T16:49:27.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.694+0000 7fa116845700 1 -- 172.21.15.69:0/1464390099 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11006a230 con 0x7fa110104040 2024-05-14T16:49:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.695+0000 7fa10d7fa700 1 -- 172.21.15.69:0/1464390099 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa10800c980 con 0x7fa110104040 2024-05-14T16:49:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.696+0000 7fa10d7fa700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa0f80b2880 0x7fa0f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.696+0000 7fa10ffff700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa0f80b2880 0x7fa0f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.696+0000 7fa10d7fa700 1 -- 172.21.15.69:0/1464390099 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fa1080e1a00 con 0x7fa110104040 2024-05-14T16:49:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.696+0000 7fa10ffff700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa0f80b2880 0x7fa0f80b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa110103500 tx=0x7fa100008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:27.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.701+0000 7fa10d7fa700 1 -- 172.21.15.69:0/1464390099 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa108021020 con 0x7fa110104040 2024-05-14T16:49:27.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.910+0000 7fa116845700 1 -- 172.21.15.69:0/1464390099 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fa110064150 con 0x7fa110104040 2024-05-14T16:49:27.910 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:27 smithi069 ceph-mon[31507]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:27.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.911+0000 7fa10d7fa700 1 -- 172.21.15.69:0/1464390099 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v80) ==== 56+0+5298 (secure 0 0 0) 0x7fa1080a97b0 con 0x7fa110104040 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 80 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:fsid 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-05-14T16:38:33.684567+0000 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2024-05-14T16:47:33.698176+0000 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:crush_version 26 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:full_ratio 0.95 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:backfillfull_ratio 0.9 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:nearfull_ratio 0.85 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:require_min_compat_client luminous 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:min_compat_client luminous 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:require_osd_release squid 2024-05-14T16:49:27.911 INFO:teuthology.orchestra.run.smithi069.stdout:stretch_mode_enabled false 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:max_osd 12 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6802/3097069954,v1:172.21.15.69:6803/3097069954] [v2:172.21.15.69:6804/3097069954,v1:172.21.15.69:6805/3097069954] exists,up f8e3c095-398a-4772-b0ef-655789199a67 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 up in weight 1 up_from 13 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/1169355341,v1:172.21.15.92:6801/1169355341] [v2:172.21.15.92:6802/1169355341,v1:172.21.15.92:6803/1169355341] exists,up 0678a843-a395-41d6-892f-7d0d24b30e40 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6800/2458206770,v1:172.21.15.203:6801/2458206770] [v2:172.21.15.203:6802/2458206770,v1:172.21.15.203:6803/2458206770] exists,up 9f4e05d2-6132-476c-9c8e-64526c9bfa08 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 up in weight 1 up_from 25 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6810/1656710465,v1:172.21.15.69:6811/1656710465] [v2:172.21.15.69:6812/1656710465,v1:172.21.15.69:6813/1656710465] exists,up 097ec507-bd81-441b-9b33-61e9c55c9f7e 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 up in weight 1 up_from 31 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/301070380,v1:172.21.15.92:6809/301070380] [v2:172.21.15.92:6810/301070380,v1:172.21.15.92:6811/301070380] exists,up 76556e0f-a08a-4c01-a662-f84af601cf5c 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 up in weight 1 up_from 36 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6808/3198746082,v1:172.21.15.203:6809/3198746082] [v2:172.21.15.203:6810/3198746082,v1:172.21.15.203:6811/3198746082] exists,up 23ebc58a-f720-4000-a6aa-a2cfa0223481 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 up in weight 1 up_from 42 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6818/2234812220,v1:172.21.15.69:6819/2234812220] [v2:172.21.15.69:6820/2234812220,v1:172.21.15.69:6821/2234812220] exists,up 0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 up in weight 1 up_from 47 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/144400151,v1:172.21.15.92:6817/144400151] [v2:172.21.15.92:6818/144400151,v1:172.21.15.92:6819/144400151] exists,up b87d086b-bce8-441b-83bb-9d21051d300c 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 up in weight 1 up_from 53 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6816/1294563875,v1:172.21.15.203:6817/1294563875] [v2:172.21.15.203:6818/1294563875,v1:172.21.15.203:6819/1294563875] exists,up 59490349-0db6-4e24-bc5b-b1cf52663838 2024-05-14T16:49:27.912 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 up in weight 1 up_from 59 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6826/1618024425,v1:172.21.15.69:6827/1618024425] [v2:172.21.15.69:6828/1618024425,v1:172.21.15.69:6829/1618024425] exists,up 0b029d15-d987-448e-8ea7-513aff17fc95 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 up in weight 1 up_from 64 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/1052364226,v1:172.21.15.92:6825/1052364226] [v2:172.21.15.92:6826/1052364226,v1:172.21.15.92:6827/1052364226] exists,up bce9e619-6203-4ae1-b02c-322eee6f809a 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 up in weight 1 up_from 70 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6824/3400874682,v1:172.21.15.203:6825/3400874682] [v2:172.21.15.203:6826/3400874682,v1:172.21.15.203:6827/3400874682] exists,up 9acab8e8-82ae-44b8-aba6-87b3a99ebe62 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.c [2,4] 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.10 [8,11] 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.13 [2,11] 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.6 [4,6] 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.17 [4,8] 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.21 [5,9] 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.22 [3,9] 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.39 [5,8] 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.3c [4,9] 2024-05-14T16:49:27.913 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3194136272 expires 2024-05-15T16:39:31.768055+0000 2024-05-14T16:49:27.914 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3983720568 expires 2024-05-15T16:39:31.768055+0000 2024-05-14T16:49:27.914 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3714306679 expires 2024-05-15T16:39:31.768055+0000 2024-05-14T16:49:27.914 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/421328414 expires 2024-05-15T16:39:31.768055+0000 2024-05-14T16:49:27.914 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3752620885 expires 2024-05-15T16:39:05.658063+0000 2024-05-14T16:49:27.914 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/4209517756 expires 2024-05-15T16:39:05.658063+0000 2024-05-14T16:49:27.914 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/421328414 expires 2024-05-15T16:39:31.768055+0000 2024-05-14T16:49:27.914 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/2746326029 expires 2024-05-15T16:39:05.658063+0000 2024-05-14T16:49:27.914 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/4187352755 expires 2024-05-15T16:39:05.658063+0000 2024-05-14T16:49:27.914 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/4209517756 expires 2024-05-15T16:39:05.658063+0000 2024-05-14T16:49:27.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.913+0000 7fa116845700 1 -- 172.21.15.69:0/1464390099 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa0f80b2880 msgr2=0x7fa0f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:27.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.913+0000 7fa116845700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa0f80b2880 0x7fa0f80b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa110103500 tx=0x7fa100008040 comp rx=0 tx=0).stop 2024-05-14T16:49:27.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.913+0000 7fa116845700 1 -- 172.21.15.69:0/1464390099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa110104040 msgr2=0x7fa1101a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:27.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.913+0000 7fa116845700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa110104040 0x7fa1101a0b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa10800ada0 tx=0x7fa10800c330 comp rx=0 tx=0).stop 2024-05-14T16:49:27.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.914+0000 7fa116845700 1 -- 172.21.15.69:0/1464390099 shutdown_connections 2024-05-14T16:49:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.914+0000 7fa116845700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fa0f80b2880 0x7fa0f80b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.914+0000 7fa116845700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa110104040 0x7fa1101a0b20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.914+0000 7fa116845700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1101024a0 0x7fa11006bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.914+0000 7fa116845700 1 --2- 172.21.15.69:0/1464390099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1101036a0 0x7fa11019e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.914+0000 7fa116845700 1 -- 172.21.15.69:0/1464390099 >> 172.21.15.69:0/1464390099 conn(0x7fa1100fda40 msgr2=0x7fa110106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.914+0000 7fa116845700 1 -- 172.21.15.69:0/1464390099 shutdown_connections 2024-05-14T16:49:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:27.914+0000 7fa116845700 1 -- 172.21.15.69:0/1464390099 wait complete. 2024-05-14T16:49:27.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:27 smithi203 ceph-mon[35787]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:28.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:27 smithi092 ceph-mon[35763]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:28.551 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-14T16:49:28.562 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:49:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:28.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:28 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1464390099' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T16:49:28.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:28 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1464390099' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T16:49:29.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:28 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1464390099' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T16:49:29.933 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:29 smithi069 ceph-mon[31507]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:29.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:29 smithi203 ceph-mon[35787]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:30.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:29 smithi092 ceph-mon[35763]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:30.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.497+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/2049854965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b141041a0 msgr2=0x7f6b141081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:30.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.497+0000 7f6b1b9ce700 1 --2- 172.21.15.69:0/2049854965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b141041a0 0x7f6b141081c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6b040099c0 tx=0x7f6b04009cd0 comp rx=0 tx=0).stop 2024-05-14T16:49:30.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.497+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/2049854965 shutdown_connections 2024-05-14T16:49:30.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.497+0000 7f6b1b9ce700 1 --2- 172.21.15.69:0/2049854965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b141088f0 0x7f6b14110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:30.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.497+0000 7f6b1b9ce700 1 --2- 172.21.15.69:0/2049854965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b141037f0 0x7f6b14103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:30.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.497+0000 7f6b1b9ce700 1 --2- 172.21.15.69:0/2049854965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b141041a0 0x7f6b141081c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:30.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.497+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/2049854965 >> 172.21.15.69:0/2049854965 conn(0x7f6b1407a6a0 msgr2=0x7f6b1407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:30.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.497+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/2049854965 shutdown_connections 2024-05-14T16:49:30.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.497+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/2049854965 wait complete. 2024-05-14T16:49:30.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b1b9ce700 1 Processor -- start 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b1b9ce700 1 -- start start 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b1b9ce700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b141037f0 0x7f6b14075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b1b9ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b141041a0 0x7f6b14075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b1b9ce700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b141088f0 0x7f6b140762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b1b9ce700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14113730 con 0x7f6b141088f0 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b1b9ce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6b141135b0 con 0x7f6b141041a0 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b1b9ce700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6b141138b0 con 0x7f6b141037f0 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b19f6b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b141088f0 0x7f6b140762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b19f6b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b141088f0 0x7f6b140762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59930/0 (socket says 172.21.15.69:59930) 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b19f6b700 1 -- 172.21.15.69:0/3658107503 learned_addr learned my addr 172.21.15.69:0/3658107503 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b18f69700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b141041a0 0x7f6b14075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b1976a700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b141037f0 0x7f6b14075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b19f6b700 1 -- 172.21.15.69:0/3658107503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b141037f0 msgr2=0x7f6b14075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:30.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b19f6b700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b141037f0 0x7f6b14075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b19f6b700 1 -- 172.21.15.69:0/3658107503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b141041a0 msgr2=0x7f6b14075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b19f6b700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b141041a0 0x7f6b14075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b19f6b700 1 -- 172.21.15.69:0/3658107503 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b141a8120 con 0x7f6b141088f0 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.498+0000 7f6b1976a700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b141037f0 0x7f6b14075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.499+0000 7f6b19f6b700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b141088f0 0x7f6b140762c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c007d70 tx=0x7f6b0c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.499+0000 7f6b0a7fc700 1 -- 172.21.15.69:0/3658107503 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c016020 con 0x7f6b141088f0 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.499+0000 7f6b0a7fc700 1 -- 172.21.15.69:0/3658107503 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0c00ad40 con 0x7f6b141088f0 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.499+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/3658107503 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b141a8410 con 0x7f6b141088f0 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.499+0000 7f6b0a7fc700 1 -- 172.21.15.69:0/3658107503 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0115e0 con 0x7f6b141088f0 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.499+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/3658107503 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b141a88d0 con 0x7f6b141088f0 2024-05-14T16:49:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.500+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/3658107503 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1406a230 con 0x7f6b141088f0 2024-05-14T16:49:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.501+0000 7f6b0a7fc700 1 -- 172.21.15.69:0/3658107503 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6b0c019020 con 0x7f6b141088f0 2024-05-14T16:49:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.502+0000 7f6b0a7fc700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b000b2880 0x7f6b000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.502+0000 7f6b1976a700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b000b2880 0x7f6b000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.502+0000 7f6b0a7fc700 1 -- 172.21.15.69:0/3658107503 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6b0c0e2700 con 0x7f6b141088f0 2024-05-14T16:49:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.502+0000 7f6b1976a700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b000b2880 0x7f6b000b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6b10007900 tx=0x7f6b10008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:30.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.506+0000 7f6b0a7fc700 1 -- 172.21.15.69:0/3658107503 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b0c020020 con 0x7f6b141088f0 2024-05-14T16:49:30.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.755+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/3658107503 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b1410bb30 con 0x7f6b141088f0 2024-05-14T16:49:30.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.756+0000 7f6b0a7fc700 1 -- 172.21.15.69:0/3658107503 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7f6b0c0aa420 con 0x7f6b141088f0 2024-05-14T16:49:30.756 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:49:30.756 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:47:04.867362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24433,24446,24422]},"id":1}]} 2024-05-14T16:49:30.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/3658107503 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b000b2880 msgr2=0x7f6b000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:30.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b000b2880 0x7f6b000b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6b10007900 tx=0x7f6b10008040 comp rx=0 tx=0).stop 2024-05-14T16:49:30.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/3658107503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b141088f0 msgr2=0x7f6b140762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:30.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b141088f0 0x7f6b140762c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c007d70 tx=0x7f6b0c00a340 comp rx=0 tx=0).stop 2024-05-14T16:49:30.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/3658107503 shutdown_connections 2024-05-14T16:49:30.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b000b2880 0x7f6b000b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:30.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b141088f0 0x7f6b140762c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:30.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b141037f0 0x7f6b14075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:30.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 --2- 172.21.15.69:0/3658107503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b141041a0 0x7f6b14075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:30.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/3658107503 >> 172.21.15.69:0/3658107503 conn(0x7f6b1407a6a0 msgr2=0x7f6b14109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:30.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/3658107503 shutdown_connections 2024-05-14T16:49:30.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:30.758+0000 7f6b1b9ce700 1 -- 172.21.15.69:0/3658107503 wait complete. 2024-05-14T16:49:30.759 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-05-14T16:49:31.479 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:49:31.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:31 smithi069 ceph-mon[31507]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:31.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:31 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3658107503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:49:31.723 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:31 smithi203 ceph-mon[35787]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:31.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:31 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3658107503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:49:32.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:31 smithi092 ceph-mon[35763]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:32.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:31 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3658107503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:49:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.364+0000 7f6b240bd700 1 -- 172.21.15.69:0/1618415024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b1c102120 msgr2=0x7f6b1c10e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.364+0000 7f6b240bd700 1 --2- 172.21.15.69:0/1618415024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b1c102120 0x7f6b1c10e630 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400b0a0 tx=0x7f6b1400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:49:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.364+0000 7f6b240bd700 1 -- 172.21.15.69:0/1618415024 shutdown_connections 2024-05-14T16:49:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.364+0000 7f6b240bd700 1 --2- 172.21.15.69:0/1618415024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b1c102120 0x7f6b1c10e630 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:33.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.364+0000 7f6b240bd700 1 --2- 172.21.15.69:0/1618415024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b1c107760 0x7f6b1c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:33.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.364+0000 7f6b240bd700 1 --2- 172.21.15.69:0/1618415024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b1c101760 0x7f6b1c101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:33.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.364+0000 7f6b240bd700 1 -- 172.21.15.69:0/1618415024 >> 172.21.15.69:0/1618415024 conn(0x7f6b1c078640 msgr2=0x7f6b1c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:33.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.365+0000 7f6b240bd700 1 -- 172.21.15.69:0/1618415024 shutdown_connections 2024-05-14T16:49:33.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.365+0000 7f6b240bd700 1 -- 172.21.15.69:0/1618415024 wait complete. 2024-05-14T16:49:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.365+0000 7f6b240bd700 1 Processor -- start 2024-05-14T16:49:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.365+0000 7f6b240bd700 1 -- start start 2024-05-14T16:49:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b240bd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b1c101760 0x7f6b1c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b240bd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b1c102120 0x7f6b1c19af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b240bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b1c107760 0x7f6b1c1a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b240bd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c112670 con 0x7f6b1c101760 2024-05-14T16:49:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b240bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c1124f0 con 0x7f6b1c107760 2024-05-14T16:49:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b240bd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c1127f0 con 0x7f6b1c102120 2024-05-14T16:49:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b21e59700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b1c101760 0x7f6b1c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b21e59700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b1c101760 0x7f6b1c19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59950/0 (socket says 172.21.15.69:59950) 2024-05-14T16:49:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b21e59700 1 -- 172.21.15.69:0/2980678249 learned_addr learned my addr 172.21.15.69:0/2980678249 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b21658700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b1c102120 0x7f6b1c19af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b2265a700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b1c107760 0x7f6b1c1a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b21e59700 1 -- 172.21.15.69:0/2980678249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b1c102120 msgr2=0x7f6b1c19af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b21e59700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b1c102120 0x7f6b1c19af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b21e59700 1 -- 172.21.15.69:0/2980678249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b1c107760 msgr2=0x7f6b1c1a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b21e59700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b1c107760 0x7f6b1c1a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b21e59700 1 -- 172.21.15.69:0/2980678249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c1a5f90 con 0x7f6b1c101760 2024-05-14T16:49:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b2265a700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b1c107760 0x7f6b1c1a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b21e59700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b1c101760 0x7f6b1c19a9d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800d980 tx=0x7f6b1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b12ffd700 1 -- 172.21.15.69:0/2980678249 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18017070 con 0x7f6b1c101760 2024-05-14T16:49:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b12ffd700 1 -- 172.21.15.69:0/2980678249 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b18005020 con 0x7f6b1c101760 2024-05-14T16:49:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b12ffd700 1 -- 172.21.15.69:0/2980678249 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18009b50 con 0x7f6b1c101760 2024-05-14T16:49:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b240bd700 1 -- 172.21.15.69:0/2980678249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c1a6160 con 0x7f6b1c101760 2024-05-14T16:49:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.366+0000 7f6b240bd700 1 -- 172.21.15.69:0/2980678249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c1a6670 con 0x7f6b1c101760 2024-05-14T16:49:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.367+0000 7f6b240bd700 1 -- 172.21.15.69:0/2980678249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c102ea0 con 0x7f6b1c101760 2024-05-14T16:49:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.369+0000 7f6b12ffd700 1 -- 172.21.15.69:0/2980678249 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6b180051d0 con 0x7f6b1c101760 2024-05-14T16:49:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.369+0000 7f6b12ffd700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b080b2900 0x7f6b080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.369+0000 7f6b21658700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b080b2900 0x7f6b080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.369+0000 7f6b12ffd700 1 -- 172.21.15.69:0/2980678249 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6b180e1fd0 con 0x7f6b1c101760 2024-05-14T16:49:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.370+0000 7f6b21658700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b080b2900 0x7f6b080b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c100ca0 tx=0x7f6b0c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:33.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.374+0000 7f6b12ffd700 1 -- 172.21.15.69:0/2980678249 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b18020020 con 0x7f6b1c101760 2024-05-14T16:49:33.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.627+0000 7f6b240bd700 1 -- 172.21.15.69:0/2980678249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b1c02e720 con 0x7f6b1c101760 2024-05-14T16:49:33.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.628+0000 7f6b12ffd700 1 -- 172.21.15.69:0/2980678249 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11859 (secure 0 0 0) 0x7f6b180a9d90 con 0x7f6b1c101760 2024-05-14T16:49:33.628 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:49:33.628 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:47:04.867362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24433,24446,24422]},"id":1}]} 2024-05-14T16:49:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 -- 172.21.15.69:0/2980678249 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b080b2900 msgr2=0x7f6b080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b080b2900 0x7f6b080b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c100ca0 tx=0x7f6b0c00a380 comp rx=0 tx=0).stop 2024-05-14T16:49:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 -- 172.21.15.69:0/2980678249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b1c101760 msgr2=0x7f6b1c19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b1c101760 0x7f6b1c19a9d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800d980 tx=0x7f6b1800dc90 comp rx=0 tx=0).stop 2024-05-14T16:49:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 -- 172.21.15.69:0/2980678249 shutdown_connections 2024-05-14T16:49:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f6b080b2900 0x7f6b080b4dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b1c101760 0x7f6b1c19a9d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b1c107760 0x7f6b1c1a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 --2- 172.21.15.69:0/2980678249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b1c102120 0x7f6b1c19af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 -- 172.21.15.69:0/2980678249 >> 172.21.15.69:0/2980678249 conn(0x7f6b1c078640 msgr2=0x7f6b1c1045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 -- 172.21.15.69:0/2980678249 shutdown_connections 2024-05-14T16:49:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:33.630+0000 7f6b240bd700 1 -- 172.21.15.69:0/2980678249 wait complete. 2024-05-14T16:49:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-05-14T16:49:33.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:33 smithi069 ceph-mon[31507]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:33.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:33 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2980678249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:49:33.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:33 smithi203 ceph-mon[35787]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:33.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:33 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2980678249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:49:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:33 smithi092 ceph-mon[35763]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:34.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:33 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2980678249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:49:34.431 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:49:34.670 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:35.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:35 smithi069 ceph-mon[31507]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:35.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:35 smithi203 ceph-mon[35787]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:36.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:35 smithi092 ceph-mon[35763]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:36.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.401+0000 7f0daf0ff700 1 -- 172.21.15.69:0/3883809406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0da81095b0 msgr2=0x7f0da8109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:36.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.401+0000 7f0daf0ff700 1 --2- 172.21.15.69:0/3883809406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0da81095b0 0x7f0da8109990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009990 tx=0x7f0d9c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 -- 172.21.15.69:0/3883809406 shutdown_connections 2024-05-14T16:49:36.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 --2- 172.21.15.69:0/3883809406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0da81095b0 0x7f0da8109990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 --2- 172.21.15.69:0/3883809406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0da806bbe0 0x7f0da81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 --2- 172.21.15.69:0/3883809406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0da806b220 0x7f0da806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 -- 172.21.15.69:0/3883809406 >> 172.21.15.69:0/3883809406 conn(0x7f0da80793f0 msgr2=0x7f0da8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 -- 172.21.15.69:0/3883809406 shutdown_connections 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 -- 172.21.15.69:0/3883809406 wait complete. 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 Processor -- start 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 -- start start 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0da806b220 0x7f0da819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0da806bbe0 0x7f0da819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0da81095b0 0x7f0da8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8113740 con 0x7f0da806bbe0 2024-05-14T16:49:36.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0da81135c0 con 0x7f0da81095b0 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0daf0ff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0da81138c0 con 0x7f0da806b220 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.402+0000 7f0da7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0da806bbe0 0x7f0da819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0da806bbe0 0x7f0da819fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38686/0 (socket says 172.21.15.69:38686) 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da7fff700 1 -- 172.21.15.69:0/2787250484 learned_addr learned my addr 172.21.15.69:0/2787250484 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0dad69c700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0da81095b0 0x7f0da8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0dace9b700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0da806b220 0x7f0da819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da7fff700 1 -- 172.21.15.69:0/2787250484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0da806b220 msgr2=0x7f0da819f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da7fff700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0da806b220 0x7f0da819f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da7fff700 1 -- 172.21.15.69:0/2787250484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0da81095b0 msgr2=0x7f0da8199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da7fff700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0da81095b0 0x7f0da8199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da7fff700 1 -- 172.21.15.69:0/2787250484 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da819a260 con 0x7f0da806bbe0 2024-05-14T16:49:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0dace9b700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0da806b220 0x7f0da819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:36.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da7fff700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0da806bbe0 0x7f0da819fe80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400b530 tx=0x7f0d9400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:36.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da5ffb700 1 -- 172.21.15.69:0/2787250484 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94015020 con 0x7f0da806bbe0 2024-05-14T16:49:36.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da5ffb700 1 -- 172.21.15.69:0/2787250484 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d94011ce0 con 0x7f0da806bbe0 2024-05-14T16:49:36.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0daf0ff700 1 -- 172.21.15.69:0/2787250484 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da819a550 con 0x7f0da806bbe0 2024-05-14T16:49:36.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0da5ffb700 1 -- 172.21.15.69:0/2787250484 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94010e40 con 0x7f0da806bbe0 2024-05-14T16:49:36.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.403+0000 7f0daf0ff700 1 -- 172.21.15.69:0/2787250484 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da81a6810 con 0x7f0da806bbe0 2024-05-14T16:49:36.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.404+0000 7f0daf0ff700 1 -- 172.21.15.69:0/2787250484 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da806a230 con 0x7f0da806bbe0 2024-05-14T16:49:36.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.405+0000 7f0da5ffb700 1 -- 172.21.15.69:0/2787250484 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0d94018020 con 0x7f0da806bbe0 2024-05-14T16:49:36.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.406+0000 7f0da5ffb700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0d980b2880 0x7f0d980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:36.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.406+0000 7f0dace9b700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0d980b2880 0x7f0d980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:36.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.406+0000 7f0da5ffb700 1 -- 172.21.15.69:0/2787250484 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f0d940e1a40 con 0x7f0da806bbe0 2024-05-14T16:49:36.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.407+0000 7f0dace9b700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0d980b2880 0x7f0d980b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00b5c0 tx=0x7f0d9c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:36.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.411+0000 7f0da5ffb700 1 -- 172.21.15.69:0/2787250484 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d940e3e20 con 0x7f0da806bbe0 2024-05-14T16:49:36.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.618+0000 7f0daf0ff700 1 -- 172.21.15.69:0/2787250484 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0da819aff0 con 0x7f0da806bbe0 2024-05-14T16:49:36.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.620+0000 7f0da5ffb700 1 -- 172.21.15.69:0/2787250484 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22664 (secure 0 0 0) 0x7f0d9401f020 con 0x7f0da806bbe0 2024-05-14T16:49:36.620 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:49:36.620 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T16:47:33.698176+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:46:38.843325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:46:42.395401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:49:36.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 -- 172.21.15.69:0/2787250484 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0d980b2880 msgr2=0x7f0d980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:36.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0d980b2880 0x7f0d980b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00b5c0 tx=0x7f0d9c005fb0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 -- 172.21.15.69:0/2787250484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0da806bbe0 msgr2=0x7f0da819fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:36.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0da806bbe0 0x7f0da819fe80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400b530 tx=0x7f0d9400b840 comp rx=0 tx=0).stop 2024-05-14T16:49:36.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 -- 172.21.15.69:0/2787250484 shutdown_connections 2024-05-14T16:49:36.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0da81095b0 0x7f0da8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0da806b220 0x7f0da819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f0d980b2880 0x7f0d980b4d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 --2- 172.21.15.69:0/2787250484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0da806bbe0 0x7f0da819fe80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:36.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 -- 172.21.15.69:0/2787250484 >> 172.21.15.69:0/2787250484 conn(0x7f0da80793f0 msgr2=0x7f0da8101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:36.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 -- 172.21.15.69:0/2787250484 shutdown_connections 2024-05-14T16:49:36.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:36.622+0000 7f0daf0ff700 1 -- 172.21.15.69:0/2787250484 wait complete. 2024-05-14T16:49:36.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:36 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2787250484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:49:36.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:36 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2787250484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:49:37.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:36 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2787250484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:49:37.407 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T16:46:45.333110+0000', 'modified': '2024-05-14T16:47:04.867362+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24446, 'mds_1': 24433, 'mds_2': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24395': {'gid': 24395, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6835/2453543352', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 2453543352}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 2453543352}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/3334129544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 3334129544}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 3334129544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6837/1824212740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 1824212740}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 1824212740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.92:6837/1194779586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 1194779586}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1194779586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.203:6839/398835794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 398835794}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 398835794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6839/2169416341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 2169416341}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 2169416341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24446, 'qdb_cluster': [24433, 24446, 24422]} 2024-05-14T16:49:37.407 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T16:49:37.408 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-14T16:49:37.644 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:37.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:37 smithi203 ceph-mon[35787]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:37.999 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:37 smithi069 ceph-mon[31507]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:38.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:37 smithi092 ceph-mon[35763]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:39.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.327+0000 7fe058005700 1 -- 172.21.15.69:0/2501561517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0501024d0 msgr2=0x7fe050102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.327+0000 7fe058005700 1 --2- 172.21.15.69:0/2501561517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0501024d0 0x7fe050102950 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe040009990 tx=0x7fe040009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.328+0000 7fe058005700 1 -- 172.21.15.69:0/2501561517 shutdown_connections 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.328+0000 7fe058005700 1 --2- 172.21.15.69:0/2501561517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050102e90 0x7fe05010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.328+0000 7fe058005700 1 --2- 172.21.15.69:0/2501561517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0501084d0 0x7fe0501088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.328+0000 7fe058005700 1 --2- 172.21.15.69:0/2501561517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0501024d0 0x7fe050102950 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.328+0000 7fe058005700 1 -- 172.21.15.69:0/2501561517 >> 172.21.15.69:0/2501561517 conn(0x7fe050078490 msgr2=0x7fe0500788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.328+0000 7fe058005700 1 -- 172.21.15.69:0/2501561517 shutdown_connections 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.328+0000 7fe058005700 1 -- 172.21.15.69:0/2501561517 wait complete. 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.328+0000 7fe058005700 1 Processor -- start 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.328+0000 7fe058005700 1 -- start start 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.328+0000 7fe058005700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0501024d0 0x7fe0501040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe058005700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050102e90 0x7fe0501045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe058005700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0501084d0 0x7fe050104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe058005700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0501126b0 con 0x7fe050102e90 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe058005700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe050112530 con 0x7fe0501084d0 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe058005700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe050112830 con 0x7fe0501024d0 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe0555a0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050102e90 0x7fe0501045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe0555a0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050102e90 0x7fe0501045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38708/0 (socket says 172.21.15.69:38708) 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe0555a0700 1 -- 172.21.15.69:0/1571576952 learned_addr learned my addr 172.21.15.69:0/1571576952 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe055da1700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0501024d0 0x7fe0501040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe0565a2700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0501084d0 0x7fe050104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe0555a0700 1 -- 172.21.15.69:0/1571576952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0501024d0 msgr2=0x7fe0501040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe0555a0700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0501024d0 0x7fe0501040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe0555a0700 1 -- 172.21.15.69:0/1571576952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0501084d0 msgr2=0x7fe050104b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe0555a0700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0501084d0 0x7fe050104b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe0555a0700 1 -- 172.21.15.69:0/1571576952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0501a8220 con 0x7fe050102e90 2024-05-14T16:49:39.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe0555a0700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050102e90 0x7fe0501045e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe040005910 tx=0x7fe0400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:39.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe046ffd700 1 -- 172.21.15.69:0/1571576952 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040027070 con 0x7fe050102e90 2024-05-14T16:49:39.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe046ffd700 1 -- 172.21.15.69:0/1571576952 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe040003fa0 con 0x7fe050102e90 2024-05-14T16:49:39.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe058005700 1 -- 172.21.15.69:0/1571576952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0501a84b0 con 0x7fe050102e90 2024-05-14T16:49:39.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe046ffd700 1 -- 172.21.15.69:0/1571576952 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040022d50 con 0x7fe050102e90 2024-05-14T16:49:39.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.329+0000 7fe058005700 1 -- 172.21.15.69:0/1571576952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0501a8960 con 0x7fe050102e90 2024-05-14T16:49:39.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.332+0000 7fe046ffd700 1 -- 172.21.15.69:0/1571576952 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe04002a070 con 0x7fe050102e90 2024-05-14T16:49:39.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.333+0000 7fe046ffd700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe03c0b2900 0x7fe03c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.333+0000 7fe055da1700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe03c0b2900 0x7fe03c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.333+0000 7fe046ffd700 1 -- 172.21.15.69:0/1571576952 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe0400e30a0 con 0x7fe050102e90 2024-05-14T16:49:39.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.333+0000 7fe055da1700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe03c0b2900 0x7fe03c0b4dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009a10 tx=0x7fe04c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:39.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.333+0000 7fe06289bb80 1 Processor -- start 2024-05-14T16:49:39.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.333+0000 7fe06289bb80 1 -- start start 2024-05-14T16:49:39.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 0x5599d80656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d80693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 0x5599d8069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe06289bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5599d7fc3c40 con 0x5599d8065c10 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe06289bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5599d7fb37e0 con 0x5599d80652f0 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe06289bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599d7fb7820 con 0x5599d8069910 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0555a0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d80693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0555a0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d80693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38714/0 (socket says 172.21.15.69:38714) 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0555a0700 1 -- 172.21.15.69:0/94801357 learned_addr learned my addr 172.21.15.69:0/94801357 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0565a2700 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 0x5599d8069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe055da1700 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 0x5599d80656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0555a0700 1 -- 172.21.15.69:0/94801357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 msgr2=0x5599d8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0555a0700 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 0x5599d8069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0555a0700 1 -- 172.21.15.69:0/94801357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 msgr2=0x5599d80656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0555a0700 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 0x5599d80656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0555a0700 1 -- 172.21.15.69:0/94801357 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d7f03880 con 0x5599d8065c10 2024-05-14T16:49:39.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe055da1700 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 0x5599d80656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:39.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0565a2700 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 0x5599d8069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:39.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe0555a0700 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d80693d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe0400a67d0 tx=0x7fe040009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48e1bec496192cb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:39.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe03bfff700 1 -- 172.21.15.69:0/94801357 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400aa750 con 0x5599d8065c10 2024-05-14T16:49:39.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe03bfff700 1 -- 172.21.15.69:0/94801357 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0400aeb00 con 0x5599d8065c10 2024-05-14T16:49:39.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.334+0000 7fe03bfff700 1 -- 172.21.15.69:0/94801357 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400aa9b0 con 0x5599d8065c10 2024-05-14T16:49:39.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.335+0000 7fe06289bb80 1 -- 172.21.15.69:0/94801357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 msgr2=0x5599d80693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.335+0000 7fe06289bb80 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d80693d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe0400a67d0 tx=0x7fe040009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.335+0000 7fe06289bb80 1 -- 172.21.15.69:0/94801357 shutdown_connections 2024-05-14T16:49:39.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.335+0000 7fe06289bb80 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 0x5599d8069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.335+0000 7fe06289bb80 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 0x5599d80656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.335+0000 7fe06289bb80 1 --2- 172.21.15.69:0/94801357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d80693d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.335+0000 7fe06289bb80 1 -- 172.21.15.69:0/94801357 >> 172.21.15.69:0/94801357 conn(0x5599d7f90200 msgr2=0x5599d805f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.335+0000 7fe06289bb80 1 -- 172.21.15.69:0/94801357 shutdown_connections 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.335+0000 7fe06289bb80 1 -- 172.21.15.69:0/94801357 wait complete. 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.335+0000 7fe06289bb80 1 -- start start 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe06289bb80 1 Processor -- start 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 0x5599d8070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d806e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 0x5599d806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe06289bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5599d7eb9f50 con 0x5599d8065c10 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe06289bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5599d7f55770 con 0x5599d80652f0 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe06289bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599d7fc3c40 con 0x5599d8069910 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe06289bb80 10 client.0 _ll_register_callbacks cb 0x5599d7f4a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe0555a0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d806e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe0555a0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d806e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38728/0 (socket says 172.21.15.69:38728) 2024-05-14T16:49:39.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe0555a0700 1 -- 172.21.15.69:0/1879502154 learned_addr learned my addr 172.21.15.69:0/1879502154 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe055da1700 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 0x5599d8070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe0565a2700 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 0x5599d806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe0555a0700 1 -- 172.21.15.69:0/1879502154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 msgr2=0x5599d806e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe0555a0700 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 0x5599d806e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe0555a0700 1 -- 172.21.15.69:0/1879502154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 msgr2=0x5599d8070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe0555a0700 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 0x5599d8070dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe0555a0700 1 -- 172.21.15.69:0/1879502154 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d7f52640 con 0x5599d8065c10 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe055da1700 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 0x5599d8070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.336+0000 7fe0565a2700 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 0x5599d806e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe0555a0700 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d806e170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe0400b1160 tx=0x7fe0400b1190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9685cf14fba80103 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe0397fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe0397fa700 1 -- 172.21.15.69:0/1879502154 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400aabc0 con 0x5599d8065c10 2024-05-14T16:49:39.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe0397fa700 1 -- 172.21.15.69:0/1879502154 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe0400a6d40 con 0x5599d8065c10 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe06289bb80 1 -- 172.21.15.69:0/1879502154 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0400aeb00 con 0x5599d8065c10 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe0397fa700 1 -- 172.21.15.69:0/1879502154 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe0400b3090 con 0x5599d8065c10 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe06289bb80 10 client.15090 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe06289bb80 1 -- 172.21.15.69:0/1879502154 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5599d807beb0 con 0x5599d8065c10 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe0397fa700 1 -- 172.21.15.69:0/1879502154 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0400aeb00 con 0x5599d8065c10 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe0397fa700 1 -- 172.21.15.69:0/1879502154 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe0400bfb80 con 0x5599d8065c10 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe06289bb80 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe06289bb80 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe06289bb80 20 client.15090 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe06289bb80 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x5599d807e310 0x5599d807f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe06289bb80 4 client.15090 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.337+0000 7fe06289bb80 1 -- 172.21.15.69:0/1879502154 --> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599d8031310 con 0x5599d807e310 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.338+0000 7fe055da1700 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x5599d807e310 0x5599d807f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.338+0000 7fe055da1700 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x5599d807e310 0x5599d807f780 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c365d62f180d4f81 server_cookie=21bcf7ddf376a46e in_seq=0 out_seq=0 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.338+0000 7fe0397fa700 10 client.15090 ms_handle_connect on v2:172.21.15.92:6836/1194779586 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe0397fa700 1 -- 172.21.15.69:0/1879502154 <== mds.2 v2:172.21.15.92:6836/1194779586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5599d8031310 con 0x5599d807e310 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe0397fa700 10 client.15090 handle_command_reply: tid=0 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 client.15090 shutdown 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 -- 172.21.15.69:0/1879502154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 msgr2=0x5599d806e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d806e170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe0400b1160 tx=0x7fe0400b1190 comp rx=0 tx=0).stop 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 -- 172.21.15.69:0/1879502154 shutdown_connections 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x5599d807e310 0x5599d807f780 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d8069910 0x5599d806e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d80652f0 0x5599d8070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 --2- 172.21.15.69:0/1879502154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8065c10 0x5599d806e170 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 -- 172.21.15.69:0/1879502154 >> 172.21.15.69:0/1879502154 conn(0x5599d7f90200 msgr2=0x5599d806ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe0397fa700 0 client.15090 ms_handle_reset on v2:172.21.15.92:6836/1194779586 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 -- 172.21.15.69:0/1879502154 shutdown_connections 2024-05-14T16:49:39.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 1 -- 172.21.15.69:0/1879502154 wait complete. 2024-05-14T16:49:39.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.339+0000 7fe06289bb80 20 client.15090 trim_cache size 0 max 16384 2024-05-14T16:49:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe06289bb80 1 Processor -- start 2024-05-14T16:49:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe06289bb80 1 -- start start 2024-05-14T16:49:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d80652f0 0x5599d8068e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 0x5599d8069380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d8067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe06289bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5599d807dac0 con 0x5599d8069910 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe06289bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5599d807e190 con 0x5599d8065c10 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe06289bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599d807e860 con 0x5599d80652f0 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0565a2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d8067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0565a2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d8067340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38740/0 (socket says 172.21.15.69:38740) 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0565a2700 1 -- 172.21.15.69:0/2803215904 learned_addr learned my addr 172.21.15.69:0/2803215904 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0555a0700 1 --2- 172.21.15.69:0/2803215904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 0x5599d8069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe055da1700 1 --2- 172.21.15.69:0/2803215904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d80652f0 0x5599d8068e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0565a2700 1 -- 172.21.15.69:0/2803215904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d80652f0 msgr2=0x5599d8068e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0565a2700 1 --2- 172.21.15.69:0/2803215904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d80652f0 0x5599d8068e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0565a2700 1 -- 172.21.15.69:0/2803215904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 msgr2=0x5599d8069380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0565a2700 1 --2- 172.21.15.69:0/2803215904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 0x5599d8069380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0565a2700 1 -- 172.21.15.69:0/2803215904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d806ea90 con 0x5599d8069910 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0555a0700 1 --2- 172.21.15.69:0/2803215904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 0x5599d8069380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.360+0000 7fe0565a2700 1 --2- 172.21.15.69:0/2803215904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d8067340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe048008710 tx=0x7fe04800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e2148ba3ddfcfb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe03b7fe700 1 -- 172.21.15.69:0/2803215904 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048016600 con 0x5599d8069910 2024-05-14T16:49:39.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe03b7fe700 1 -- 172.21.15.69:0/2803215904 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe048016c50 con 0x5599d8069910 2024-05-14T16:49:39.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe03b7fe700 1 -- 172.21.15.69:0/2803215904 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048015be0 con 0x5599d8069910 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe06289bb80 1 -- 172.21.15.69:0/2803215904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 msgr2=0x5599d8067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe06289bb80 1 --2- 172.21.15.69:0/2803215904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d8067340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe048008710 tx=0x7fe04800c040 comp rx=0 tx=0).stop 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe06289bb80 1 -- 172.21.15.69:0/2803215904 shutdown_connections 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe06289bb80 1 --2- 172.21.15.69:0/2803215904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d8067340 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe06289bb80 1 --2- 172.21.15.69:0/2803215904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d80652f0 0x5599d8068e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe06289bb80 1 --2- 172.21.15.69:0/2803215904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 0x5599d8069380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe06289bb80 1 -- 172.21.15.69:0/2803215904 >> 172.21.15.69:0/2803215904 conn(0x5599d7f90200 msgr2=0x5599d8068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe06289bb80 1 -- 172.21.15.69:0/2803215904 shutdown_connections 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.361+0000 7fe06289bb80 1 -- 172.21.15.69:0/2803215904 wait complete. 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe06289bb80 1 -- start start 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe06289bb80 1 Processor -- start 2024-05-14T16:49:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d80652f0 0x5599d8088dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 0x5599d8085160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe06289bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d80856a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe06289bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5599d807f600 con 0x5599d8069910 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe06289bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5599d807ef30 con 0x5599d8065c10 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe06289bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599d807fcd0 con 0x5599d80652f0 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe06289bb80 10 client.0 _ll_register_callbacks cb 0x5599d7f4a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe0565a2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d80856a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.362+0000 7fe0565a2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d80856a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38744/0 (socket says 172.21.15.69:38744) 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0555a0700 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 0x5599d8085160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0565a2700 1 -- 172.21.15.69:0/2270631884 learned_addr learned my addr 172.21.15.69:0/2270631884 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe055da1700 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d80652f0 0x5599d8088dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0565a2700 1 -- 172.21.15.69:0/2270631884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d80652f0 msgr2=0x5599d8088dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0565a2700 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d80652f0 0x5599d8088dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0565a2700 1 -- 172.21.15.69:0/2270631884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 msgr2=0x5599d8085160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:39.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0565a2700 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 0x5599d8085160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0565a2700 1 -- 172.21.15.69:0/2270631884 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d8085be0 con 0x5599d8069910 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0555a0700 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 0x5599d8085160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0565a2700 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d80856a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe048018040 tx=0x7fe048007480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb62e9424834cdc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0397fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0397fa700 1 -- 172.21.15.69:0/2270631884 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048016600 con 0x5599d8069910 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0397fa700 1 -- 172.21.15.69:0/2270631884 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe0480167a0 con 0x5599d8069910 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe06289bb80 1 -- 172.21.15.69:0/2270631884 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe048016c50 con 0x5599d8069910 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0397fa700 1 -- 172.21.15.69:0/2270631884 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe0480169a0 con 0x5599d8069910 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe06289bb80 10 client.15096 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe06289bb80 1 -- 172.21.15.69:0/2270631884 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5599d80ea4a0 con 0x5599d8069910 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.363+0000 7fe0397fa700 1 -- 172.21.15.69:0/2270631884 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe048016c50 con 0x5599d8069910 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.364+0000 7fe0397fa700 1 -- 172.21.15.69:0/2270631884 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe048020810 con 0x5599d8069910 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.364+0000 7fe06289bb80 10 client.15096 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.364+0000 7fe06289bb80 10 client.15096 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:49:39.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.364+0000 7fe06289bb80 20 client.15096 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.364+0000 7fe06289bb80 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x5599d80eec90 0x5599d80ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.364+0000 7fe06289bb80 4 client.15096 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.364+0000 7fe06289bb80 1 -- 172.21.15.69:0/2270631884 --> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] -- command(tid 0: {"prefix": "status"}) -- 0x5599d7fc5b90 con 0x5599d80eec90 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.364+0000 7fe055da1700 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x5599d80eec90 0x5599d80ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.364+0000 7fe055da1700 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x5599d80eec90 0x5599d80ed9a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbc6af59932da2e server_cookie=94b5eadc13c6dbad in_seq=0 out_seq=0 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.364+0000 7fe0397fa700 10 client.15096 ms_handle_connect on v2:172.21.15.92:6836/1194779586 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe0397fa700 1 -- 172.21.15.69:0/2270631884 <== mds.2 v2:172.21.15.92:6836/1194779586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5599d7fc5b90 con 0x5599d80eec90 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe0397fa700 10 client.15096 handle_command_reply: tid=0 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 client.15096 shutdown 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 -- 172.21.15.69:0/2270631884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 msgr2=0x5599d80856a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d80856a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe048018040 tx=0x7fe048007480 comp rx=0 tx=0).stop 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 -- 172.21.15.69:0/2270631884 shutdown_connections 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x5599d80eec90 0x5599d80ed9a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599d8069910 0x5599d80856a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe0397fa700 0 client.15096 ms_handle_reset on v2:172.21.15.92:6836/1194779586 2024-05-14T16:49:39.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599d80652f0 0x5599d8088dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 --2- 172.21.15.69:0/2270631884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599d8065c10 0x5599d8085160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 -- 172.21.15.69:0/2270631884 >> 172.21.15.69:0/2270631884 conn(0x5599d7f90200 msgr2=0x5599d8068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:39.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 -- 172.21.15.69:0/2270631884 shutdown_connections 2024-05-14T16:49:39.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 1 -- 172.21.15.69:0/2270631884 wait complete. 2024-05-14T16:49:39.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.365+0000 7fe06289bb80 20 client.15096 trim_cache size 0 max 16384 2024-05-14T16:49:39.370 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:49:39.370 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:49:39.370 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-05-14T16:49:39.370 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24422, 2024-05-14T16:49:39.371 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-05-14T16:49:39.371 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-05-14T16:49:39.371 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-05-14T16:49:39.371 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 159.837521103, 2024-05-14T16:49:39.371 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 24, 2024-05-14T16:49:39.371 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 80, 2024-05-14T16:49:39.371 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:49:39.371 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 203.63927960300001 2024-05-14T16:49:39.371 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:49:39.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 -- 172.21.15.69:0/1571576952 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe03c0b2900 msgr2=0x7fe03c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe03c0b2900 0x7fe03c0b4dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009a10 tx=0x7fe04c009190 comp rx=0 tx=0).stop 2024-05-14T16:49:39.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 -- 172.21.15.69:0/1571576952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050102e90 msgr2=0x7fe0501045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050102e90 0x7fe0501045e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe040005910 tx=0x7fe0400047b0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 -- 172.21.15.69:0/1571576952 shutdown_connections 2024-05-14T16:49:39.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0501084d0 0x7fe050104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0501024d0 0x7fe0501040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fe03c0b2900 0x7fe03c0b4dc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 --2- 172.21.15.69:0/1571576952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050102e90 0x7fe0501045e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 -- 172.21.15.69:0/1571576952 >> 172.21.15.69:0/1571576952 conn(0x7fe050078490 msgr2=0x7fe05006bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:39.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 -- 172.21.15.69:0/1571576952 shutdown_connections 2024-05-14T16:49:39.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:39.366+0000 7fe058005700 1 -- 172.21.15.69:0/1571576952 wait complete. 2024-05-14T16:49:39.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:39 smithi069 ceph-mon[31507]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:39 smithi092 ceph-mon[35763]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:40.109 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-14T16:49:40.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:39 smithi203 ceph-mon[35787]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:40.350 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:41.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:41 smithi069 ceph-mon[31507]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:42.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:41 smithi092 ceph-mon[35763]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:42.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:41 smithi203 ceph-mon[35787]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T16:49:42.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.250+0000 7ffa24b61700 1 -- 172.21.15.69:0/2529440176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa2006c1f0 msgr2=0x7ffa2006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:42.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.250+0000 7ffa24b61700 1 --2- 172.21.15.69:0/2529440176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa2006c1f0 0x7ffa2006c670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009990 tx=0x7ffa14009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.251+0000 7ffa24b61700 1 -- 172.21.15.69:0/2529440176 shutdown_connections 2024-05-14T16:49:42.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.251+0000 7ffa24b61700 1 --2- 172.21.15.69:0/2529440176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa20105fd0 0x7ffa20106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.251+0000 7ffa24b61700 1 --2- 172.21.15.69:0/2529440176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20108560 0x7ffa2006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.251+0000 7ffa24b61700 1 --2- 172.21.15.69:0/2529440176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa2006c1f0 0x7ffa2006c670 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.251+0000 7ffa24b61700 1 -- 172.21.15.69:0/2529440176 >> 172.21.15.69:0/2529440176 conn(0x7ffa20078490 msgr2=0x7ffa200788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:42.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.251+0000 7ffa24b61700 1 -- 172.21.15.69:0/2529440176 shutdown_connections 2024-05-14T16:49:42.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.251+0000 7ffa24b61700 1 -- 172.21.15.69:0/2529440176 wait complete. 2024-05-14T16:49:42.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.251+0000 7ffa24b61700 1 Processor -- start 2024-05-14T16:49:42.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.251+0000 7ffa24b61700 1 -- start start 2024-05-14T16:49:42.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa24b61700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa2006c1f0 0x7ffa201a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa24b61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20105fd0 0x7ffa201a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa24b61700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa20108560 0x7ffa201a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa24b61700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20119d70 con 0x7ffa20108560 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa24b61700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20119bf0 con 0x7ffa20105fd0 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa24b61700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20119ef0 con 0x7ffa2006c1f0 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa20108560 0x7ffa201a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa20108560 0x7ffa201a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38762/0 (socket says 172.21.15.69:38762) 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1ed9d700 1 -- 172.21.15.69:0/3968298554 learned_addr learned my addr 172.21.15.69:0/3968298554 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1dd9b700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20105fd0 0x7ffa201a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa2006c1f0 0x7ffa201a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1ed9d700 1 -- 172.21.15.69:0/3968298554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa2006c1f0 msgr2=0x7ffa201a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1ed9d700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa2006c1f0 0x7ffa201a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1ed9d700 1 -- 172.21.15.69:0/3968298554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20105fd0 msgr2=0x7ffa201a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:42.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1ed9d700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20105fd0 0x7ffa201a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1ed9d700 1 -- 172.21.15.69:0/3968298554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa201a51f0 con 0x7ffa20108560 2024-05-14T16:49:42.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa1ed9d700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa20108560 0x7ffa201a4ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800ada0 tx=0x7ffa1800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:42.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa0f7fe700 1 -- 172.21.15.69:0/3968298554 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18017020 con 0x7ffa20108560 2024-05-14T16:49:42.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa0f7fe700 1 -- 172.21.15.69:0/3968298554 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1800ce30 con 0x7ffa20108560 2024-05-14T16:49:42.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa24b61700 1 -- 172.21.15.69:0/3968298554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa201a54e0 con 0x7ffa20108560 2024-05-14T16:49:42.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa0f7fe700 1 -- 172.21.15.69:0/3968298554 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa180054a0 con 0x7ffa20108560 2024-05-14T16:49:42.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.252+0000 7ffa24b61700 1 -- 172.21.15.69:0/3968298554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa201acd60 con 0x7ffa20108560 2024-05-14T16:49:42.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.255+0000 7ffa0f7fe700 1 -- 172.21.15.69:0/3968298554 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffa1800c980 con 0x7ffa20108560 2024-05-14T16:49:42.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.255+0000 7ffa0f7fe700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa080b2880 0x7ffa080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.256+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa080b2880 0x7ffa080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.256+0000 7ffa0f7fe700 1 -- 172.21.15.69:0/3968298554 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ffa180e1a00 con 0x7ffa20108560 2024-05-14T16:49:42.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.256+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa080b2880 0x7ffa080b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa10006fd0 tx=0x7ffa10008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:42.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.257+0000 7ffa2f3f7b80 1 Processor -- start 2024-05-14T16:49:42.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa2f3f7b80 1 -- start start 2024-05-14T16:49:42.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a32f0 0x562b337a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a3c10 0x562b337a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a7910 0x562b337a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b33701c40 con 0x562b337a3c10 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b336f17e0 con 0x562b337a7910 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b336f5820 con 0x562b337a32f0 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa1dd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a3c10 0x562b337a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa1dd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a3c10 0x562b337a73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38778/0 (socket says 172.21.15.69:38778) 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa1dd9b700 1 -- 172.21.15.69:0/1878267543 learned_addr learned my addr 172.21.15.69:0/1878267543 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/1878267543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a32f0 0x562b337a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa1ed9d700 1 --2- 172.21.15.69:0/1878267543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a7910 0x562b337a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa1dd9b700 1 -- 172.21.15.69:0/1878267543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a32f0 msgr2=0x562b337a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa1dd9b700 1 --2- 172.21.15.69:0/1878267543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a32f0 0x562b337a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa1dd9b700 1 -- 172.21.15.69:0/1878267543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a7910 msgr2=0x562b337a7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:42.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa1dd9b700 1 --2- 172.21.15.69:0/1878267543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a7910 0x562b337a7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.258+0000 7ffa1dd9b700 1 -- 172.21.15.69:0/1878267543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b33641880 con 0x562b337a3c10 2024-05-14T16:49:42.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.259+0000 7ffa1dd9b700 1 --2- 172.21.15.69:0/1878267543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a3c10 0x562b337a73d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa14015000 tx=0x7ffa14005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244d402d062c3c8b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:42.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.259+0000 7ffa0cff9700 1 -- 172.21.15.69:0/1878267543 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14027070 con 0x562b337a3c10 2024-05-14T16:49:42.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.259+0000 7ffa0cff9700 1 -- 172.21.15.69:0/1878267543 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1400bca0 con 0x562b337a3c10 2024-05-14T16:49:42.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.259+0000 7ffa0cff9700 1 -- 172.21.15.69:0/1878267543 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14003d10 con 0x562b337a3c10 2024-05-14T16:49:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1878267543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a3c10 msgr2=0x562b337a73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/1878267543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a3c10 0x562b337a73d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa14015000 tx=0x7ffa14005c80 comp rx=0 tx=0).stop 2024-05-14T16:49:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1878267543 shutdown_connections 2024-05-14T16:49:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/1878267543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a7910 0x562b337a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/1878267543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a32f0 0x562b337a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/1878267543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a3c10 0x562b337a73d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1878267543 >> 172.21.15.69:0/1878267543 conn(0x562b336ce200 msgr2=0x562b3379d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1878267543 shutdown_connections 2024-05-14T16:49:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1878267543 wait complete. 2024-05-14T16:49:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 -- start start 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.260+0000 7ffa2f3f7b80 1 Processor -- start 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a32f0 0x562b337aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a3c10 0x562b337ac150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a7910 0x562b337ac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b335f7f50 con 0x562b337a7910 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b33693770 con 0x562b337a32f0 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b337afb10 con 0x562b337a3c10 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa2f3f7b80 10 client.0 _ll_register_callbacks cb 0x562b33688530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a7910 0x562b337ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a7910 0x562b337ac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38782/0 (socket says 172.21.15.69:38782) 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1ed9d700 1 -- 172.21.15.69:0/946901147 learned_addr learned my addr 172.21.15.69:0/946901147 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1dd9b700 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a3c10 0x562b337ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a32f0 0x562b337aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1ed9d700 1 -- 172.21.15.69:0/946901147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a3c10 msgr2=0x562b337ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1ed9d700 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a3c10 0x562b337ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1ed9d700 1 -- 172.21.15.69:0/946901147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a32f0 msgr2=0x562b337aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1ed9d700 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a32f0 0x562b337aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1ed9d700 1 -- 172.21.15.69:0/946901147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b33690640 con 0x562b337a7910 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa1ed9d700 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a7910 0x562b337ac690 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800a0a0 tx=0x7ffa1800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45967ba81ecb186 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.261+0000 7ffa05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa05ffb700 1 -- 172.21.15.69:0/946901147 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa180a97b0 con 0x562b337a7910 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa05ffb700 1 -- 172.21.15.69:0/946901147 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ffa180ad110 con 0x562b337a7910 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/946901147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1400bca0 con 0x562b337a7910 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa05ffb700 1 -- 172.21.15.69:0/946901147 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffa180a90b0 con 0x562b337a7910 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa2f3f7b80 10 client.15108 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/946901147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562b337ba2b0 con 0x562b337a7910 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa05ffb700 1 -- 172.21.15.69:0/946901147 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1400bca0 con 0x562b337a7910 2024-05-14T16:49:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa05ffb700 1 -- 172.21.15.69:0/946901147 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ffa18012070 con 0x562b337a7910 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa2f3f7b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa2f3f7b80 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa2f3f7b80 20 client.15108 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x562b337bc880 0x562b337bdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa2f3f7b80 4 client.15108 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.262+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/946901147 --> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b3376f310 con 0x562b337bc880 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.263+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x562b337bc880 0x562b337bdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.263+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x562b337bc880 0x562b337bdcf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7e20b18d308d0a1 server_cookie=13ed8c535179a0bb in_seq=0 out_seq=0 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.263+0000 7ffa05ffb700 10 client.15108 ms_handle_connect on v2:172.21.15.203:6838/398835794 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa05ffb700 1 -- 172.21.15.69:0/946901147 <== mds.1 v2:172.21.15.203:6838/398835794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b3376f310 con 0x562b337bc880 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa05ffb700 10 client.15108 handle_command_reply: tid=0 2024-05-14T16:49:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 client.15108 shutdown 2024-05-14T16:49:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/946901147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a7910 msgr2=0x562b337ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a7910 0x562b337ac690 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800a0a0 tx=0x7ffa1800ccf0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/946901147 shutdown_connections 2024-05-14T16:49:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x562b337bc880 0x562b337bdcf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a7910 0x562b337ac690 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa05ffb700 0 client.15108 ms_handle_reset on v2:172.21.15.203:6838/398835794 2024-05-14T16:49:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a32f0 0x562b337aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/946901147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a3c10 0x562b337ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/946901147 >> 172.21.15.69:0/946901147 conn(0x562b336ce200 msgr2=0x562b337a8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:42.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/946901147 shutdown_connections 2024-05-14T16:49:42.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/946901147 wait complete. 2024-05-14T16:49:42.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.264+0000 7ffa2f3f7b80 20 client.15108 trim_cache size 0 max 16384 2024-05-14T16:49:42.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.285+0000 7ffa2f3f7b80 1 Processor -- start 2024-05-14T16:49:42.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.285+0000 7ffa2f3f7b80 1 -- start start 2024-05-14T16:49:42.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.285+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337ac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.285+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a3c10 0x562b337a2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.285+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a7910 0x562b337a9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.285+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b337bc030 con 0x562b337a32f0 2024-05-14T16:49:42.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.285+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b337bc700 con 0x562b337a3c10 2024-05-14T16:49:42.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.285+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b337bcdd0 con 0x562b337a7910 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337ac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38790/0 (socket says 172.21.15.69:38790) 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1e59c700 1 -- 172.21.15.69:0/2132453249 learned_addr learned my addr 172.21.15.69:0/2132453249 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1dd9b700 1 --2- 172.21.15.69:0/2132453249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a3c10 0x562b337a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1e59c700 1 -- 172.21.15.69:0/2132453249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a7910 msgr2=0x562b337a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1ed9d700 1 --2- 172.21.15.69:0/2132453249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a7910 0x562b337a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/2132453249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a7910 0x562b337a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1e59c700 1 -- 172.21.15.69:0/2132453249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a3c10 msgr2=0x562b337a2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/2132453249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a3c10 0x562b337a2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1e59c700 1 -- 172.21.15.69:0/2132453249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b33798a30 con 0x562b337a32f0 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/2132453249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337ac700 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x562b337be550 tx=0x7ffa10013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3b07fcb2278944 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa07fff700 1 -- 172.21.15.69:0/2132453249 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10014790 con 0x562b337a32f0 2024-05-14T16:49:42.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa07fff700 1 -- 172.21.15.69:0/2132453249 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa10014de0 con 0x562b337a32f0 2024-05-14T16:49:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.286+0000 7ffa07fff700 1 -- 172.21.15.69:0/2132453249 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10028920 con 0x562b337a32f0 2024-05-14T16:49:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/2132453249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 msgr2=0x562b337ac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/2132453249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337ac700 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x562b337be550 tx=0x7ffa10013000 comp rx=0 tx=0).stop 2024-05-14T16:49:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/2132453249 shutdown_connections 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/2132453249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337ac700 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/2132453249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a7910 0x562b337a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/2132453249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a3c10 0x562b337a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/2132453249 >> 172.21.15.69:0/2132453249 conn(0x562b336ce200 msgr2=0x562b337a2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/2132453249 shutdown_connections 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/2132453249 wait complete. 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 -- start start 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.287+0000 7ffa2f3f7b80 1 Processor -- start 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a3c10 0x562b337c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa2f3f7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a7910 0x562b337c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b337bdb70 con 0x562b337a32f0 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b337bd4a0 con 0x562b337a7910 2024-05-14T16:49:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa2f3f7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b337be240 con 0x562b337a3c10 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa2f3f7b80 10 client.0 _ll_register_callbacks cb 0x562b33688530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38794/0 (socket says 172.21.15.69:38794) 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1e59c700 1 -- 172.21.15.69:0/1397934437 learned_addr learned my addr 172.21.15.69:0/1397934437 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1dd9b700 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a3c10 0x562b337c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1ed9d700 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a7910 0x562b337c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1e59c700 1 -- 172.21.15.69:0/1397934437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a3c10 msgr2=0x562b337c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a3c10 0x562b337c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1e59c700 1 -- 172.21.15.69:0/1397934437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a7910 msgr2=0x562b337c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a7910 0x562b337c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1e59c700 1 -- 172.21.15.69:0/1397934437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b337c0b00 con 0x562b337a32f0 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa1e59c700 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337c7170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffa10028740 tx=0x7ffa10028a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e880c94a9c3f0fa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa05ffb700 1 -- 172.21.15.69:0/1397934437 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa100149a0 con 0x562b337a32f0 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa05ffb700 1 -- 172.21.15.69:0/1397934437 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ffa10014b40 con 0x562b337a32f0 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.288+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1397934437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa10014de0 con 0x562b337a32f0 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa05ffb700 1 -- 172.21.15.69:0/1397934437 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffa10028220 con 0x562b337a32f0 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa2f3f7b80 10 client.15114 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1397934437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562b338289d0 con 0x562b337a32f0 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa05ffb700 1 -- 172.21.15.69:0/1397934437 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa10014de0 con 0x562b337a32f0 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa05ffb700 1 -- 172.21.15.69:0/1397934437 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ffa10013070 con 0x562b337a32f0 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa2f3f7b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa2f3f7b80 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa2f3f7b80 20 client.15114 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x562b3382d1d0 0x562b3382bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:42.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa2f3f7b80 4 client.15114 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1397934437 --> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] -- command(tid 0: {"prefix": "status"}) -- 0x562b33703b90 con 0x562b3382d1d0 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.289+0000 7ffa1dd9b700 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x562b3382d1d0 0x562b3382bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa1dd9b700 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x562b3382d1d0 0x562b3382bee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd9666f2c42b6e4 server_cookie=9dbf24f5e92cfe76 in_seq=0 out_seq=0 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa05ffb700 10 client.15114 ms_handle_connect on v2:172.21.15.203:6838/398835794 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa05ffb700 1 -- 172.21.15.69:0/1397934437 <== mds.1 v2:172.21.15.203:6838/398835794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562b33703b90 con 0x562b3382d1d0 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa05ffb700 10 client.15114 handle_command_reply: tid=0 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 client.15114 shutdown 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1397934437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 msgr2=0x562b337c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337c7170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffa10028740 tx=0x7ffa10028a40 comp rx=0 tx=0).stop 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1397934437 shutdown_connections 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x562b3382d1d0 0x562b3382bee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b337a32f0 0x562b337c7170 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b337a7910 0x562b337c02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa05ffb700 0 client.15114 ms_handle_reset on v2:172.21.15.203:6838/398835794 2024-05-14T16:49:42.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 --2- 172.21.15.69:0/1397934437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b337a3c10 0x562b337c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1397934437 >> 172.21.15.69:0/1397934437 conn(0x562b336ce200 msgr2=0x562b337a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:42.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1397934437 shutdown_connections 2024-05-14T16:49:42.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 1 -- 172.21.15.69:0/1397934437 wait complete. 2024-05-14T16:49:42.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.290+0000 7ffa2f3f7b80 20 client.15114 trim_cache size 0 max 16384 2024-05-14T16:49:42.296 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:49:42.296 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:49:42.296 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-05-14T16:49:42.296 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24433, 2024-05-14T16:49:42.296 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-05-14T16:49:42.297 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-05-14T16:49:42.297 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-05-14T16:49:42.297 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 163.76860744300001, 2024-05-14T16:49:42.297 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 24, 2024-05-14T16:49:42.297 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 80, 2024-05-14T16:49:42.297 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:49:42.297 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 203.594675142 2024-05-14T16:49:42.297 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:49:42.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 -- 172.21.15.69:0/3968298554 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa080b2880 msgr2=0x7ffa080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:42.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa080b2880 0x7ffa080b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa10006fd0 tx=0x7ffa10008040 comp rx=0 tx=0).stop 2024-05-14T16:49:42.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 -- 172.21.15.69:0/3968298554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa20108560 msgr2=0x7ffa201a4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:42.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa20108560 0x7ffa201a4ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800ada0 tx=0x7ffa1800c330 comp rx=0 tx=0).stop 2024-05-14T16:49:42.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 -- 172.21.15.69:0/3968298554 shutdown_connections 2024-05-14T16:49:42.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7ffa080b2880 0x7ffa080b4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa20108560 0x7ffa201a4ab0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa2006c1f0 0x7ffa201a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 --2- 172.21.15.69:0/3968298554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20105fd0 0x7ffa201a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:42.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 -- 172.21.15.69:0/3968298554 >> 172.21.15.69:0/3968298554 conn(0x7ffa20078490 msgr2=0x7ffa201014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:42.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 -- 172.21.15.69:0/3968298554 shutdown_connections 2024-05-14T16:49:42.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:42.291+0000 7ffa24b61700 1 -- 172.21.15.69:0/3968298554 wait complete. 2024-05-14T16:49:43.088 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T16:49:43.328 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:44.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:43 smithi092 ceph-mon[35763]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:44.011 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:43 smithi203 ceph-mon[35787]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:44.134 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:43 smithi069 ceph-mon[31507]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:49:44.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:44 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:45.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:44 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:45.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.072+0000 7f3837531700 1 -- 172.21.15.69:0/1449029605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f383006b220 msgr2=0x7f383006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:45.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.072+0000 7f3837531700 1 --2- 172.21.15.69:0/1449029605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f383006b220 0x7f383006b6a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3820009990 tx=0x7f3820009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.072+0000 7f3837531700 1 -- 172.21.15.69:0/1449029605 shutdown_connections 2024-05-14T16:49:45.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.072+0000 7f3837531700 1 --2- 172.21.15.69:0/1449029605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f383006bbe0 0x7f38301103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.072+0000 7f3837531700 1 --2- 172.21.15.69:0/1449029605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38301095b0 0x7f3830109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.072+0000 7f3837531700 1 --2- 172.21.15.69:0/1449029605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f383006b220 0x7f383006b6a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.072+0000 7f3837531700 1 -- 172.21.15.69:0/1449029605 >> 172.21.15.69:0/1449029605 conn(0x7f38300793f0 msgr2=0x7f3830079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.072+0000 7f3837531700 1 -- 172.21.15.69:0/1449029605 shutdown_connections 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.072+0000 7f3837531700 1 -- 172.21.15.69:0/1449029605 wait complete. 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3837531700 1 Processor -- start 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3837531700 1 -- start start 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3837531700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f383006b220 0x7f3830102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3837531700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f383006bbe0 0x7f3830102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3837531700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38301095b0 0x7f3830102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3837531700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f38301136c0 con 0x7f38301095b0 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3837531700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3830113540 con 0x7f383006bbe0 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3837531700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3830113840 con 0x7f383006b220 2024-05-14T16:49:45.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3835ace700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38301095b0 0x7f3830102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3835ace700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38301095b0 0x7f3830102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34432/0 (socket says 172.21.15.69:34432) 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3835ace700 1 -- 172.21.15.69:0/2980048819 learned_addr learned my addr 172.21.15.69:0/2980048819 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f38352cd700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f383006b220 0x7f3830102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3834acc700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f383006bbe0 0x7f3830102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3835ace700 1 -- 172.21.15.69:0/2980048819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f383006b220 msgr2=0x7f3830102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3835ace700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f383006b220 0x7f3830102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3835ace700 1 -- 172.21.15.69:0/2980048819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f383006bbe0 msgr2=0x7f3830102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3835ace700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f383006bbe0 0x7f3830102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3835ace700 1 -- 172.21.15.69:0/2980048819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38301a8160 con 0x7f38301095b0 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f38352cd700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f383006b220 0x7f3830102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3834acc700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f383006bbe0 0x7f3830102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.073+0000 7f3835ace700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38301095b0 0x7f3830102f90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f382800ef30 tx=0x7f382800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.074+0000 7f38267fc700 1 -- 172.21.15.69:0/2980048819 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38280092e0 con 0x7f38301095b0 2024-05-14T16:49:45.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.074+0000 7f38267fc700 1 -- 172.21.15.69:0/2980048819 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3828004bb0 con 0x7f38301095b0 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.074+0000 7f38267fc700 1 -- 172.21.15.69:0/2980048819 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828004280 con 0x7f38301095b0 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.074+0000 7f3837531700 1 -- 172.21.15.69:0/2980048819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38301a83f0 con 0x7f38301095b0 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.074+0000 7f3837531700 1 -- 172.21.15.69:0/2980048819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38301a88e0 con 0x7f38301095b0 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.076+0000 7f38267fc700 1 -- 172.21.15.69:0/2980048819 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f382801b030 con 0x7f38301095b0 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.077+0000 7f38267fc700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f381c0b2900 0x7f381c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.077+0000 7f38352cd700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f381c0b2900 0x7f381c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.077+0000 7f38267fc700 1 -- 172.21.15.69:0/2980048819 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f38280e1d00 con 0x7f38301095b0 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.077+0000 7f38352cd700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f381c0b2900 0x7f381c0b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f382c005950 tx=0x7f382c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3841dc7b80 1 Processor -- start 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3841dc7b80 1 -- start start 2024-05-14T16:49:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 0x564210b4f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b497a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b4a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564210aa7d20 con 0x564210b4f8c0 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564210a978c0 con 0x564210b49dc0 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564210a9b900 con 0x564210b4efa0 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3834acc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3834acc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b497a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34446/0 (socket says 172.21.15.69:34446) 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.079+0000 7f3834acc700 1 -- 172.21.15.69:0/2548053416 learned_addr learned my addr 172.21.15.69:0/2548053416 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f3835ace700 1 --2- 172.21.15.69:0/2548053416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b4a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f38352cd700 1 --2- 172.21.15.69:0/2548053416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 0x564210b4f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f3834acc700 1 -- 172.21.15.69:0/2548053416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 msgr2=0x564210b4f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f3834acc700 1 --2- 172.21.15.69:0/2548053416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 0x564210b4f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f3834acc700 1 -- 172.21.15.69:0/2548053416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 msgr2=0x564210b4a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f3834acc700 1 --2- 172.21.15.69:0/2548053416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b4a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f3834acc700 1 -- 172.21.15.69:0/2548053416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564210a40f60 con 0x564210b4f8c0 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f38352cd700 1 --2- 172.21.15.69:0/2548053416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 0x564210b4f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f3834acc700 1 --2- 172.21.15.69:0/2548053416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b497a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3820009990 tx=0x7f38200046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8319d3521c9c5db9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f381b7fe700 1 -- 172.21.15.69:0/2548053416 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382000bd80 con 0x564210b4f8c0 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f381b7fe700 1 -- 172.21.15.69:0/2548053416 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38200044d0 con 0x564210b4f8c0 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.080+0000 7f381b7fe700 1 -- 172.21.15.69:0/2548053416 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38200049a0 con 0x564210b4f8c0 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.081+0000 7f3841dc7b80 1 -- 172.21.15.69:0/2548053416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 msgr2=0x564210b497a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.081+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/2548053416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b497a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3820009990 tx=0x7f38200046d0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.081+0000 7f3841dc7b80 1 -- 172.21.15.69:0/2548053416 shutdown_connections 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.081+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/2548053416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b4a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.081+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/2548053416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 0x564210b4f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.081+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/2548053416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b497a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.081+0000 7f3841dc7b80 1 -- 172.21.15.69:0/2548053416 >> 172.21.15.69:0/2548053416 conn(0x564210a76d60 msgr2=0x564210b43e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:45.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.081+0000 7f3841dc7b80 1 -- 172.21.15.69:0/2548053416 shutdown_connections 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.081+0000 7f3841dc7b80 1 -- 172.21.15.69:0/2548053416 wait complete. 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.082+0000 7f3841dc7b80 1 -- start start 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.082+0000 7f3841dc7b80 1 Processor -- start 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.082+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b49dc0 0x564210b55560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.082+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b52900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.082+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b4f8c0 0x564210b52e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.082+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56421099df50 con 0x564210b4efa0 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.082+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5642109ffd50 con 0x564210b4f8c0 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.082+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564210b56210 con 0x564210b49dc0 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3841dc7b80 10 client.0 _ll_register_callbacks cb 0x564210a32900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3834acc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b52900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3834acc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b52900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34448/0 (socket says 172.21.15.69:34448) 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3834acc700 1 -- 172.21.15.69:0/166639963 learned_addr learned my addr 172.21.15.69:0/166639963 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3835ace700 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b4f8c0 0x564210b52e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f38352cd700 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b49dc0 0x564210b55560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3834acc700 1 -- 172.21.15.69:0/166639963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b49dc0 msgr2=0x564210b55560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3834acc700 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b49dc0 0x564210b55560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3834acc700 1 -- 172.21.15.69:0/166639963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b4f8c0 msgr2=0x564210b52e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3834acc700 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b4f8c0 0x564210b52e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3834acc700 1 -- 172.21.15.69:0/166639963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642109e7880 con 0x564210b4efa0 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3835ace700 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b4f8c0 0x564210b52e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f38352cd700 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b49dc0 0x564210b55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3834acc700 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b52900 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f38200046a0 tx=0x7f3820005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598920c08d675b95 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3818ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3818ff9700 1 -- 172.21.15.69:0/166639963 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820022850 con 0x564210b4efa0 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3818ff9700 1 -- 172.21.15.69:0/166639963 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f38200229f0 con 0x564210b4efa0 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3841dc7b80 1 -- 172.21.15.69:0/166639963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38200044d0 con 0x564210b4efa0 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.083+0000 7f3818ff9700 1 -- 172.21.15.69:0/166639963 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3820022bf0 con 0x564210b4efa0 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f3841dc7b80 10 client.15126 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f3841dc7b80 1 -- 172.21.15.69:0/166639963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564210a51710 con 0x564210b4efa0 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f3818ff9700 1 -- 172.21.15.69:0/166639963 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38200044d0 con 0x564210b4efa0 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f3818ff9700 1 -- 172.21.15.69:0/166639963 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3820037020 con 0x564210b4efa0 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f3841dc7b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f3841dc7b80 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f3841dc7b80 20 client.15126 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564210b61a70 0x564210b63bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f3841dc7b80 4 client.15126 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:49:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f3841dc7b80 1 -- 172.21.15.69:0/166639963 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564210aa9c70 con 0x564210b61a70 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.084+0000 7f38352cd700 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564210b61a70 0x564210b63bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.085+0000 7f3818ff9700 10 client.15126 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.085+0000 7f38352cd700 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564210b61a70 0x564210b63bb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2a7fa0ff9872ca9 server_cookie=2983a926df9e8828 in_seq=0 out_seq=0 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.085+0000 7f3818ff9700 1 -- 172.21.15.69:0/166639963 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564210aa9c70 con 0x564210b61a70 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.085+0000 7f3818ff9700 10 client.15126 handle_command_reply: tid=0 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.085+0000 7f3841dc7b80 1 client.15126 shutdown 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.085+0000 7f3841dc7b80 1 -- 172.21.15.69:0/166639963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 msgr2=0x564210b52900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.085+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b52900 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f38200046a0 tx=0x7f3820005e80 comp rx=0 tx=0).stop 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.086+0000 7f3841dc7b80 1 -- 172.21.15.69:0/166639963 shutdown_connections 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.086+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564210b61a70 0x564210b63bb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.086+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b4f8c0 0x564210b52e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.086+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b49dc0 0x564210b55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.086+0000 7f3818ff9700 0 client.15126 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:49:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.086+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/166639963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b52900 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.086+0000 7f3841dc7b80 1 -- 172.21.15.69:0/166639963 >> 172.21.15.69:0/166639963 conn(0x564210a76d60 msgr2=0x564210b43630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.086+0000 7f3841dc7b80 1 -- 172.21.15.69:0/166639963 shutdown_connections 2024-05-14T16:49:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.086+0000 7f3841dc7b80 1 -- 172.21.15.69:0/166639963 wait complete. 2024-05-14T16:49:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.086+0000 7f3841dc7b80 20 client.15126 trim_cache size 0 max 16384 2024-05-14T16:49:45.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3841dc7b80 1 Processor -- start 2024-05-14T16:49:45.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3841dc7b80 1 -- start start 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b497a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 0x564210b503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b43db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564210b61ef0 con 0x564210b4f8c0 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564210b625c0 con 0x564210b49dc0 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564210b62c90 con 0x564210b4efa0 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3835ace700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b43db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3835ace700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b43db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34450/0 (socket says 172.21.15.69:34450) 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3835ace700 1 -- 172.21.15.69:0/2480680351 learned_addr learned my addr 172.21.15.69:0/2480680351 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f3834acc700 1 --2- 172.21.15.69:0/2480680351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 0x564210b503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.107+0000 7f38352cd700 1 --2- 172.21.15.69:0/2480680351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.108+0000 7f3835ace700 1 -- 172.21.15.69:0/2480680351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 msgr2=0x564210b503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.108+0000 7f3835ace700 1 --2- 172.21.15.69:0/2480680351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 0x564210b503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.108+0000 7f3835ace700 1 -- 172.21.15.69:0/2480680351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 msgr2=0x564210b497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.108+0000 7f3835ace700 1 --2- 172.21.15.69:0/2480680351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.108+0000 7f3835ace700 1 -- 172.21.15.69:0/2480680351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564210b508f0 con 0x564210b4f8c0 2024-05-14T16:49:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.108+0000 7f38352cd700 1 --2- 172.21.15.69:0/2480680351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.108+0000 7f3835ace700 1 --2- 172.21.15.69:0/2480680351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b43db0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f38280a65e0 tx=0x7f38280a5360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a157eefc22b8bb88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.108+0000 7f381affd700 1 -- 172.21.15.69:0/2480680351 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38280a9a70 con 0x564210b4f8c0 2024-05-14T16:49:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.108+0000 7f381affd700 1 -- 172.21.15.69:0/2480680351 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38280a9370 con 0x564210b4f8c0 2024-05-14T16:49:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.108+0000 7f381affd700 1 -- 172.21.15.69:0/2480680351 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38280a4f40 con 0x564210b4f8c0 2024-05-14T16:49:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.109+0000 7f3841dc7b80 1 -- 172.21.15.69:0/2480680351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 msgr2=0x564210b43db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.109+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/2480680351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b43db0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f38280a65e0 tx=0x7f38280a5360 comp rx=0 tx=0).stop 2024-05-14T16:49:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.109+0000 7f3841dc7b80 1 -- 172.21.15.69:0/2480680351 shutdown_connections 2024-05-14T16:49:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.109+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/2480680351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4f8c0 0x564210b43db0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.109+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/2480680351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.109+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/2480680351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4efa0 0x564210b503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.109+0000 7f3841dc7b80 1 -- 172.21.15.69:0/2480680351 >> 172.21.15.69:0/2480680351 conn(0x564210a76d60 msgr2=0x564210b4fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.109+0000 7f3841dc7b80 1 -- 172.21.15.69:0/2480680351 shutdown_connections 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.109+0000 7f3841dc7b80 1 -- 172.21.15.69:0/2480680351 wait complete. 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3841dc7b80 1 -- start start 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3841dc7b80 1 Processor -- start 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b6d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b67fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3841dc7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4f8c0 0x564210b68520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564210b63a30 con 0x564210b4efa0 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564210b63360 con 0x564210b49dc0 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3841dc7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564210b64100 con 0x564210b4f8c0 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3841dc7b80 10 client.0 _ll_register_callbacks cb 0x564210a32900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3834acc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b67fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3834acc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b67fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34454/0 (socket says 172.21.15.69:34454) 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f3834acc700 1 -- 172.21.15.69:0/1317324775 learned_addr learned my addr 172.21.15.69:0/1317324775 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.110+0000 7f38352cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b6d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3835ace700 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4f8c0 0x564210b68520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3834acc700 1 -- 172.21.15.69:0/1317324775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4f8c0 msgr2=0x564210b68520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3834acc700 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4f8c0 0x564210b68520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3834acc700 1 -- 172.21.15.69:0/1317324775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 msgr2=0x564210b6d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3834acc700 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b6d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3834acc700 1 -- 172.21.15.69:0/1317324775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564210b68a60 con 0x564210b4efa0 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f38352cd700 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b6d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3835ace700 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4f8c0 0x564210b68520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3834acc700 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b67fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3820001eb0 tx=0x7f382000f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85905cd77b4201e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3818ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3818ff9700 1 -- 172.21.15.69:0/1317324775 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382000fb70 con 0x564210b4efa0 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3841dc7b80 1 -- 172.21.15.69:0/1317324775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38280a9370 con 0x564210b4efa0 2024-05-14T16:49:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3818ff9700 1 -- 172.21.15.69:0/1317324775 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f382000fd10 con 0x564210b4efa0 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3818ff9700 1 -- 172.21.15.69:0/1317324775 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3820005c00 con 0x564210b4efa0 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3841dc7b80 10 client.15132 fetch_fsmap learned FSMap version 24 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.111+0000 7f3841dc7b80 1 -- 172.21.15.69:0/1317324775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564210bce810 con 0x564210b4efa0 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f3818ff9700 1 -- 172.21.15.69:0/1317324775 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38280a9370 con 0x564210b4efa0 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f3818ff9700 1 -- 172.21.15.69:0/1317324775 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f38200395d0 con 0x564210b4efa0 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f3841dc7b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f3841dc7b80 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f3841dc7b80 20 client.15132 populate_metadata read hostname 'smithi069' 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564210bd3120 0x564210bd1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f3841dc7b80 4 client.15132 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f3841dc7b80 1 -- 172.21.15.69:0/1317324775 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "status"}) -- 0x564210aa3450 con 0x564210bd3120 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f38352cd700 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564210bd3120 0x564210bd1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f3818ff9700 10 client.15132 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.112+0000 7f38352cd700 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564210bd3120 0x564210bd1e30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce45dc31dfe2a087 server_cookie=7137940e2ad5b26c in_seq=0 out_seq=0 2024-05-14T16:49:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3818ff9700 1 -- 172.21.15.69:0/1317324775 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564210aa3450 con 0x564210bd3120 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3818ff9700 10 client.15132 handle_command_reply: tid=0 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 client.15132 shutdown 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 -- 172.21.15.69:0/1317324775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 msgr2=0x564210b67fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b67fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3820001eb0 tx=0x7f382000f820 comp rx=0 tx=0).stop 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 -- 172.21.15.69:0/1317324775 shutdown_connections 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564210bd3120 0x564210bd1e30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564210b4f8c0 0x564210b68520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3818ff9700 0 client.15132 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564210b49dc0 0x564210b6d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 --2- 172.21.15.69:0/1317324775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564210b4efa0 0x564210b67fe0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 -- 172.21.15.69:0/1317324775 >> 172.21.15.69:0/1317324775 conn(0x564210a76d60 msgr2=0x564210b4da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 -- 172.21.15.69:0/1317324775 shutdown_connections 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 1 -- 172.21.15.69:0/1317324775 wait complete. 2024-05-14T16:49:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.113+0000 7f3841dc7b80 20 client.15132 trim_cache size 0 max 16384 2024-05-14T16:49:45.118 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:49:45.118 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:49:45.118 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-05-14T16:49:45.118 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24446, 2024-05-14T16:49:45.118 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-05-14T16:49:45.118 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-05-14T16:49:45.118 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-05-14T16:49:45.118 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 179.55061732999999, 2024-05-14T16:49:45.118 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 24, 2024-05-14T16:49:45.118 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 80, 2024-05-14T16:49:45.119 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:49:45.119 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 196.51553737200001 2024-05-14T16:49:45.119 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:49:45.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 -- 172.21.15.69:0/2980048819 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f381c0b2900 msgr2=0x7f381c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:45.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f381c0b2900 0x7f381c0b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f382c005950 tx=0x7f382c00a300 comp rx=0 tx=0).stop 2024-05-14T16:49:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 -- 172.21.15.69:0/2980048819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38301095b0 msgr2=0x7f3830102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38301095b0 0x7f3830102f90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f382800ef30 tx=0x7f382800c3a0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 -- 172.21.15.69:0/2980048819 shutdown_connections 2024-05-14T16:49:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f381c0b2900 0x7f381c0b4dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38301095b0 0x7f3830102f90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f383006b220 0x7f3830102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 --2- 172.21.15.69:0/2980048819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f383006bbe0 0x7f3830102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 -- 172.21.15.69:0/2980048819 >> 172.21.15.69:0/2980048819 conn(0x7f38300793f0 msgr2=0x7f3830101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 -- 172.21.15.69:0/2980048819 shutdown_connections 2024-05-14T16:49:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:45.114+0000 7f3837531700 1 -- 172.21.15.69:0/2980048819 wait complete. 2024-05-14T16:49:45.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:44 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:45.936 INFO:teuthology.run_tasks:Running task sequential... 2024-05-14T16:49:45.945 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-14T16:49:45.945 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-14T16:49:45.945 INFO:teuthology.task.print:Enabling mgr modules 2024-05-14T16:49:45.945 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-14T16:49:45.946 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T16:49:45.946 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T16:49:45.946 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-14T16:49:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:45 smithi092 ceph-mon[35763]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:49:46.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:45 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.054+0000 7f2880064640 1 -- 172.21.15.69:0/3885625676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878103cc0 msgr2=0x7f2878104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.054+0000 7f2880064640 1 --2- 172.21.15.69:0/3885625676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878103cc0 0x7f2878104140 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2868009900 tx=0x7f286802a410 comp rx=0 tx=0).stop 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.054+0000 7f2880064640 1 -- 172.21.15.69:0/3885625676 shutdown_connections 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.054+0000 7f2880064640 1 --2- 172.21.15.69:0/3885625676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2878104680 0x7f287810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.054+0000 7f2880064640 1 --2- 172.21.15.69:0/3885625676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878103cc0 0x7f2878104140 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.054+0000 7f2880064640 1 --2- 172.21.15.69:0/3885625676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2878102ad0 0x7f2878102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.054+0000 7f2880064640 1 -- 172.21.15.69:0/3885625676 >> 172.21.15.69:0/3885625676 conn(0x7f28780fe040 msgr2=0x7f2878100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.054+0000 7f2880064640 1 -- 172.21.15.69:0/3885625676 shutdown_connections 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.054+0000 7f2880064640 1 -- 172.21.15.69:0/3885625676 wait complete. 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f2880064640 1 Processor -- start 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f2880064640 1 -- start start 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f2880064640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878102ad0 0x7f2878198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:46.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f2880064640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2878103cc0 0x7f28781991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f2880064640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2878104680 0x7f28781a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f2880064640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f287810e630 con 0x7f2878102ad0 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f2880064640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f287810e4b0 con 0x7f2878104680 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f2880064640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f287810e7b0 con 0x7f2878103cc0 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878102ad0 0x7f2878198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878102ad0 0x7f2878198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34480/0 (socket says 172.21.15.69:34480) 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287de00640 1 -- 172.21.15.69:0/1189784406 learned_addr learned my addr 172.21.15.69:0/1189784406 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287d5ff640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2878103cc0 0x7f28781991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287e601640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2878104680 0x7f28781a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287de00640 1 -- 172.21.15.69:0/1189784406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2878103cc0 msgr2=0x7f28781991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287de00640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2878103cc0 0x7f28781991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287de00640 1 -- 172.21.15.69:0/1189784406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2878104680 msgr2=0x7f28781a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287de00640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2878104680 0x7f28781a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287de00640 1 -- 172.21.15.69:0/1189784406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28781a0880 con 0x7f2878102ad0 2024-05-14T16:49:46.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.055+0000 7f287de00640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878102ad0 0x7f2878198c70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f287400d950 tx=0x7f287400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:46.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.056+0000 7f286effd640 1 -- 172.21.15.69:0/1189784406 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874017020 con 0x7f2878102ad0 2024-05-14T16:49:46.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.056+0000 7f286effd640 1 -- 172.21.15.69:0/1189784406 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2874009c30 con 0x7f2878102ad0 2024-05-14T16:49:46.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.056+0000 7f2880064640 1 -- 172.21.15.69:0/1189784406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28781a0b70 con 0x7f2878102ad0 2024-05-14T16:49:46.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.056+0000 7f286effd640 1 -- 172.21.15.69:0/1189784406 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287400f6b0 con 0x7f2878102ad0 2024-05-14T16:49:46.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.056+0000 7f2880064640 1 -- 172.21.15.69:0/1189784406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28781a10c0 con 0x7f2878102ad0 2024-05-14T16:49:46.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.057+0000 7f2880064640 1 -- 172.21.15.69:0/1189784406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2878109df0 con 0x7f2878102ad0 2024-05-14T16:49:46.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.058+0000 7f286effd640 1 -- 172.21.15.69:0/1189784406 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f287401a020 con 0x7f2878102ad0 2024-05-14T16:49:46.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.060+0000 7f286effd640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28640b28a0 0x7f28640b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:46.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.060+0000 7f287d5ff640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28640b28a0 0x7f28640b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:46.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.060+0000 7f286effd640 1 -- 172.21.15.69:0/1189784406 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f28740e22d0 con 0x7f2878102ad0 2024-05-14T16:49:46.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.060+0000 7f287d5ff640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28640b28a0 0x7f28640b4d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2868031500 tx=0x7f2868005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:46.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.062+0000 7f286effd640 1 -- 172.21.15.69:0/1189784406 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2874009de0 con 0x7f2878102ad0 2024-05-14T16:49:46.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:45 smithi069 ceph-mon[31507]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:46.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:49:46.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:45 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:49:46.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:45 smithi203 ceph-mon[35787]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:49:46.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:49:46.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:45 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:49:46.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:46.302+0000 7f2880064640 1 -- 172.21.15.69:0/1189784406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f287802de10 con 0x7f2878102ad0 2024-05-14T16:49:47.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:46 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:47.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:46 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:47.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:46 smithi092 ceph-mon[35763]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:49:47.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:46 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1189784406' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T16:49:47.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.093+0000 7f286effd640 1 -- 172.21.15.69:0/1189784406 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f28740a9fb0 con 0x7f2878102ad0 2024-05-14T16:49:47.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.095+0000 7f2880064640 1 -- 172.21.15.69:0/1189784406 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28640b28a0 msgr2=0x7f28640b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:47.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.095+0000 7f2880064640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28640b28a0 0x7f28640b4d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2868031500 tx=0x7f2868005a20 comp rx=0 tx=0).stop 2024-05-14T16:49:47.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.096+0000 7f2880064640 1 -- 172.21.15.69:0/1189784406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878102ad0 msgr2=0x7f2878198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:47.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.096+0000 7f2880064640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878102ad0 0x7f2878198c70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f287400d950 tx=0x7f287400de10 comp rx=0 tx=0).stop 2024-05-14T16:49:47.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.096+0000 7f2880064640 1 -- 172.21.15.69:0/1189784406 shutdown_connections 2024-05-14T16:49:47.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.096+0000 7f2880064640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f28640b28a0 0x7f28640b4d60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:47.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.096+0000 7f2880064640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2878104680 0x7f28781a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:47.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.096+0000 7f2880064640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2878103cc0 0x7f28781991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:47.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.096+0000 7f2880064640 1 --2- 172.21.15.69:0/1189784406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878102ad0 0x7f2878198c70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:47.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.096+0000 7f2880064640 1 -- 172.21.15.69:0/1189784406 >> 172.21.15.69:0/1189784406 conn(0x7f28780fe040 msgr2=0x7f2878106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:47.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.096+0000 7f2880064640 1 -- 172.21.15.69:0/1189784406 shutdown_connections 2024-05-14T16:49:47.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:47.096+0000 7f2880064640 1 -- 172.21.15.69:0/1189784406 wait complete. 2024-05-14T16:49:47.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:46 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:47.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:46 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:47.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:46 smithi069 ceph-mon[31507]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:49:47.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:46 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1189784406' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T16:49:47.114 INFO:teuthology.run_tasks:Running task kclient... 2024-05-14T16:49:47.137 INFO:tasks.kclient:Mounting kernel clients... 2024-05-14T16:49:47.137 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-14T16:49:47.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:49:47.138 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-05-14T16:49:47.138 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:46 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:47.138 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:46 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:47.138 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:46 smithi203 ceph-mon[35787]: from='mgr.14150 172.21.15.69:0/2035051102' entity='mgr.x' 2024-05-14T16:49:47.139 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:46 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1189784406' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T16:49:47.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:49:47.154 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-05-14T16:49:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2024-05-14T16:49:47.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:49:47.224 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-05-14T16:49:47.224 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T16:49:47.224 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-14T16:49:47.224 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-14T16:49:47.224 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-14T16:49:47.453 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: ignoring --setuser ceph since I am not root 2024-05-14T16:49:47.453 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: ignoring --setgroup ceph since I am not root 2024-05-14T16:49:47.453 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:47.283+0000 7f1357aa0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:49:47.455 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: ignoring --setuser ceph since I am not root 2024-05-14T16:49:47.455 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: ignoring --setgroup ceph since I am not root 2024-05-14T16:49:47.455 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:47 smithi092 ceph-mgr[37134]: -- 172.21.15.92:0/2509579610 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562080d7e5a0 con 0x562080d53000 2024-05-14T16:49:47.455 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:47.272+0000 7f6a9f3a2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:49:47.455 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:47.455+0000 7f6a9f3a2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:49:47.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: ignoring --setuser ceph since I am not root 2024-05-14T16:49:47.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: ignoring --setgroup ceph since I am not root 2024-05-14T16:49:47.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:47 smithi203 ceph-mgr[37640]: -- 172.21.15.203:0/2734493503 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e94be05a0 con 0x556e94bb4c00 2024-05-14T16:49:47.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:47.278+0000 7fafe7b36200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:49:47.511 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:47.729 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:47.462+0000 7fafe7b36200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:49:47.729 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:47.566+0000 7fafe7b36200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:49:47.729 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:47.559+0000 7f6a9f3a2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:49:47.773 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:47 smithi069 ceph-mon[31507]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:47.773 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:47 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1189784406' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T16:49:47.773 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:47 smithi069 ceph-mon[31507]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-14T16:49:47.773 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:47.483+0000 7f1357aa0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:49:47.773 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:47.598+0000 7f1357aa0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:49:48.000 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:47.873+0000 7f6a9f3a2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:49:48.000 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:47.959+0000 7f6a9f3a2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:49:48.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:47 smithi092 ceph-mon[35763]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:48.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:47 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1189784406' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T16:49:48.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:47 smithi092 ceph-mon[35763]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-14T16:49:48.203 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:47.923+0000 7f1357aa0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:49:48.203 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:48 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:48.011+0000 7f1357aa0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:49:48.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:47 smithi203 ceph-mon[35787]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:49:48.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:47 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1189784406' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T16:49:48.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:47 smithi203 ceph-mon[35787]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-14T16:49:48.209 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:47.878+0000 7fafe7b36200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:49:48.209 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:47.965+0000 7fafe7b36200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:49:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.329+0000 7f66abe77700 1 -- 172.21.15.69:0/1336030849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a4102df0 msgr2=0x7f66a410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.329+0000 7f66abe77700 1 --2- 172.21.15.69:0/1336030849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a4102df0 0x7f66a410f2e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f669c00b0a0 tx=0x7f669c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.329+0000 7f66abe77700 1 -- 172.21.15.69:0/1336030849 shutdown_connections 2024-05-14T16:49:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.329+0000 7f66abe77700 1 --2- 172.21.15.69:0/1336030849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a4102df0 0x7f66a410f2e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.329+0000 7f66abe77700 1 --2- 172.21.15.69:0/1336030849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a4108510 0x7f66a41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.329+0000 7f66abe77700 1 --2- 172.21.15.69:0/1336030849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4102430 0x7f66a41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.329+0000 7f66abe77700 1 -- 172.21.15.69:0/1336030849 >> 172.21.15.69:0/1336030849 conn(0x7f66a40793b0 msgr2=0x7f66a40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.329+0000 7f66abe77700 1 -- 172.21.15.69:0/1336030849 shutdown_connections 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.329+0000 7f66abe77700 1 -- 172.21.15.69:0/1336030849 wait complete. 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66abe77700 1 Processor -- start 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66abe77700 1 -- start start 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66abe77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4102430 0x7f66a419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66abe77700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a4102df0 0x7f66a419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66abe77700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a4108510 0x7f66a4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66abe77700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41125c0 con 0x7f66a4102df0 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66abe77700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112440 con 0x7f66a4102430 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66abe77700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112740 con 0x7f66a4108510 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66a9412700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a4102df0 0x7f66a419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66a9412700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a4102df0 0x7f66a419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34522/0 (socket says 172.21.15.69:34522) 2024-05-14T16:49:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66a9c13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4102430 0x7f66a419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.330+0000 7f66a9412700 1 -- 172.21.15.69:0/3982149410 learned_addr learned my addr 172.21.15.69:0/3982149410 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f66aa414700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a4108510 0x7f66a4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f66a9c13700 1 -- 172.21.15.69:0/3982149410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a4108510 msgr2=0x7f66a4199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f66a9c13700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a4108510 0x7f66a4199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f66a9c13700 1 -- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a4102df0 msgr2=0x7f66a419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f66a9c13700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a4102df0 0x7f66a419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f66a9c13700 1 -- 172.21.15.69:0/3982149410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a419a190 con 0x7f66a4102430 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f66a9412700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a4102df0 0x7f66a419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f66a9c13700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4102430 0x7f66a419f870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66a000d950 tx=0x7f66a000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f669affd700 1 -- 172.21.15.69:0/3982149410 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0016070 con 0x7f66a4102430 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f669affd700 1 -- 172.21.15.69:0/3982149410 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a000fd40 con 0x7f66a4102430 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f66abe77700 1 -- 172.21.15.69:0/3982149410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a419a480 con 0x7f66a4102430 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f669affd700 1 -- 172.21.15.69:0/3982149410 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000e3b0 con 0x7f66a4102430 2024-05-14T16:49:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.331+0000 7f66abe77700 1 -- 172.21.15.69:0/3982149410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a41a6740 con 0x7f66a4102430 2024-05-14T16:49:49.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.332+0000 7f66abe77700 1 -- 172.21.15.69:0/3982149410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a406a230 con 0x7f66a4102430 2024-05-14T16:49:49.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.334+0000 7f669affd700 1 -- 172.21.15.69:0/3982149410 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f66a0019070 con 0x7f66a4102430 2024-05-14T16:49:49.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.334+0000 7f669affd700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f66900b28d0 0x7f66900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:49.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.334+0000 7f66a9412700 1 -- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f66900b28d0 msgr2=0x7f66900b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:49.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.334+0000 7f669affd700 1 -- 172.21.15.69:0/3982149410 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f66a00e2130 con 0x7f66a4102430 2024-05-14T16:49:49.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.334+0000 7f66a9412700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f66900b28d0 0x7f66900b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:49:49.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.338+0000 7f669affd700 1 -- 172.21.15.69:0/3982149410 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66a00a9e10 con 0x7f66a4102430 2024-05-14T16:49:49.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:49.153+0000 7fafe7b36200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:49:49.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:49.326+0000 7fafe7b36200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:49:49.471 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:49.133+0000 7f6a9f3a2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:49:49.471 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:49.302+0000 7f6a9f3a2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:49:49.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.535+0000 7f66a9412700 1 -- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f66900b28d0 msgr2=0x7f66900b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:49.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.535+0000 7f66a9412700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f66900b28d0 0x7f66900b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:49:49.535 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:49.239+0000 7f1357aa0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:49:49.535 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:49.440+0000 7f1357aa0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:49:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.616+0000 7f66abe77700 1 -- 172.21.15.69:0/3982149410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f66a419af60 con 0x7f66a4102430 2024-05-14T16:49:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.617+0000 7f669affd700 1 -- 172.21.15.69:0/3982149410 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f66a00a9710 con 0x7f66a4102430 2024-05-14T16:49:49.617 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T16:49:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 -- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f66900b28d0 msgr2=0x7f66900b4d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:49:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f66900b28d0 0x7f66900b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 -- 172.21.15.69:0/3982149410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4102430 msgr2=0x7f66a419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4102430 0x7f66a419f870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66a000d950 tx=0x7f66a000dc60 comp rx=0 tx=0).stop 2024-05-14T16:49:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 -- 172.21.15.69:0/3982149410 shutdown_connections 2024-05-14T16:49:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a4108510 0x7f66a4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4102430 0x7f66a419f870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f66900b28d0 0x7f66900b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 --2- 172.21.15.69:0/3982149410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a4102df0 0x7f66a419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:49.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 -- 172.21.15.69:0/3982149410 >> 172.21.15.69:0/3982149410 conn(0x7f66a40793b0 msgr2=0x7f66a406be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:49.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 -- 172.21.15.69:0/3982149410 shutdown_connections 2024-05-14T16:49:49.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:49.619+0000 7f66abe77700 1 -- 172.21.15.69:0/3982149410 wait complete. 2024-05-14T16:49:49.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:49 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3982149410' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T16:49:49.750 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:49.472+0000 7f6a9f3a2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:49:49.792 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:49 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3982149410' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T16:49:49.792 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:49.497+0000 7fafe7b36200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:49:49.922 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:49 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3982149410' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T16:49:49.922 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:49.617+0000 7f1357aa0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:49:50.058 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:49.766+0000 7f6a9f3a2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:49:50.059 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:49.857+0000 7f6a9f3a2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:49:50.092 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:49.792+0000 7fafe7b36200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:49:50.092 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:49.887+0000 7fafe7b36200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:49:50.203 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:49.922+0000 7f1357aa0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:49:50.203 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:50 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:50.018+0000 7f1357aa0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:49:50.323 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T16:49:50.323 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T16:49:50.323 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-05-14T16:49:50.323 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T16:49:50.323 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-14T16:49:50.323 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T16:49:50.323 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T16:49:50.323 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T16:49:50.323 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-14T16:49:50.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:49:50.324 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-05-14T16:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T16:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T16:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T16:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-05-14T16:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-05-14T16:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-05-14T16:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T16:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-05-14T16:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-05-14T16:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T16:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-05-14T16:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-05-14T16:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T16:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-05-14T16:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T16:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-05-14T16:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-14T16:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42013sec preferred_lft 42013sec 2024-05-14T16:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-05-14T16:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-05-14T16:49:50.342 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-14T16:49:50.342 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T16:49:50.342 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-05-14T16:49:50.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-05-14T16:49:50.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-05-14T16:49:50.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-05-14T16:49:50.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-14T16:49:50.342 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-05-14T16:49:50.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:49:50.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:50.092+0000 7fafe7b36200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:49:50.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:50.289+0000 7fafe7b36200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:49:50.498 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:50 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:50.227+0000 7f1357aa0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:49:50.498 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:50 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:50.430+0000 7f1357aa0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:49:50.500 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:50 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:50.059+0000 7f6a9f3a2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:49:50.500 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:50 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:50.251+0000 7f6a9f3a2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:49:50.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:49:50.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:49:50.553 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T16:49:50.579 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-05-14T16:49:50.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:49:50.581 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-05-14T16:49:50.661 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-05-14T16:49:50.662 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T16:49:50.662 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T16:49:50.662 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T16:49:50.666 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T16:49:50.667 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T16:49:50.667 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-05-14T16:49:50.667 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T16:49:50.667 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T16:49:50.667 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T16:49:50.667 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-05-14T16:49:50.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:49:50.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:49:50.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:49:50.984 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-05-14T16:49:51.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:49:51.000 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-05-14T16:49:51.055 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T16:49:51.055 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-05-14T16:49:51.055 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-14T16:49:51.055 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-14T16:49:51.055 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-05-14T16:49:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:49:51.153 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:51.079+0000 7f1357aa0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:49:51.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:49:51.179 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-14T16:49:51.180 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T16:49:51.180 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-05-14T16:49:51.180 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-14T16:49:51.180 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-14T16:49:51.180 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-14T16:49:51.180 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-14T16:49:51.180 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-14T16:49:51.180 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-05-14T16:49:51.209 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:50.895+0000 7fafe7b36200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:49:51.209 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:51 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:51.118+0000 7fafe7b36200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:49:51.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:49:51.250 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:50 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:50.852+0000 7f6a9f3a2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:49:51.250 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:51 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:51.074+0000 7f6a9f3a2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:49:51.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:49:51.311 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T16:49:51.311 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-05-14T16:49:51.311 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-05-14T16:49:51.311 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-05-14T16:49:51.311 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-05-14T16:49:51.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:49:51.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:49:51.390 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T16:49:51.390 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /tmp/cephfs 2024-05-14T16:49:51.404 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-14T16:49:51.405 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T16:49:51.405 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /tmp/cephfs 2024-05-14T16:49:51.439 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:51.329+0000 7f1357aa0200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:49:51.454 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-14T16:49:51.455 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-14T16:49:51.455 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/265d528f-6ed7-4d79-b5c4-99c32193f887 2024-05-14T16:49:51.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:49:51.455 DEBUG:teuthology.orchestra.run.smithi069:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/265d528f-6ed7-4d79-b5c4-99c32193f887 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-05-14T16:49:51.613 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-05-14T16:49:51.613 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-05-14T16:49:51.730 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-05-14T16:49:51.731 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-05-14T16:49:51.731 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: resolved to: "172.21.15.69:3300,172.21.15.92:3300,172.21.15.203:3300" 2024-05-14T16:49:51.731 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: trying mount with new device syntax: 0@356e185e-1210-11ef-bc98-c7b262605968.cephfs=/volumes/qa/sv_1/265d528f-6ed7-4d79-b5c4-99c32193f887 2024-05-14T16:49:51.731 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.69:3300/172.21.15.92:3300/172.21.15.203:3300" will pass to kernel 2024-05-14T16:49:51.734 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-14T16:49:51.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:49:51.734 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /tmp/cephfs 2024-05-14T16:49:51.750 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:51 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:51.495+0000 7f6a9f3a2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:49:51.751 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:51 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:51.592+0000 7f6a9f3a2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:49:51.765 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T16:49:51.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T16:49:51.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:49:51.866 INFO:teuthology.orchestra.run.smithi069.stdout:client15165 2024-05-14T16:49:51.866 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T16:49:51.866 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T16:49:51.883 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T16:49:51.959 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:51 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:51.546+0000 7fafe7b36200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:49:51.959 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:51 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:51.644+0000 7fafe7b36200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:49:51.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T16:49:51.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:49:51.981 INFO:teuthology.orchestra.run.smithi069.stdout:client15165 2024-05-14T16:49:51.982 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T16:49:52.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T16:49:52.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:49:52.043 INFO:teuthology.orchestra.run.smithi069.stdout:client15165 2024-05-14T16:49:52.044 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/356e185e-1210-11ef-bc98-c7b262605968.client15165/status') 2024-05-14T16:49:52.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:49:52.103 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:51.798+0000 7f1357aa0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:49:52.103 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:51.908+0000 7f1357aa0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:49:52.106 INFO:teuthology.orchestra.run.smithi069.stdout:instance: client.15165 (3)192.168.0.1:0/2698519602 2024-05-14T16:49:52.106 INFO:teuthology.orchestra.run.smithi069.stdout:blocklisted: false 2024-05-14T16:49:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records in 2024-05-14T16:49:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records out 2024-05-14T16:49:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:70 bytes copied, 2.3463e-05 s, 3.0 MB/s 2024-05-14T16:49:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:49:52.114 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T16:49:52.123 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T16:49:52.123 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T16:49:52.123 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-14T16:49:52.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.229+0000 7f484f809640 1 -- 172.21.15.69:0/1486166555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48481035d0 msgr2=0x7f48481059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:52.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.229+0000 7f484f809640 1 --2- 172.21.15.69:0/1486166555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48481035d0 0x7f48481059c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4844009900 tx=0x7f484402a410 comp rx=0 tx=0).stop 2024-05-14T16:49:52.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.229+0000 7f484f809640 1 -- 172.21.15.69:0/1486166555 shutdown_connections 2024-05-14T16:49:52.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.229+0000 7f484f809640 1 --2- 172.21.15.69:0/1486166555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4848108830 0x7f484810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.229+0000 7f484f809640 1 --2- 172.21.15.69:0/1486166555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4848105f00 0x7f48481082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.229+0000 7f484f809640 1 --2- 172.21.15.69:0/1486166555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48481035d0 0x7f48481059c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.229+0000 7f484f809640 1 -- 172.21.15.69:0/1486166555 >> 172.21.15.69:0/1486166555 conn(0x7f48480fcf20 msgr2=0x7f48480ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:52.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.229+0000 7f484f809640 1 -- 172.21.15.69:0/1486166555 shutdown_connections 2024-05-14T16:49:52.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.229+0000 7f484f809640 1 -- 172.21.15.69:0/1486166555 wait complete. 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484f809640 1 Processor -- start 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484f809640 1 -- start start 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484f809640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48481035d0 0x7f4848198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484f809640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4848105f00 0x7f48481991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484f809640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4848108830 0x7f48481a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484f809640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f484810e5f0 con 0x7f4848105f00 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484f809640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f484810e470 con 0x7f48481035d0 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484f809640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f484810e770 con 0x7f4848108830 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484cda4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4848105f00 0x7f48481991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484d5a5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48481035d0 0x7f4848198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484cda4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4848105f00 0x7f48481991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34542/0 (socket says 172.21.15.69:34542) 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484cda4640 1 -- 172.21.15.69:0/198912049 learned_addr learned my addr 172.21.15.69:0/198912049 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.230+0000 7f484dda6640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4848108830 0x7f48481a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f484cda4640 1 -- 172.21.15.69:0/198912049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4848108830 msgr2=0x7f48481a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f484cda4640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4848108830 0x7f48481a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f484cda4640 1 -- 172.21.15.69:0/198912049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48481035d0 msgr2=0x7f4848198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f484cda4640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48481035d0 0x7f4848198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f484cda4640 1 -- 172.21.15.69:0/198912049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48481a08d0 con 0x7f4848105f00 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f484d5a5640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48481035d0 0x7f4848198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f484cda4640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4848105f00 0x7f48481991d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f483800e9c0 tx=0x7f483800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f483e7fc640 1 -- 172.21.15.69:0/198912049 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483800cd80 con 0x7f4848105f00 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f484f809640 1 -- 172.21.15.69:0/198912049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48481a0bc0 con 0x7f4848105f00 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f483e7fc640 1 -- 172.21.15.69:0/198912049 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48380054d0 con 0x7f4848105f00 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f483e7fc640 1 -- 172.21.15.69:0/198912049 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838010430 con 0x7f4848105f00 2024-05-14T16:49:52.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.231+0000 7f484f809640 1 -- 172.21.15.69:0/198912049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48481a1160 con 0x7f4848105f00 2024-05-14T16:49:52.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.232+0000 7f484f809640 1 -- 172.21.15.69:0/198912049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48480fea10 con 0x7f4848105f00 2024-05-14T16:49:52.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.234+0000 7f483e7fc640 1 -- 172.21.15.69:0/198912049 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f4838004fd0 con 0x7f4848105f00 2024-05-14T16:49:52.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.234+0000 7f483e7fc640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48340b28f0 0x7f48340b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:52.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.234+0000 7f484d5a5640 1 -- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48340b28f0 msgr2=0x7f48340b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:52.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.234+0000 7f483e7fc640 1 -- 172.21.15.69:0/198912049 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f48380e26d0 con 0x7f4848105f00 2024-05-14T16:49:52.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.234+0000 7f484d5a5640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48340b28f0 0x7f48340b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:49:52.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.239+0000 7f483e7fc640 1 -- 172.21.15.69:0/198912049 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48380aa3f0 con 0x7f4848105f00 2024-05-14T16:49:52.250 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:51 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:51.965+0000 7f6a9f3a2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:49:52.379 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:52.018+0000 7fafe7b36200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:49:52.434 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:52.302+0000 7f1357aa0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.435+0000 7f484d5a5640 1 -- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48340b28f0 msgr2=0x7f48340b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.435+0000 7f484d5a5640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48340b28f0 0x7f48340b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:49:52.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.472+0000 7f484f809640 1 -- 172.21.15.69:0/198912049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f4848063650 con 0x7f4848105f00 2024-05-14T16:49:52.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.523+0000 7f483e7fc640 1 -- 172.21.15.69:0/198912049 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f48380a9cf0 con 0x7f4848105f00 2024-05-14T16:49:52.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 -- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48340b28f0 msgr2=0x7f48340b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:49:52.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48340b28f0 0x7f48340b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 -- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4848105f00 msgr2=0x7f48481991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4848105f00 0x7f48481991d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f483800e9c0 tx=0x7f483800ee80 comp rx=0 tx=0).stop 2024-05-14T16:49:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 -- 172.21.15.69:0/198912049 shutdown_connections 2024-05-14T16:49:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f48340b28f0 0x7f48340b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4848108830 0x7f48481a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4848105f00 0x7f48481991d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 --2- 172.21.15.69:0/198912049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48481035d0 0x7f4848198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 -- 172.21.15.69:0/198912049 >> 172.21.15.69:0/198912049 conn(0x7f48480fcf20 msgr2=0x7f4848103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 -- 172.21.15.69:0/198912049 shutdown_connections 2024-05-14T16:49:52.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.526+0000 7f484f809640 1 -- 172.21.15.69:0/198912049 wait complete. 2024-05-14T16:49:52.538 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-14T16:49:52.555 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T16:49:52.568 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T16:49:52.568 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T16:49:52.569 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-14T16:49:52.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:52 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/198912049' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T16:49:52.656 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:52 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:52.319+0000 7f6a9f3a2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:49:52.657 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:52 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:52.411+0000 7f6a9f3a2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:49:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.674+0000 7fb337b89640 1 -- 172.21.15.69:0/1391420857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3301019b0 msgr2=0x7fb330101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.674+0000 7fb337b89640 1 --2- 172.21.15.69:0/1391420857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3301019b0 0x7fb330101dd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009900 tx=0x7fb32c02a410 comp rx=0 tx=0).stop 2024-05-14T16:49:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.674+0000 7fb337b89640 1 -- 172.21.15.69:0/1391420857 shutdown_connections 2024-05-14T16:49:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.674+0000 7fb337b89640 1 --2- 172.21.15.69:0/1391420857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330103560 0x7fb330109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.674+0000 7fb337b89640 1 --2- 172.21.15.69:0/1391420857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb330102ba0 0x7fb330103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.674+0000 7fb337b89640 1 --2- 172.21.15.69:0/1391420857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3301019b0 0x7fb330101dd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.674+0000 7fb337b89640 1 -- 172.21.15.69:0/1391420857 >> 172.21.15.69:0/1391420857 conn(0x7fb3300fcf20 msgr2=0x7fb3300ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.674+0000 7fb337b89640 1 -- 172.21.15.69:0/1391420857 shutdown_connections 2024-05-14T16:49:52.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.674+0000 7fb337b89640 1 -- 172.21.15.69:0/1391420857 wait complete. 2024-05-14T16:49:52.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb337b89640 1 Processor -- start 2024-05-14T16:49:52.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb337b89640 1 -- start start 2024-05-14T16:49:52.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb337b89640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3301019b0 0x7fb330197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:52.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb337b89640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb330102ba0 0x7fb330198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:52.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb337b89640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330103560 0x7fb33019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:52.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb337b89640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb33010d4b0 con 0x7fb330102ba0 2024-05-14T16:49:52.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb337b89640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb33010d330 con 0x7fb330103560 2024-05-14T16:49:52.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb337b89640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb33010d630 con 0x7fb3301019b0 2024-05-14T16:49:52.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb335124640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb330102ba0 0x7fb330198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:52.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb335124640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb330102ba0 0x7fb330198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34558/0 (socket says 172.21.15.69:34558) 2024-05-14T16:49:52.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb335124640 1 -- 172.21.15.69:0/872567782 learned_addr learned my addr 172.21.15.69:0/872567782 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:52.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb335925640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3301019b0 0x7fb330197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:52.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb336126640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330103560 0x7fb33019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:52.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb335124640 1 -- 172.21.15.69:0/872567782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3301019b0 msgr2=0x7fb330197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:52.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb335124640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3301019b0 0x7fb330197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb335124640 1 -- 172.21.15.69:0/872567782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330103560 msgr2=0x7fb33019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:52.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb335124640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330103560 0x7fb33019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb335124640 1 -- 172.21.15.69:0/872567782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33019f770 con 0x7fb330102ba0 2024-05-14T16:49:52.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.675+0000 7fb335124640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb330102ba0 0x7fb330198090 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb32000d950 tx=0x7fb32000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:52.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.676+0000 7fb326ffd640 1 -- 172.21.15.69:0/872567782 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb320017030 con 0x7fb330102ba0 2024-05-14T16:49:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.676+0000 7fb326ffd640 1 -- 172.21.15.69:0/872567782 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb320004020 con 0x7fb330102ba0 2024-05-14T16:49:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.676+0000 7fb337b89640 1 -- 172.21.15.69:0/872567782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33019fa60 con 0x7fb330102ba0 2024-05-14T16:49:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.676+0000 7fb326ffd640 1 -- 172.21.15.69:0/872567782 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3200047a0 con 0x7fb330102ba0 2024-05-14T16:49:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.676+0000 7fb337b89640 1 -- 172.21.15.69:0/872567782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33019ffb0 con 0x7fb330102ba0 2024-05-14T16:49:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.677+0000 7fb337b89640 1 -- 172.21.15.69:0/872567782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb330108dc0 con 0x7fb330102ba0 2024-05-14T16:49:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.678+0000 7fb326ffd640 1 -- 172.21.15.69:0/872567782 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fb320009a30 con 0x7fb330102ba0 2024-05-14T16:49:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.678+0000 7fb326ffd640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb31c0b28f0 0x7fb31c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:52.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.679+0000 7fb335925640 1 -- 172.21.15.69:0/872567782 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb31c0b28f0 msgr2=0x7fb31c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:52.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.679+0000 7fb326ffd640 1 -- 172.21.15.69:0/872567782 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb3200e2340 con 0x7fb330102ba0 2024-05-14T16:49:52.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.679+0000 7fb335925640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb31c0b28f0 0x7fb31c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:49:52.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.682+0000 7fb326ffd640 1 -- 172.21.15.69:0/872567782 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3200aa060 con 0x7fb330102ba0 2024-05-14T16:49:52.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:52 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/198912049' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T16:49:52.711 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:52 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/198912049' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T16:49:52.711 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:52.379+0000 7fafe7b36200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:49:52.711 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:52.475+0000 7fafe7b36200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:49:52.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.871+0000 7fb337b89640 1 -- 172.21.15.69:0/872567782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fb3300636c0 con 0x7fb330102ba0 2024-05-14T16:49:52.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.877+0000 7fb326ffd640 1 -- 172.21.15.69:0/872567782 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v27)=0 v27) ==== 129+0+0 (secure 0 0 0) 0x7fb3200a9960 con 0x7fb330102ba0 2024-05-14T16:49:52.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.879+0000 7fb337b89640 1 -- 172.21.15.69:0/872567782 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb31c0b28f0 msgr2=0x7fb31c0b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:49:52.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.879+0000 7fb337b89640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb31c0b28f0 0x7fb31c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.879+0000 7fb337b89640 1 -- 172.21.15.69:0/872567782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb330102ba0 msgr2=0x7fb330198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:52.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.879+0000 7fb337b89640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb330102ba0 0x7fb330198090 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb32000d950 tx=0x7fb32000de10 comp rx=0 tx=0).stop 2024-05-14T16:49:52.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.880+0000 7fb337b89640 1 -- 172.21.15.69:0/872567782 shutdown_connections 2024-05-14T16:49:52.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.880+0000 7fb337b89640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fb31c0b28f0 0x7fb31c0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.880+0000 7fb337b89640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330103560 0x7fb33019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.880+0000 7fb337b89640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb330102ba0 0x7fb330198090 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.880+0000 7fb337b89640 1 --2- 172.21.15.69:0/872567782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3301019b0 0x7fb330197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:52.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.880+0000 7fb337b89640 1 -- 172.21.15.69:0/872567782 >> 172.21.15.69:0/872567782 conn(0x7fb3300fcf20 msgr2=0x7fb330105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:52.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.880+0000 7fb337b89640 1 -- 172.21.15.69:0/872567782 shutdown_connections 2024-05-14T16:49:52.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:52.880+0000 7fb337b89640 1 -- 172.21.15.69:0/872567782 wait complete. 2024-05-14T16:49:52.891 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-14T16:49:52.954 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:52.685+0000 7f1357aa0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:49:52.954 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:52.784+0000 7f1357aa0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:49:53.000 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:52 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:52.656+0000 7f6a9f3a2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:49:53.000 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:52 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:52.790+0000 7f6a9f3a2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:49:53.000 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:52 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:52.885+0000 7f6a9f3a2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:49:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.001+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/817558048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c8105e60 msgr2=0x7fd2c8108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.001+0000 7fd2cd2dc640 1 --2- 172.21.15.69:0/817558048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c8105e60 0x7fd2c8108250 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009900 tx=0x7fd2bc02a410 comp rx=0 tx=0).stop 2024-05-14T16:49:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.001+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/817558048 shutdown_connections 2024-05-14T16:49:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.001+0000 7fd2cd2dc640 1 --2- 172.21.15.69:0/817558048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8108790 0x7fd2c810abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.001+0000 7fd2cd2dc640 1 --2- 172.21.15.69:0/817558048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c8105e60 0x7fd2c8108250 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.001+0000 7fd2cd2dc640 1 --2- 172.21.15.69:0/817558048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c8103530 0x7fd2c8105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.001+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/817558048 >> 172.21.15.69:0/817558048 conn(0x7fd2c80fcf20 msgr2=0x7fd2c80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.001+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/817558048 shutdown_connections 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.001+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/817558048 wait complete. 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2cd2dc640 1 Processor -- start 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2cd2dc640 1 -- start start 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2cd2dc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c8103530 0x7fd2c8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2cd2dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c8105e60 0x7fd2c81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2cd2dc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8108790 0x7fd2c81a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2cd2dc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c810e5f0 con 0x7fd2c8105e60 2024-05-14T16:49:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2cd2dc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c810e470 con 0x7fd2c8108790 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2cd2dc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c810e770 con 0x7fd2c8103530 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2c67fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c8105e60 0x7fd2c81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2c67fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c8105e60 0x7fd2c81991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34582/0 (socket says 172.21.15.69:34582) 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2c67fc640 1 -- 172.21.15.69:0/1061191573 learned_addr learned my addr 172.21.15.69:0/1061191573 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2c6ffd640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c8103530 0x7fd2c8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.002+0000 7fd2c77fe640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8108790 0x7fd2c81a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2c67fc640 1 -- 172.21.15.69:0/1061191573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c8103530 msgr2=0x7fd2c8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2c67fc640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c8103530 0x7fd2c8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2c67fc640 1 -- 172.21.15.69:0/1061191573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8108790 msgr2=0x7fd2c81a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2c67fc640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8108790 0x7fd2c81a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2c67fc640 1 -- 172.21.15.69:0/1061191573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c81a08b0 con 0x7fd2c8105e60 2024-05-14T16:49:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2c6ffd640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c8103530 0x7fd2c8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:49:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2c67fc640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c8105e60 0x7fd2c81991b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc006010 tx=0x7fd2bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2affff640 1 -- 172.21.15.69:0/1061191573 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc03f030 con 0x7fd2c8105e60 2024-05-14T16:49:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2affff640 1 -- 172.21.15.69:0/1061191573 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2bc031910 con 0x7fd2c8105e60 2024-05-14T16:49:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/1061191573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c81a0b40 con 0x7fd2c8105e60 2024-05-14T16:49:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2affff640 1 -- 172.21.15.69:0/1061191573 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc035070 con 0x7fd2c8105e60 2024-05-14T16:49:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.003+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/1061191573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c81a1030 con 0x7fd2c8105e60 2024-05-14T16:49:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.004+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/1061191573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c80fea10 con 0x7fd2c8105e60 2024-05-14T16:49:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.005+0000 7fd2affff640 1 -- 172.21.15.69:0/1061191573 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fd2bc042030 con 0x7fd2c8105e60 2024-05-14T16:49:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.006+0000 7fd2affff640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd2b00b28f0 0x7fd2b00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.007+0000 7fd2c6ffd640 1 -- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd2b00b28f0 msgr2=0x7fd2b00b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.007+0000 7fd2affff640 1 -- 172.21.15.69:0/1061191573 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd2bc102760 con 0x7fd2c8105e60 2024-05-14T16:49:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.007+0000 7fd2c6ffd640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd2b00b28f0 0x7fd2b00b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:49:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.011+0000 7fd2affff640 1 -- 172.21.15.69:0/1061191573 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2bc0c9e30 con 0x7fd2c8105e60 2024-05-14T16:49:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.200+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/1061191573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fd2c8063650 con 0x7fd2c8105e60 2024-05-14T16:49:53.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.205+0000 7fd2affff640 1 -- 172.21.15.69:0/1061191573 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fd2bc0ce130 con 0x7fd2c8105e60 2024-05-14T16:49:53.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2c6ffd640 1 -- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd2b00b28f0 msgr2=0x7fd2b00b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:53.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2c6ffd640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd2b00b28f0 0x7fd2b00b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:49:53.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd2b00b28f0 msgr2=0x7fd2b00b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:49:53.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd2b00b28f0 0x7fd2b00b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c8105e60 msgr2=0x7fd2c81991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c8105e60 0x7fd2c81991b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc006010 tx=0x7fd2bc0042f0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/1061191573 shutdown_connections 2024-05-14T16:49:53.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd2b00b28f0 0x7fd2b00b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c8108790 0x7fd2c81a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c8105e60 0x7fd2c81991b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 --2- 172.21.15.69:0/1061191573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c8103530 0x7fd2c8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/1061191573 >> 172.21.15.69:0/1061191573 conn(0x7fd2c80fcf20 msgr2=0x7fd2c8103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:53.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/1061191573 shutdown_connections 2024-05-14T16:49:53.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.208+0000 7fd2cd2dc640 1 -- 172.21.15.69:0/1061191573 wait complete. 2024-05-14T16:49:53.209 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:52.724+0000 7fafe7b36200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:49:53.209 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:52.858+0000 7fafe7b36200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:49:53.209 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:52.957+0000 7fafe7b36200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:49:53.219 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-14T16:49:53.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.323+0000 7f57b62e8640 1 -- 172.21.15.69:0/1851689100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57b0102be0 msgr2=0x7f57b0103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.323+0000 7f57b62e8640 1 --2- 172.21.15.69:0/1851689100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57b0102be0 0x7f57b0103060 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5798009900 tx=0x7f579802a410 comp rx=0 tx=0).stop 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.323+0000 7f57b62e8640 1 -- 172.21.15.69:0/1851689100 shutdown_connections 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.323+0000 7f57b62e8640 1 --2- 172.21.15.69:0/1851689100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b01035a0 0x7f57b0109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.323+0000 7f57b62e8640 1 --2- 172.21.15.69:0/1851689100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57b0102be0 0x7f57b0103060 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.323+0000 7f57b62e8640 1 --2- 172.21.15.69:0/1851689100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b01019f0 0x7f57b0101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.323+0000 7f57b62e8640 1 -- 172.21.15.69:0/1851689100 >> 172.21.15.69:0/1851689100 conn(0x7f57b00fcf60 msgr2=0x7f57b00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.323+0000 7f57b62e8640 1 -- 172.21.15.69:0/1851689100 shutdown_connections 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.323+0000 7f57b62e8640 1 -- 172.21.15.69:0/1851689100 wait complete. 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b62e8640 1 Processor -- start 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b62e8640 1 -- start start 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b62e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b01019f0 0x7f57b006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b62e8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b0102be0 0x7f57b019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b62e8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57b01035a0 0x7f57b019ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b62e8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f57b010d520 con 0x7f57b01035a0 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b62e8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f57b010d3a0 con 0x7f57b0102be0 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b62e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f57b010d6a0 con 0x7f57b01019f0 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b4885640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57b01035a0 0x7f57b019ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b0102be0 0x7f57b019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b4885640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57b01035a0 0x7f57b019ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34600/0 (socket says 172.21.15.69:34600) 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.324+0000 7f57b4885640 1 -- 172.21.15.69:0/3075318123 learned_addr learned my addr 172.21.15.69:0/3075318123 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57affff640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b01019f0 0x7f57b006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57b4885640 1 -- 172.21.15.69:0/3075318123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b01019f0 msgr2=0x7f57b006b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57b4885640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b01019f0 0x7f57b006b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57b4885640 1 -- 172.21.15.69:0/3075318123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b0102be0 msgr2=0x7f57b019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57b4885640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b0102be0 0x7f57b019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57b4885640 1 -- 172.21.15.69:0/3075318123 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b01a0840 con 0x7f57b01035a0 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57b4885640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57b01035a0 0x7f57b019ff80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f57a800eb10 tx=0x7f57a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57adffb640 1 -- 172.21.15.69:0/3075318123 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8004b20 con 0x7f57b01035a0 2024-05-14T16:49:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57adffb640 1 -- 172.21.15.69:0/3075318123 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a8004cc0 con 0x7f57b01035a0 2024-05-14T16:49:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57b62e8640 1 -- 172.21.15.69:0/3075318123 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b01a0b30 con 0x7f57b01035a0 2024-05-14T16:49:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57adffb640 1 -- 172.21.15.69:0/3075318123 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8010670 con 0x7f57b01035a0 2024-05-14T16:49:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.325+0000 7f57b62e8640 1 -- 172.21.15.69:0/3075318123 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b01a1050 con 0x7f57b01035a0 2024-05-14T16:49:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.326+0000 7f57b62e8640 1 -- 172.21.15.69:0/3075318123 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b0108cb0 con 0x7f57b01035a0 2024-05-14T16:49:53.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.327+0000 7f57adffb640 1 -- 172.21.15.69:0/3075318123 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f57a8005020 con 0x7f57b01035a0 2024-05-14T16:49:53.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.328+0000 7f57adffb640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f579c0b28f0 0x7f579c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.328+0000 7f57affff640 1 -- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f579c0b28f0 msgr2=0x7f579c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:53.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.328+0000 7f57adffb640 1 -- 172.21.15.69:0/3075318123 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f57a80e2750 con 0x7f57b01035a0 2024-05-14T16:49:53.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.328+0000 7f57affff640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f579c0b28f0 0x7f579c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:49:53.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.332+0000 7f57adffb640 1 -- 172.21.15.69:0/3075318123 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f57a80aa470 con 0x7f57b01035a0 2024-05-14T16:49:53.454 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:53.049+0000 7f1357aa0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:49:53.454 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:53.192+0000 7f1357aa0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:49:53.454 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:53.298+0000 7f1357aa0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:49:53.500 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:53 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:53.210+0000 7f6a9f3a2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:49:53.500 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:53 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:53.301+0000 7f6a9f3a2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:49:53.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.521+0000 7f57b62e8640 1 -- 172.21.15.69:0/3075318123 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f57b002dba0 con 0x7f57b01035a0 2024-05-14T16:49:53.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.527+0000 7f57adffb640 1 -- 172.21.15.69:0/3075318123 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f57a80a9d70 con 0x7f57b01035a0 2024-05-14T16:49:53.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57affff640 1 -- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f579c0b28f0 msgr2=0x7f579c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:53.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57affff640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f579c0b28f0 0x7f579c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:49:53.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 -- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f579c0b28f0 msgr2=0x7f579c0b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:49:53.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f579c0b28f0 0x7f579c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 -- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57b01035a0 msgr2=0x7f57b019ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57b01035a0 0x7f57b019ff80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f57a800eb10 tx=0x7f57a800c2d0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 -- 172.21.15.69:0/3075318123 shutdown_connections 2024-05-14T16:49:53.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7f579c0b28f0 0x7f579c0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57b01035a0 0x7f57b019ff80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b0102be0 0x7f57b019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 --2- 172.21.15.69:0/3075318123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b01019f0 0x7f57b006b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 -- 172.21.15.69:0/3075318123 >> 172.21.15.69:0/3075318123 conn(0x7f57b00fcf60 msgr2=0x7f57b0105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:53.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.529+0000 7f57b62e8640 1 -- 172.21.15.69:0/3075318123 shutdown_connections 2024-05-14T16:49:53.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.530+0000 7f57b62e8640 1 -- 172.21.15.69:0/3075318123 wait complete. 2024-05-14T16:49:53.540 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-14T16:49:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.646+0000 7fd8b8823640 1 -- 172.21.15.69:0/4244934613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8b4103560 msgr2=0x7fd8b4109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.646+0000 7fd8b8823640 1 --2- 172.21.15.69:0/4244934613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8b4103560 0x7fd8b4109df0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009ca0 tx=0x7fd8ac02a8c0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.646+0000 7fd8b8823640 1 -- 172.21.15.69:0/4244934613 shutdown_connections 2024-05-14T16:49:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.646+0000 7fd8b8823640 1 --2- 172.21.15.69:0/4244934613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8b4103560 0x7fd8b4109df0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.646+0000 7fd8b8823640 1 --2- 172.21.15.69:0/4244934613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4102ba0 0x7fd8b4103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.646+0000 7fd8b8823640 1 --2- 172.21.15.69:0/4244934613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8b41019b0 0x7fd8b4101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.646+0000 7fd8b8823640 1 -- 172.21.15.69:0/4244934613 >> 172.21.15.69:0/4244934613 conn(0x7fd8b40fcf40 msgr2=0x7fd8b40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.646+0000 7fd8b8823640 1 -- 172.21.15.69:0/4244934613 shutdown_connections 2024-05-14T16:49:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.646+0000 7fd8b8823640 1 -- 172.21.15.69:0/4244934613 wait complete. 2024-05-14T16:49:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.647+0000 7fd8b8823640 1 Processor -- start 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.647+0000 7fd8b8823640 1 -- start start 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.647+0000 7fd8b8823640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8b41019b0 0x7fd8b419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.647+0000 7fd8b8823640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8b4102ba0 0x7fd8b419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.647+0000 7fd8b8823640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4103560 0x7fd8b41a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.647+0000 7fd8b8823640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b410d4c0 con 0x7fd8b4102ba0 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.647+0000 7fd8b8823640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b410d340 con 0x7fd8b41019b0 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.647+0000 7fd8b8823640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b410d640 con 0x7fd8b4103560 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.647+0000 7fd8abfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8b4102ba0 0x7fd8b419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8abfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8b4102ba0 0x7fd8b419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34620/0 (socket says 172.21.15.69:34620) 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8abfff640 1 -- 172.21.15.69:0/4145271354 learned_addr learned my addr 172.21.15.69:0/4145271354 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8b2d9d640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4103560 0x7fd8b41a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8b259c640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8b41019b0 0x7fd8b419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8abfff640 1 -- 172.21.15.69:0/4145271354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4103560 msgr2=0x7fd8b41a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8abfff640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4103560 0x7fd8b41a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8abfff640 1 -- 172.21.15.69:0/4145271354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8b41019b0 msgr2=0x7fd8b419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8abfff640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8b41019b0 0x7fd8b419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8abfff640 1 -- 172.21.15.69:0/4145271354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b41a29f0 con 0x7fd8b4102ba0 2024-05-14T16:49:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8abfff640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8b4102ba0 0x7fd8b419b310 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e9f0 tx=0x7fd89c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00cd80 con 0x7fd8b4102ba0 2024-05-14T16:49:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd89c0108e0 con 0x7fd8b4102ba0 2024-05-14T16:49:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8b8823640 1 -- 172.21.15.69:0/4145271354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b41a2ce0 con 0x7fd8b4102ba0 2024-05-14T16:49:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c018980 con 0x7fd8b4102ba0 2024-05-14T16:49:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.648+0000 7fd8b8823640 1 -- 172.21.15.69:0/4145271354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b41a31d0 con 0x7fd8b4102ba0 2024-05-14T16:49:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.649+0000 7fd8b8823640 1 -- 172.21.15.69:0/4145271354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b41a3400 con 0x7fd8b4102ba0 2024-05-14T16:49:53.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.650+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fd89c010430 con 0x7fd8b4102ba0 2024-05-14T16:49:53.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.651+0000 7fd8ab7fe640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8a00b28f0 0x7fd8a00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:53.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.651+0000 7fd8b259c640 1 -- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8a00b28f0 msgr2=0x7fd8a00b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:53.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.651+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd89c0e2590 con 0x7fd8b4102ba0 2024-05-14T16:49:53.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.651+0000 7fd8b259c640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8a00b28f0 0x7fd8a00b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:49:53.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.656+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd89c0a9c60 con 0x7fd8b4102ba0 2024-05-14T16:49:53.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:53 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/198912049' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T16:49:53.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:53 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:49:53.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:53 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/872567782' entity='client.admin' 2024-05-14T16:49:53.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:53 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1061191573' entity='client.admin' 2024-05-14T16:49:53.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:53 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3075318123' entity='client.admin' 2024-05-14T16:49:53.709 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:53 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:53.285+0000 7fafe7b36200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:49:53.710 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:53 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:53.379+0000 7fafe7b36200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:49:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.852+0000 7fd8b259c640 1 -- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8a00b28f0 msgr2=0x7fd8a00b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.852+0000 7fd8b259c640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8a00b28f0 0x7fd8a00b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:49:53.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:53.868+0000 7fd8b8823640 1 -- 172.21.15.69:0/4145271354 --> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd8b40ff060 con 0x7fd8a00b28f0 2024-05-14T16:49:53.904 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:53 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/198912049' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T16:49:53.904 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:53 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:49:53.904 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:53 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/872567782' entity='client.admin' 2024-05-14T16:49:53.904 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:53 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1061191573' entity='client.admin' 2024-05-14T16:49:53.904 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:53 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3075318123' entity='client.admin' 2024-05-14T16:49:53.953 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:53.646+0000 7f1357aa0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:49:53.953 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:53.748+0000 7f1357aa0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:49:53.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:53 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/198912049' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T16:49:53.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:53 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:49:53.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:53 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/872567782' entity='client.admin' 2024-05-14T16:49:53.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:53 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1061191573' entity='client.admin' 2024-05-14T16:49:53.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:53 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3075318123' entity='client.admin' 2024-05-14T16:49:54.250 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:53 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:53.904+0000 7f6a9f3a2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:49:54.250 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 16:49:54 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y[37130]: 2024-05-14T16:49:54.120+0000 7f6a9f3a2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:49:54.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:54.253+0000 7fd8b259c640 1 -- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8a00b28f0 msgr2=0x7fd8a00b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4012740750 2024-05-14T16:49:54.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:54.253+0000 7fd8b259c640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8a00b28f0 0x7fd8a00b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T16:49:54.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:53 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:53.992+0000 7fafe7b36200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:49:54.459 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 16:49:54 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z[37636]: 2024-05-14T16:49:54.209+0000 7fafe7b36200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:49:54.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:54.546+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fd89c0a5ef0 con 0x7fd8b4102ba0 2024-05-14T16:49:54.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:54.611+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7fd89c0e0110 con 0x7fd8b4102ba0 2024-05-14T16:49:54.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:54.612+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8a00b28f0 msgr2=0x7fd8a00b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:49:54.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:54.612+0000 7fd8ab7fe640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4012740750,v1:172.21.15.69:6801/4012740750] conn(0x7fd8a00b28f0 0x7fd8a00b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:54.703 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.366+0000 7f1357aa0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:49:54.703 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.597+0000 7f1357aa0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:49:54.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: Standby manager daemon y restarted 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: Standby manager daemon y started 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: Standby manager daemon z restarted 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: Standby manager daemon z started 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:49:54.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:54 smithi069 ceph-mon[31507]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:49:54.822 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: Standby manager daemon y restarted 2024-05-14T16:49:54.822 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: Standby manager daemon y started 2024-05-14T16:49:54.822 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:49:54.822 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:49:54.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:49:54.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:49:54.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: Standby manager daemon z restarted 2024-05-14T16:49:54.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: Standby manager daemon z started 2024-05-14T16:49:54.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:49:54.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:49:54.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:49:54.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:54 smithi092 ceph-mon[35763]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: Standby manager daemon y restarted 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: Standby manager daemon y started 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.92:0/616987801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: Standby manager daemon z restarted 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: Standby manager daemon z started 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:49:54.830 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:54 smithi203 ceph-mon[35787]: from='mgr.? 172.21.15.203:0/86887687' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:49:55.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.830+0000 7f12b82f8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:49:55.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.830+0000 7f12b82f8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:49:55.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.830+0000 7f12b82f8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:49:55.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.830+0000 7f12b82f8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:49:55.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.830+0000 7f12b82f8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:49:55.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.831+0000 7f12b4af1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:49:55.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.831+0000 7f12b4af1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:49:55.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.831+0000 7f12b4af1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:49:55.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.831+0000 7f12b4af1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:49:55.204 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:49:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:49:54.831+0000 7f12b4af1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:49:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.620+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 <== mon.0 v2:172.21.15.69:3300/0 9 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fd89c052180 con 0x7fd8b4102ba0 2024-05-14T16:49:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.621+0000 7fd8ab7fe640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8a003a520 0x7fd8a003a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.621+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 --> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd8a0043a60 con 0x7fd8a003a520 2024-05-14T16:49:55.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.623+0000 7fd8b259c640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8a003a520 0x7fd8a003a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:55.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.625+0000 7fd8b259c640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8a003a520 0x7fd8a003a900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4102a00 tx=0x7fd8a4005f90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.686+0000 7fd8ab7fe640 1 -- 172.21.15.69:0/4145271354 <== mgr.24764 v2:172.21.15.69:6800/4217409403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd8a0043a60 con 0x7fd8a003a520 2024-05-14T16:49:55.686 INFO:teuthology.orchestra.run.smithi069.stdout:Schedule set for path / 2024-05-14T16:49:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.688+0000 7fd8b8823640 1 -- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8a003a520 msgr2=0x7fd8a003a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.688+0000 7fd8b8823640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8a003a520 0x7fd8a003a900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4102a00 tx=0x7fd8a4005f90 comp rx=0 tx=0).stop 2024-05-14T16:49:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.688+0000 7fd8b8823640 1 -- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8b4102ba0 msgr2=0x7fd8b419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.688+0000 7fd8b8823640 1 --2- 172.21.15.69:0/4145271354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8b4102ba0 0x7fd8b419b310 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e9f0 tx=0x7fd89c00eeb0 comp rx=0 tx=0).stop 2024-05-14T16:49:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.688+0000 7fd8b2d9d640 1 -- 172.21.15.69:0/4145271354 reap_dead start 2024-05-14T16:49:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.688+0000 7fd8b8823640 1 -- 172.21.15.69:0/4145271354 shutdown_connections 2024-05-14T16:49:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.688+0000 7fd8b8823640 1 -- 172.21.15.69:0/4145271354 >> 172.21.15.69:0/4145271354 conn(0x7fd8b40fcf40 msgr2=0x7fd8b40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.688+0000 7fd8b8823640 1 -- 172.21.15.69:0/4145271354 shutdown_connections 2024-05-14T16:49:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.689+0000 7fd8b8823640 1 -- 172.21.15.69:0/4145271354 wait complete. 2024-05-14T16:49:55.699 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-14T16:49:55.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.804+0000 7f4fb9f67640 1 -- 172.21.15.69:0/3752535859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fb4103560 msgr2=0x7f4fb4109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:55.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.804+0000 7f4fb9f67640 1 --2- 172.21.15.69:0/3752535859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fb4103560 0x7f4fb4109df0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009ca0 tx=0x7f4fac02a8c0 comp rx=0 tx=0).stop 2024-05-14T16:49:55.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.804+0000 7f4fb9f67640 1 -- 172.21.15.69:0/3752535859 shutdown_connections 2024-05-14T16:49:55.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.804+0000 7f4fb9f67640 1 --2- 172.21.15.69:0/3752535859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fb4103560 0x7f4fb4109df0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:55.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.804+0000 7f4fb9f67640 1 --2- 172.21.15.69:0/3752535859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb4102ba0 0x7f4fb4103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:55.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.804+0000 7f4fb9f67640 1 --2- 172.21.15.69:0/3752535859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb41019b0 0x7f4fb4101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:55.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.804+0000 7f4fb9f67640 1 -- 172.21.15.69:0/3752535859 >> 172.21.15.69:0/3752535859 conn(0x7f4fb40fcf20 msgr2=0x7f4fb40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.804+0000 7f4fb9f67640 1 -- 172.21.15.69:0/3752535859 shutdown_connections 2024-05-14T16:49:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.804+0000 7f4fb9f67640 1 -- 172.21.15.69:0/3752535859 wait complete. 2024-05-14T16:49:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb9f67640 1 Processor -- start 2024-05-14T16:49:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb9f67640 1 -- start start 2024-05-14T16:49:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb9f67640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fb41019b0 0x7f4fb4198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb9f67640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb4102ba0 0x7f4fb41991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb9f67640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb4103560 0x7f4fb41a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb9f67640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb410e600 con 0x7f4fb41019b0 2024-05-14T16:49:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb9f67640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb410e480 con 0x7f4fb4103560 2024-05-14T16:49:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb9f67640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb410e780 con 0x7f4fb4102ba0 2024-05-14T16:49:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fb41019b0 0x7f4fb4198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fb41019b0 0x7f4fb4198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44752/0 (socket says 172.21.15.69:44752) 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb37fe640 1 -- 172.21.15.69:0/2742122159 learned_addr learned my addr 172.21.15.69:0/2742122159 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb2ffd640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb4102ba0 0x7f4fb41991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb3fff640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb4103560 0x7f4fb41a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb37fe640 1 -- 172.21.15.69:0/2742122159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb4102ba0 msgr2=0x7f4fb41991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb37fe640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb4102ba0 0x7f4fb41991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb37fe640 1 -- 172.21.15.69:0/2742122159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb4103560 msgr2=0x7f4fb41a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.805+0000 7f4fb37fe640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb4103560 0x7f4fb41a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.806+0000 7f4fb37fe640 1 -- 172.21.15.69:0/2742122159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb41a0860 con 0x7f4fb41019b0 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.806+0000 7f4fb37fe640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fb41019b0 0x7f4fb4198ca0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400e9c0 tx=0x7f4fa400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.806+0000 7f4fb0ff9640 1 -- 172.21.15.69:0/2742122159 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400cd80 con 0x7f4fb41019b0 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.806+0000 7f4fb9f67640 1 -- 172.21.15.69:0/2742122159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb41a0b50 con 0x7f4fb41019b0 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.806+0000 7f4fb0ff9640 1 -- 172.21.15.69:0/2742122159 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa40035f0 con 0x7f4fb41019b0 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.806+0000 7f4fb0ff9640 1 -- 172.21.15.69:0/2742122159 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4010690 con 0x7f4fb41019b0 2024-05-14T16:49:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.806+0000 7f4fb9f67640 1 -- 172.21.15.69:0/2742122159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb41a10f0 con 0x7f4fb41019b0 2024-05-14T16:49:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.807+0000 7f4fb9f67640 1 -- 172.21.15.69:0/2742122159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb4108d20 con 0x7f4fb41019b0 2024-05-14T16:49:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.808+0000 7f4fb0ff9640 1 -- 172.21.15.69:0/2742122159 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f4fa40037a0 con 0x7f4fb41019b0 2024-05-14T16:49:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.809+0000 7f4fb0ff9640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4f9c0b2690 0x7f4f9c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.809+0000 7f4fb2ffd640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4f9c0b2690 0x7f4f9c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:55.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.809+0000 7f4fb0ff9640 1 -- 172.21.15.69:0/2742122159 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f4fa40e2390 con 0x7f4fb41019b0 2024-05-14T16:49:55.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.809+0000 7f4fb2ffd640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4f9c0b2690 0x7f4f9c0b4b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009bd0 tx=0x7f4fa8005f90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: Active manager daemon x restarted 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: Activating manager daemon x 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: mgrmap e21: x(active, starting, since 0.0127798s), standbys: z, y 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:49:55.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:49:55.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: Manager daemon x is now available 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:49:55.815 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:49:55.816 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:55.816 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:55 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:55.813+0000 7f4fb0ff9640 1 -- 172.21.15.69:0/2742122159 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4fa40aa290 con 0x7f4fb41019b0 2024-05-14T16:49:55.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-14T16:49:55.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: Active manager daemon x restarted 2024-05-14T16:49:55.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: Activating manager daemon x 2024-05-14T16:49:55.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T16:49:55.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: mgrmap e21: x(active, starting, since 0.0127798s), standbys: z, y 2024-05-14T16:49:55.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:49:55.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:49:55.887 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:49:55.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:49:55.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:49:55.890 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: Manager daemon x is now available 2024-05-14T16:49:55.890 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:55.890 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:49:55.890 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:49:55.890 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:55.890 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:55 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:56.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: Active manager daemon x restarted 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: Activating manager daemon x 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: mgrmap e21: x(active, starting, since 0.0127798s), standbys: z, y 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:49:56.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:49:56.002 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: Manager daemon x is now available 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:56.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:55 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:56.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.032+0000 7f4fb9f67640 1 -- 172.21.15.69:0/2742122159 --> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f4fb4062f30 con 0x7f4f9c0b2690 2024-05-14T16:49:56.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.086+0000 7f4fb0ff9640 1 -- 172.21.15.69:0/2742122159 <== mgr.24764 v2:172.21.15.69:6800/4217409403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f4fb4062f30 con 0x7f4f9c0b2690 2024-05-14T16:49:56.086 INFO:teuthology.orchestra.run.smithi069.stdout:Retention added to path / 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 -- 172.21.15.69:0/2742122159 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4f9c0b2690 msgr2=0x7f4f9c0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4f9c0b2690 0x7f4f9c0b4b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009bd0 tx=0x7f4fa8005f90 comp rx=0 tx=0).stop 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 -- 172.21.15.69:0/2742122159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fb41019b0 msgr2=0x7f4fb4198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fb41019b0 0x7f4fb4198ca0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400e9c0 tx=0x7f4fa400ee80 comp rx=0 tx=0).stop 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 -- 172.21.15.69:0/2742122159 shutdown_connections 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4f9c0b2690 0x7f4f9c0b4b50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb4103560 0x7f4fb41a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb4102ba0 0x7f4fb41991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 --2- 172.21.15.69:0/2742122159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fb41019b0 0x7f4fb4198ca0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 -- 172.21.15.69:0/2742122159 >> 172.21.15.69:0/2742122159 conn(0x7f4fb40fcf20 msgr2=0x7f4fb4105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 -- 172.21.15.69:0/2742122159 shutdown_connections 2024-05-14T16:49:56.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.088+0000 7f4fb9f67640 1 -- 172.21.15.69:0/2742122159 wait complete. 2024-05-14T16:49:56.098 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-14T16:49:56.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.202+0000 7f90f109c640 1 -- 172.21.15.69:0/1272744686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90ec103530 msgr2=0x7f90ec105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.202+0000 7f90f109c640 1 --2- 172.21.15.69:0/1272744686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90ec103530 0x7f90ec105920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009ca0 tx=0x7f90dc02a8c0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.202+0000 7f90f109c640 1 -- 172.21.15.69:0/1272744686 shutdown_connections 2024-05-14T16:49:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.202+0000 7f90f109c640 1 --2- 172.21.15.69:0/1272744686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90ec108790 0x7f90ec10abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.202+0000 7f90f109c640 1 --2- 172.21.15.69:0/1272744686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90ec105e60 0x7f90ec108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.202+0000 7f90f109c640 1 --2- 172.21.15.69:0/1272744686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90ec103530 0x7f90ec105920 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.202+0000 7f90f109c640 1 -- 172.21.15.69:0/1272744686 >> 172.21.15.69:0/1272744686 conn(0x7f90ec0fcf20 msgr2=0x7f90ec0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.202+0000 7f90f109c640 1 -- 172.21.15.69:0/1272744686 shutdown_connections 2024-05-14T16:49:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.202+0000 7f90f109c640 1 -- 172.21.15.69:0/1272744686 wait complete. 2024-05-14T16:49:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.203+0000 7f90f109c640 1 Processor -- start 2024-05-14T16:49:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.203+0000 7f90f109c640 1 -- start start 2024-05-14T16:49:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.203+0000 7f90f109c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90ec103530 0x7f90ec102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90f109c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90ec105e60 0x7f90ec1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90f109c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90ec108790 0x7f90ec101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90f109c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec10e5e0 con 0x7f90ec108790 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90f109c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec10e460 con 0x7f90ec105e60 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90f109c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec10e760 con 0x7f90ec103530 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90eb59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90ec108790 0x7f90ec101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90eb59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90ec108790 0x7f90ec101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44778/0 (socket says 172.21.15.69:44778) 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90eb59e640 1 -- 172.21.15.69:0/1787774652 learned_addr learned my addr 172.21.15.69:0/1787774652 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90ead9d640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90ec103530 0x7f90ec102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90ea59c640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90ec105e60 0x7f90ec1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90eb59e640 1 -- 172.21.15.69:0/1787774652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90ec103530 msgr2=0x7f90ec102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90eb59e640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90ec103530 0x7f90ec102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90eb59e640 1 -- 172.21.15.69:0/1787774652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90ec105e60 msgr2=0x7f90ec1014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90eb59e640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90ec105e60 0x7f90ec1014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90eb59e640 1 -- 172.21.15.69:0/1787774652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec101f60 con 0x7f90ec108790 2024-05-14T16:49:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90eb59e640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90ec108790 0x7f90ec101a20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f90e400d950 tx=0x7f90e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90e3fff640 1 -- 172.21.15.69:0/1787774652 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4017030 con 0x7f90ec108790 2024-05-14T16:49:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90e3fff640 1 -- 172.21.15.69:0/1787774652 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e4004020 con 0x7f90ec108790 2024-05-14T16:49:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90e3fff640 1 -- 172.21.15.69:0/1787774652 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4003b50 con 0x7f90ec108790 2024-05-14T16:49:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90f109c640 1 -- 172.21.15.69:0/1787774652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ec102250 con 0x7f90ec108790 2024-05-14T16:49:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.204+0000 7f90f109c640 1 -- 172.21.15.69:0/1787774652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ec1a76b0 con 0x7f90ec108790 2024-05-14T16:49:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.205+0000 7f90f109c640 1 -- 172.21.15.69:0/1787774652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ec0fea10 con 0x7f90ec108790 2024-05-14T16:49:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.207+0000 7f90e3fff640 1 -- 172.21.15.69:0/1787774652 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f90e401a070 con 0x7f90ec108790 2024-05-14T16:49:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.208+0000 7f90e3fff640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f90d80b2690 0x7f90d80b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.208+0000 7f90ead9d640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f90d80b2690 0x7f90d80b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.208+0000 7f90e3fff640 1 -- 172.21.15.69:0/1787774652 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f90e40e1a30 con 0x7f90ec108790 2024-05-14T16:49:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.208+0000 7f90ead9d640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f90d80b2690 0x7f90d80b4b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f90dc005e30 tx=0x7f90dc005dc0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:56.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.211+0000 7f90e3fff640 1 -- 172.21.15.69:0/1787774652 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f90e4021070 con 0x7f90ec108790 2024-05-14T16:49:56.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.454+0000 7f90f109c640 1 -- 172.21.15.69:0/1787774652 --> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f90ec062e90 con 0x7f90d80b2690 2024-05-14T16:49:56.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.463+0000 7f90e3fff640 1 -- 172.21.15.69:0/1787774652 <== mgr.24764 v2:172.21.15.69:6800/4217409403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f90ec062e90 con 0x7f90d80b2690 2024-05-14T16:49:56.463 INFO:teuthology.orchestra.run.smithi069.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-14T00:00:00", "created": "2024-05-14T16:49:55", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-14T16:49:56.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.466+0000 7f90f109c640 1 -- 172.21.15.69:0/1787774652 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f90d80b2690 msgr2=0x7f90d80b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.466+0000 7f90f109c640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f90d80b2690 0x7f90d80b4b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f90dc005e30 tx=0x7f90dc005dc0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.466+0000 7f90f109c640 1 -- 172.21.15.69:0/1787774652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90ec108790 msgr2=0x7f90ec101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.466+0000 7f90f109c640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90ec108790 0x7f90ec101a20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f90e400d950 tx=0x7f90e400de10 comp rx=0 tx=0).stop 2024-05-14T16:49:56.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.466+0000 7f90f109c640 1 -- 172.21.15.69:0/1787774652 shutdown_connections 2024-05-14T16:49:56.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.466+0000 7f90f109c640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f90d80b2690 0x7f90d80b4b50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.466+0000 7f90f109c640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90ec108790 0x7f90ec101a20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.466+0000 7f90f109c640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90ec105e60 0x7f90ec1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.466+0000 7f90f109c640 1 --2- 172.21.15.69:0/1787774652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90ec103530 0x7f90ec102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.466+0000 7f90f109c640 1 -- 172.21.15.69:0/1787774652 >> 172.21.15.69:0/1787774652 conn(0x7f90ec0fcf20 msgr2=0x7f90ec103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:56.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.467+0000 7f90f109c640 1 -- 172.21.15.69:0/1787774652 shutdown_connections 2024-05-14T16:49:56.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.467+0000 7f90f109c640 1 -- 172.21.15.69:0/1787774652 wait complete. 2024-05-14T16:49:56.478 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-14T16:49:56.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 -- 172.21.15.69:0/3618698163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc0101a50 msgr2=0x7f8fc0101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 --2- 172.21.15.69:0/3618698163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc0101a50 0x7f8fc0101e70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009900 tx=0x7f8fb002a410 comp rx=0 tx=0).stop 2024-05-14T16:49:56.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 -- 172.21.15.69:0/3618698163 shutdown_connections 2024-05-14T16:49:56.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 --2- 172.21.15.69:0/3618698163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0103600 0x7f8fc0109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 --2- 172.21.15.69:0/3618698163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc0102c40 0x7f8fc01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 --2- 172.21.15.69:0/3618698163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc0101a50 0x7f8fc0101e70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 -- 172.21.15.69:0/3618698163 >> 172.21.15.69:0/3618698163 conn(0x7f8fc00fcfc0 msgr2=0x7f8fc00ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:56.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 -- 172.21.15.69:0/3618698163 shutdown_connections 2024-05-14T16:49:56.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 -- 172.21.15.69:0/3618698163 wait complete. 2024-05-14T16:49:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 Processor -- start 2024-05-14T16:49:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.588+0000 7f8fc83da640 1 -- start start 2024-05-14T16:49:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc83da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc0101a50 0x7f8fc01059e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc83da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc0102c40 0x7f8fc0104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc83da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0103600 0x7f8fc0104560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc83da640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc010d4e0 con 0x7f8fc0102c40 2024-05-14T16:49:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc83da640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc010d360 con 0x7f8fc0101a50 2024-05-14T16:49:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc83da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc010d660 con 0x7f8fc0103600 2024-05-14T16:49:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc5975640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc0102c40 0x7f8fc0104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc5975640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc0102c40 0x7f8fc0104020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44800/0 (socket says 172.21.15.69:44800) 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc5975640 1 -- 172.21.15.69:0/726908349 learned_addr learned my addr 172.21.15.69:0/726908349 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc6977640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0103600 0x7f8fc0104560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc6176640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc0101a50 0x7f8fc01059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc6977640 1 -- 172.21.15.69:0/726908349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc0101a50 msgr2=0x7f8fc01059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc6977640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc0101a50 0x7f8fc01059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc6977640 1 -- 172.21.15.69:0/726908349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc0102c40 msgr2=0x7f8fc0104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc6977640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc0102c40 0x7f8fc0104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc6977640 1 -- 172.21.15.69:0/726908349 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc0104df0 con 0x7f8fc0103600 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc6176640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc0101a50 0x7f8fc01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc5975640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc0102c40 0x7f8fc0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:49:56.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.589+0000 7f8fc6977640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0103600 0x7f8fc0104560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009fd0 tx=0x7f8fb800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:56.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.590+0000 7f8fb77fe640 1 -- 172.21.15.69:0/726908349 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800ce20 con 0x7f8fc0103600 2024-05-14T16:49:56.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.590+0000 7f8fc83da640 1 -- 172.21.15.69:0/726908349 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc01a2d60 con 0x7f8fc0103600 2024-05-14T16:49:56.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.590+0000 7f8fb77fe640 1 -- 172.21.15.69:0/726908349 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb8005020 con 0x7f8fc0103600 2024-05-14T16:49:56.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.590+0000 7f8fb77fe640 1 -- 172.21.15.69:0/726908349 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8010690 con 0x7f8fc0103600 2024-05-14T16:49:56.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.590+0000 7f8fc83da640 1 -- 172.21.15.69:0/726908349 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc01a31f0 con 0x7f8fc0103600 2024-05-14T16:49:56.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.591+0000 7f8fc83da640 1 -- 172.21.15.69:0/726908349 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc01a34e0 con 0x7f8fc0103600 2024-05-14T16:49:56.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.592+0000 7f8fb77fe640 1 -- 172.21.15.69:0/726908349 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f8fb80054d0 con 0x7f8fc0103600 2024-05-14T16:49:56.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.593+0000 7f8fb77fe640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8fac0b2690 0x7f8fac0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:56.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.593+0000 7f8fb77fe640 1 -- 172.21.15.69:0/726908349 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f8fb80e2640 con 0x7f8fc0103600 2024-05-14T16:49:56.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.593+0000 7f8fc6176640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8fac0b2690 0x7f8fac0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:56.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.593+0000 7f8fc6176640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8fac0b2690 0x7f8fac0b4b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009230 tx=0x7f8fb00314a0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:56.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.597+0000 7f8fb77fe640 1 -- 172.21.15.69:0/726908349 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8fb80aa540 con 0x7f8fc0103600 2024-05-14T16:49:56.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.627+0000 7f8fb77fe640 1 -- 172.21.15.69:0/726908349 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7f8fb80a60d0 con 0x7f8fc0103600 2024-05-14T16:49:56.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:56 smithi092 ceph-mon[35763]: [14/May/2024:16:49:55] ENGINE Bus STARTING 2024-05-14T16:49:56.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:56 smithi092 ceph-mon[35763]: [14/May/2024:16:49:55] ENGINE Serving on https://172.21.15.69:7150 2024-05-14T16:49:56.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:56 smithi092 ceph-mon[35763]: [14/May/2024:16:49:55] ENGINE Serving on http://172.21.15.69:8765 2024-05-14T16:49:56.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:56 smithi092 ceph-mon[35763]: [14/May/2024:16:49:55] ENGINE Bus STARTED 2024-05-14T16:49:56.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:56 smithi092 ceph-mon[35763]: mgrmap e22: x(active, since 1.0208s), standbys: z, y 2024-05-14T16:49:56.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:56 smithi092 ceph-mon[35763]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:56.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:56 smithi092 ceph-mon[35763]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:49:56.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:56 smithi092 ceph-mon[35763]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.844+0000 7f8fc83da640 1 -- 172.21.15.69:0/726908349 --> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8fc01a3770 con 0x7f8fac0b2690 2024-05-14T16:49:56.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.851+0000 7f8fb77fe640 1 -- 172.21.15.69:0/726908349 <== mgr.24764 v2:172.21.15.69:6800/4217409403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8fc01a3770 con 0x7f8fac0b2690 2024-05-14T16:49:56.851 INFO:teuthology.orchestra.run.smithi069.stdout:/ 1m 6m3h 2024-05-14T16:49:56.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 -- 172.21.15.69:0/726908349 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8fac0b2690 msgr2=0x7f8fac0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8fac0b2690 0x7f8fac0b4b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009230 tx=0x7f8fb00314a0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 -- 172.21.15.69:0/726908349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0103600 msgr2=0x7f8fc0104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:56.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0103600 0x7f8fc0104560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009fd0 tx=0x7f8fb800ef10 comp rx=0 tx=0).stop 2024-05-14T16:49:56.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 -- 172.21.15.69:0/726908349 shutdown_connections 2024-05-14T16:49:56.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8fac0b2690 0x7f8fac0b4b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0103600 0x7f8fc0104560 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc0102c40 0x7f8fc0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 --2- 172.21.15.69:0/726908349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc0101a50 0x7f8fc01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:56.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 -- 172.21.15.69:0/726908349 >> 172.21.15.69:0/726908349 conn(0x7f8fc00fcfc0 msgr2=0x7f8fc00fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:56.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 -- 172.21.15.69:0/726908349 shutdown_connections 2024-05-14T16:49:56.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:56.853+0000 7f8fc83da640 1 -- 172.21.15.69:0/726908349 wait complete. 2024-05-14T16:49:56.863 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:56 smithi069 ceph-mon[31507]: [14/May/2024:16:49:55] ENGINE Bus STARTING 2024-05-14T16:49:56.863 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:56 smithi069 ceph-mon[31507]: [14/May/2024:16:49:55] ENGINE Serving on https://172.21.15.69:7150 2024-05-14T16:49:56.863 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:56 smithi069 ceph-mon[31507]: [14/May/2024:16:49:55] ENGINE Serving on http://172.21.15.69:8765 2024-05-14T16:49:56.863 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:56 smithi069 ceph-mon[31507]: [14/May/2024:16:49:55] ENGINE Bus STARTED 2024-05-14T16:49:56.863 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:56 smithi069 ceph-mon[31507]: mgrmap e22: x(active, since 1.0208s), standbys: z, y 2024-05-14T16:49:56.863 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:56 smithi069 ceph-mon[31507]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:56.864 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:56 smithi069 ceph-mon[31507]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:49:56.864 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:56 smithi069 ceph-mon[31507]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:56.865 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-14T16:49:56.891 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-14T16:49:56.901 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-14T16:49:56.911 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T16:49:56.911 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-14T16:49:56.934 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-14T16:49:56.946 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T16:49:56.946 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:49:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:56 smithi203 ceph-mon[35787]: [14/May/2024:16:49:55] ENGINE Bus STARTING 2024-05-14T16:49:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:56 smithi203 ceph-mon[35787]: [14/May/2024:16:49:55] ENGINE Serving on https://172.21.15.69:7150 2024-05-14T16:49:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:56 smithi203 ceph-mon[35787]: [14/May/2024:16:49:55] ENGINE Serving on http://172.21.15.69:8765 2024-05-14T16:49:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:56 smithi203 ceph-mon[35787]: [14/May/2024:16:49:55] ENGINE Bus STARTED 2024-05-14T16:49:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:56 smithi203 ceph-mon[35787]: mgrmap e22: x(active, since 1.0208s), standbys: z, y 2024-05-14T16:49:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:56 smithi203 ceph-mon[35787]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:56 smithi203 ceph-mon[35787]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:49:56.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:56 smithi203 ceph-mon[35787]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:57.199 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:57.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:57 smithi069 ceph-mon[31507]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:57 smithi069 ceph-mon[31507]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:49:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:57 smithi069 ceph-mon[31507]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-14T16:49:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:57 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:57 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:57 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:57 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:57 smithi069 ceph-mon[31507]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:57 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:57 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:57 smithi203 ceph-mon[35787]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:57 smithi203 ceph-mon[35787]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:49:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:57 smithi203 ceph-mon[35787]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-14T16:49:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:57 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:57 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:57 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:57 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:57 smithi203 ceph-mon[35787]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:57.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:57 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:57.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:57 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:57 smithi092 ceph-mon[35763]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:57 smithi092 ceph-mon[35763]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:49:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:57 smithi092 ceph-mon[35763]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-14T16:49:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:57 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:57 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:57 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:57 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:57 smithi092 ceph-mon[35763]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:57 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:58.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:57 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:58.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.944+0000 7f44942e7700 1 -- 172.21.15.69:0/1724466893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f448c102430 msgr2=0x7f448c1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:58.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.944+0000 7f44942e7700 1 --2- 172.21.15.69:0/1724466893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f448c102430 0x7f448c1028b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f447c009990 tx=0x7f447c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:49:58.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.944+0000 7f44942e7700 1 -- 172.21.15.69:0/1724466893 shutdown_connections 2024-05-14T16:49:58.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.944+0000 7f44942e7700 1 --2- 172.21.15.69:0/1724466893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448c102df0 0x7f448c10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:58.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.944+0000 7f44942e7700 1 --2- 172.21.15.69:0/1724466893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f448c108510 0x7f448c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:58.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.944+0000 7f44942e7700 1 --2- 172.21.15.69:0/1724466893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f448c102430 0x7f448c1028b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.944+0000 7f44942e7700 1 -- 172.21.15.69:0/1724466893 >> 172.21.15.69:0/1724466893 conn(0x7f448c0793b0 msgr2=0x7f448c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.944+0000 7f44942e7700 1 -- 172.21.15.69:0/1724466893 shutdown_connections 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.944+0000 7f44942e7700 1 -- 172.21.15.69:0/1724466893 wait complete. 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.945+0000 7f44942e7700 1 Processor -- start 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.945+0000 7f44942e7700 1 -- start start 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f44942e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448c102430 0x7f448c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f44942e7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f448c102df0 0x7f448c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f44942e7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f448c108510 0x7f448c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f44942e7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1125e0 con 0x7f448c102df0 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f44942e7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f448c112460 con 0x7f448c102430 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f44942e7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f448c112760 con 0x7f448c108510 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4491882700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f448c102df0 0x7f448c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:58.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4491882700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f448c102df0 0x7f448c19fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44834/0 (socket says 172.21.15.69:44834) 2024-05-14T16:49:58.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4491882700 1 -- 172.21.15.69:0/2074153658 learned_addr learned my addr 172.21.15.69:0/2074153658 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:49:58.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4492083700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448c102430 0x7f448c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:58.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4492884700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f448c108510 0x7f448c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:58.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4492884700 1 -- 172.21.15.69:0/2074153658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448c102430 msgr2=0x7f448c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:58.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4492884700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448c102430 0x7f448c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:58.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4492884700 1 -- 172.21.15.69:0/2074153658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f448c102df0 msgr2=0x7f448c19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:58.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4492884700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f448c102df0 0x7f448c19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:58.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4492884700 1 -- 172.21.15.69:0/2074153658 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c19a170 con 0x7f448c108510 2024-05-14T16:49:58.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f4492884700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f448c108510 0x7f448c199a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f448400b330 tx=0x7f448400b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:58.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.946+0000 7f44837fe700 1 -- 172.21.15.69:0/2074153658 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448400bdd0 con 0x7f448c108510 2024-05-14T16:49:58.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.947+0000 7f44837fe700 1 -- 172.21.15.69:0/2074153658 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4484004bb0 con 0x7f448c108510 2024-05-14T16:49:58.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.947+0000 7f44942e7700 1 -- 172.21.15.69:0/2074153658 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448c19a400 con 0x7f448c108510 2024-05-14T16:49:58.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.947+0000 7f44837fe700 1 -- 172.21.15.69:0/2074153658 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484007dc0 con 0x7f448c108510 2024-05-14T16:49:58.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.947+0000 7f44942e7700 1 -- 172.21.15.69:0/2074153658 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448c103fa0 con 0x7f448c108510 2024-05-14T16:49:58.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.948+0000 7f44942e7700 1 -- 172.21.15.69:0/2074153658 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448c06a230 con 0x7f448c108510 2024-05-14T16:49:58.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.949+0000 7f44837fe700 1 -- 172.21.15.69:0/2074153658 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7f4484020020 con 0x7f448c108510 2024-05-14T16:49:58.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.950+0000 7f44837fe700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44780b2a70 0x7f44780b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:58.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.950+0000 7f4492083700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44780b2a70 0x7f44780b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:58.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.950+0000 7f44837fe700 1 -- 172.21.15.69:0/2074153658 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f44840e5cc0 con 0x7f448c108510 2024-05-14T16:49:58.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.950+0000 7f4492083700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44780b2a70 0x7f44780b4f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f448c19ab50 tx=0x7f4488009260 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:58.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:58.953+0000 7f44837fe700 1 -- 172.21.15.69:0/2074153658 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44840ad650 con 0x7f448c108510 2024-05-14T16:49:59.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.168+0000 7f44942e7700 1 -- 172.21.15.69:0/2074153658 --> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f448c06bab0 con 0x7f44780b2a70 2024-05-14T16:49:59.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:59.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:58 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.182+0000 7f44837fe700 1 -- 172.21.15.69:0/2074153658 <== mgr.24764 v2:172.21.15.69:6800/4217409403 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258029 (secure 0 0 0) 0x7f448c06bab0 con 0x7f44780b2a70 2024-05-14T16:49:59.186 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:49:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.185+0000 7f44942e7700 1 -- 172.21.15.69:0/2074153658 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44780b2a70 msgr2=0x7f44780b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.185+0000 7f44942e7700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44780b2a70 0x7f44780b4f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f448c19ab50 tx=0x7f4488009260 comp rx=0 tx=0).stop 2024-05-14T16:49:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.185+0000 7f44942e7700 1 -- 172.21.15.69:0/2074153658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f448c108510 msgr2=0x7f448c199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.185+0000 7f44942e7700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f448c108510 0x7f448c199a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f448400b330 tx=0x7f448400b640 comp rx=0 tx=0).stop 2024-05-14T16:49:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.185+0000 7f44942e7700 1 -- 172.21.15.69:0/2074153658 shutdown_connections 2024-05-14T16:49:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.185+0000 7f44942e7700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f448c108510 0x7f448c199a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.185+0000 7f44942e7700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f448c102430 0x7f448c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.185+0000 7f44942e7700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44780b2a70 0x7f44780b4f30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.185+0000 7f44942e7700 1 --2- 172.21.15.69:0/2074153658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f448c102df0 0x7f448c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.185+0000 7f44942e7700 1 -- 172.21.15.69:0/2074153658 >> 172.21.15.69:0/2074153658 conn(0x7f448c0793b0 msgr2=0x7f448c1062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.186+0000 7f44942e7700 1 -- 172.21.15.69:0/2074153658 shutdown_connections 2024-05-14T16:49:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:49:59.186+0000 7f44942e7700 1 -- 172.21.15.69:0/2074153658 wait complete. 2024-05-14T16:49:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-05-14T16:49:59.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-05-14T16:49:59.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-05-14T16:49:59.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:59.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:58 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:59.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:59.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:58 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:59.995 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-14T16:49:58.621683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187521,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1249,"num_read_kb":761,"num_write":420,"num_write_kb":3590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952568,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":823,"ondisk_log_size":823,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351836,"kb_used_data":10956,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589988,"statfs":{"total":1151940427776,"available":1151580147712,"internally_reserved":0,"allocated":11218944,"data_stored":8764482,"data_compressed":287472,"data_compressed_allocated":380928,"data_compressed_original":3612672,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990122"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607961+0000","last_change":"2024-05-14T16:46:43.572738+0000","last_active":"2024-05-14T16:49:54.607961+0000","last_peered":"2024-05-14T16:49:54.607961+0000","last_clean":"2024-05-14T16:49:54.607961+0000","last_became_active":"2024-05-14T16:46:43.572400+0000","last_became_peered":"2024-05-14T16:46:43.572400+0000","last_unstale":"2024-05-14T16:49:54.607961+0000","last_undegraded":"2024-05-14T16:49:54.607961+0000","last_fullsized":"2024-05-14T16:49:54.607961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:44.332862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615350+0000","last_change":"2024-05-14T16:46:43.566193+0000","last_active":"2024-05-14T16:49:54.615350+0000","last_peered":"2024-05-14T16:49:54.615350+0000","last_clean":"2024-05-14T16:49:54.615350+0000","last_became_active":"2024-05-14T16:46:43.565853+0000","last_became_peered":"2024-05-14T16:46:43.565853+0000","last_unstale":"2024-05-14T16:49:54.615350+0000","last_undegraded":"2024-05-14T16:49:54.615350+0000","last_fullsized":"2024-05-14T16:49:54.615350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:19:34.481225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615237+0000","last_change":"2024-05-14T16:46:43.574937+0000","last_active":"2024-05-14T16:49:54.615237+0000","last_peered":"2024-05-14T16:49:54.615237+0000","last_clean":"2024-05-14T16:49:54.615237+0000","last_became_active":"2024-05-14T16:46:43.574317+0000","last_became_peered":"2024-05-14T16:46:43.574317+0000","last_unstale":"2024-05-14T16:49:54.615237+0000","last_undegraded":"2024-05-14T16:49:54.615237+0000","last_fullsized":"2024-05-14T16:49:54.615237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:35:49.933658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614517+0000","last_change":"2024-05-14T16:47:33.711809+0000","last_active":"2024-05-14T16:49:54.614517+0000","last_peered":"2024-05-14T16:49:54.614517+0000","last_clean":"2024-05-14T16:49:54.614517+0000","last_became_active":"2024-05-14T16:47:33.711422+0000","last_became_peered":"2024-05-14T16:47:33.711422+0000","last_unstale":"2024-05-14T16:49:54.614517+0000","last_undegraded":"2024-05-14T16:49:54.614517+0000","last_fullsized":"2024-05-14T16:49:54.614517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:44:26.005189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616108+0000","last_change":"2024-05-14T16:46:43.567104+0000","last_active":"2024-05-14T16:49:54.616108+0000","last_peered":"2024-05-14T16:49:54.616108+0000","last_clean":"2024-05-14T16:49:54.616108+0000","last_became_active":"2024-05-14T16:46:43.566762+0000","last_became_peered":"2024-05-14T16:46:43.566762+0000","last_unstale":"2024-05-14T16:49:54.616108+0000","last_undegraded":"2024-05-14T16:49:54.616108+0000","last_fullsized":"2024-05-14T16:49:54.616108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:08.819856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607201+0000","last_change":"2024-05-14T16:46:43.565055+0000","last_active":"2024-05-14T16:49:54.607201+0000","last_peered":"2024-05-14T16:49:54.607201+0000","last_clean":"2024-05-14T16:49:54.607201+0000","last_became_active":"2024-05-14T16:46:43.564699+0000","last_became_peered":"2024-05-14T16:46:43.564699+0000","last_unstale":"2024-05-14T16:49:54.607201+0000","last_undegraded":"2024-05-14T16:49:54.607201+0000","last_fullsized":"2024-05-14T16:49:54.607201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:01.903025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.835993+0000","last_change":"2024-05-14T16:47:33.711866+0000","last_active":"2024-05-14T16:49:54.835993+0000","last_peered":"2024-05-14T16:49:54.835993+0000","last_clean":"2024-05-14T16:49:54.835993+0000","last_became_active":"2024-05-14T16:47:33.711481+0000","last_became_peered":"2024-05-14T16:47:33.711481+0000","last_unstale":"2024-05-14T16:49:54.835993+0000","last_undegraded":"2024-05-14T16:49:54.835993+0000","last_fullsized":"2024-05-14T16:49:54.835993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:01:22.724678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614120+0000","last_change":"2024-05-14T16:46:43.564470+0000","last_active":"2024-05-14T16:49:54.614120+0000","last_peered":"2024-05-14T16:49:54.614120+0000","last_clean":"2024-05-14T16:49:54.614120+0000","last_became_active":"2024-05-14T16:46:43.564169+0000","last_became_peered":"2024-05-14T16:46:43.564169+0000","last_unstale":"2024-05-14T16:49:54.614120+0000","last_undegraded":"2024-05-14T16:49:54.614120+0000","last_fullsized":"2024-05-14T16:49:54.614120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:38.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614961+0000","last_change":"2024-05-14T16:46:43.580988+0000","last_active":"2024-05-14T16:49:54.614961+0000","last_peered":"2024-05-14T16:49:54.614961+0000","last_clean":"2024-05-14T16:49:54.614961+0000","last_became_active":"2024-05-14T16:46:43.580657+0000","last_became_peered":"2024-05-14T16:46:43.580657+0000","last_unstale":"2024-05-14T16:49:54.614961+0000","last_undegraded":"2024-05-14T16:49:54.614961+0000","last_fullsized":"2024-05-14T16:49:54.614961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:39:01.568822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607883+0000","last_change":"2024-05-14T16:46:43.565742+0000","last_active":"2024-05-14T16:49:54.607883+0000","last_peered":"2024-05-14T16:49:54.607883+0000","last_clean":"2024-05-14T16:49:54.607883+0000","last_became_active":"2024-05-14T16:46:43.565260+0000","last_became_peered":"2024-05-14T16:46:43.565260+0000","last_unstale":"2024-05-14T16:49:54.607883+0000","last_undegraded":"2024-05-14T16:49:54.607883+0000","last_fullsized":"2024-05-14T16:49:54.607883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:25.718174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836633+0000","last_change":"2024-05-14T16:46:43.576268+0000","last_active":"2024-05-14T16:49:54.836633+0000","last_peered":"2024-05-14T16:49:54.836633+0000","last_clean":"2024-05-14T16:49:54.836633+0000","last_became_active":"2024-05-14T16:46:43.575856+0000","last_became_peered":"2024-05-14T16:46:43.575856+0000","last_unstale":"2024-05-14T16:49:54.836633+0000","last_undegraded":"2024-05-14T16:49:54.836633+0000","last_fullsized":"2024-05-14T16:49:54.836633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:12:16.549127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614667+0000","last_change":"2024-05-14T16:46:43.573315+0000","last_active":"2024-05-14T16:49:54.614667+0000","last_peered":"2024-05-14T16:49:54.614667+0000","last_clean":"2024-05-14T16:49:54.614667+0000","last_became_active":"2024-05-14T16:46:43.572981+0000","last_became_peered":"2024-05-14T16:46:43.572981+0000","last_unstale":"2024-05-14T16:49:54.614667+0000","last_undegraded":"2024-05-14T16:49:54.614667+0000","last_fullsized":"2024-05-14T16:49:54.614667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:07.757539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605825+0000","last_change":"2024-05-14T16:46:43.569388+0000","last_active":"2024-05-14T16:49:54.605825+0000","last_peered":"2024-05-14T16:49:54.605825+0000","last_clean":"2024-05-14T16:49:54.605825+0000","last_became_active":"2024-05-14T16:46:43.568992+0000","last_became_peered":"2024-05-14T16:46:43.568992+0000","last_unstale":"2024-05-14T16:49:54.605825+0000","last_undegraded":"2024-05-14T16:49:54.605825+0000","last_fullsized":"2024-05-14T16:49:54.605825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:08:06.236797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615319+0000","last_change":"2024-05-14T16:46:43.564137+0000","last_active":"2024-05-14T16:49:54.615319+0000","last_peered":"2024-05-14T16:49:54.615319+0000","last_clean":"2024-05-14T16:49:54.615319+0000","last_became_active":"2024-05-14T16:46:43.563831+0000","last_became_peered":"2024-05-14T16:46:43.563831+0000","last_unstale":"2024-05-14T16:49:54.615319+0000","last_undegraded":"2024-05-14T16:49:54.615319+0000","last_fullsized":"2024-05-14T16:49:54.615319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:58.473863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615197+0000","last_change":"2024-05-14T16:46:43.571789+0000","last_active":"2024-05-14T16:49:54.615197+0000","last_peered":"2024-05-14T16:49:54.615197+0000","last_clean":"2024-05-14T16:49:54.615197+0000","last_became_active":"2024-05-14T16:46:43.571521+0000","last_became_peered":"2024-05-14T16:46:43.571521+0000","last_unstale":"2024-05-14T16:49:54.615197+0000","last_undegraded":"2024-05-14T16:49:54.615197+0000","last_fullsized":"2024-05-14T16:49:54.615197+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:23.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614202+0000","last_change":"2024-05-14T16:46:43.569532+0000","last_active":"2024-05-14T16:49:54.614202+0000","last_peered":"2024-05-14T16:49:54.614202+0000","last_clean":"2024-05-14T16:49:54.614202+0000","last_became_active":"2024-05-14T16:46:43.569132+0000","last_became_peered":"2024-05-14T16:46:43.569132+0000","last_unstale":"2024-05-14T16:49:54.614202+0000","last_undegraded":"2024-05-14T16:49:54.614202+0000","last_fullsized":"2024-05-14T16:49:54.614202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:09.328241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.608089+0000","last_change":"2024-05-14T16:46:43.574451+0000","last_active":"2024-05-14T16:49:54.608089+0000","last_peered":"2024-05-14T16:49:54.608089+0000","last_clean":"2024-05-14T16:49:54.608089+0000","last_became_active":"2024-05-14T16:46:43.574143+0000","last_became_peered":"2024-05-14T16:46:43.574143+0000","last_unstale":"2024-05-14T16:49:54.608089+0000","last_undegraded":"2024-05-14T16:49:54.608089+0000","last_fullsized":"2024-05-14T16:49:54.608089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:01:42.831262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614829+0000","last_change":"2024-05-14T16:46:43.568799+0000","last_active":"2024-05-14T16:49:54.614829+0000","last_peered":"2024-05-14T16:49:54.614829+0000","last_clean":"2024-05-14T16:49:54.614829+0000","last_became_active":"2024-05-14T16:46:43.568467+0000","last_became_peered":"2024-05-14T16:46:43.568467+0000","last_unstale":"2024-05-14T16:49:54.614829+0000","last_undegraded":"2024-05-14T16:49:54.614829+0000","last_fullsized":"2024-05-14T16:49:54.614829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:59.036372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615487+0000","last_change":"2024-05-14T16:46:43.566879+0000","last_active":"2024-05-14T16:49:54.615487+0000","last_peered":"2024-05-14T16:49:54.615487+0000","last_clean":"2024-05-14T16:49:54.615487+0000","last_became_active":"2024-05-14T16:46:43.566346+0000","last_became_peered":"2024-05-14T16:46:43.566346+0000","last_unstale":"2024-05-14T16:49:54.615487+0000","last_undegraded":"2024-05-14T16:49:54.615487+0000","last_fullsized":"2024-05-14T16:49:54.615487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:11.027173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614436+0000","last_change":"2024-05-14T16:46:40.546990+0000","last_active":"2024-05-14T16:49:54.614436+0000","last_peered":"2024-05-14T16:49:54.614436+0000","last_clean":"2024-05-14T16:49:54.614436+0000","last_became_active":"2024-05-14T16:46:40.546646+0000","last_became_peered":"2024-05-14T16:46:40.546646+0000","last_unstale":"2024-05-14T16:49:54.614436+0000","last_undegraded":"2024-05-14T16:49:54.614436+0000","last_fullsized":"2024-05-14T16:49:54.614436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:47.001760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614919+0000","last_change":"2024-05-14T16:46:43.570352+0000","last_active":"2024-05-14T16:49:54.614919+0000","last_peered":"2024-05-14T16:49:54.614919+0000","last_clean":"2024-05-14T16:49:54.614919+0000","last_became_active":"2024-05-14T16:46:43.569748+0000","last_became_peered":"2024-05-14T16:46:43.569748+0000","last_unstale":"2024-05-14T16:49:54.614919+0000","last_undegraded":"2024-05-14T16:49:54.614919+0000","last_fullsized":"2024-05-14T16:49:54.614919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:24.707586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615028+0000","last_change":"2024-05-14T16:46:40.855034+0000","last_active":"2024-05-14T16:49:54.615028+0000","last_peered":"2024-05-14T16:49:54.615028+0000","last_clean":"2024-05-14T16:49:54.615028+0000","last_became_active":"2024-05-14T16:46:40.854686+0000","last_became_peered":"2024-05-14T16:46:40.854686+0000","last_unstale":"2024-05-14T16:49:54.615028+0000","last_undegraded":"2024-05-14T16:49:54.615028+0000","last_fullsized":"2024-05-14T16:49:54.615028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:56.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615572+0000","last_change":"2024-05-14T16:46:43.574682+0000","last_active":"2024-05-14T16:49:54.615572+0000","last_peered":"2024-05-14T16:49:54.615572+0000","last_clean":"2024-05-14T16:49:54.615572+0000","last_became_active":"2024-05-14T16:46:43.574322+0000","last_became_peered":"2024-05-14T16:46:43.574322+0000","last_unstale":"2024-05-14T16:49:54.615572+0000","last_undegraded":"2024-05-14T16:49:54.615572+0000","last_fullsized":"2024-05-14T16:49:54.615572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:20.169428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616589+0000","last_change":"2024-05-14T16:46:40.551307+0000","last_active":"2024-05-14T16:49:54.616589+0000","last_peered":"2024-05-14T16:49:54.616589+0000","last_clean":"2024-05-14T16:49:54.616589+0000","last_became_active":"2024-05-14T16:46:40.550987+0000","last_became_peered":"2024-05-14T16:46:40.550987+0000","last_unstale":"2024-05-14T16:49:54.616589+0000","last_undegraded":"2024-05-14T16:49:54.616589+0000","last_fullsized":"2024-05-14T16:49:54.616589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:27.819457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614524+0000","last_change":"2024-05-14T16:46:43.575200+0000","last_active":"2024-05-14T16:49:54.614524+0000","last_peered":"2024-05-14T16:49:54.614524+0000","last_clean":"2024-05-14T16:49:54.614524+0000","last_became_active":"2024-05-14T16:46:43.574864+0000","last_became_peered":"2024-05-14T16:46:43.574864+0000","last_unstale":"2024-05-14T16:49:54.614524+0000","last_undegraded":"2024-05-14T16:49:54.614524+0000","last_fullsized":"2024-05-14T16:49:54.614524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:48.630101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616383+0000","last_change":"2024-05-14T16:46:40.548397+0000","last_active":"2024-05-14T16:49:54.616383+0000","last_peered":"2024-05-14T16:49:54.616383+0000","last_clean":"2024-05-14T16:49:54.616383+0000","last_became_active":"2024-05-14T16:46:40.547803+0000","last_became_peered":"2024-05-14T16:46:40.547803+0000","last_unstale":"2024-05-14T16:49:54.616383+0000","last_undegraded":"2024-05-14T16:49:54.616383+0000","last_fullsized":"2024-05-14T16:49:54.616383+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:21:19.812413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615180+0000","last_change":"2024-05-14T16:46:43.578301+0000","last_active":"2024-05-14T16:49:54.615180+0000","last_peered":"2024-05-14T16:49:54.615180+0000","last_clean":"2024-05-14T16:49:54.615180+0000","last_became_active":"2024-05-14T16:46:43.577796+0000","last_became_peered":"2024-05-14T16:46:43.577796+0000","last_unstale":"2024-05-14T16:49:54.615180+0000","last_undegraded":"2024-05-14T16:49:54.615180+0000","last_fullsized":"2024-05-14T16:49:54.615180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:13.159570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":211,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.870130+0000","last_change":"2024-05-14T16:46:40.547818+0000","last_active":"2024-05-14T16:49:54.870130+0000","last_peered":"2024-05-14T16:49:54.870130+0000","last_clean":"2024-05-14T16:49:54.870130+0000","last_became_active":"2024-05-14T16:46:40.547454+0000","last_became_peered":"2024-05-14T16:46:40.547454+0000","last_unstale":"2024-05-14T16:49:54.870130+0000","last_undegraded":"2024-05-14T16:49:54.870130+0000","last_fullsized":"2024-05-14T16:49:54.870130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:16.362415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616051+0000","last_change":"2024-05-14T16:46:43.570743+0000","last_active":"2024-05-14T16:49:54.616051+0000","last_peered":"2024-05-14T16:49:54.616051+0000","last_clean":"2024-05-14T16:49:54.616051+0000","last_became_active":"2024-05-14T16:46:43.570427+0000","last_became_peered":"2024-05-14T16:46:43.570427+0000","last_unstale":"2024-05-14T16:49:54.616051+0000","last_undegraded":"2024-05-14T16:49:54.616051+0000","last_fullsized":"2024-05-14T16:49:54.616051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:04:11.091285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607185+0000","last_change":"2024-05-14T16:46:40.546675+0000","last_active":"2024-05-14T16:49:54.607185+0000","last_peered":"2024-05-14T16:49:54.607185+0000","last_clean":"2024-05-14T16:49:54.607185+0000","last_became_active":"2024-05-14T16:46:40.546250+0000","last_became_peered":"2024-05-14T16:46:40.546250+0000","last_unstale":"2024-05-14T16:49:54.607185+0000","last_undegraded":"2024-05-14T16:49:54.607185+0000","last_fullsized":"2024-05-14T16:49:54.607185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:42:52.367719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614882+0000","last_change":"2024-05-14T16:46:43.579119+0000","last_active":"2024-05-14T16:49:54.614882+0000","last_peered":"2024-05-14T16:49:54.614882+0000","last_clean":"2024-05-14T16:49:54.614882+0000","last_became_active":"2024-05-14T16:46:43.578808+0000","last_became_peered":"2024-05-14T16:46:43.578808+0000","last_unstale":"2024-05-14T16:49:54.614882+0000","last_undegraded":"2024-05-14T16:49:54.614882+0000","last_fullsized":"2024-05-14T16:49:54.614882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:57.852982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605792+0000","last_change":"2024-05-14T16:46:40.538827+0000","last_active":"2024-05-14T16:49:54.605792+0000","last_peered":"2024-05-14T16:49:54.605792+0000","last_clean":"2024-05-14T16:49:54.605792+0000","last_became_active":"2024-05-14T16:46:40.538369+0000","last_became_peered":"2024-05-14T16:46:40.538369+0000","last_unstale":"2024-05-14T16:49:54.605792+0000","last_undegraded":"2024-05-14T16:49:54.605792+0000","last_fullsized":"2024-05-14T16:49:54.605792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:48:03.599457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614307+0000","last_change":"2024-05-14T16:46:43.558411+0000","last_active":"2024-05-14T16:49:54.614307+0000","last_peered":"2024-05-14T16:49:54.614307+0000","last_clean":"2024-05-14T16:49:54.614307+0000","last_became_active":"2024-05-14T16:46:43.558101+0000","last_became_peered":"2024-05-14T16:46:43.558101+0000","last_unstale":"2024-05-14T16:49:54.614307+0000","last_undegraded":"2024-05-14T16:49:54.614307+0000","last_fullsized":"2024-05-14T16:49:54.614307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:10.595833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614356+0000","last_change":"2024-05-14T16:46:40.544858+0000","last_active":"2024-05-14T16:49:54.614356+0000","last_peered":"2024-05-14T16:49:54.614356+0000","last_clean":"2024-05-14T16:49:54.614356+0000","last_became_active":"2024-05-14T16:46:40.544197+0000","last_became_peered":"2024-05-14T16:46:40.544197+0000","last_unstale":"2024-05-14T16:49:54.614356+0000","last_undegraded":"2024-05-14T16:49:54.614356+0000","last_fullsized":"2024-05-14T16:49:54.614356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:16:45.249626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614048+0000","last_change":"2024-05-14T16:46:43.567019+0000","last_active":"2024-05-14T16:49:54.614048+0000","last_peered":"2024-05-14T16:49:54.614048+0000","last_clean":"2024-05-14T16:49:54.614048+0000","last_became_active":"2024-05-14T16:46:43.566700+0000","last_became_peered":"2024-05-14T16:46:43.566700+0000","last_unstale":"2024-05-14T16:49:54.614048+0000","last_undegraded":"2024-05-14T16:49:54.614048+0000","last_fullsized":"2024-05-14T16:49:54.614048+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:43.373467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":207,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.839019+0000","last_change":"2024-05-14T16:46:40.541876+0000","last_active":"2024-05-14T16:49:54.839019+0000","last_peered":"2024-05-14T16:49:54.839019+0000","last_clean":"2024-05-14T16:49:54.839019+0000","last_became_active":"2024-05-14T16:46:40.541460+0000","last_became_peered":"2024-05-14T16:46:40.541460+0000","last_unstale":"2024-05-14T16:49:54.839019+0000","last_undegraded":"2024-05-14T16:49:54.839019+0000","last_fullsized":"2024-05-14T16:49:54.839019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:07:22.536429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614958+0000","last_change":"2024-05-14T16:46:43.577015+0000","last_active":"2024-05-14T16:49:54.614958+0000","last_peered":"2024-05-14T16:49:54.614958+0000","last_clean":"2024-05-14T16:49:54.614958+0000","last_became_active":"2024-05-14T16:46:43.576657+0000","last_became_peered":"2024-05-14T16:46:43.576657+0000","last_unstale":"2024-05-14T16:49:54.614958+0000","last_undegraded":"2024-05-14T16:49:54.614958+0000","last_fullsized":"2024-05-14T16:49:54.614958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:29.242445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836348+0000","last_change":"2024-05-14T16:46:40.550727+0000","last_active":"2024-05-14T16:49:54.836348+0000","last_peered":"2024-05-14T16:49:54.836348+0000","last_clean":"2024-05-14T16:49:54.836348+0000","last_became_active":"2024-05-14T16:46:40.550310+0000","last_became_peered":"2024-05-14T16:46:40.550310+0000","last_unstale":"2024-05-14T16:49:54.836348+0000","last_undegraded":"2024-05-14T16:49:54.836348+0000","last_fullsized":"2024-05-14T16:49:54.836348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:04.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615580+0000","last_change":"2024-05-14T16:47:33.711766+0000","last_active":"2024-05-14T16:49:54.615580+0000","last_peered":"2024-05-14T16:49:54.615580+0000","last_clean":"2024-05-14T16:49:54.615580+0000","last_became_active":"2024-05-14T16:47:33.711352+0000","last_became_peered":"2024-05-14T16:47:33.711352+0000","last_unstale":"2024-05-14T16:49:54.615580+0000","last_undegraded":"2024-05-14T16:49:54.615580+0000","last_fullsized":"2024-05-14T16:49:54.615580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:47.924117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615228+0000","last_change":"2024-05-14T16:46:40.855680+0000","last_active":"2024-05-14T16:49:54.615228+0000","last_peered":"2024-05-14T16:49:54.615228+0000","last_clean":"2024-05-14T16:49:54.615228+0000","last_became_active":"2024-05-14T16:46:40.855265+0000","last_became_peered":"2024-05-14T16:46:40.855265+0000","last_unstale":"2024-05-14T16:49:54.615228+0000","last_undegraded":"2024-05-14T16:49:54.615228+0000","last_fullsized":"2024-05-14T16:49:54.615228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:26.065298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614667+0000","last_change":"2024-05-14T16:46:43.579423+0000","last_active":"2024-05-14T16:49:54.614667+0000","last_peered":"2024-05-14T16:49:54.614667+0000","last_clean":"2024-05-14T16:49:54.614667+0000","last_became_active":"2024-05-14T16:46:43.579023+0000","last_became_peered":"2024-05-14T16:46:43.579023+0000","last_unstale":"2024-05-14T16:49:54.614667+0000","last_undegraded":"2024-05-14T16:49:54.614667+0000","last_fullsized":"2024-05-14T16:49:54.614667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:21.311018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'38","reported_seq":428,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.870707+0000","last_change":"2024-05-14T16:46:40.550871+0000","last_active":"2024-05-14T16:49:54.870707+0000","last_peered":"2024-05-14T16:49:54.870707+0000","last_clean":"2024-05-14T16:49:54.870707+0000","last_became_active":"2024-05-14T16:46:40.550419+0000","last_became_peered":"2024-05-14T16:46:40.550419+0000","last_unstale":"2024-05-14T16:49:54.870707+0000","last_undegraded":"2024-05-14T16:49:54.870707+0000","last_fullsized":"2024-05-14T16:49:54.870707+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:53:09.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67403,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":246,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616462+0000","last_change":"2024-05-14T16:46:43.569996+0000","last_active":"2024-05-14T16:49:54.616462+0000","last_peered":"2024-05-14T16:49:54.616462+0000","last_clean":"2024-05-14T16:49:54.616462+0000","last_became_active":"2024-05-14T16:46:43.569501+0000","last_became_peered":"2024-05-14T16:46:43.569501+0000","last_unstale":"2024-05-14T16:49:54.616462+0000","last_undegraded":"2024-05-14T16:49:54.616462+0000","last_fullsized":"2024-05-14T16:49:54.616462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:55.135103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607804+0000","last_change":"2024-05-14T16:47:33.733316+0000","last_active":"2024-05-14T16:49:54.607804+0000","last_peered":"2024-05-14T16:49:54.607804+0000","last_clean":"2024-05-14T16:49:54.607804+0000","last_became_active":"2024-05-14T16:47:33.709886+0000","last_became_peered":"2024-05-14T16:47:33.709886+0000","last_unstale":"2024-05-14T16:49:54.607804+0000","last_undegraded":"2024-05-14T16:49:54.607804+0000","last_fullsized":"2024-05-14T16:49:54.607804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:00:47.464686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605666+0000","last_change":"2024-05-14T16:46:43.562773+0000","last_active":"2024-05-14T16:49:54.605666+0000","last_peered":"2024-05-14T16:49:54.605666+0000","last_clean":"2024-05-14T16:49:54.605666+0000","last_became_active":"2024-05-14T16:46:43.562449+0000","last_became_peered":"2024-05-14T16:46:43.562449+0000","last_unstale":"2024-05-14T16:49:54.605666+0000","last_undegraded":"2024-05-14T16:49:54.605666+0000","last_fullsized":"2024-05-14T16:49:54.605666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:45.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614731+0000","last_change":"2024-05-14T16:46:40.545882+0000","last_active":"2024-05-14T16:49:54.614731+0000","last_peered":"2024-05-14T16:49:54.614731+0000","last_clean":"2024-05-14T16:49:54.614731+0000","last_became_active":"2024-05-14T16:46:40.545439+0000","last_became_peered":"2024-05-14T16:46:40.545439+0000","last_unstale":"2024-05-14T16:49:54.614731+0000","last_undegraded":"2024-05-14T16:49:54.614731+0000","last_fullsized":"2024-05-14T16:49:54.614731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:07.983329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836397+0000","last_change":"2024-05-14T16:46:43.568109+0000","last_active":"2024-05-14T16:49:54.836397+0000","last_peered":"2024-05-14T16:49:54.836397+0000","last_clean":"2024-05-14T16:49:54.836397+0000","last_became_active":"2024-05-14T16:46:43.567106+0000","last_became_peered":"2024-05-14T16:46:43.567106+0000","last_unstale":"2024-05-14T16:49:54.836397+0000","last_undegraded":"2024-05-14T16:49:54.836397+0000","last_fullsized":"2024-05-14T16:49:54.836397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:29.705387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":212,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.870503+0000","last_change":"2024-05-14T16:46:40.542204+0000","last_active":"2024-05-14T16:49:54.870503+0000","last_peered":"2024-05-14T16:49:54.870503+0000","last_clean":"2024-05-14T16:49:54.870503+0000","last_became_active":"2024-05-14T16:46:40.541807+0000","last_became_peered":"2024-05-14T16:46:40.541807+0000","last_unstale":"2024-05-14T16:49:54.870503+0000","last_undegraded":"2024-05-14T16:49:54.870503+0000","last_fullsized":"2024-05-14T16:49:54.870503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:30:45.115647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836336+0000","last_change":"2024-05-14T16:46:43.573546+0000","last_active":"2024-05-14T16:49:54.836336+0000","last_peered":"2024-05-14T16:49:54.836336+0000","last_clean":"2024-05-14T16:49:54.836336+0000","last_became_active":"2024-05-14T16:46:43.573183+0000","last_became_peered":"2024-05-14T16:46:43.573183+0000","last_unstale":"2024-05-14T16:49:54.836336+0000","last_undegraded":"2024-05-14T16:49:54.836336+0000","last_fullsized":"2024-05-14T16:49:54.836336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:49.873097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607658+0000","last_change":"2024-05-14T16:47:33.726700+0000","last_active":"2024-05-14T16:49:54.607658+0000","last_peered":"2024-05-14T16:49:54.607658+0000","last_clean":"2024-05-14T16:49:54.607658+0000","last_became_active":"2024-05-14T16:47:33.709297+0000","last_became_peered":"2024-05-14T16:47:33.709297+0000","last_unstale":"2024-05-14T16:49:54.607658+0000","last_undegraded":"2024-05-14T16:49:54.607658+0000","last_fullsized":"2024-05-14T16:49:54.607658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:35.028197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614569+0000","last_change":"2024-05-14T16:46:43.579059+0000","last_active":"2024-05-14T16:49:54.614569+0000","last_peered":"2024-05-14T16:49:54.614569+0000","last_clean":"2024-05-14T16:49:54.614569+0000","last_became_active":"2024-05-14T16:46:43.578748+0000","last_became_peered":"2024-05-14T16:46:43.578748+0000","last_unstale":"2024-05-14T16:49:54.614569+0000","last_undegraded":"2024-05-14T16:49:54.614569+0000","last_fullsized":"2024-05-14T16:49:54.614569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:42.446422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616551+0000","last_change":"2024-05-14T16:46:40.549047+0000","last_active":"2024-05-14T16:49:54.616551+0000","last_peered":"2024-05-14T16:49:54.616551+0000","last_clean":"2024-05-14T16:49:54.616551+0000","last_became_active":"2024-05-14T16:46:40.548648+0000","last_became_peered":"2024-05-14T16:46:40.548648+0000","last_unstale":"2024-05-14T16:49:54.616551+0000","last_undegraded":"2024-05-14T16:49:54.616551+0000","last_fullsized":"2024-05-14T16:49:54.616551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:52:56.370871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607790+0000","last_change":"2024-05-14T16:46:43.569624+0000","last_active":"2024-05-14T16:49:54.607790+0000","last_peered":"2024-05-14T16:49:54.607790+0000","last_clean":"2024-05-14T16:49:54.607790+0000","last_became_active":"2024-05-14T16:46:43.569077+0000","last_became_peered":"2024-05-14T16:46:43.569077+0000","last_unstale":"2024-05-14T16:49:54.607790+0000","last_undegraded":"2024-05-14T16:49:54.607790+0000","last_fullsized":"2024-05-14T16:49:54.607790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:45:50.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614415+0000","last_change":"2024-05-14T16:46:40.856302+0000","last_active":"2024-05-14T16:49:54.614415+0000","last_peered":"2024-05-14T16:49:54.614415+0000","last_clean":"2024-05-14T16:49:54.614415+0000","last_became_active":"2024-05-14T16:46:40.855964+0000","last_became_peered":"2024-05-14T16:46:40.855964+0000","last_unstale":"2024-05-14T16:49:54.614415+0000","last_undegraded":"2024-05-14T16:49:54.614415+0000","last_fullsized":"2024-05-14T16:49:54.614415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:38.501418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607375+0000","last_change":"2024-05-14T16:46:43.562847+0000","last_active":"2024-05-14T16:49:54.607375+0000","last_peered":"2024-05-14T16:49:54.607375+0000","last_clean":"2024-05-14T16:49:54.607375+0000","last_became_active":"2024-05-14T16:46:43.560978+0000","last_became_peered":"2024-05-14T16:46:43.560978+0000","last_unstale":"2024-05-14T16:49:54.607375+0000","last_undegraded":"2024-05-14T16:49:54.607375+0000","last_fullsized":"2024-05-14T16:49:54.607375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:12:33.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836124+0000","last_change":"2024-05-14T16:46:40.544017+0000","last_active":"2024-05-14T16:49:54.836124+0000","last_peered":"2024-05-14T16:49:54.836124+0000","last_clean":"2024-05-14T16:49:54.836124+0000","last_became_active":"2024-05-14T16:46:40.543176+0000","last_became_peered":"2024-05-14T16:46:40.543176+0000","last_unstale":"2024-05-14T16:49:54.836124+0000","last_undegraded":"2024-05-14T16:49:54.836124+0000","last_fullsized":"2024-05-14T16:49:54.836124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:09.817891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607232+0000","last_change":"2024-05-14T16:46:43.564395+0000","last_active":"2024-05-14T16:49:54.607232+0000","last_peered":"2024-05-14T16:49:54.607232+0000","last_clean":"2024-05-14T16:49:54.607232+0000","last_became_active":"2024-05-14T16:46:43.564065+0000","last_became_peered":"2024-05-14T16:46:43.564065+0000","last_unstale":"2024-05-14T16:49:54.607232+0000","last_undegraded":"2024-05-14T16:49:54.607232+0000","last_fullsized":"2024-05-14T16:49:54.607232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:52.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614590+0000","last_change":"2024-05-14T16:46:43.577067+0000","last_active":"2024-05-14T16:49:54.614590+0000","last_peered":"2024-05-14T16:49:54.614590+0000","last_clean":"2024-05-14T16:49:54.614590+0000","last_became_active":"2024-05-14T16:46:43.576750+0000","last_became_peered":"2024-05-14T16:46:43.576750+0000","last_unstale":"2024-05-14T16:49:54.614590+0000","last_undegraded":"2024-05-14T16:49:54.614590+0000","last_fullsized":"2024-05-14T16:49:54.614590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:25.961124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":207,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605674+0000","last_change":"2024-05-14T16:46:40.855073+0000","last_active":"2024-05-14T16:49:54.605674+0000","last_peered":"2024-05-14T16:49:54.605674+0000","last_clean":"2024-05-14T16:49:54.605674+0000","last_became_active":"2024-05-14T16:46:40.854641+0000","last_became_peered":"2024-05-14T16:46:40.854641+0000","last_unstale":"2024-05-14T16:49:54.605674+0000","last_undegraded":"2024-05-14T16:49:54.605674+0000","last_fullsized":"2024-05-14T16:49:54.605674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:58:11.670926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614618+0000","last_change":"2024-05-14T16:46:43.569030+0000","last_active":"2024-05-14T16:49:54.614618+0000","last_peered":"2024-05-14T16:49:54.614618+0000","last_clean":"2024-05-14T16:49:54.614618+0000","last_became_active":"2024-05-14T16:46:43.568646+0000","last_became_peered":"2024-05-14T16:46:43.568646+0000","last_unstale":"2024-05-14T16:49:54.614618+0000","last_undegraded":"2024-05-14T16:49:54.614618+0000","last_fullsized":"2024-05-14T16:49:54.614618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:38.086818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614520+0000","last_change":"2024-05-14T16:46:40.542624+0000","last_active":"2024-05-14T16:49:54.614520+0000","last_peered":"2024-05-14T16:49:54.614520+0000","last_clean":"2024-05-14T16:49:54.614520+0000","last_became_active":"2024-05-14T16:46:40.542230+0000","last_became_peered":"2024-05-14T16:46:40.542230+0000","last_unstale":"2024-05-14T16:49:54.614520+0000","last_undegraded":"2024-05-14T16:49:54.614520+0000","last_fullsized":"2024-05-14T16:49:54.614520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:54:50.083362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836053+0000","last_change":"2024-05-14T16:45:15.724994+0000","last_active":"2024-05-14T16:49:54.836053+0000","last_peered":"2024-05-14T16:49:54.836053+0000","last_clean":"2024-05-14T16:49:54.836053+0000","last_became_active":"2024-05-14T16:45:15.720080+0000","last_became_peered":"2024-05-14T16:45:15.720080+0000","last_unstale":"2024-05-14T16:49:54.836053+0000","last_undegraded":"2024-05-14T16:49:54.836053+0000","last_fullsized":"2024-05-14T16:49:54.836053+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_clean_scrub_stamp":"2024-05-14T16:41:54.615675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:32:43.569611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605606+0000","last_change":"2024-05-14T16:46:43.567143+0000","last_active":"2024-05-14T16:49:54.605606+0000","last_peered":"2024-05-14T16:49:54.605606+0000","last_clean":"2024-05-14T16:49:54.605606+0000","last_became_active":"2024-05-14T16:46:43.564717+0000","last_became_peered":"2024-05-14T16:46:43.564717+0000","last_unstale":"2024-05-14T16:49:54.605606+0000","last_undegraded":"2024-05-14T16:49:54.605606+0000","last_fullsized":"2024-05-14T16:49:54.605606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:59.025279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836233+0000","last_change":"2024-05-14T16:46:40.546709+0000","last_active":"2024-05-14T16:49:54.836233+0000","last_peered":"2024-05-14T16:49:54.836233+0000","last_clean":"2024-05-14T16:49:54.836233+0000","last_became_active":"2024-05-14T16:46:40.546295+0000","last_became_peered":"2024-05-14T16:46:40.546295+0000","last_unstale":"2024-05-14T16:49:54.836233+0000","last_undegraded":"2024-05-14T16:49:54.836233+0000","last_fullsized":"2024-05-14T16:49:54.836233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:42:50.255289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616443+0000","last_change":"2024-05-14T16:46:40.856822+0000","last_active":"2024-05-14T16:49:54.616443+0000","last_peered":"2024-05-14T16:49:54.616443+0000","last_clean":"2024-05-14T16:49:54.616443+0000","last_became_active":"2024-05-14T16:46:40.856434+0000","last_became_peered":"2024-05-14T16:46:40.856434+0000","last_unstale":"2024-05-14T16:49:54.616443+0000","last_undegraded":"2024-05-14T16:49:54.616443+0000","last_fullsized":"2024-05-14T16:49:54.616443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:38.997679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605968+0000","last_change":"2024-05-14T16:46:43.563918+0000","last_active":"2024-05-14T16:49:54.605968+0000","last_peered":"2024-05-14T16:49:54.605968+0000","last_clean":"2024-05-14T16:49:54.605968+0000","last_became_active":"2024-05-14T16:46:43.563527+0000","last_became_peered":"2024-05-14T16:46:43.563527+0000","last_unstale":"2024-05-14T16:49:54.605968+0000","last_undegraded":"2024-05-14T16:49:54.605968+0000","last_fullsized":"2024-05-14T16:49:54.605968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:58:52.984695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'48","reported_seq":100,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.806047+0000","last_change":"2024-05-14T16:46:40.547480+0000","last_active":"2024-05-14T16:49:54.806047+0000","last_peered":"2024-05-14T16:49:54.806047+0000","last_clean":"2024-05-14T16:49:54.806047+0000","last_became_active":"2024-05-14T16:46:40.547062+0000","last_became_peered":"2024-05-14T16:46:40.547062+0000","last_unstale":"2024-05-14T16:49:54.806047+0000","last_undegraded":"2024-05-14T16:49:54.806047+0000","last_fullsized":"2024-05-14T16:49:54.806047+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:30.211421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":87,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615053+0000","last_change":"2024-05-14T16:46:43.563870+0000","last_active":"2024-05-14T16:49:54.615053+0000","last_peered":"2024-05-14T16:49:54.615053+0000","last_clean":"2024-05-14T16:49:54.615053+0000","last_became_active":"2024-05-14T16:46:43.563542+0000","last_became_peered":"2024-05-14T16:46:43.563542+0000","last_unstale":"2024-05-14T16:49:54.615053+0000","last_undegraded":"2024-05-14T16:49:54.615053+0000","last_fullsized":"2024-05-14T16:49:54.615053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:50.699601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614489+0000","last_change":"2024-05-14T16:46:40.665192+0000","last_active":"2024-05-14T16:49:54.614489+0000","last_peered":"2024-05-14T16:49:54.614489+0000","last_clean":"2024-05-14T16:49:54.614489+0000","last_became_active":"2024-05-14T16:46:40.664827+0000","last_became_peered":"2024-05-14T16:46:40.664827+0000","last_unstale":"2024-05-14T16:49:54.614489+0000","last_undegraded":"2024-05-14T16:49:54.614489+0000","last_fullsized":"2024-05-14T16:49:54.614489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:41:47.179677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615488+0000","last_change":"2024-05-14T16:47:33.717718+0000","last_active":"2024-05-14T16:49:54.615488+0000","last_peered":"2024-05-14T16:49:54.615488+0000","last_clean":"2024-05-14T16:49:54.615488+0000","last_became_active":"2024-05-14T16:47:33.717368+0000","last_became_peered":"2024-05-14T16:47:33.717368+0000","last_unstale":"2024-05-14T16:49:54.615488+0000","last_undegraded":"2024-05-14T16:49:54.615488+0000","last_fullsized":"2024-05-14T16:49:54.615488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:07:43.990388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614203+0000","last_change":"2024-05-14T16:46:40.855458+0000","last_active":"2024-05-14T16:49:54.614203+0000","last_peered":"2024-05-14T16:49:54.614203+0000","last_clean":"2024-05-14T16:49:54.614203+0000","last_became_active":"2024-05-14T16:46:40.855122+0000","last_became_peered":"2024-05-14T16:46:40.855122+0000","last_unstale":"2024-05-14T16:49:54.614203+0000","last_undegraded":"2024-05-14T16:49:54.614203+0000","last_fullsized":"2024-05-14T16:49:54.614203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:20:43.435860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615935+0000","last_change":"2024-05-14T16:46:43.566482+0000","last_active":"2024-05-14T16:49:54.615935+0000","last_peered":"2024-05-14T16:49:54.615935+0000","last_clean":"2024-05-14T16:49:54.615935+0000","last_became_active":"2024-05-14T16:46:43.566145+0000","last_became_peered":"2024-05-14T16:46:43.566145+0000","last_unstale":"2024-05-14T16:49:54.615935+0000","last_undegraded":"2024-05-14T16:49:54.615935+0000","last_fullsized":"2024-05-14T16:49:54.615935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:14:22.731629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614710+0000","last_change":"2024-05-14T16:46:40.541655+0000","last_active":"2024-05-14T16:49:54.614710+0000","last_peered":"2024-05-14T16:49:54.614710+0000","last_clean":"2024-05-14T16:49:54.614710+0000","last_became_active":"2024-05-14T16:46:40.541331+0000","last_became_peered":"2024-05-14T16:46:40.541331+0000","last_unstale":"2024-05-14T16:49:54.614710+0000","last_undegraded":"2024-05-14T16:49:54.614710+0000","last_fullsized":"2024-05-14T16:49:54.614710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:47:27.950532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607663+0000","last_change":"2024-05-14T16:46:43.564216+0000","last_active":"2024-05-14T16:49:54.607663+0000","last_peered":"2024-05-14T16:49:54.607663+0000","last_clean":"2024-05-14T16:49:54.607663+0000","last_became_active":"2024-05-14T16:46:43.563894+0000","last_became_peered":"2024-05-14T16:46:43.563894+0000","last_unstale":"2024-05-14T16:49:54.607663+0000","last_undegraded":"2024-05-14T16:49:54.607663+0000","last_fullsized":"2024-05-14T16:49:54.607663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:16:09.256909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615063+0000","last_change":"2024-05-14T16:46:40.551034+0000","last_active":"2024-05-14T16:49:54.615063+0000","last_peered":"2024-05-14T16:49:54.615063+0000","last_clean":"2024-05-14T16:49:54.615063+0000","last_became_active":"2024-05-14T16:46:40.550211+0000","last_became_peered":"2024-05-14T16:46:40.550211+0000","last_unstale":"2024-05-14T16:49:54.615063+0000","last_undegraded":"2024-05-14T16:49:54.615063+0000","last_fullsized":"2024-05-14T16:49:54.615063+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:15.720942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614751+0000","last_change":"2024-05-14T16:46:43.558264+0000","last_active":"2024-05-14T16:49:54.614751+0000","last_peered":"2024-05-14T16:49:54.614751+0000","last_clean":"2024-05-14T16:49:54.614751+0000","last_became_active":"2024-05-14T16:46:43.557907+0000","last_became_peered":"2024-05-14T16:46:43.557907+0000","last_unstale":"2024-05-14T16:49:54.614751+0000","last_undegraded":"2024-05-14T16:49:54.614751+0000","last_fullsized":"2024-05-14T16:49:54.614751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:20:52.752482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616215+0000","last_change":"2024-05-14T16:46:40.551013+0000","last_active":"2024-05-14T16:49:54.616215+0000","last_peered":"2024-05-14T16:49:54.616215+0000","last_clean":"2024-05-14T16:49:54.616215+0000","last_became_active":"2024-05-14T16:46:40.550658+0000","last_became_peered":"2024-05-14T16:46:40.550658+0000","last_unstale":"2024-05-14T16:49:54.616215+0000","last_undegraded":"2024-05-14T16:49:54.616215+0000","last_fullsized":"2024-05-14T16:49:54.616215+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:36.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836187+0000","last_change":"2024-05-14T16:46:43.572270+0000","last_active":"2024-05-14T16:49:54.836187+0000","last_peered":"2024-05-14T16:49:54.836187+0000","last_clean":"2024-05-14T16:49:54.836187+0000","last_became_active":"2024-05-14T16:46:43.571886+0000","last_became_peered":"2024-05-14T16:46:43.571886+0000","last_unstale":"2024-05-14T16:49:54.836187+0000","last_undegraded":"2024-05-14T16:49:54.836187+0000","last_fullsized":"2024-05-14T16:49:54.836187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:11.890881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614625+0000","last_change":"2024-05-14T16:47:33.715246+0000","last_active":"2024-05-14T16:49:54.614625+0000","last_peered":"2024-05-14T16:49:54.614625+0000","last_clean":"2024-05-14T16:49:54.614625+0000","last_became_active":"2024-05-14T16:47:33.708725+0000","last_became_peered":"2024-05-14T16:47:33.708725+0000","last_unstale":"2024-05-14T16:49:54.614625+0000","last_undegraded":"2024-05-14T16:49:54.614625+0000","last_fullsized":"2024-05-14T16:49:54.614625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:19.118454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836720+0000","last_change":"2024-05-14T16:46:43.572455+0000","last_active":"2024-05-14T16:49:54.836720+0000","last_peered":"2024-05-14T16:49:54.836720+0000","last_clean":"2024-05-14T16:49:54.836720+0000","last_became_active":"2024-05-14T16:46:43.571930+0000","last_became_peered":"2024-05-14T16:46:43.571930+0000","last_unstale":"2024-05-14T16:49:54.836720+0000","last_undegraded":"2024-05-14T16:49:54.836720+0000","last_fullsized":"2024-05-14T16:49:54.836720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:35:39.643239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'39","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.800806+0000","last_change":"2024-05-14T16:46:40.541708+0000","last_active":"2024-05-14T16:49:54.800806+0000","last_peered":"2024-05-14T16:49:54.800806+0000","last_clean":"2024-05-14T16:49:54.800806+0000","last_became_active":"2024-05-14T16:46:40.541202+0000","last_became_peered":"2024-05-14T16:46:40.541202+0000","last_unstale":"2024-05-14T16:49:54.800806+0000","last_undegraded":"2024-05-14T16:49:54.800806+0000","last_fullsized":"2024-05-14T16:49:54.800806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:50:01.821013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836382+0000","last_change":"2024-05-14T16:46:43.574039+0000","last_active":"2024-05-14T16:49:54.836382+0000","last_peered":"2024-05-14T16:49:54.836382+0000","last_clean":"2024-05-14T16:49:54.836382+0000","last_became_active":"2024-05-14T16:46:43.573237+0000","last_became_peered":"2024-05-14T16:46:43.573237+0000","last_unstale":"2024-05-14T16:49:54.836382+0000","last_undegraded":"2024-05-14T16:49:54.836382+0000","last_fullsized":"2024-05-14T16:49:54.836382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:13.234495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605951+0000","last_change":"2024-05-14T16:46:40.549519+0000","last_active":"2024-05-14T16:49:54.605951+0000","last_peered":"2024-05-14T16:49:54.605951+0000","last_clean":"2024-05-14T16:49:54.605951+0000","last_became_active":"2024-05-14T16:46:40.549169+0000","last_became_peered":"2024-05-14T16:46:40.549169+0000","last_unstale":"2024-05-14T16:49:54.605951+0000","last_undegraded":"2024-05-14T16:49:54.605951+0000","last_fullsized":"2024-05-14T16:49:54.605951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:22.559724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836719+0000","last_change":"2024-05-14T16:46:43.567674+0000","last_active":"2024-05-14T16:49:54.836719+0000","last_peered":"2024-05-14T16:49:54.836719+0000","last_clean":"2024-05-14T16:49:54.836719+0000","last_became_active":"2024-05-14T16:46:43.567337+0000","last_became_peered":"2024-05-14T16:46:43.567337+0000","last_unstale":"2024-05-14T16:49:54.836719+0000","last_undegraded":"2024-05-14T16:49:54.836719+0000","last_fullsized":"2024-05-14T16:49:54.836719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:02.321955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607773+0000","last_change":"2024-05-14T16:46:43.567627+0000","last_active":"2024-05-14T16:49:54.607773+0000","last_peered":"2024-05-14T16:49:54.607773+0000","last_clean":"2024-05-14T16:49:54.607773+0000","last_became_active":"2024-05-14T16:46:43.567313+0000","last_became_peered":"2024-05-14T16:46:43.567313+0000","last_unstale":"2024-05-14T16:49:54.607773+0000","last_undegraded":"2024-05-14T16:49:54.607773+0000","last_fullsized":"2024-05-14T16:49:54.607773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:47.396923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615757+0000","last_change":"2024-05-14T16:47:33.708354+0000","last_active":"2024-05-14T16:49:54.615757+0000","last_peered":"2024-05-14T16:49:54.615757+0000","last_clean":"2024-05-14T16:49:54.615757+0000","last_became_active":"2024-05-14T16:47:33.707842+0000","last_became_peered":"2024-05-14T16:47:33.707842+0000","last_unstale":"2024-05-14T16:49:54.615757+0000","last_undegraded":"2024-05-14T16:49:54.615757+0000","last_fullsized":"2024-05-14T16:49:54.615757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:56:26.845782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614936+0000","last_change":"2024-05-14T16:47:33.708332+0000","last_active":"2024-05-14T16:49:54.614936+0000","last_peered":"2024-05-14T16:49:54.614936+0000","last_clean":"2024-05-14T16:49:54.614936+0000","last_became_active":"2024-05-14T16:47:33.707774+0000","last_became_peered":"2024-05-14T16:47:33.707774+0000","last_unstale":"2024-05-14T16:49:54.614936+0000","last_undegraded":"2024-05-14T16:49:54.614936+0000","last_fullsized":"2024-05-14T16:49:54.614936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:44:47.699303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615706+0000","last_change":"2024-05-14T16:46:43.572168+0000","last_active":"2024-05-14T16:49:54.615706+0000","last_peered":"2024-05-14T16:49:54.615706+0000","last_clean":"2024-05-14T16:49:54.615706+0000","last_became_active":"2024-05-14T16:46:43.571822+0000","last_became_peered":"2024-05-14T16:46:43.571822+0000","last_unstale":"2024-05-14T16:49:54.615706+0000","last_undegraded":"2024-05-14T16:49:54.615706+0000","last_fullsized":"2024-05-14T16:49:54.615706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:57.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605874+0000","last_change":"2024-05-14T16:46:43.570935+0000","last_active":"2024-05-14T16:49:54.605874+0000","last_peered":"2024-05-14T16:49:54.605874+0000","last_clean":"2024-05-14T16:49:54.605874+0000","last_became_active":"2024-05-14T16:46:43.570605+0000","last_became_peered":"2024-05-14T16:46:43.570605+0000","last_unstale":"2024-05-14T16:49:54.605874+0000","last_undegraded":"2024-05-14T16:49:54.605874+0000","last_fullsized":"2024-05-14T16:49:54.605874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:17:55.161195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.608013+0000","last_change":"2024-05-14T16:46:43.568442+0000","last_active":"2024-05-14T16:49:54.608013+0000","last_peered":"2024-05-14T16:49:54.608013+0000","last_clean":"2024-05-14T16:49:54.608013+0000","last_became_active":"2024-05-14T16:46:43.568118+0000","last_became_peered":"2024-05-14T16:46:43.568118+0000","last_unstale":"2024-05-14T16:49:54.608013+0000","last_undegraded":"2024-05-14T16:49:54.608013+0000","last_fullsized":"2024-05-14T16:49:54.608013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:52.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615674+0000","last_change":"2024-05-14T16:46:43.576174+0000","last_active":"2024-05-14T16:49:54.615674+0000","last_peered":"2024-05-14T16:49:54.615674+0000","last_clean":"2024-05-14T16:49:54.615674+0000","last_became_active":"2024-05-14T16:46:43.575737+0000","last_became_peered":"2024-05-14T16:46:43.575737+0000","last_unstale":"2024-05-14T16:49:54.615674+0000","last_undegraded":"2024-05-14T16:49:54.615674+0000","last_fullsized":"2024-05-14T16:49:54.615674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:08.453070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614955+0000","last_change":"2024-05-14T16:46:43.566256+0000","last_active":"2024-05-14T16:49:54.614955+0000","last_peered":"2024-05-14T16:49:54.614955+0000","last_clean":"2024-05-14T16:49:54.614955+0000","last_became_active":"2024-05-14T16:46:43.565886+0000","last_became_peered":"2024-05-14T16:46:43.565886+0000","last_unstale":"2024-05-14T16:49:54.614955+0000","last_undegraded":"2024-05-14T16:49:54.614955+0000","last_fullsized":"2024-05-14T16:49:54.614955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:50.021286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614405+0000","last_change":"2024-05-14T16:46:43.569819+0000","last_active":"2024-05-14T16:49:54.614405+0000","last_peered":"2024-05-14T16:49:54.614405+0000","last_clean":"2024-05-14T16:49:54.614405+0000","last_became_active":"2024-05-14T16:46:43.569461+0000","last_became_peered":"2024-05-14T16:46:43.569461+0000","last_unstale":"2024-05-14T16:49:54.614405+0000","last_undegraded":"2024-05-14T16:49:54.614405+0000","last_fullsized":"2024-05-14T16:49:54.614405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:52.013604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614556+0000","last_change":"2024-05-14T16:46:43.567569+0000","last_active":"2024-05-14T16:49:54.614556+0000","last_peered":"2024-05-14T16:49:54.614556+0000","last_clean":"2024-05-14T16:49:54.614556+0000","last_became_active":"2024-05-14T16:46:43.564808+0000","last_became_peered":"2024-05-14T16:46:43.564808+0000","last_unstale":"2024-05-14T16:49:54.614556+0000","last_undegraded":"2024-05-14T16:49:54.614556+0000","last_fullsized":"2024-05-14T16:49:54.614556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:13.744730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614465+0000","last_change":"2024-05-14T16:46:43.578417+0000","last_active":"2024-05-14T16:49:54.614465+0000","last_peered":"2024-05-14T16:49:54.614465+0000","last_clean":"2024-05-14T16:49:54.614465+0000","last_became_active":"2024-05-14T16:46:43.577687+0000","last_became_peered":"2024-05-14T16:46:43.577687+0000","last_unstale":"2024-05-14T16:49:54.614465+0000","last_undegraded":"2024-05-14T16:49:54.614465+0000","last_fullsized":"2024-05-14T16:49:54.614465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:32.060934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.606013+0000","last_change":"2024-05-14T16:46:43.564114+0000","last_active":"2024-05-14T16:49:54.606013+0000","last_peered":"2024-05-14T16:49:54.606013+0000","last_clean":"2024-05-14T16:49:54.606013+0000","last_became_active":"2024-05-14T16:46:43.563734+0000","last_became_peered":"2024-05-14T16:46:43.563734+0000","last_unstale":"2024-05-14T16:49:54.606013+0000","last_undegraded":"2024-05-14T16:49:54.606013+0000","last_fullsized":"2024-05-14T16:49:54.606013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:19.016741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607626+0000","last_change":"2024-05-14T16:46:43.566490+0000","last_active":"2024-05-14T16:49:54.607626+0000","last_peered":"2024-05-14T16:49:54.607626+0000","last_clean":"2024-05-14T16:49:54.607626+0000","last_became_active":"2024-05-14T16:46:43.566086+0000","last_became_peered":"2024-05-14T16:46:43.566086+0000","last_unstale":"2024-05-14T16:49:54.607626+0000","last_undegraded":"2024-05-14T16:49:54.607626+0000","last_fullsized":"2024-05-14T16:49:54.607626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:39:09.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107202,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596919,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1168,"num_read_kb":692,"num_write":295,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":728,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1790757,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710774,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":434319,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1056768,"data_stored":1224183,"data_compressed":50587,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070528,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":827625,"data_compressed":27843,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":827433,"data_compressed":27843,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":1026958,"data_compressed":44431,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626512,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":433798,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":500994,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":435069,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":502119,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":958925,"data_compressed":33999,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986265,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":888832,"data_stored":435516,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822740,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":1157543,"data_compressed":50587,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133467,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":393842,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201280,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:50:00.002 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:50:00.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-14T16:50:00.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-14T16:50:00.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-14T16:50:00.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Unable to set osd_memory_target on smithi069 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-14T16:50:00.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-14T16:50:00.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-14T16:50:00.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-14T16:50:00.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:00.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:50:00.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:00.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:00.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:00.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:49:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Unable to set osd_memory_target on smithi069 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:50:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:00.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:49:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-14T16:50:00.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Unable to set osd_memory_target on smithi069 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Updating smithi203:/etc/ceph/ceph.conf 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-14T16:50:00.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:00.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:00.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:00.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:49:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:50:00.257 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:00.380 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:50:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:50:00.095+0000 7f1296e36700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:50:00.380 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:50:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:50:00.095+0000 7f1296e36700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:50:00.380 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:50:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:50:00.095+0000 7f1296e36700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:50:00.380 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:50:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:50:00.095+0000 7f1296e36700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:50:00.380 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 16:50:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T16:50:00.095+0000 7f1296e36700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:50:00.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:00 smithi069 ceph-mon[31507]: from='client.24838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:50:00.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:00 smithi069 ceph-mon[31507]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:50:00.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:00 smithi069 ceph-mon[31507]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:50:00.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:00 smithi069 ceph-mon[31507]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:50:00.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:00 smithi069 ceph-mon[31507]: overall HEALTH_OK 2024-05-14T16:50:01.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:00 smithi203 ceph-mon[35787]: from='client.24838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:50:01.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:00 smithi203 ceph-mon[35787]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:50:01.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:00 smithi203 ceph-mon[35787]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:50:01.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:00 smithi203 ceph-mon[35787]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:50:01.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:00 smithi203 ceph-mon[35787]: overall HEALTH_OK 2024-05-14T16:50:01.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:00 smithi092 ceph-mon[35763]: from='client.24838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:50:01.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:00 smithi092 ceph-mon[35763]: Updating smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:50:01.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:00 smithi092 ceph-mon[35763]: Updating smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:50:01.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:00 smithi092 ceph-mon[35763]: Updating smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:50:01.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:00 smithi092 ceph-mon[35763]: overall HEALTH_OK 2024-05-14T16:50:01.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:01 smithi069 ceph-mon[31507]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 115 KiB/s wr, 98 op/s 2024-05-14T16:50:01.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:01 smithi069 ceph-mon[31507]: mgrmap e25: x(active, since 6s), standbys: z, y 2024-05-14T16:50:02.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 -- 172.21.15.69:0/604517743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52f406c650 msgr2=0x7f52f41103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:02.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 --2- 172.21.15.69:0/604517743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52f406c650 0x7f52f41103e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00b0d0 tx=0x7f52ec00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 -- 172.21.15.69:0/604517743 shutdown_connections 2024-05-14T16:50:02.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 --2- 172.21.15.69:0/604517743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52f406c650 0x7f52f41103e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 --2- 172.21.15.69:0/604517743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f4109650 0x7f52f4109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 --2- 172.21.15.69:0/604517743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52f406bc90 0x7f52f406c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 -- 172.21.15.69:0/604517743 >> 172.21.15.69:0/604517743 conn(0x7f52f4079330 msgr2=0x7f52f4079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 -- 172.21.15.69:0/604517743 shutdown_connections 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 -- 172.21.15.69:0/604517743 wait complete. 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 Processor -- start 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.103+0000 7f52faf3c700 1 -- start start 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52faf3c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52f406bc90 0x7f52f419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52faf3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f406c650 0x7f52f419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52faf3c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52f4109650 0x7f52f4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52faf3c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52f4113770 con 0x7f52f406bc90 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52faf3c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f52f41135f0 con 0x7f52f406c650 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52faf3c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f52f41138f0 con 0x7f52f4109650 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f8cd8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52f406bc90 0x7f52f419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f8cd8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52f406bc90 0x7f52f419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44876/0 (socket says 172.21.15.69:44876) 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f8cd8700 1 -- 172.21.15.69:0/320533643 learned_addr learned my addr 172.21.15.69:0/320533643 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f94d9700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52f4109650 0x7f52f4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f3fff700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f406c650 0x7f52f419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f8cd8700 1 -- 172.21.15.69:0/320533643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52f4109650 msgr2=0x7f52f4199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f8cd8700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52f4109650 0x7f52f4199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f8cd8700 1 -- 172.21.15.69:0/320533643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f406c650 msgr2=0x7f52f419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f8cd8700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f406c650 0x7f52f419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f8cd8700 1 -- 172.21.15.69:0/320533643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f419a0f0 con 0x7f52f406bc90 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f8cd8700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52f406bc90 0x7f52f419f780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c920 tx=0x7f52e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f1ffb700 1 -- 172.21.15.69:0/320533643 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8007b00 con 0x7f52f406bc90 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f1ffb700 1 -- 172.21.15.69:0/320533643 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e8004ce0 con 0x7f52f406bc90 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.104+0000 7f52f1ffb700 1 -- 172.21.15.69:0/320533643 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8005550 con 0x7f52f406bc90 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.105+0000 7f52faf3c700 1 -- 172.21.15.69:0/320533643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f419a3e0 con 0x7f52f406bc90 2024-05-14T16:50:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.105+0000 7f52faf3c700 1 -- 172.21.15.69:0/320533643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f41a6650 con 0x7f52f406bc90 2024-05-14T16:50:02.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.106+0000 7f52faf3c700 1 -- 172.21.15.69:0/320533643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f4104c80 con 0x7f52f406bc90 2024-05-14T16:50:02.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.107+0000 7f52f1ffb700 1 -- 172.21.15.69:0/320533643 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f52e8007ca0 con 0x7f52f406bc90 2024-05-14T16:50:02.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.107+0000 7f52f1ffb700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e40b2b10 0x7f52e40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:02.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.108+0000 7f52f3fff700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e40b2b10 0x7f52e40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:02.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.108+0000 7f52f1ffb700 1 -- 172.21.15.69:0/320533643 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f52e80e27c0 con 0x7f52f406bc90 2024-05-14T16:50:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.108+0000 7f52f3fff700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e40b2b10 0x7f52e40b4fd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005950 tx=0x7f52e00058e0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:02.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.112+0000 7f52f1ffb700 1 -- 172.21.15.69:0/320533643 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52e8014030 con 0x7f52f406bc90 2024-05-14T16:50:02.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:01 smithi203 ceph-mon[35787]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 115 KiB/s wr, 98 op/s 2024-05-14T16:50:02.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:01 smithi203 ceph-mon[35787]: mgrmap e25: x(active, since 6s), standbys: z, y 2024-05-14T16:50:02.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:01 smithi092 ceph-mon[35763]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 115 KiB/s wr, 98 op/s 2024-05-14T16:50:02.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:01 smithi092 ceph-mon[35763]: mgrmap e25: x(active, since 6s), standbys: z, y 2024-05-14T16:50:02.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.314+0000 7f52faf3c700 1 -- 172.21.15.69:0/320533643 --> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52f41017e0 con 0x7f52e40b2b10 2024-05-14T16:50:02.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.322+0000 7f52f1ffb700 1 -- 172.21.15.69:0/320533643 <== mgr.24764 v2:172.21.15.69:6800/4217409403 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258048 (secure 0 0 0) 0x7f52f41017e0 con 0x7f52e40b2b10 2024-05-14T16:50:02.325 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:02.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 -- 172.21.15.69:0/320533643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e40b2b10 msgr2=0x7f52e40b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:02.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e40b2b10 0x7f52e40b4fd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005950 tx=0x7f52e00058e0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 -- 172.21.15.69:0/320533643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52f406bc90 msgr2=0x7f52f419f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52f406bc90 0x7f52f419f780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c920 tx=0x7f52e800cc30 comp rx=0 tx=0).stop 2024-05-14T16:50:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 -- 172.21.15.69:0/320533643 shutdown_connections 2024-05-14T16:50:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e40b2b10 0x7f52e40b4fd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52f406bc90 0x7f52f419f780 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52f4109650 0x7f52f4199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 --2- 172.21.15.69:0/320533643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f406c650 0x7f52f419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 -- 172.21.15.69:0/320533643 >> 172.21.15.69:0/320533643 conn(0x7f52f4079330 msgr2=0x7f52f41010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 -- 172.21.15.69:0/320533643 shutdown_connections 2024-05-14T16:50:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:02.326+0000 7f52faf3c700 1 -- 172.21.15.69:0/320533643 wait complete. 2024-05-14T16:50:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-05-14T16:50:03.008 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-14T16:50:00.622022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187521,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1594,"num_read_kb":1072,"num_write":565,"num_write_kb":4166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952568,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":900,"ondisk_log_size":900,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351864,"kb_used_data":10980,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589960,"statfs":{"total":1151940427776,"available":1151580119040,"internally_reserved":0,"allocated":11243520,"data_stored":8961114,"data_compressed":296818,"data_compressed_allocated":393216,"data_compressed_original":3796992,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":345,"num_read_kb":311,"num_write":145,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.990461"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607961+0000","last_change":"2024-05-14T16:46:43.572738+0000","last_active":"2024-05-14T16:49:54.607961+0000","last_peered":"2024-05-14T16:49:54.607961+0000","last_clean":"2024-05-14T16:49:54.607961+0000","last_became_active":"2024-05-14T16:46:43.572400+0000","last_became_peered":"2024-05-14T16:46:43.572400+0000","last_unstale":"2024-05-14T16:49:54.607961+0000","last_undegraded":"2024-05-14T16:49:54.607961+0000","last_fullsized":"2024-05-14T16:49:54.607961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:44.332862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615350+0000","last_change":"2024-05-14T16:46:43.566193+0000","last_active":"2024-05-14T16:49:54.615350+0000","last_peered":"2024-05-14T16:49:54.615350+0000","last_clean":"2024-05-14T16:49:54.615350+0000","last_became_active":"2024-05-14T16:46:43.565853+0000","last_became_peered":"2024-05-14T16:46:43.565853+0000","last_unstale":"2024-05-14T16:49:54.615350+0000","last_undegraded":"2024-05-14T16:49:54.615350+0000","last_fullsized":"2024-05-14T16:49:54.615350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:19:34.481225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615237+0000","last_change":"2024-05-14T16:46:43.574937+0000","last_active":"2024-05-14T16:49:54.615237+0000","last_peered":"2024-05-14T16:49:54.615237+0000","last_clean":"2024-05-14T16:49:54.615237+0000","last_became_active":"2024-05-14T16:46:43.574317+0000","last_became_peered":"2024-05-14T16:46:43.574317+0000","last_unstale":"2024-05-14T16:49:54.615237+0000","last_undegraded":"2024-05-14T16:49:54.615237+0000","last_fullsized":"2024-05-14T16:49:54.615237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:35:49.933658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614517+0000","last_change":"2024-05-14T16:47:33.711809+0000","last_active":"2024-05-14T16:49:54.614517+0000","last_peered":"2024-05-14T16:49:54.614517+0000","last_clean":"2024-05-14T16:49:54.614517+0000","last_became_active":"2024-05-14T16:47:33.711422+0000","last_became_peered":"2024-05-14T16:47:33.711422+0000","last_unstale":"2024-05-14T16:49:54.614517+0000","last_undegraded":"2024-05-14T16:49:54.614517+0000","last_fullsized":"2024-05-14T16:49:54.614517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:44:26.005189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616108+0000","last_change":"2024-05-14T16:46:43.567104+0000","last_active":"2024-05-14T16:49:54.616108+0000","last_peered":"2024-05-14T16:49:54.616108+0000","last_clean":"2024-05-14T16:49:54.616108+0000","last_became_active":"2024-05-14T16:46:43.566762+0000","last_became_peered":"2024-05-14T16:46:43.566762+0000","last_unstale":"2024-05-14T16:49:54.616108+0000","last_undegraded":"2024-05-14T16:49:54.616108+0000","last_fullsized":"2024-05-14T16:49:54.616108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:08.819856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607201+0000","last_change":"2024-05-14T16:46:43.565055+0000","last_active":"2024-05-14T16:49:54.607201+0000","last_peered":"2024-05-14T16:49:54.607201+0000","last_clean":"2024-05-14T16:49:54.607201+0000","last_became_active":"2024-05-14T16:46:43.564699+0000","last_became_peered":"2024-05-14T16:46:43.564699+0000","last_unstale":"2024-05-14T16:49:54.607201+0000","last_undegraded":"2024-05-14T16:49:54.607201+0000","last_fullsized":"2024-05-14T16:49:54.607201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:01.903025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.835993+0000","last_change":"2024-05-14T16:47:33.711866+0000","last_active":"2024-05-14T16:49:54.835993+0000","last_peered":"2024-05-14T16:49:54.835993+0000","last_clean":"2024-05-14T16:49:54.835993+0000","last_became_active":"2024-05-14T16:47:33.711481+0000","last_became_peered":"2024-05-14T16:47:33.711481+0000","last_unstale":"2024-05-14T16:49:54.835993+0000","last_undegraded":"2024-05-14T16:49:54.835993+0000","last_fullsized":"2024-05-14T16:49:54.835993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:01:22.724678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614120+0000","last_change":"2024-05-14T16:46:43.564470+0000","last_active":"2024-05-14T16:49:54.614120+0000","last_peered":"2024-05-14T16:49:54.614120+0000","last_clean":"2024-05-14T16:49:54.614120+0000","last_became_active":"2024-05-14T16:46:43.564169+0000","last_became_peered":"2024-05-14T16:46:43.564169+0000","last_unstale":"2024-05-14T16:49:54.614120+0000","last_undegraded":"2024-05-14T16:49:54.614120+0000","last_fullsized":"2024-05-14T16:49:54.614120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:38.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614961+0000","last_change":"2024-05-14T16:46:43.580988+0000","last_active":"2024-05-14T16:49:54.614961+0000","last_peered":"2024-05-14T16:49:54.614961+0000","last_clean":"2024-05-14T16:49:54.614961+0000","last_became_active":"2024-05-14T16:46:43.580657+0000","last_became_peered":"2024-05-14T16:46:43.580657+0000","last_unstale":"2024-05-14T16:49:54.614961+0000","last_undegraded":"2024-05-14T16:49:54.614961+0000","last_fullsized":"2024-05-14T16:49:54.614961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:39:01.568822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607883+0000","last_change":"2024-05-14T16:46:43.565742+0000","last_active":"2024-05-14T16:49:54.607883+0000","last_peered":"2024-05-14T16:49:54.607883+0000","last_clean":"2024-05-14T16:49:54.607883+0000","last_became_active":"2024-05-14T16:46:43.565260+0000","last_became_peered":"2024-05-14T16:46:43.565260+0000","last_unstale":"2024-05-14T16:49:54.607883+0000","last_undegraded":"2024-05-14T16:49:54.607883+0000","last_fullsized":"2024-05-14T16:49:54.607883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:25.718174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836633+0000","last_change":"2024-05-14T16:46:43.576268+0000","last_active":"2024-05-14T16:49:54.836633+0000","last_peered":"2024-05-14T16:49:54.836633+0000","last_clean":"2024-05-14T16:49:54.836633+0000","last_became_active":"2024-05-14T16:46:43.575856+0000","last_became_peered":"2024-05-14T16:46:43.575856+0000","last_unstale":"2024-05-14T16:49:54.836633+0000","last_undegraded":"2024-05-14T16:49:54.836633+0000","last_fullsized":"2024-05-14T16:49:54.836633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:12:16.549127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614667+0000","last_change":"2024-05-14T16:46:43.573315+0000","last_active":"2024-05-14T16:49:54.614667+0000","last_peered":"2024-05-14T16:49:54.614667+0000","last_clean":"2024-05-14T16:49:54.614667+0000","last_became_active":"2024-05-14T16:46:43.572981+0000","last_became_peered":"2024-05-14T16:46:43.572981+0000","last_unstale":"2024-05-14T16:49:54.614667+0000","last_undegraded":"2024-05-14T16:49:54.614667+0000","last_fullsized":"2024-05-14T16:49:54.614667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:07.757539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605825+0000","last_change":"2024-05-14T16:46:43.569388+0000","last_active":"2024-05-14T16:49:54.605825+0000","last_peered":"2024-05-14T16:49:54.605825+0000","last_clean":"2024-05-14T16:49:54.605825+0000","last_became_active":"2024-05-14T16:46:43.568992+0000","last_became_peered":"2024-05-14T16:46:43.568992+0000","last_unstale":"2024-05-14T16:49:54.605825+0000","last_undegraded":"2024-05-14T16:49:54.605825+0000","last_fullsized":"2024-05-14T16:49:54.605825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:08:06.236797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615319+0000","last_change":"2024-05-14T16:46:43.564137+0000","last_active":"2024-05-14T16:49:54.615319+0000","last_peered":"2024-05-14T16:49:54.615319+0000","last_clean":"2024-05-14T16:49:54.615319+0000","last_became_active":"2024-05-14T16:46:43.563831+0000","last_became_peered":"2024-05-14T16:46:43.563831+0000","last_unstale":"2024-05-14T16:49:54.615319+0000","last_undegraded":"2024-05-14T16:49:54.615319+0000","last_fullsized":"2024-05-14T16:49:54.615319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:58.473863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615197+0000","last_change":"2024-05-14T16:46:43.571789+0000","last_active":"2024-05-14T16:49:54.615197+0000","last_peered":"2024-05-14T16:49:54.615197+0000","last_clean":"2024-05-14T16:49:54.615197+0000","last_became_active":"2024-05-14T16:46:43.571521+0000","last_became_peered":"2024-05-14T16:46:43.571521+0000","last_unstale":"2024-05-14T16:49:54.615197+0000","last_undegraded":"2024-05-14T16:49:54.615197+0000","last_fullsized":"2024-05-14T16:49:54.615197+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:23.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614202+0000","last_change":"2024-05-14T16:46:43.569532+0000","last_active":"2024-05-14T16:49:54.614202+0000","last_peered":"2024-05-14T16:49:54.614202+0000","last_clean":"2024-05-14T16:49:54.614202+0000","last_became_active":"2024-05-14T16:46:43.569132+0000","last_became_peered":"2024-05-14T16:46:43.569132+0000","last_unstale":"2024-05-14T16:49:54.614202+0000","last_undegraded":"2024-05-14T16:49:54.614202+0000","last_fullsized":"2024-05-14T16:49:54.614202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:09.328241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.608089+0000","last_change":"2024-05-14T16:46:43.574451+0000","last_active":"2024-05-14T16:49:54.608089+0000","last_peered":"2024-05-14T16:49:54.608089+0000","last_clean":"2024-05-14T16:49:54.608089+0000","last_became_active":"2024-05-14T16:46:43.574143+0000","last_became_peered":"2024-05-14T16:46:43.574143+0000","last_unstale":"2024-05-14T16:49:54.608089+0000","last_undegraded":"2024-05-14T16:49:54.608089+0000","last_fullsized":"2024-05-14T16:49:54.608089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:01:42.831262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614829+0000","last_change":"2024-05-14T16:46:43.568799+0000","last_active":"2024-05-14T16:49:54.614829+0000","last_peered":"2024-05-14T16:49:54.614829+0000","last_clean":"2024-05-14T16:49:54.614829+0000","last_became_active":"2024-05-14T16:46:43.568467+0000","last_became_peered":"2024-05-14T16:46:43.568467+0000","last_unstale":"2024-05-14T16:49:54.614829+0000","last_undegraded":"2024-05-14T16:49:54.614829+0000","last_fullsized":"2024-05-14T16:49:54.614829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:59.036372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615487+0000","last_change":"2024-05-14T16:46:43.566879+0000","last_active":"2024-05-14T16:49:54.615487+0000","last_peered":"2024-05-14T16:49:54.615487+0000","last_clean":"2024-05-14T16:49:54.615487+0000","last_became_active":"2024-05-14T16:46:43.566346+0000","last_became_peered":"2024-05-14T16:46:43.566346+0000","last_unstale":"2024-05-14T16:49:54.615487+0000","last_undegraded":"2024-05-14T16:49:54.615487+0000","last_fullsized":"2024-05-14T16:49:54.615487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:11.027173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614436+0000","last_change":"2024-05-14T16:46:40.546990+0000","last_active":"2024-05-14T16:49:54.614436+0000","last_peered":"2024-05-14T16:49:54.614436+0000","last_clean":"2024-05-14T16:49:54.614436+0000","last_became_active":"2024-05-14T16:46:40.546646+0000","last_became_peered":"2024-05-14T16:46:40.546646+0000","last_unstale":"2024-05-14T16:49:54.614436+0000","last_undegraded":"2024-05-14T16:49:54.614436+0000","last_fullsized":"2024-05-14T16:49:54.614436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:47.001760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614919+0000","last_change":"2024-05-14T16:46:43.570352+0000","last_active":"2024-05-14T16:49:54.614919+0000","last_peered":"2024-05-14T16:49:54.614919+0000","last_clean":"2024-05-14T16:49:54.614919+0000","last_became_active":"2024-05-14T16:46:43.569748+0000","last_became_peered":"2024-05-14T16:46:43.569748+0000","last_unstale":"2024-05-14T16:49:54.614919+0000","last_undegraded":"2024-05-14T16:49:54.614919+0000","last_fullsized":"2024-05-14T16:49:54.614919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:24.707586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615028+0000","last_change":"2024-05-14T16:46:40.855034+0000","last_active":"2024-05-14T16:49:54.615028+0000","last_peered":"2024-05-14T16:49:54.615028+0000","last_clean":"2024-05-14T16:49:54.615028+0000","last_became_active":"2024-05-14T16:46:40.854686+0000","last_became_peered":"2024-05-14T16:46:40.854686+0000","last_unstale":"2024-05-14T16:49:54.615028+0000","last_undegraded":"2024-05-14T16:49:54.615028+0000","last_fullsized":"2024-05-14T16:49:54.615028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:56.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615572+0000","last_change":"2024-05-14T16:46:43.574682+0000","last_active":"2024-05-14T16:49:54.615572+0000","last_peered":"2024-05-14T16:49:54.615572+0000","last_clean":"2024-05-14T16:49:54.615572+0000","last_became_active":"2024-05-14T16:46:43.574322+0000","last_became_peered":"2024-05-14T16:46:43.574322+0000","last_unstale":"2024-05-14T16:49:54.615572+0000","last_undegraded":"2024-05-14T16:49:54.615572+0000","last_fullsized":"2024-05-14T16:49:54.615572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:20.169428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616589+0000","last_change":"2024-05-14T16:46:40.551307+0000","last_active":"2024-05-14T16:49:54.616589+0000","last_peered":"2024-05-14T16:49:54.616589+0000","last_clean":"2024-05-14T16:49:54.616589+0000","last_became_active":"2024-05-14T16:46:40.550987+0000","last_became_peered":"2024-05-14T16:46:40.550987+0000","last_unstale":"2024-05-14T16:49:54.616589+0000","last_undegraded":"2024-05-14T16:49:54.616589+0000","last_fullsized":"2024-05-14T16:49:54.616589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:27.819457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614524+0000","last_change":"2024-05-14T16:46:43.575200+0000","last_active":"2024-05-14T16:49:54.614524+0000","last_peered":"2024-05-14T16:49:54.614524+0000","last_clean":"2024-05-14T16:49:54.614524+0000","last_became_active":"2024-05-14T16:46:43.574864+0000","last_became_peered":"2024-05-14T16:46:43.574864+0000","last_unstale":"2024-05-14T16:49:54.614524+0000","last_undegraded":"2024-05-14T16:49:54.614524+0000","last_fullsized":"2024-05-14T16:49:54.614524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:48.630101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616383+0000","last_change":"2024-05-14T16:46:40.548397+0000","last_active":"2024-05-14T16:49:54.616383+0000","last_peered":"2024-05-14T16:49:54.616383+0000","last_clean":"2024-05-14T16:49:54.616383+0000","last_became_active":"2024-05-14T16:46:40.547803+0000","last_became_peered":"2024-05-14T16:46:40.547803+0000","last_unstale":"2024-05-14T16:49:54.616383+0000","last_undegraded":"2024-05-14T16:49:54.616383+0000","last_fullsized":"2024-05-14T16:49:54.616383+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:21:19.812413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615180+0000","last_change":"2024-05-14T16:46:43.578301+0000","last_active":"2024-05-14T16:49:54.615180+0000","last_peered":"2024-05-14T16:49:54.615180+0000","last_clean":"2024-05-14T16:49:54.615180+0000","last_became_active":"2024-05-14T16:46:43.577796+0000","last_became_peered":"2024-05-14T16:46:43.577796+0000","last_unstale":"2024-05-14T16:49:54.615180+0000","last_undegraded":"2024-05-14T16:49:54.615180+0000","last_fullsized":"2024-05-14T16:49:54.615180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:13.159570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":211,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.870130+0000","last_change":"2024-05-14T16:46:40.547818+0000","last_active":"2024-05-14T16:49:54.870130+0000","last_peered":"2024-05-14T16:49:54.870130+0000","last_clean":"2024-05-14T16:49:54.870130+0000","last_became_active":"2024-05-14T16:46:40.547454+0000","last_became_peered":"2024-05-14T16:46:40.547454+0000","last_unstale":"2024-05-14T16:49:54.870130+0000","last_undegraded":"2024-05-14T16:49:54.870130+0000","last_fullsized":"2024-05-14T16:49:54.870130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:16.362415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616051+0000","last_change":"2024-05-14T16:46:43.570743+0000","last_active":"2024-05-14T16:49:54.616051+0000","last_peered":"2024-05-14T16:49:54.616051+0000","last_clean":"2024-05-14T16:49:54.616051+0000","last_became_active":"2024-05-14T16:46:43.570427+0000","last_became_peered":"2024-05-14T16:46:43.570427+0000","last_unstale":"2024-05-14T16:49:54.616051+0000","last_undegraded":"2024-05-14T16:49:54.616051+0000","last_fullsized":"2024-05-14T16:49:54.616051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:04:11.091285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607185+0000","last_change":"2024-05-14T16:46:40.546675+0000","last_active":"2024-05-14T16:49:54.607185+0000","last_peered":"2024-05-14T16:49:54.607185+0000","last_clean":"2024-05-14T16:49:54.607185+0000","last_became_active":"2024-05-14T16:46:40.546250+0000","last_became_peered":"2024-05-14T16:46:40.546250+0000","last_unstale":"2024-05-14T16:49:54.607185+0000","last_undegraded":"2024-05-14T16:49:54.607185+0000","last_fullsized":"2024-05-14T16:49:54.607185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:42:52.367719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614882+0000","last_change":"2024-05-14T16:46:43.579119+0000","last_active":"2024-05-14T16:49:54.614882+0000","last_peered":"2024-05-14T16:49:54.614882+0000","last_clean":"2024-05-14T16:49:54.614882+0000","last_became_active":"2024-05-14T16:46:43.578808+0000","last_became_peered":"2024-05-14T16:46:43.578808+0000","last_unstale":"2024-05-14T16:49:54.614882+0000","last_undegraded":"2024-05-14T16:49:54.614882+0000","last_fullsized":"2024-05-14T16:49:54.614882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:57.852982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605792+0000","last_change":"2024-05-14T16:46:40.538827+0000","last_active":"2024-05-14T16:49:54.605792+0000","last_peered":"2024-05-14T16:49:54.605792+0000","last_clean":"2024-05-14T16:49:54.605792+0000","last_became_active":"2024-05-14T16:46:40.538369+0000","last_became_peered":"2024-05-14T16:46:40.538369+0000","last_unstale":"2024-05-14T16:49:54.605792+0000","last_undegraded":"2024-05-14T16:49:54.605792+0000","last_fullsized":"2024-05-14T16:49:54.605792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:48:03.599457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614307+0000","last_change":"2024-05-14T16:46:43.558411+0000","last_active":"2024-05-14T16:49:54.614307+0000","last_peered":"2024-05-14T16:49:54.614307+0000","last_clean":"2024-05-14T16:49:54.614307+0000","last_became_active":"2024-05-14T16:46:43.558101+0000","last_became_peered":"2024-05-14T16:46:43.558101+0000","last_unstale":"2024-05-14T16:49:54.614307+0000","last_undegraded":"2024-05-14T16:49:54.614307+0000","last_fullsized":"2024-05-14T16:49:54.614307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:10.595833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614356+0000","last_change":"2024-05-14T16:46:40.544858+0000","last_active":"2024-05-14T16:49:54.614356+0000","last_peered":"2024-05-14T16:49:54.614356+0000","last_clean":"2024-05-14T16:49:54.614356+0000","last_became_active":"2024-05-14T16:46:40.544197+0000","last_became_peered":"2024-05-14T16:46:40.544197+0000","last_unstale":"2024-05-14T16:49:54.614356+0000","last_undegraded":"2024-05-14T16:49:54.614356+0000","last_fullsized":"2024-05-14T16:49:54.614356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:16:45.249626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614048+0000","last_change":"2024-05-14T16:46:43.567019+0000","last_active":"2024-05-14T16:49:54.614048+0000","last_peered":"2024-05-14T16:49:54.614048+0000","last_clean":"2024-05-14T16:49:54.614048+0000","last_became_active":"2024-05-14T16:46:43.566700+0000","last_became_peered":"2024-05-14T16:46:43.566700+0000","last_unstale":"2024-05-14T16:49:54.614048+0000","last_undegraded":"2024-05-14T16:49:54.614048+0000","last_fullsized":"2024-05-14T16:49:54.614048+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:43.373467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":207,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.839019+0000","last_change":"2024-05-14T16:46:40.541876+0000","last_active":"2024-05-14T16:49:54.839019+0000","last_peered":"2024-05-14T16:49:54.839019+0000","last_clean":"2024-05-14T16:49:54.839019+0000","last_became_active":"2024-05-14T16:46:40.541460+0000","last_became_peered":"2024-05-14T16:46:40.541460+0000","last_unstale":"2024-05-14T16:49:54.839019+0000","last_undegraded":"2024-05-14T16:49:54.839019+0000","last_fullsized":"2024-05-14T16:49:54.839019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:07:22.536429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614958+0000","last_change":"2024-05-14T16:46:43.577015+0000","last_active":"2024-05-14T16:49:54.614958+0000","last_peered":"2024-05-14T16:49:54.614958+0000","last_clean":"2024-05-14T16:49:54.614958+0000","last_became_active":"2024-05-14T16:46:43.576657+0000","last_became_peered":"2024-05-14T16:46:43.576657+0000","last_unstale":"2024-05-14T16:49:54.614958+0000","last_undegraded":"2024-05-14T16:49:54.614958+0000","last_fullsized":"2024-05-14T16:49:54.614958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:29.242445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836348+0000","last_change":"2024-05-14T16:46:40.550727+0000","last_active":"2024-05-14T16:49:54.836348+0000","last_peered":"2024-05-14T16:49:54.836348+0000","last_clean":"2024-05-14T16:49:54.836348+0000","last_became_active":"2024-05-14T16:46:40.550310+0000","last_became_peered":"2024-05-14T16:46:40.550310+0000","last_unstale":"2024-05-14T16:49:54.836348+0000","last_undegraded":"2024-05-14T16:49:54.836348+0000","last_fullsized":"2024-05-14T16:49:54.836348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:04.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615580+0000","last_change":"2024-05-14T16:47:33.711766+0000","last_active":"2024-05-14T16:49:54.615580+0000","last_peered":"2024-05-14T16:49:54.615580+0000","last_clean":"2024-05-14T16:49:54.615580+0000","last_became_active":"2024-05-14T16:47:33.711352+0000","last_became_peered":"2024-05-14T16:47:33.711352+0000","last_unstale":"2024-05-14T16:49:54.615580+0000","last_undegraded":"2024-05-14T16:49:54.615580+0000","last_fullsized":"2024-05-14T16:49:54.615580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:47.924117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615228+0000","last_change":"2024-05-14T16:46:40.855680+0000","last_active":"2024-05-14T16:49:54.615228+0000","last_peered":"2024-05-14T16:49:54.615228+0000","last_clean":"2024-05-14T16:49:54.615228+0000","last_became_active":"2024-05-14T16:46:40.855265+0000","last_became_peered":"2024-05-14T16:46:40.855265+0000","last_unstale":"2024-05-14T16:49:54.615228+0000","last_undegraded":"2024-05-14T16:49:54.615228+0000","last_fullsized":"2024-05-14T16:49:54.615228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:26.065298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614667+0000","last_change":"2024-05-14T16:46:43.579423+0000","last_active":"2024-05-14T16:49:54.614667+0000","last_peered":"2024-05-14T16:49:54.614667+0000","last_clean":"2024-05-14T16:49:54.614667+0000","last_became_active":"2024-05-14T16:46:43.579023+0000","last_became_peered":"2024-05-14T16:46:43.579023+0000","last_unstale":"2024-05-14T16:49:54.614667+0000","last_undegraded":"2024-05-14T16:49:54.614667+0000","last_fullsized":"2024-05-14T16:49:54.614667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:21.311018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'38","reported_seq":428,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.870707+0000","last_change":"2024-05-14T16:46:40.550871+0000","last_active":"2024-05-14T16:49:54.870707+0000","last_peered":"2024-05-14T16:49:54.870707+0000","last_clean":"2024-05-14T16:49:54.870707+0000","last_became_active":"2024-05-14T16:46:40.550419+0000","last_became_peered":"2024-05-14T16:46:40.550419+0000","last_unstale":"2024-05-14T16:49:54.870707+0000","last_undegraded":"2024-05-14T16:49:54.870707+0000","last_fullsized":"2024-05-14T16:49:54.870707+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:53:09.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67403,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":246,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616462+0000","last_change":"2024-05-14T16:46:43.569996+0000","last_active":"2024-05-14T16:49:54.616462+0000","last_peered":"2024-05-14T16:49:54.616462+0000","last_clean":"2024-05-14T16:49:54.616462+0000","last_became_active":"2024-05-14T16:46:43.569501+0000","last_became_peered":"2024-05-14T16:46:43.569501+0000","last_unstale":"2024-05-14T16:49:54.616462+0000","last_undegraded":"2024-05-14T16:49:54.616462+0000","last_fullsized":"2024-05-14T16:49:54.616462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:55.135103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607804+0000","last_change":"2024-05-14T16:47:33.733316+0000","last_active":"2024-05-14T16:49:54.607804+0000","last_peered":"2024-05-14T16:49:54.607804+0000","last_clean":"2024-05-14T16:49:54.607804+0000","last_became_active":"2024-05-14T16:47:33.709886+0000","last_became_peered":"2024-05-14T16:47:33.709886+0000","last_unstale":"2024-05-14T16:49:54.607804+0000","last_undegraded":"2024-05-14T16:49:54.607804+0000","last_fullsized":"2024-05-14T16:49:54.607804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:00:47.464686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605666+0000","last_change":"2024-05-14T16:46:43.562773+0000","last_active":"2024-05-14T16:49:54.605666+0000","last_peered":"2024-05-14T16:49:54.605666+0000","last_clean":"2024-05-14T16:49:54.605666+0000","last_became_active":"2024-05-14T16:46:43.562449+0000","last_became_peered":"2024-05-14T16:46:43.562449+0000","last_unstale":"2024-05-14T16:49:54.605666+0000","last_undegraded":"2024-05-14T16:49:54.605666+0000","last_fullsized":"2024-05-14T16:49:54.605666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:45.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614731+0000","last_change":"2024-05-14T16:46:40.545882+0000","last_active":"2024-05-14T16:49:54.614731+0000","last_peered":"2024-05-14T16:49:54.614731+0000","last_clean":"2024-05-14T16:49:54.614731+0000","last_became_active":"2024-05-14T16:46:40.545439+0000","last_became_peered":"2024-05-14T16:46:40.545439+0000","last_unstale":"2024-05-14T16:49:54.614731+0000","last_undegraded":"2024-05-14T16:49:54.614731+0000","last_fullsized":"2024-05-14T16:49:54.614731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:07.983329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836397+0000","last_change":"2024-05-14T16:46:43.568109+0000","last_active":"2024-05-14T16:49:54.836397+0000","last_peered":"2024-05-14T16:49:54.836397+0000","last_clean":"2024-05-14T16:49:54.836397+0000","last_became_active":"2024-05-14T16:46:43.567106+0000","last_became_peered":"2024-05-14T16:46:43.567106+0000","last_unstale":"2024-05-14T16:49:54.836397+0000","last_undegraded":"2024-05-14T16:49:54.836397+0000","last_fullsized":"2024-05-14T16:49:54.836397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:29.705387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":212,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.870503+0000","last_change":"2024-05-14T16:46:40.542204+0000","last_active":"2024-05-14T16:49:54.870503+0000","last_peered":"2024-05-14T16:49:54.870503+0000","last_clean":"2024-05-14T16:49:54.870503+0000","last_became_active":"2024-05-14T16:46:40.541807+0000","last_became_peered":"2024-05-14T16:46:40.541807+0000","last_unstale":"2024-05-14T16:49:54.870503+0000","last_undegraded":"2024-05-14T16:49:54.870503+0000","last_fullsized":"2024-05-14T16:49:54.870503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:30:45.115647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836336+0000","last_change":"2024-05-14T16:46:43.573546+0000","last_active":"2024-05-14T16:49:54.836336+0000","last_peered":"2024-05-14T16:49:54.836336+0000","last_clean":"2024-05-14T16:49:54.836336+0000","last_became_active":"2024-05-14T16:46:43.573183+0000","last_became_peered":"2024-05-14T16:46:43.573183+0000","last_unstale":"2024-05-14T16:49:54.836336+0000","last_undegraded":"2024-05-14T16:49:54.836336+0000","last_fullsized":"2024-05-14T16:49:54.836336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:49.873097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607658+0000","last_change":"2024-05-14T16:47:33.726700+0000","last_active":"2024-05-14T16:49:54.607658+0000","last_peered":"2024-05-14T16:49:54.607658+0000","last_clean":"2024-05-14T16:49:54.607658+0000","last_became_active":"2024-05-14T16:47:33.709297+0000","last_became_peered":"2024-05-14T16:47:33.709297+0000","last_unstale":"2024-05-14T16:49:54.607658+0000","last_undegraded":"2024-05-14T16:49:54.607658+0000","last_fullsized":"2024-05-14T16:49:54.607658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:35.028197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614569+0000","last_change":"2024-05-14T16:46:43.579059+0000","last_active":"2024-05-14T16:49:54.614569+0000","last_peered":"2024-05-14T16:49:54.614569+0000","last_clean":"2024-05-14T16:49:54.614569+0000","last_became_active":"2024-05-14T16:46:43.578748+0000","last_became_peered":"2024-05-14T16:46:43.578748+0000","last_unstale":"2024-05-14T16:49:54.614569+0000","last_undegraded":"2024-05-14T16:49:54.614569+0000","last_fullsized":"2024-05-14T16:49:54.614569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:42.446422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616551+0000","last_change":"2024-05-14T16:46:40.549047+0000","last_active":"2024-05-14T16:49:54.616551+0000","last_peered":"2024-05-14T16:49:54.616551+0000","last_clean":"2024-05-14T16:49:54.616551+0000","last_became_active":"2024-05-14T16:46:40.548648+0000","last_became_peered":"2024-05-14T16:46:40.548648+0000","last_unstale":"2024-05-14T16:49:54.616551+0000","last_undegraded":"2024-05-14T16:49:54.616551+0000","last_fullsized":"2024-05-14T16:49:54.616551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:52:56.370871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607790+0000","last_change":"2024-05-14T16:46:43.569624+0000","last_active":"2024-05-14T16:49:54.607790+0000","last_peered":"2024-05-14T16:49:54.607790+0000","last_clean":"2024-05-14T16:49:54.607790+0000","last_became_active":"2024-05-14T16:46:43.569077+0000","last_became_peered":"2024-05-14T16:46:43.569077+0000","last_unstale":"2024-05-14T16:49:54.607790+0000","last_undegraded":"2024-05-14T16:49:54.607790+0000","last_fullsized":"2024-05-14T16:49:54.607790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:45:50.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614415+0000","last_change":"2024-05-14T16:46:40.856302+0000","last_active":"2024-05-14T16:49:54.614415+0000","last_peered":"2024-05-14T16:49:54.614415+0000","last_clean":"2024-05-14T16:49:54.614415+0000","last_became_active":"2024-05-14T16:46:40.855964+0000","last_became_peered":"2024-05-14T16:46:40.855964+0000","last_unstale":"2024-05-14T16:49:54.614415+0000","last_undegraded":"2024-05-14T16:49:54.614415+0000","last_fullsized":"2024-05-14T16:49:54.614415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:38.501418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607375+0000","last_change":"2024-05-14T16:46:43.562847+0000","last_active":"2024-05-14T16:49:54.607375+0000","last_peered":"2024-05-14T16:49:54.607375+0000","last_clean":"2024-05-14T16:49:54.607375+0000","last_became_active":"2024-05-14T16:46:43.560978+0000","last_became_peered":"2024-05-14T16:46:43.560978+0000","last_unstale":"2024-05-14T16:49:54.607375+0000","last_undegraded":"2024-05-14T16:49:54.607375+0000","last_fullsized":"2024-05-14T16:49:54.607375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:12:33.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836124+0000","last_change":"2024-05-14T16:46:40.544017+0000","last_active":"2024-05-14T16:49:54.836124+0000","last_peered":"2024-05-14T16:49:54.836124+0000","last_clean":"2024-05-14T16:49:54.836124+0000","last_became_active":"2024-05-14T16:46:40.543176+0000","last_became_peered":"2024-05-14T16:46:40.543176+0000","last_unstale":"2024-05-14T16:49:54.836124+0000","last_undegraded":"2024-05-14T16:49:54.836124+0000","last_fullsized":"2024-05-14T16:49:54.836124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:09.817891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607232+0000","last_change":"2024-05-14T16:46:43.564395+0000","last_active":"2024-05-14T16:49:54.607232+0000","last_peered":"2024-05-14T16:49:54.607232+0000","last_clean":"2024-05-14T16:49:54.607232+0000","last_became_active":"2024-05-14T16:46:43.564065+0000","last_became_peered":"2024-05-14T16:46:43.564065+0000","last_unstale":"2024-05-14T16:49:54.607232+0000","last_undegraded":"2024-05-14T16:49:54.607232+0000","last_fullsized":"2024-05-14T16:49:54.607232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:52.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614590+0000","last_change":"2024-05-14T16:46:43.577067+0000","last_active":"2024-05-14T16:49:54.614590+0000","last_peered":"2024-05-14T16:49:54.614590+0000","last_clean":"2024-05-14T16:49:54.614590+0000","last_became_active":"2024-05-14T16:46:43.576750+0000","last_became_peered":"2024-05-14T16:46:43.576750+0000","last_unstale":"2024-05-14T16:49:54.614590+0000","last_undegraded":"2024-05-14T16:49:54.614590+0000","last_fullsized":"2024-05-14T16:49:54.614590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:25.961124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":212,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:58.874644+0000","last_change":"2024-05-14T16:46:40.855073+0000","last_active":"2024-05-14T16:49:58.874644+0000","last_peered":"2024-05-14T16:49:58.874644+0000","last_clean":"2024-05-14T16:49:58.874644+0000","last_became_active":"2024-05-14T16:46:40.854641+0000","last_became_peered":"2024-05-14T16:46:40.854641+0000","last_unstale":"2024-05-14T16:49:58.874644+0000","last_undegraded":"2024-05-14T16:49:58.874644+0000","last_fullsized":"2024-05-14T16:49:58.874644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:58:11.670926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614618+0000","last_change":"2024-05-14T16:46:43.569030+0000","last_active":"2024-05-14T16:49:54.614618+0000","last_peered":"2024-05-14T16:49:54.614618+0000","last_clean":"2024-05-14T16:49:54.614618+0000","last_became_active":"2024-05-14T16:46:43.568646+0000","last_became_peered":"2024-05-14T16:46:43.568646+0000","last_unstale":"2024-05-14T16:49:54.614618+0000","last_undegraded":"2024-05-14T16:49:54.614618+0000","last_fullsized":"2024-05-14T16:49:54.614618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:38.086818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614520+0000","last_change":"2024-05-14T16:46:40.542624+0000","last_active":"2024-05-14T16:49:54.614520+0000","last_peered":"2024-05-14T16:49:54.614520+0000","last_clean":"2024-05-14T16:49:54.614520+0000","last_became_active":"2024-05-14T16:46:40.542230+0000","last_became_peered":"2024-05-14T16:46:40.542230+0000","last_unstale":"2024-05-14T16:49:54.614520+0000","last_undegraded":"2024-05-14T16:49:54.614520+0000","last_fullsized":"2024-05-14T16:49:54.614520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:54:50.083362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836053+0000","last_change":"2024-05-14T16:45:15.724994+0000","last_active":"2024-05-14T16:49:54.836053+0000","last_peered":"2024-05-14T16:49:54.836053+0000","last_clean":"2024-05-14T16:49:54.836053+0000","last_became_active":"2024-05-14T16:45:15.720080+0000","last_became_peered":"2024-05-14T16:45:15.720080+0000","last_unstale":"2024-05-14T16:49:54.836053+0000","last_undegraded":"2024-05-14T16:49:54.836053+0000","last_fullsized":"2024-05-14T16:49:54.836053+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_clean_scrub_stamp":"2024-05-14T16:41:54.615675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:32:43.569611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605606+0000","last_change":"2024-05-14T16:46:43.567143+0000","last_active":"2024-05-14T16:49:54.605606+0000","last_peered":"2024-05-14T16:49:54.605606+0000","last_clean":"2024-05-14T16:49:54.605606+0000","last_became_active":"2024-05-14T16:46:43.564717+0000","last_became_peered":"2024-05-14T16:46:43.564717+0000","last_unstale":"2024-05-14T16:49:54.605606+0000","last_undegraded":"2024-05-14T16:49:54.605606+0000","last_fullsized":"2024-05-14T16:49:54.605606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:59.025279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836233+0000","last_change":"2024-05-14T16:46:40.546709+0000","last_active":"2024-05-14T16:49:54.836233+0000","last_peered":"2024-05-14T16:49:54.836233+0000","last_clean":"2024-05-14T16:49:54.836233+0000","last_became_active":"2024-05-14T16:46:40.546295+0000","last_became_peered":"2024-05-14T16:46:40.546295+0000","last_unstale":"2024-05-14T16:49:54.836233+0000","last_undegraded":"2024-05-14T16:49:54.836233+0000","last_fullsized":"2024-05-14T16:49:54.836233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:42:50.255289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616443+0000","last_change":"2024-05-14T16:46:40.856822+0000","last_active":"2024-05-14T16:49:54.616443+0000","last_peered":"2024-05-14T16:49:54.616443+0000","last_clean":"2024-05-14T16:49:54.616443+0000","last_became_active":"2024-05-14T16:46:40.856434+0000","last_became_peered":"2024-05-14T16:46:40.856434+0000","last_unstale":"2024-05-14T16:49:54.616443+0000","last_undegraded":"2024-05-14T16:49:54.616443+0000","last_fullsized":"2024-05-14T16:49:54.616443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:38.997679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605968+0000","last_change":"2024-05-14T16:46:43.563918+0000","last_active":"2024-05-14T16:49:54.605968+0000","last_peered":"2024-05-14T16:49:54.605968+0000","last_clean":"2024-05-14T16:49:54.605968+0000","last_became_active":"2024-05-14T16:46:43.563527+0000","last_became_peered":"2024-05-14T16:46:43.563527+0000","last_unstale":"2024-05-14T16:49:54.605968+0000","last_undegraded":"2024-05-14T16:49:54.605968+0000","last_fullsized":"2024-05-14T16:49:54.605968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:58:52.984695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'125","reported_seq":279,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:50:00.156171+0000","last_change":"2024-05-14T16:46:40.547480+0000","last_active":"2024-05-14T16:50:00.156171+0000","last_peered":"2024-05-14T16:50:00.156171+0000","last_clean":"2024-05-14T16:50:00.156171+0000","last_became_active":"2024-05-14T16:46:40.547062+0000","last_became_peered":"2024-05-14T16:46:40.547062+0000","last_unstale":"2024-05-14T16:50:00.156171+0000","last_undegraded":"2024-05-14T16:50:00.156171+0000","last_fullsized":"2024-05-14T16:50:00.156171+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:30.211421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":232,"num_write_kb":1415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615053+0000","last_change":"2024-05-14T16:46:43.563870+0000","last_active":"2024-05-14T16:49:54.615053+0000","last_peered":"2024-05-14T16:49:54.615053+0000","last_clean":"2024-05-14T16:49:54.615053+0000","last_became_active":"2024-05-14T16:46:43.563542+0000","last_became_peered":"2024-05-14T16:46:43.563542+0000","last_unstale":"2024-05-14T16:49:54.615053+0000","last_undegraded":"2024-05-14T16:49:54.615053+0000","last_fullsized":"2024-05-14T16:49:54.615053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:50.699601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614489+0000","last_change":"2024-05-14T16:46:40.665192+0000","last_active":"2024-05-14T16:49:54.614489+0000","last_peered":"2024-05-14T16:49:54.614489+0000","last_clean":"2024-05-14T16:49:54.614489+0000","last_became_active":"2024-05-14T16:46:40.664827+0000","last_became_peered":"2024-05-14T16:46:40.664827+0000","last_unstale":"2024-05-14T16:49:54.614489+0000","last_undegraded":"2024-05-14T16:49:54.614489+0000","last_fullsized":"2024-05-14T16:49:54.614489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:41:47.179677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615488+0000","last_change":"2024-05-14T16:47:33.717718+0000","last_active":"2024-05-14T16:49:54.615488+0000","last_peered":"2024-05-14T16:49:54.615488+0000","last_clean":"2024-05-14T16:49:54.615488+0000","last_became_active":"2024-05-14T16:47:33.717368+0000","last_became_peered":"2024-05-14T16:47:33.717368+0000","last_unstale":"2024-05-14T16:49:54.615488+0000","last_undegraded":"2024-05-14T16:49:54.615488+0000","last_fullsized":"2024-05-14T16:49:54.615488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:07:43.990388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614203+0000","last_change":"2024-05-14T16:46:40.855458+0000","last_active":"2024-05-14T16:49:54.614203+0000","last_peered":"2024-05-14T16:49:54.614203+0000","last_clean":"2024-05-14T16:49:54.614203+0000","last_became_active":"2024-05-14T16:46:40.855122+0000","last_became_peered":"2024-05-14T16:46:40.855122+0000","last_unstale":"2024-05-14T16:49:54.614203+0000","last_undegraded":"2024-05-14T16:49:54.614203+0000","last_fullsized":"2024-05-14T16:49:54.614203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:20:43.435860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615935+0000","last_change":"2024-05-14T16:46:43.566482+0000","last_active":"2024-05-14T16:49:54.615935+0000","last_peered":"2024-05-14T16:49:54.615935+0000","last_clean":"2024-05-14T16:49:54.615935+0000","last_became_active":"2024-05-14T16:46:43.566145+0000","last_became_peered":"2024-05-14T16:46:43.566145+0000","last_unstale":"2024-05-14T16:49:54.615935+0000","last_undegraded":"2024-05-14T16:49:54.615935+0000","last_fullsized":"2024-05-14T16:49:54.615935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:14:22.731629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614710+0000","last_change":"2024-05-14T16:46:40.541655+0000","last_active":"2024-05-14T16:49:54.614710+0000","last_peered":"2024-05-14T16:49:54.614710+0000","last_clean":"2024-05-14T16:49:54.614710+0000","last_became_active":"2024-05-14T16:46:40.541331+0000","last_became_peered":"2024-05-14T16:46:40.541331+0000","last_unstale":"2024-05-14T16:49:54.614710+0000","last_undegraded":"2024-05-14T16:49:54.614710+0000","last_fullsized":"2024-05-14T16:49:54.614710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:47:27.950532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607663+0000","last_change":"2024-05-14T16:46:43.564216+0000","last_active":"2024-05-14T16:49:54.607663+0000","last_peered":"2024-05-14T16:49:54.607663+0000","last_clean":"2024-05-14T16:49:54.607663+0000","last_became_active":"2024-05-14T16:46:43.563894+0000","last_became_peered":"2024-05-14T16:46:43.563894+0000","last_unstale":"2024-05-14T16:49:54.607663+0000","last_undegraded":"2024-05-14T16:49:54.607663+0000","last_fullsized":"2024-05-14T16:49:54.607663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:16:09.256909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615063+0000","last_change":"2024-05-14T16:46:40.551034+0000","last_active":"2024-05-14T16:49:54.615063+0000","last_peered":"2024-05-14T16:49:54.615063+0000","last_clean":"2024-05-14T16:49:54.615063+0000","last_became_active":"2024-05-14T16:46:40.550211+0000","last_became_peered":"2024-05-14T16:46:40.550211+0000","last_unstale":"2024-05-14T16:49:54.615063+0000","last_undegraded":"2024-05-14T16:49:54.615063+0000","last_fullsized":"2024-05-14T16:49:54.615063+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:15.720942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614751+0000","last_change":"2024-05-14T16:46:43.558264+0000","last_active":"2024-05-14T16:49:54.614751+0000","last_peered":"2024-05-14T16:49:54.614751+0000","last_clean":"2024-05-14T16:49:54.614751+0000","last_became_active":"2024-05-14T16:46:43.557907+0000","last_became_peered":"2024-05-14T16:46:43.557907+0000","last_unstale":"2024-05-14T16:49:54.614751+0000","last_undegraded":"2024-05-14T16:49:54.614751+0000","last_fullsized":"2024-05-14T16:49:54.614751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:20:52.752482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616215+0000","last_change":"2024-05-14T16:46:40.551013+0000","last_active":"2024-05-14T16:49:54.616215+0000","last_peered":"2024-05-14T16:49:54.616215+0000","last_clean":"2024-05-14T16:49:54.616215+0000","last_became_active":"2024-05-14T16:46:40.550658+0000","last_became_peered":"2024-05-14T16:46:40.550658+0000","last_unstale":"2024-05-14T16:49:54.616215+0000","last_undegraded":"2024-05-14T16:49:54.616215+0000","last_fullsized":"2024-05-14T16:49:54.616215+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:36.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836187+0000","last_change":"2024-05-14T16:46:43.572270+0000","last_active":"2024-05-14T16:49:54.836187+0000","last_peered":"2024-05-14T16:49:54.836187+0000","last_clean":"2024-05-14T16:49:54.836187+0000","last_became_active":"2024-05-14T16:46:43.571886+0000","last_became_peered":"2024-05-14T16:46:43.571886+0000","last_unstale":"2024-05-14T16:49:54.836187+0000","last_undegraded":"2024-05-14T16:49:54.836187+0000","last_fullsized":"2024-05-14T16:49:54.836187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:11.890881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614625+0000","last_change":"2024-05-14T16:47:33.715246+0000","last_active":"2024-05-14T16:49:54.614625+0000","last_peered":"2024-05-14T16:49:54.614625+0000","last_clean":"2024-05-14T16:49:54.614625+0000","last_became_active":"2024-05-14T16:47:33.708725+0000","last_became_peered":"2024-05-14T16:47:33.708725+0000","last_unstale":"2024-05-14T16:49:54.614625+0000","last_undegraded":"2024-05-14T16:49:54.614625+0000","last_fullsized":"2024-05-14T16:49:54.614625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:19.118454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836720+0000","last_change":"2024-05-14T16:46:43.572455+0000","last_active":"2024-05-14T16:49:54.836720+0000","last_peered":"2024-05-14T16:49:54.836720+0000","last_clean":"2024-05-14T16:49:54.836720+0000","last_became_active":"2024-05-14T16:46:43.571930+0000","last_became_peered":"2024-05-14T16:46:43.571930+0000","last_unstale":"2024-05-14T16:49:54.836720+0000","last_undegraded":"2024-05-14T16:49:54.836720+0000","last_fullsized":"2024-05-14T16:49:54.836720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:35:39.643239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'39","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.800806+0000","last_change":"2024-05-14T16:46:40.541708+0000","last_active":"2024-05-14T16:49:54.800806+0000","last_peered":"2024-05-14T16:49:54.800806+0000","last_clean":"2024-05-14T16:49:54.800806+0000","last_became_active":"2024-05-14T16:46:40.541202+0000","last_became_peered":"2024-05-14T16:46:40.541202+0000","last_unstale":"2024-05-14T16:49:54.800806+0000","last_undegraded":"2024-05-14T16:49:54.800806+0000","last_fullsized":"2024-05-14T16:49:54.800806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:50:01.821013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836382+0000","last_change":"2024-05-14T16:46:43.574039+0000","last_active":"2024-05-14T16:49:54.836382+0000","last_peered":"2024-05-14T16:49:54.836382+0000","last_clean":"2024-05-14T16:49:54.836382+0000","last_became_active":"2024-05-14T16:46:43.573237+0000","last_became_peered":"2024-05-14T16:46:43.573237+0000","last_unstale":"2024-05-14T16:49:54.836382+0000","last_undegraded":"2024-05-14T16:49:54.836382+0000","last_fullsized":"2024-05-14T16:49:54.836382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:13.234495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605951+0000","last_change":"2024-05-14T16:46:40.549519+0000","last_active":"2024-05-14T16:49:54.605951+0000","last_peered":"2024-05-14T16:49:54.605951+0000","last_clean":"2024-05-14T16:49:54.605951+0000","last_became_active":"2024-05-14T16:46:40.549169+0000","last_became_peered":"2024-05-14T16:46:40.549169+0000","last_unstale":"2024-05-14T16:49:54.605951+0000","last_undegraded":"2024-05-14T16:49:54.605951+0000","last_fullsized":"2024-05-14T16:49:54.605951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:22.559724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836719+0000","last_change":"2024-05-14T16:46:43.567674+0000","last_active":"2024-05-14T16:49:54.836719+0000","last_peered":"2024-05-14T16:49:54.836719+0000","last_clean":"2024-05-14T16:49:54.836719+0000","last_became_active":"2024-05-14T16:46:43.567337+0000","last_became_peered":"2024-05-14T16:46:43.567337+0000","last_unstale":"2024-05-14T16:49:54.836719+0000","last_undegraded":"2024-05-14T16:49:54.836719+0000","last_fullsized":"2024-05-14T16:49:54.836719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:02.321955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607773+0000","last_change":"2024-05-14T16:46:43.567627+0000","last_active":"2024-05-14T16:49:54.607773+0000","last_peered":"2024-05-14T16:49:54.607773+0000","last_clean":"2024-05-14T16:49:54.607773+0000","last_became_active":"2024-05-14T16:46:43.567313+0000","last_became_peered":"2024-05-14T16:46:43.567313+0000","last_unstale":"2024-05-14T16:49:54.607773+0000","last_undegraded":"2024-05-14T16:49:54.607773+0000","last_fullsized":"2024-05-14T16:49:54.607773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:47.396923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615757+0000","last_change":"2024-05-14T16:47:33.708354+0000","last_active":"2024-05-14T16:49:54.615757+0000","last_peered":"2024-05-14T16:49:54.615757+0000","last_clean":"2024-05-14T16:49:54.615757+0000","last_became_active":"2024-05-14T16:47:33.707842+0000","last_became_peered":"2024-05-14T16:47:33.707842+0000","last_unstale":"2024-05-14T16:49:54.615757+0000","last_undegraded":"2024-05-14T16:49:54.615757+0000","last_fullsized":"2024-05-14T16:49:54.615757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:56:26.845782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614936+0000","last_change":"2024-05-14T16:47:33.708332+0000","last_active":"2024-05-14T16:49:54.614936+0000","last_peered":"2024-05-14T16:49:54.614936+0000","last_clean":"2024-05-14T16:49:54.614936+0000","last_became_active":"2024-05-14T16:47:33.707774+0000","last_became_peered":"2024-05-14T16:47:33.707774+0000","last_unstale":"2024-05-14T16:49:54.614936+0000","last_undegraded":"2024-05-14T16:49:54.614936+0000","last_fullsized":"2024-05-14T16:49:54.614936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:44:47.699303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615706+0000","last_change":"2024-05-14T16:46:43.572168+0000","last_active":"2024-05-14T16:49:54.615706+0000","last_peered":"2024-05-14T16:49:54.615706+0000","last_clean":"2024-05-14T16:49:54.615706+0000","last_became_active":"2024-05-14T16:46:43.571822+0000","last_became_peered":"2024-05-14T16:46:43.571822+0000","last_unstale":"2024-05-14T16:49:54.615706+0000","last_undegraded":"2024-05-14T16:49:54.615706+0000","last_fullsized":"2024-05-14T16:49:54.615706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:57.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605874+0000","last_change":"2024-05-14T16:46:43.570935+0000","last_active":"2024-05-14T16:49:54.605874+0000","last_peered":"2024-05-14T16:49:54.605874+0000","last_clean":"2024-05-14T16:49:54.605874+0000","last_became_active":"2024-05-14T16:46:43.570605+0000","last_became_peered":"2024-05-14T16:46:43.570605+0000","last_unstale":"2024-05-14T16:49:54.605874+0000","last_undegraded":"2024-05-14T16:49:54.605874+0000","last_fullsized":"2024-05-14T16:49:54.605874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:17:55.161195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.608013+0000","last_change":"2024-05-14T16:46:43.568442+0000","last_active":"2024-05-14T16:49:54.608013+0000","last_peered":"2024-05-14T16:49:54.608013+0000","last_clean":"2024-05-14T16:49:54.608013+0000","last_became_active":"2024-05-14T16:46:43.568118+0000","last_became_peered":"2024-05-14T16:46:43.568118+0000","last_unstale":"2024-05-14T16:49:54.608013+0000","last_undegraded":"2024-05-14T16:49:54.608013+0000","last_fullsized":"2024-05-14T16:49:54.608013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:52.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615674+0000","last_change":"2024-05-14T16:46:43.576174+0000","last_active":"2024-05-14T16:49:54.615674+0000","last_peered":"2024-05-14T16:49:54.615674+0000","last_clean":"2024-05-14T16:49:54.615674+0000","last_became_active":"2024-05-14T16:46:43.575737+0000","last_became_peered":"2024-05-14T16:46:43.575737+0000","last_unstale":"2024-05-14T16:49:54.615674+0000","last_undegraded":"2024-05-14T16:49:54.615674+0000","last_fullsized":"2024-05-14T16:49:54.615674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:08.453070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614955+0000","last_change":"2024-05-14T16:46:43.566256+0000","last_active":"2024-05-14T16:49:54.614955+0000","last_peered":"2024-05-14T16:49:54.614955+0000","last_clean":"2024-05-14T16:49:54.614955+0000","last_became_active":"2024-05-14T16:46:43.565886+0000","last_became_peered":"2024-05-14T16:46:43.565886+0000","last_unstale":"2024-05-14T16:49:54.614955+0000","last_undegraded":"2024-05-14T16:49:54.614955+0000","last_fullsized":"2024-05-14T16:49:54.614955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:50.021286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614405+0000","last_change":"2024-05-14T16:46:43.569819+0000","last_active":"2024-05-14T16:49:54.614405+0000","last_peered":"2024-05-14T16:49:54.614405+0000","last_clean":"2024-05-14T16:49:54.614405+0000","last_became_active":"2024-05-14T16:46:43.569461+0000","last_became_peered":"2024-05-14T16:46:43.569461+0000","last_unstale":"2024-05-14T16:49:54.614405+0000","last_undegraded":"2024-05-14T16:49:54.614405+0000","last_fullsized":"2024-05-14T16:49:54.614405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:52.013604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614556+0000","last_change":"2024-05-14T16:46:43.567569+0000","last_active":"2024-05-14T16:49:54.614556+0000","last_peered":"2024-05-14T16:49:54.614556+0000","last_clean":"2024-05-14T16:49:54.614556+0000","last_became_active":"2024-05-14T16:46:43.564808+0000","last_became_peered":"2024-05-14T16:46:43.564808+0000","last_unstale":"2024-05-14T16:49:54.614556+0000","last_undegraded":"2024-05-14T16:49:54.614556+0000","last_fullsized":"2024-05-14T16:49:54.614556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:13.744730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614465+0000","last_change":"2024-05-14T16:46:43.578417+0000","last_active":"2024-05-14T16:49:54.614465+0000","last_peered":"2024-05-14T16:49:54.614465+0000","last_clean":"2024-05-14T16:49:54.614465+0000","last_became_active":"2024-05-14T16:46:43.577687+0000","last_became_peered":"2024-05-14T16:46:43.577687+0000","last_unstale":"2024-05-14T16:49:54.614465+0000","last_undegraded":"2024-05-14T16:49:54.614465+0000","last_fullsized":"2024-05-14T16:49:54.614465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:32.060934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.606013+0000","last_change":"2024-05-14T16:46:43.564114+0000","last_active":"2024-05-14T16:49:54.606013+0000","last_peered":"2024-05-14T16:49:54.606013+0000","last_clean":"2024-05-14T16:49:54.606013+0000","last_became_active":"2024-05-14T16:46:43.563734+0000","last_became_peered":"2024-05-14T16:46:43.563734+0000","last_unstale":"2024-05-14T16:49:54.606013+0000","last_undegraded":"2024-05-14T16:49:54.606013+0000","last_fullsized":"2024-05-14T16:49:54.606013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:19.016741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607626+0000","last_change":"2024-05-14T16:46:43.566490+0000","last_active":"2024-05-14T16:49:54.607626+0000","last_peered":"2024-05-14T16:49:54.607626+0000","last_clean":"2024-05-14T16:49:54.607626+0000","last_became_active":"2024-05-14T16:46:43.566086+0000","last_became_peered":"2024-05-14T16:46:43.566086+0000","last_unstale":"2024-05-14T16:49:54.607626+0000","last_undegraded":"2024-05-14T16:49:54.607626+0000","last_fullsized":"2024-05-14T16:49:54.607626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:39:09.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107202,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596919,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1513,"num_read_kb":1003,"num_write":440,"num_write_kb":2014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":728,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":1987389,"data_compressed":85252,"data_compressed_allocated":110592,"data_compressed_original":1732608,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710774,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":434319,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1056768,"data_stored":1224183,"data_compressed":50587,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070529,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":827625,"data_compressed":27993,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":827433,"data_compressed":27843,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":1026958,"data_compressed":44431,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626513,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":433798,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":500994,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":435069,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":502119,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1155557,"data_compressed":43195,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986265,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":888832,"data_stored":435516,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822740,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":1157543,"data_compressed":50587,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34498,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133467,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":393842,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19296,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201280,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:50:03.012 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T16:50:03.012 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:50:03.287 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:04.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:03 smithi069 ceph-mon[31507]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:50:04.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:03 smithi069 ceph-mon[31507]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 169 KiB/s wr, 94 op/s 2024-05-14T16:50:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:03 smithi203 ceph-mon[35787]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:50:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:03 smithi203 ceph-mon[35787]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 169 KiB/s wr, 94 op/s 2024-05-14T16:50:04.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:03 smithi092 ceph-mon[35763]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:50:04.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:03 smithi092 ceph-mon[35763]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 169 KiB/s wr, 94 op/s 2024-05-14T16:50:04.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.960+0000 7fd4f4231700 1 -- 172.21.15.69:0/427352555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4ec114240 msgr2=0x7fd4ec116630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:04.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.960+0000 7fd4f4231700 1 --2- 172.21.15.69:0/427352555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4ec114240 0x7fd4ec116630 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400b0a0 tx=0x7fd4e400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:50:04.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.960+0000 7fd4f4231700 1 -- 172.21.15.69:0/427352555 shutdown_connections 2024-05-14T16:50:04.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.960+0000 7fd4f4231700 1 --2- 172.21.15.69:0/427352555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4ec114240 0x7fd4ec116630 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:04.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.960+0000 7fd4f4231700 1 --2- 172.21.15.69:0/427352555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec077520 0x7fd4ec075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:04.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.960+0000 7fd4f4231700 1 --2- 172.21.15.69:0/427352555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec075cb0 0x7fd4ec076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:04.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.960+0000 7fd4f4231700 1 -- 172.21.15.69:0/427352555 >> 172.21.15.69:0/427352555 conn(0x7fd4ec07a700 msgr2=0x7fd4ec07ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:04.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.960+0000 7fd4f4231700 1 -- 172.21.15.69:0/427352555 shutdown_connections 2024-05-14T16:50:04.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.960+0000 7fd4f4231700 1 -- 172.21.15.69:0/427352555 wait complete. 2024-05-14T16:50:04.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f4231700 1 Processor -- start 2024-05-14T16:50:04.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f4231700 1 -- start start 2024-05-14T16:50:04.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f4231700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec075cb0 0x7fd4ec1a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f4231700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4ec077520 0x7fd4ec1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f4231700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec114240 0x7fd4ec1a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f4231700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec119d10 con 0x7fd4ec077520 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f4231700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec119b90 con 0x7fd4ec114240 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f4231700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec119e90 con 0x7fd4ec075cb0 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f17cc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4ec077520 0x7fd4ec1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f17cc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4ec077520 0x7fd4ec1a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42916/0 (socket says 172.21.15.69:42916) 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f17cc700 1 -- 172.21.15.69:0/575402127 learned_addr learned my addr 172.21.15.69:0/575402127 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f1fcd700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec075cb0 0x7fd4ec1a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f27ce700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec114240 0x7fd4ec1a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f17cc700 1 -- 172.21.15.69:0/575402127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec075cb0 msgr2=0x7fd4ec1a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f17cc700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec075cb0 0x7fd4ec1a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f17cc700 1 -- 172.21.15.69:0/575402127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec114240 msgr2=0x7fd4ec1a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:04.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f17cc700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec114240 0x7fd4ec1a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:04.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f17cc700 1 -- 172.21.15.69:0/575402127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec1a51f0 con 0x7fd4ec077520 2024-05-14T16:50:04.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.961+0000 7fd4f17cc700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4ec077520 0x7fd4ec1a0720 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00d950 tx=0x7fd4dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:04.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.962+0000 7fd4e2ffd700 1 -- 172.21.15.69:0/575402127 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc016070 con 0x7fd4ec077520 2024-05-14T16:50:04.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.962+0000 7fd4e2ffd700 1 -- 172.21.15.69:0/575402127 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4dc009df0 con 0x7fd4ec077520 2024-05-14T16:50:04.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.962+0000 7fd4e2ffd700 1 -- 172.21.15.69:0/575402127 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00f3c0 con 0x7fd4ec077520 2024-05-14T16:50:04.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.962+0000 7fd4f4231700 1 -- 172.21.15.69:0/575402127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec1a54e0 con 0x7fd4ec077520 2024-05-14T16:50:04.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.962+0000 7fd4f4231700 1 -- 172.21.15.69:0/575402127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec1acd60 con 0x7fd4ec077520 2024-05-14T16:50:04.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.963+0000 7fd4f4231700 1 -- 172.21.15.69:0/575402127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ec06a230 con 0x7fd4ec077520 2024-05-14T16:50:04.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.964+0000 7fd4e2ffd700 1 -- 172.21.15.69:0/575402127 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fd4dc019070 con 0x7fd4ec077520 2024-05-14T16:50:04.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.965+0000 7fd4e2ffd700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4d80b2b90 0x7fd4d80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:04.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.965+0000 7fd4f1fcd700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4d80b2b90 0x7fd4d80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:04.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.965+0000 7fd4e2ffd700 1 -- 172.21.15.69:0/575402127 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7fd4dc0e2460 con 0x7fd4ec077520 2024-05-14T16:50:04.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.965+0000 7fd4f1fcd700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4d80b2b90 0x7fd4d80b5050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8005950 tx=0x7fd4e800a300 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:04.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:04.969+0000 7fd4e2ffd700 1 -- 172.21.15.69:0/575402127 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4dc0a9c00 con 0x7fd4ec077520 2024-05-14T16:50:05.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.168+0000 7fd4f4231700 1 -- 172.21.15.69:0/575402127 --> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4ec113710 con 0x7fd4d80b2b90 2024-05-14T16:50:05.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.175+0000 7fd4e2ffd700 1 -- 172.21.15.69:0/575402127 <== mgr.24764 v2:172.21.15.69:6800/4217409403 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258041 (secure 0 0 0) 0x7fd4ec113710 con 0x7fd4d80b2b90 2024-05-14T16:50:05.179 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:05.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 -- 172.21.15.69:0/575402127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4d80b2b90 msgr2=0x7fd4d80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4d80b2b90 0x7fd4d80b5050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8005950 tx=0x7fd4e800a300 comp rx=0 tx=0).stop 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 -- 172.21.15.69:0/575402127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4ec077520 msgr2=0x7fd4ec1a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4ec077520 0x7fd4ec1a0720 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00d950 tx=0x7fd4dc00dc60 comp rx=0 tx=0).stop 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 -- 172.21.15.69:0/575402127 shutdown_connections 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec114240 0x7fd4ec1a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec075cb0 0x7fd4ec1a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4d80b2b90 0x7fd4d80b5050 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 --2- 172.21.15.69:0/575402127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4ec077520 0x7fd4ec1a0720 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 -- 172.21.15.69:0/575402127 >> 172.21.15.69:0/575402127 conn(0x7fd4ec07a700 msgr2=0x7fd4ec113000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 -- 172.21.15.69:0/575402127 shutdown_connections 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:05.178+0000 7fd4f4231700 1 -- 172.21.15.69:0/575402127 wait complete. 2024-05-14T16:50:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-05-14T16:50:05.859 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-05-14T16:50:04.622976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1385982,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1716,"num_read_kb":1295,"num_write":616,"num_write_kb":4770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952568,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":945,"ondisk_log_size":945,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351908,"kb_used_data":11028,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589916,"statfs":{"total":1151940427776,"available":1151580073984,"internally_reserved":0,"allocated":11292672,"data_stored":9355942,"data_compressed":315060,"data_compressed_allocated":417792,"data_compressed_original":4165632,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":198461,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":534,"num_write":196,"num_write_kb":1180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991416"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607961+0000","last_change":"2024-05-14T16:46:43.572738+0000","last_active":"2024-05-14T16:49:54.607961+0000","last_peered":"2024-05-14T16:49:54.607961+0000","last_clean":"2024-05-14T16:49:54.607961+0000","last_became_active":"2024-05-14T16:46:43.572400+0000","last_became_peered":"2024-05-14T16:46:43.572400+0000","last_unstale":"2024-05-14T16:49:54.607961+0000","last_undegraded":"2024-05-14T16:49:54.607961+0000","last_fullsized":"2024-05-14T16:49:54.607961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:44.332862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615350+0000","last_change":"2024-05-14T16:46:43.566193+0000","last_active":"2024-05-14T16:49:54.615350+0000","last_peered":"2024-05-14T16:49:54.615350+0000","last_clean":"2024-05-14T16:49:54.615350+0000","last_became_active":"2024-05-14T16:46:43.565853+0000","last_became_peered":"2024-05-14T16:46:43.565853+0000","last_unstale":"2024-05-14T16:49:54.615350+0000","last_undegraded":"2024-05-14T16:49:54.615350+0000","last_fullsized":"2024-05-14T16:49:54.615350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:19:34.481225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615237+0000","last_change":"2024-05-14T16:46:43.574937+0000","last_active":"2024-05-14T16:49:54.615237+0000","last_peered":"2024-05-14T16:49:54.615237+0000","last_clean":"2024-05-14T16:49:54.615237+0000","last_became_active":"2024-05-14T16:46:43.574317+0000","last_became_peered":"2024-05-14T16:46:43.574317+0000","last_unstale":"2024-05-14T16:49:54.615237+0000","last_undegraded":"2024-05-14T16:49:54.615237+0000","last_fullsized":"2024-05-14T16:49:54.615237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:35:49.933658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614517+0000","last_change":"2024-05-14T16:47:33.711809+0000","last_active":"2024-05-14T16:49:54.614517+0000","last_peered":"2024-05-14T16:49:54.614517+0000","last_clean":"2024-05-14T16:49:54.614517+0000","last_became_active":"2024-05-14T16:47:33.711422+0000","last_became_peered":"2024-05-14T16:47:33.711422+0000","last_unstale":"2024-05-14T16:49:54.614517+0000","last_undegraded":"2024-05-14T16:49:54.614517+0000","last_fullsized":"2024-05-14T16:49:54.614517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:44:26.005189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616108+0000","last_change":"2024-05-14T16:46:43.567104+0000","last_active":"2024-05-14T16:49:54.616108+0000","last_peered":"2024-05-14T16:49:54.616108+0000","last_clean":"2024-05-14T16:49:54.616108+0000","last_became_active":"2024-05-14T16:46:43.566762+0000","last_became_peered":"2024-05-14T16:46:43.566762+0000","last_unstale":"2024-05-14T16:49:54.616108+0000","last_undegraded":"2024-05-14T16:49:54.616108+0000","last_fullsized":"2024-05-14T16:49:54.616108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:08.819856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607201+0000","last_change":"2024-05-14T16:46:43.565055+0000","last_active":"2024-05-14T16:49:54.607201+0000","last_peered":"2024-05-14T16:49:54.607201+0000","last_clean":"2024-05-14T16:49:54.607201+0000","last_became_active":"2024-05-14T16:46:43.564699+0000","last_became_peered":"2024-05-14T16:46:43.564699+0000","last_unstale":"2024-05-14T16:49:54.607201+0000","last_undegraded":"2024-05-14T16:49:54.607201+0000","last_fullsized":"2024-05-14T16:49:54.607201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:01.903025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.835993+0000","last_change":"2024-05-14T16:47:33.711866+0000","last_active":"2024-05-14T16:49:54.835993+0000","last_peered":"2024-05-14T16:49:54.835993+0000","last_clean":"2024-05-14T16:49:54.835993+0000","last_became_active":"2024-05-14T16:47:33.711481+0000","last_became_peered":"2024-05-14T16:47:33.711481+0000","last_unstale":"2024-05-14T16:49:54.835993+0000","last_undegraded":"2024-05-14T16:49:54.835993+0000","last_fullsized":"2024-05-14T16:49:54.835993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:01:22.724678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614120+0000","last_change":"2024-05-14T16:46:43.564470+0000","last_active":"2024-05-14T16:49:54.614120+0000","last_peered":"2024-05-14T16:49:54.614120+0000","last_clean":"2024-05-14T16:49:54.614120+0000","last_became_active":"2024-05-14T16:46:43.564169+0000","last_became_peered":"2024-05-14T16:46:43.564169+0000","last_unstale":"2024-05-14T16:49:54.614120+0000","last_undegraded":"2024-05-14T16:49:54.614120+0000","last_fullsized":"2024-05-14T16:49:54.614120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:38.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614961+0000","last_change":"2024-05-14T16:46:43.580988+0000","last_active":"2024-05-14T16:49:54.614961+0000","last_peered":"2024-05-14T16:49:54.614961+0000","last_clean":"2024-05-14T16:49:54.614961+0000","last_became_active":"2024-05-14T16:46:43.580657+0000","last_became_peered":"2024-05-14T16:46:43.580657+0000","last_unstale":"2024-05-14T16:49:54.614961+0000","last_undegraded":"2024-05-14T16:49:54.614961+0000","last_fullsized":"2024-05-14T16:49:54.614961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:39:01.568822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607883+0000","last_change":"2024-05-14T16:46:43.565742+0000","last_active":"2024-05-14T16:49:54.607883+0000","last_peered":"2024-05-14T16:49:54.607883+0000","last_clean":"2024-05-14T16:49:54.607883+0000","last_became_active":"2024-05-14T16:46:43.565260+0000","last_became_peered":"2024-05-14T16:46:43.565260+0000","last_unstale":"2024-05-14T16:49:54.607883+0000","last_undegraded":"2024-05-14T16:49:54.607883+0000","last_fullsized":"2024-05-14T16:49:54.607883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:25.718174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836633+0000","last_change":"2024-05-14T16:46:43.576268+0000","last_active":"2024-05-14T16:49:54.836633+0000","last_peered":"2024-05-14T16:49:54.836633+0000","last_clean":"2024-05-14T16:49:54.836633+0000","last_became_active":"2024-05-14T16:46:43.575856+0000","last_became_peered":"2024-05-14T16:46:43.575856+0000","last_unstale":"2024-05-14T16:49:54.836633+0000","last_undegraded":"2024-05-14T16:49:54.836633+0000","last_fullsized":"2024-05-14T16:49:54.836633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:12:16.549127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614667+0000","last_change":"2024-05-14T16:46:43.573315+0000","last_active":"2024-05-14T16:49:54.614667+0000","last_peered":"2024-05-14T16:49:54.614667+0000","last_clean":"2024-05-14T16:49:54.614667+0000","last_became_active":"2024-05-14T16:46:43.572981+0000","last_became_peered":"2024-05-14T16:46:43.572981+0000","last_unstale":"2024-05-14T16:49:54.614667+0000","last_undegraded":"2024-05-14T16:49:54.614667+0000","last_fullsized":"2024-05-14T16:49:54.614667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:07.757539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605825+0000","last_change":"2024-05-14T16:46:43.569388+0000","last_active":"2024-05-14T16:49:54.605825+0000","last_peered":"2024-05-14T16:49:54.605825+0000","last_clean":"2024-05-14T16:49:54.605825+0000","last_became_active":"2024-05-14T16:46:43.568992+0000","last_became_peered":"2024-05-14T16:46:43.568992+0000","last_unstale":"2024-05-14T16:49:54.605825+0000","last_undegraded":"2024-05-14T16:49:54.605825+0000","last_fullsized":"2024-05-14T16:49:54.605825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:08:06.236797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615319+0000","last_change":"2024-05-14T16:46:43.564137+0000","last_active":"2024-05-14T16:49:54.615319+0000","last_peered":"2024-05-14T16:49:54.615319+0000","last_clean":"2024-05-14T16:49:54.615319+0000","last_became_active":"2024-05-14T16:46:43.563831+0000","last_became_peered":"2024-05-14T16:46:43.563831+0000","last_unstale":"2024-05-14T16:49:54.615319+0000","last_undegraded":"2024-05-14T16:49:54.615319+0000","last_fullsized":"2024-05-14T16:49:54.615319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:58.473863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615197+0000","last_change":"2024-05-14T16:46:43.571789+0000","last_active":"2024-05-14T16:49:54.615197+0000","last_peered":"2024-05-14T16:49:54.615197+0000","last_clean":"2024-05-14T16:49:54.615197+0000","last_became_active":"2024-05-14T16:46:43.571521+0000","last_became_peered":"2024-05-14T16:46:43.571521+0000","last_unstale":"2024-05-14T16:49:54.615197+0000","last_undegraded":"2024-05-14T16:49:54.615197+0000","last_fullsized":"2024-05-14T16:49:54.615197+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:23.147578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614202+0000","last_change":"2024-05-14T16:46:43.569532+0000","last_active":"2024-05-14T16:49:54.614202+0000","last_peered":"2024-05-14T16:49:54.614202+0000","last_clean":"2024-05-14T16:49:54.614202+0000","last_became_active":"2024-05-14T16:46:43.569132+0000","last_became_peered":"2024-05-14T16:46:43.569132+0000","last_unstale":"2024-05-14T16:49:54.614202+0000","last_undegraded":"2024-05-14T16:49:54.614202+0000","last_fullsized":"2024-05-14T16:49:54.614202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:09.328241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.608089+0000","last_change":"2024-05-14T16:46:43.574451+0000","last_active":"2024-05-14T16:49:54.608089+0000","last_peered":"2024-05-14T16:49:54.608089+0000","last_clean":"2024-05-14T16:49:54.608089+0000","last_became_active":"2024-05-14T16:46:43.574143+0000","last_became_peered":"2024-05-14T16:46:43.574143+0000","last_unstale":"2024-05-14T16:49:54.608089+0000","last_undegraded":"2024-05-14T16:49:54.608089+0000","last_fullsized":"2024-05-14T16:49:54.608089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:01:42.831262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614829+0000","last_change":"2024-05-14T16:46:43.568799+0000","last_active":"2024-05-14T16:49:54.614829+0000","last_peered":"2024-05-14T16:49:54.614829+0000","last_clean":"2024-05-14T16:49:54.614829+0000","last_became_active":"2024-05-14T16:46:43.568467+0000","last_became_peered":"2024-05-14T16:46:43.568467+0000","last_unstale":"2024-05-14T16:49:54.614829+0000","last_undegraded":"2024-05-14T16:49:54.614829+0000","last_fullsized":"2024-05-14T16:49:54.614829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:59.036372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615487+0000","last_change":"2024-05-14T16:46:43.566879+0000","last_active":"2024-05-14T16:49:54.615487+0000","last_peered":"2024-05-14T16:49:54.615487+0000","last_clean":"2024-05-14T16:49:54.615487+0000","last_became_active":"2024-05-14T16:46:43.566346+0000","last_became_peered":"2024-05-14T16:46:43.566346+0000","last_unstale":"2024-05-14T16:49:54.615487+0000","last_undegraded":"2024-05-14T16:49:54.615487+0000","last_fullsized":"2024-05-14T16:49:54.615487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:11.027173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614436+0000","last_change":"2024-05-14T16:46:40.546990+0000","last_active":"2024-05-14T16:49:54.614436+0000","last_peered":"2024-05-14T16:49:54.614436+0000","last_clean":"2024-05-14T16:49:54.614436+0000","last_became_active":"2024-05-14T16:46:40.546646+0000","last_became_peered":"2024-05-14T16:46:40.546646+0000","last_unstale":"2024-05-14T16:49:54.614436+0000","last_undegraded":"2024-05-14T16:49:54.614436+0000","last_fullsized":"2024-05-14T16:49:54.614436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:47.001760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614919+0000","last_change":"2024-05-14T16:46:43.570352+0000","last_active":"2024-05-14T16:49:54.614919+0000","last_peered":"2024-05-14T16:49:54.614919+0000","last_clean":"2024-05-14T16:49:54.614919+0000","last_became_active":"2024-05-14T16:46:43.569748+0000","last_became_peered":"2024-05-14T16:46:43.569748+0000","last_unstale":"2024-05-14T16:49:54.614919+0000","last_undegraded":"2024-05-14T16:49:54.614919+0000","last_fullsized":"2024-05-14T16:49:54.614919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:24.707586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615028+0000","last_change":"2024-05-14T16:46:40.855034+0000","last_active":"2024-05-14T16:49:54.615028+0000","last_peered":"2024-05-14T16:49:54.615028+0000","last_clean":"2024-05-14T16:49:54.615028+0000","last_became_active":"2024-05-14T16:46:40.854686+0000","last_became_peered":"2024-05-14T16:46:40.854686+0000","last_unstale":"2024-05-14T16:49:54.615028+0000","last_undegraded":"2024-05-14T16:49:54.615028+0000","last_fullsized":"2024-05-14T16:49:54.615028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:56.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615572+0000","last_change":"2024-05-14T16:46:43.574682+0000","last_active":"2024-05-14T16:49:54.615572+0000","last_peered":"2024-05-14T16:49:54.615572+0000","last_clean":"2024-05-14T16:49:54.615572+0000","last_became_active":"2024-05-14T16:46:43.574322+0000","last_became_peered":"2024-05-14T16:46:43.574322+0000","last_unstale":"2024-05-14T16:49:54.615572+0000","last_undegraded":"2024-05-14T16:49:54.615572+0000","last_fullsized":"2024-05-14T16:49:54.615572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:20.169428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616589+0000","last_change":"2024-05-14T16:46:40.551307+0000","last_active":"2024-05-14T16:49:54.616589+0000","last_peered":"2024-05-14T16:49:54.616589+0000","last_clean":"2024-05-14T16:49:54.616589+0000","last_became_active":"2024-05-14T16:46:40.550987+0000","last_became_peered":"2024-05-14T16:46:40.550987+0000","last_unstale":"2024-05-14T16:49:54.616589+0000","last_undegraded":"2024-05-14T16:49:54.616589+0000","last_fullsized":"2024-05-14T16:49:54.616589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:27.819457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614524+0000","last_change":"2024-05-14T16:46:43.575200+0000","last_active":"2024-05-14T16:49:54.614524+0000","last_peered":"2024-05-14T16:49:54.614524+0000","last_clean":"2024-05-14T16:49:54.614524+0000","last_became_active":"2024-05-14T16:46:43.574864+0000","last_became_peered":"2024-05-14T16:46:43.574864+0000","last_unstale":"2024-05-14T16:49:54.614524+0000","last_undegraded":"2024-05-14T16:49:54.614524+0000","last_fullsized":"2024-05-14T16:49:54.614524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:48.630101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616383+0000","last_change":"2024-05-14T16:46:40.548397+0000","last_active":"2024-05-14T16:49:54.616383+0000","last_peered":"2024-05-14T16:49:54.616383+0000","last_clean":"2024-05-14T16:49:54.616383+0000","last_became_active":"2024-05-14T16:46:40.547803+0000","last_became_peered":"2024-05-14T16:46:40.547803+0000","last_unstale":"2024-05-14T16:49:54.616383+0000","last_undegraded":"2024-05-14T16:49:54.616383+0000","last_fullsized":"2024-05-14T16:49:54.616383+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:21:19.812413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615180+0000","last_change":"2024-05-14T16:46:43.578301+0000","last_active":"2024-05-14T16:49:54.615180+0000","last_peered":"2024-05-14T16:49:54.615180+0000","last_clean":"2024-05-14T16:49:54.615180+0000","last_became_active":"2024-05-14T16:46:43.577796+0000","last_became_peered":"2024-05-14T16:46:43.577796+0000","last_unstale":"2024-05-14T16:49:54.615180+0000","last_undegraded":"2024-05-14T16:49:54.615180+0000","last_fullsized":"2024-05-14T16:49:54.615180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:13.159570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":217,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:59.876615+0000","last_change":"2024-05-14T16:46:40.547818+0000","last_active":"2024-05-14T16:49:59.876615+0000","last_peered":"2024-05-14T16:49:59.876615+0000","last_clean":"2024-05-14T16:49:59.876615+0000","last_became_active":"2024-05-14T16:46:40.547454+0000","last_became_peered":"2024-05-14T16:46:40.547454+0000","last_unstale":"2024-05-14T16:49:59.876615+0000","last_undegraded":"2024-05-14T16:49:59.876615+0000","last_fullsized":"2024-05-14T16:49:59.876615+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:16.362415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616051+0000","last_change":"2024-05-14T16:46:43.570743+0000","last_active":"2024-05-14T16:49:54.616051+0000","last_peered":"2024-05-14T16:49:54.616051+0000","last_clean":"2024-05-14T16:49:54.616051+0000","last_became_active":"2024-05-14T16:46:43.570427+0000","last_became_peered":"2024-05-14T16:46:43.570427+0000","last_unstale":"2024-05-14T16:49:54.616051+0000","last_undegraded":"2024-05-14T16:49:54.616051+0000","last_fullsized":"2024-05-14T16:49:54.616051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:04:11.091285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607185+0000","last_change":"2024-05-14T16:46:40.546675+0000","last_active":"2024-05-14T16:49:54.607185+0000","last_peered":"2024-05-14T16:49:54.607185+0000","last_clean":"2024-05-14T16:49:54.607185+0000","last_became_active":"2024-05-14T16:46:40.546250+0000","last_became_peered":"2024-05-14T16:46:40.546250+0000","last_unstale":"2024-05-14T16:49:54.607185+0000","last_undegraded":"2024-05-14T16:49:54.607185+0000","last_fullsized":"2024-05-14T16:49:54.607185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:42:52.367719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614882+0000","last_change":"2024-05-14T16:46:43.579119+0000","last_active":"2024-05-14T16:49:54.614882+0000","last_peered":"2024-05-14T16:49:54.614882+0000","last_clean":"2024-05-14T16:49:54.614882+0000","last_became_active":"2024-05-14T16:46:43.578808+0000","last_became_peered":"2024-05-14T16:46:43.578808+0000","last_unstale":"2024-05-14T16:49:54.614882+0000","last_undegraded":"2024-05-14T16:49:54.614882+0000","last_fullsized":"2024-05-14T16:49:54.614882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:57.852982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605792+0000","last_change":"2024-05-14T16:46:40.538827+0000","last_active":"2024-05-14T16:49:54.605792+0000","last_peered":"2024-05-14T16:49:54.605792+0000","last_clean":"2024-05-14T16:49:54.605792+0000","last_became_active":"2024-05-14T16:46:40.538369+0000","last_became_peered":"2024-05-14T16:46:40.538369+0000","last_unstale":"2024-05-14T16:49:54.605792+0000","last_undegraded":"2024-05-14T16:49:54.605792+0000","last_fullsized":"2024-05-14T16:49:54.605792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:48:03.599457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614307+0000","last_change":"2024-05-14T16:46:43.558411+0000","last_active":"2024-05-14T16:49:54.614307+0000","last_peered":"2024-05-14T16:49:54.614307+0000","last_clean":"2024-05-14T16:49:54.614307+0000","last_became_active":"2024-05-14T16:46:43.558101+0000","last_became_peered":"2024-05-14T16:46:43.558101+0000","last_unstale":"2024-05-14T16:49:54.614307+0000","last_undegraded":"2024-05-14T16:49:54.614307+0000","last_fullsized":"2024-05-14T16:49:54.614307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:10.595833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614356+0000","last_change":"2024-05-14T16:46:40.544858+0000","last_active":"2024-05-14T16:49:54.614356+0000","last_peered":"2024-05-14T16:49:54.614356+0000","last_clean":"2024-05-14T16:49:54.614356+0000","last_became_active":"2024-05-14T16:46:40.544197+0000","last_became_peered":"2024-05-14T16:46:40.544197+0000","last_unstale":"2024-05-14T16:49:54.614356+0000","last_undegraded":"2024-05-14T16:49:54.614356+0000","last_fullsized":"2024-05-14T16:49:54.614356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:16:45.249626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614048+0000","last_change":"2024-05-14T16:46:43.567019+0000","last_active":"2024-05-14T16:49:54.614048+0000","last_peered":"2024-05-14T16:49:54.614048+0000","last_clean":"2024-05-14T16:49:54.614048+0000","last_became_active":"2024-05-14T16:46:43.566700+0000","last_became_peered":"2024-05-14T16:46:43.566700+0000","last_unstale":"2024-05-14T16:49:54.614048+0000","last_undegraded":"2024-05-14T16:49:54.614048+0000","last_fullsized":"2024-05-14T16:49:54.614048+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:43.373467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":212,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:58.875008+0000","last_change":"2024-05-14T16:46:40.541876+0000","last_active":"2024-05-14T16:49:58.875008+0000","last_peered":"2024-05-14T16:49:58.875008+0000","last_clean":"2024-05-14T16:49:58.875008+0000","last_became_active":"2024-05-14T16:46:40.541460+0000","last_became_peered":"2024-05-14T16:46:40.541460+0000","last_unstale":"2024-05-14T16:49:58.875008+0000","last_undegraded":"2024-05-14T16:49:58.875008+0000","last_fullsized":"2024-05-14T16:49:58.875008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:07:22.536429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614958+0000","last_change":"2024-05-14T16:46:43.577015+0000","last_active":"2024-05-14T16:49:54.614958+0000","last_peered":"2024-05-14T16:49:54.614958+0000","last_clean":"2024-05-14T16:49:54.614958+0000","last_became_active":"2024-05-14T16:46:43.576657+0000","last_became_peered":"2024-05-14T16:46:43.576657+0000","last_unstale":"2024-05-14T16:49:54.614958+0000","last_undegraded":"2024-05-14T16:49:54.614958+0000","last_fullsized":"2024-05-14T16:49:54.614958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:29.242445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836348+0000","last_change":"2024-05-14T16:46:40.550727+0000","last_active":"2024-05-14T16:49:54.836348+0000","last_peered":"2024-05-14T16:49:54.836348+0000","last_clean":"2024-05-14T16:49:54.836348+0000","last_became_active":"2024-05-14T16:46:40.550310+0000","last_became_peered":"2024-05-14T16:46:40.550310+0000","last_unstale":"2024-05-14T16:49:54.836348+0000","last_undegraded":"2024-05-14T16:49:54.836348+0000","last_fullsized":"2024-05-14T16:49:54.836348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:04.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615580+0000","last_change":"2024-05-14T16:47:33.711766+0000","last_active":"2024-05-14T16:49:54.615580+0000","last_peered":"2024-05-14T16:49:54.615580+0000","last_clean":"2024-05-14T16:49:54.615580+0000","last_became_active":"2024-05-14T16:47:33.711352+0000","last_became_peered":"2024-05-14T16:47:33.711352+0000","last_unstale":"2024-05-14T16:49:54.615580+0000","last_undegraded":"2024-05-14T16:49:54.615580+0000","last_fullsized":"2024-05-14T16:49:54.615580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:47.924117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615228+0000","last_change":"2024-05-14T16:46:40.855680+0000","last_active":"2024-05-14T16:49:54.615228+0000","last_peered":"2024-05-14T16:49:54.615228+0000","last_clean":"2024-05-14T16:49:54.615228+0000","last_became_active":"2024-05-14T16:46:40.855265+0000","last_became_peered":"2024-05-14T16:46:40.855265+0000","last_unstale":"2024-05-14T16:49:54.615228+0000","last_undegraded":"2024-05-14T16:49:54.615228+0000","last_fullsized":"2024-05-14T16:49:54.615228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:26.065298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614667+0000","last_change":"2024-05-14T16:46:43.579423+0000","last_active":"2024-05-14T16:49:54.614667+0000","last_peered":"2024-05-14T16:49:54.614667+0000","last_clean":"2024-05-14T16:49:54.614667+0000","last_became_active":"2024-05-14T16:46:43.579023+0000","last_became_peered":"2024-05-14T16:46:43.579023+0000","last_unstale":"2024-05-14T16:49:54.614667+0000","last_undegraded":"2024-05-14T16:49:54.614667+0000","last_fullsized":"2024-05-14T16:49:54.614667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:21.311018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'41","reported_seq":441,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:50:00.105774+0000","last_change":"2024-05-14T16:46:40.550871+0000","last_active":"2024-05-14T16:50:00.105774+0000","last_peered":"2024-05-14T16:50:00.105774+0000","last_clean":"2024-05-14T16:50:00.105774+0000","last_became_active":"2024-05-14T16:46:40.550419+0000","last_became_peered":"2024-05-14T16:46:40.550419+0000","last_unstale":"2024-05-14T16:50:00.105774+0000","last_undegraded":"2024-05-14T16:50:00.105774+0000","last_fullsized":"2024-05-14T16:50:00.105774+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:53:09.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68841,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":371,"num_read_kb":251,"num_write":42,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616462+0000","last_change":"2024-05-14T16:46:43.569996+0000","last_active":"2024-05-14T16:49:54.616462+0000","last_peered":"2024-05-14T16:49:54.616462+0000","last_clean":"2024-05-14T16:49:54.616462+0000","last_became_active":"2024-05-14T16:46:43.569501+0000","last_became_peered":"2024-05-14T16:46:43.569501+0000","last_unstale":"2024-05-14T16:49:54.616462+0000","last_undegraded":"2024-05-14T16:49:54.616462+0000","last_fullsized":"2024-05-14T16:49:54.616462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:55.135103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607804+0000","last_change":"2024-05-14T16:47:33.733316+0000","last_active":"2024-05-14T16:49:54.607804+0000","last_peered":"2024-05-14T16:49:54.607804+0000","last_clean":"2024-05-14T16:49:54.607804+0000","last_became_active":"2024-05-14T16:47:33.709886+0000","last_became_peered":"2024-05-14T16:47:33.709886+0000","last_unstale":"2024-05-14T16:49:54.607804+0000","last_undegraded":"2024-05-14T16:49:54.607804+0000","last_fullsized":"2024-05-14T16:49:54.607804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:00:47.464686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605666+0000","last_change":"2024-05-14T16:46:43.562773+0000","last_active":"2024-05-14T16:49:54.605666+0000","last_peered":"2024-05-14T16:49:54.605666+0000","last_clean":"2024-05-14T16:49:54.605666+0000","last_became_active":"2024-05-14T16:46:43.562449+0000","last_became_peered":"2024-05-14T16:46:43.562449+0000","last_unstale":"2024-05-14T16:49:54.605666+0000","last_undegraded":"2024-05-14T16:49:54.605666+0000","last_fullsized":"2024-05-14T16:49:54.605666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:45.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614731+0000","last_change":"2024-05-14T16:46:40.545882+0000","last_active":"2024-05-14T16:49:54.614731+0000","last_peered":"2024-05-14T16:49:54.614731+0000","last_clean":"2024-05-14T16:49:54.614731+0000","last_became_active":"2024-05-14T16:46:40.545439+0000","last_became_peered":"2024-05-14T16:46:40.545439+0000","last_unstale":"2024-05-14T16:49:54.614731+0000","last_undegraded":"2024-05-14T16:49:54.614731+0000","last_fullsized":"2024-05-14T16:49:54.614731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:07.983329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836397+0000","last_change":"2024-05-14T16:46:43.568109+0000","last_active":"2024-05-14T16:49:54.836397+0000","last_peered":"2024-05-14T16:49:54.836397+0000","last_clean":"2024-05-14T16:49:54.836397+0000","last_became_active":"2024-05-14T16:46:43.567106+0000","last_became_peered":"2024-05-14T16:46:43.567106+0000","last_unstale":"2024-05-14T16:49:54.836397+0000","last_undegraded":"2024-05-14T16:49:54.836397+0000","last_fullsized":"2024-05-14T16:49:54.836397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:29.705387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":217,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:58.875129+0000","last_change":"2024-05-14T16:46:40.542204+0000","last_active":"2024-05-14T16:49:58.875129+0000","last_peered":"2024-05-14T16:49:58.875129+0000","last_clean":"2024-05-14T16:49:58.875129+0000","last_became_active":"2024-05-14T16:46:40.541807+0000","last_became_peered":"2024-05-14T16:46:40.541807+0000","last_unstale":"2024-05-14T16:49:58.875129+0000","last_undegraded":"2024-05-14T16:49:58.875129+0000","last_fullsized":"2024-05-14T16:49:58.875129+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:30:45.115647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836336+0000","last_change":"2024-05-14T16:46:43.573546+0000","last_active":"2024-05-14T16:49:54.836336+0000","last_peered":"2024-05-14T16:49:54.836336+0000","last_clean":"2024-05-14T16:49:54.836336+0000","last_became_active":"2024-05-14T16:46:43.573183+0000","last_became_peered":"2024-05-14T16:46:43.573183+0000","last_unstale":"2024-05-14T16:49:54.836336+0000","last_undegraded":"2024-05-14T16:49:54.836336+0000","last_fullsized":"2024-05-14T16:49:54.836336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:49.873097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607658+0000","last_change":"2024-05-14T16:47:33.726700+0000","last_active":"2024-05-14T16:49:54.607658+0000","last_peered":"2024-05-14T16:49:54.607658+0000","last_clean":"2024-05-14T16:49:54.607658+0000","last_became_active":"2024-05-14T16:47:33.709297+0000","last_became_peered":"2024-05-14T16:47:33.709297+0000","last_unstale":"2024-05-14T16:49:54.607658+0000","last_undegraded":"2024-05-14T16:49:54.607658+0000","last_fullsized":"2024-05-14T16:49:54.607658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:35.028197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614569+0000","last_change":"2024-05-14T16:46:43.579059+0000","last_active":"2024-05-14T16:49:54.614569+0000","last_peered":"2024-05-14T16:49:54.614569+0000","last_clean":"2024-05-14T16:49:54.614569+0000","last_became_active":"2024-05-14T16:46:43.578748+0000","last_became_peered":"2024-05-14T16:46:43.578748+0000","last_unstale":"2024-05-14T16:49:54.614569+0000","last_undegraded":"2024-05-14T16:49:54.614569+0000","last_fullsized":"2024-05-14T16:49:54.614569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:42.446422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616551+0000","last_change":"2024-05-14T16:46:40.549047+0000","last_active":"2024-05-14T16:49:54.616551+0000","last_peered":"2024-05-14T16:49:54.616551+0000","last_clean":"2024-05-14T16:49:54.616551+0000","last_became_active":"2024-05-14T16:46:40.548648+0000","last_became_peered":"2024-05-14T16:46:40.548648+0000","last_unstale":"2024-05-14T16:49:54.616551+0000","last_undegraded":"2024-05-14T16:49:54.616551+0000","last_fullsized":"2024-05-14T16:49:54.616551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:52:56.370871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607790+0000","last_change":"2024-05-14T16:46:43.569624+0000","last_active":"2024-05-14T16:49:54.607790+0000","last_peered":"2024-05-14T16:49:54.607790+0000","last_clean":"2024-05-14T16:49:54.607790+0000","last_became_active":"2024-05-14T16:46:43.569077+0000","last_became_peered":"2024-05-14T16:46:43.569077+0000","last_unstale":"2024-05-14T16:49:54.607790+0000","last_undegraded":"2024-05-14T16:49:54.607790+0000","last_fullsized":"2024-05-14T16:49:54.607790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:45:50.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614415+0000","last_change":"2024-05-14T16:46:40.856302+0000","last_active":"2024-05-14T16:49:54.614415+0000","last_peered":"2024-05-14T16:49:54.614415+0000","last_clean":"2024-05-14T16:49:54.614415+0000","last_became_active":"2024-05-14T16:46:40.855964+0000","last_became_peered":"2024-05-14T16:46:40.855964+0000","last_unstale":"2024-05-14T16:49:54.614415+0000","last_undegraded":"2024-05-14T16:49:54.614415+0000","last_fullsized":"2024-05-14T16:49:54.614415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:38.501418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607375+0000","last_change":"2024-05-14T16:46:43.562847+0000","last_active":"2024-05-14T16:49:54.607375+0000","last_peered":"2024-05-14T16:49:54.607375+0000","last_clean":"2024-05-14T16:49:54.607375+0000","last_became_active":"2024-05-14T16:46:43.560978+0000","last_became_peered":"2024-05-14T16:46:43.560978+0000","last_unstale":"2024-05-14T16:49:54.607375+0000","last_undegraded":"2024-05-14T16:49:54.607375+0000","last_fullsized":"2024-05-14T16:49:54.607375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:12:33.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836124+0000","last_change":"2024-05-14T16:46:40.544017+0000","last_active":"2024-05-14T16:49:54.836124+0000","last_peered":"2024-05-14T16:49:54.836124+0000","last_clean":"2024-05-14T16:49:54.836124+0000","last_became_active":"2024-05-14T16:46:40.543176+0000","last_became_peered":"2024-05-14T16:46:40.543176+0000","last_unstale":"2024-05-14T16:49:54.836124+0000","last_undegraded":"2024-05-14T16:49:54.836124+0000","last_fullsized":"2024-05-14T16:49:54.836124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:09.817891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607232+0000","last_change":"2024-05-14T16:46:43.564395+0000","last_active":"2024-05-14T16:49:54.607232+0000","last_peered":"2024-05-14T16:49:54.607232+0000","last_clean":"2024-05-14T16:49:54.607232+0000","last_became_active":"2024-05-14T16:46:43.564065+0000","last_became_peered":"2024-05-14T16:46:43.564065+0000","last_unstale":"2024-05-14T16:49:54.607232+0000","last_undegraded":"2024-05-14T16:49:54.607232+0000","last_fullsized":"2024-05-14T16:49:54.607232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:52.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614590+0000","last_change":"2024-05-14T16:46:43.577067+0000","last_active":"2024-05-14T16:49:54.614590+0000","last_peered":"2024-05-14T16:49:54.614590+0000","last_clean":"2024-05-14T16:49:54.614590+0000","last_became_active":"2024-05-14T16:46:43.576750+0000","last_became_peered":"2024-05-14T16:46:43.576750+0000","last_unstale":"2024-05-14T16:49:54.614590+0000","last_undegraded":"2024-05-14T16:49:54.614590+0000","last_fullsized":"2024-05-14T16:49:54.614590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:25.961124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":212,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:58.874644+0000","last_change":"2024-05-14T16:46:40.855073+0000","last_active":"2024-05-14T16:49:58.874644+0000","last_peered":"2024-05-14T16:49:58.874644+0000","last_clean":"2024-05-14T16:49:58.874644+0000","last_became_active":"2024-05-14T16:46:40.854641+0000","last_became_peered":"2024-05-14T16:46:40.854641+0000","last_unstale":"2024-05-14T16:49:58.874644+0000","last_undegraded":"2024-05-14T16:49:58.874644+0000","last_fullsized":"2024-05-14T16:49:58.874644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:58:11.670926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614618+0000","last_change":"2024-05-14T16:46:43.569030+0000","last_active":"2024-05-14T16:49:54.614618+0000","last_peered":"2024-05-14T16:49:54.614618+0000","last_clean":"2024-05-14T16:49:54.614618+0000","last_became_active":"2024-05-14T16:46:43.568646+0000","last_became_peered":"2024-05-14T16:46:43.568646+0000","last_unstale":"2024-05-14T16:49:54.614618+0000","last_undegraded":"2024-05-14T16:49:54.614618+0000","last_fullsized":"2024-05-14T16:49:54.614618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:38.086818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614520+0000","last_change":"2024-05-14T16:46:40.542624+0000","last_active":"2024-05-14T16:49:54.614520+0000","last_peered":"2024-05-14T16:49:54.614520+0000","last_clean":"2024-05-14T16:49:54.614520+0000","last_became_active":"2024-05-14T16:46:40.542230+0000","last_became_peered":"2024-05-14T16:46:40.542230+0000","last_unstale":"2024-05-14T16:49:54.614520+0000","last_undegraded":"2024-05-14T16:49:54.614520+0000","last_fullsized":"2024-05-14T16:49:54.614520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:54:50.083362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"81'99","reported_seq":130,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:56.707645+0000","last_change":"2024-05-14T16:45:15.724994+0000","last_active":"2024-05-14T16:49:56.707645+0000","last_peered":"2024-05-14T16:49:56.707645+0000","last_clean":"2024-05-14T16:49:56.707645+0000","last_became_active":"2024-05-14T16:45:15.720080+0000","last_became_peered":"2024-05-14T16:45:15.720080+0000","last_unstale":"2024-05-14T16:49:56.707645+0000","last_undegraded":"2024-05-14T16:49:56.707645+0000","last_fullsized":"2024-05-14T16:49:56.707645+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:41:54.615675+0000","last_clean_scrub_stamp":"2024-05-14T16:41:54.615675+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:32:43.569611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605606+0000","last_change":"2024-05-14T16:46:43.567143+0000","last_active":"2024-05-14T16:49:54.605606+0000","last_peered":"2024-05-14T16:49:54.605606+0000","last_clean":"2024-05-14T16:49:54.605606+0000","last_became_active":"2024-05-14T16:46:43.564717+0000","last_became_peered":"2024-05-14T16:46:43.564717+0000","last_unstale":"2024-05-14T16:49:54.605606+0000","last_undegraded":"2024-05-14T16:49:54.605606+0000","last_fullsized":"2024-05-14T16:49:54.605606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:53:59.025279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836233+0000","last_change":"2024-05-14T16:46:40.546709+0000","last_active":"2024-05-14T16:49:54.836233+0000","last_peered":"2024-05-14T16:49:54.836233+0000","last_clean":"2024-05-14T16:49:54.836233+0000","last_became_active":"2024-05-14T16:46:40.546295+0000","last_became_peered":"2024-05-14T16:46:40.546295+0000","last_unstale":"2024-05-14T16:49:54.836233+0000","last_undegraded":"2024-05-14T16:49:54.836233+0000","last_fullsized":"2024-05-14T16:49:54.836233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:42:50.255289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616443+0000","last_change":"2024-05-14T16:46:40.856822+0000","last_active":"2024-05-14T16:49:54.616443+0000","last_peered":"2024-05-14T16:49:54.616443+0000","last_clean":"2024-05-14T16:49:54.616443+0000","last_became_active":"2024-05-14T16:46:40.856434+0000","last_became_peered":"2024-05-14T16:46:40.856434+0000","last_unstale":"2024-05-14T16:49:54.616443+0000","last_undegraded":"2024-05-14T16:49:54.616443+0000","last_fullsized":"2024-05-14T16:49:54.616443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:38.997679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605968+0000","last_change":"2024-05-14T16:46:43.563918+0000","last_active":"2024-05-14T16:49:54.605968+0000","last_peered":"2024-05-14T16:49:54.605968+0000","last_clean":"2024-05-14T16:49:54.605968+0000","last_became_active":"2024-05-14T16:46:43.563527+0000","last_became_peered":"2024-05-14T16:46:43.563527+0000","last_unstale":"2024-05-14T16:49:54.605968+0000","last_undegraded":"2024-05-14T16:49:54.605968+0000","last_fullsized":"2024-05-14T16:49:54.605968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:58:52.984695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'125","reported_seq":279,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:50:00.156171+0000","last_change":"2024-05-14T16:46:40.547480+0000","last_active":"2024-05-14T16:50:00.156171+0000","last_peered":"2024-05-14T16:50:00.156171+0000","last_clean":"2024-05-14T16:50:00.156171+0000","last_became_active":"2024-05-14T16:46:40.547062+0000","last_became_peered":"2024-05-14T16:46:40.547062+0000","last_unstale":"2024-05-14T16:50:00.156171+0000","last_undegraded":"2024-05-14T16:50:00.156171+0000","last_fullsized":"2024-05-14T16:50:00.156171+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:30.211421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":232,"num_write_kb":1415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615053+0000","last_change":"2024-05-14T16:46:43.563870+0000","last_active":"2024-05-14T16:49:54.615053+0000","last_peered":"2024-05-14T16:49:54.615053+0000","last_clean":"2024-05-14T16:49:54.615053+0000","last_became_active":"2024-05-14T16:46:43.563542+0000","last_became_peered":"2024-05-14T16:46:43.563542+0000","last_unstale":"2024-05-14T16:49:54.615053+0000","last_undegraded":"2024-05-14T16:49:54.615053+0000","last_fullsized":"2024-05-14T16:49:54.615053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:50.699601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614489+0000","last_change":"2024-05-14T16:46:40.665192+0000","last_active":"2024-05-14T16:49:54.614489+0000","last_peered":"2024-05-14T16:49:54.614489+0000","last_clean":"2024-05-14T16:49:54.614489+0000","last_became_active":"2024-05-14T16:46:40.664827+0000","last_became_peered":"2024-05-14T16:46:40.664827+0000","last_unstale":"2024-05-14T16:49:54.614489+0000","last_undegraded":"2024-05-14T16:49:54.614489+0000","last_fullsized":"2024-05-14T16:49:54.614489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:41:47.179677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615488+0000","last_change":"2024-05-14T16:47:33.717718+0000","last_active":"2024-05-14T16:49:54.615488+0000","last_peered":"2024-05-14T16:49:54.615488+0000","last_clean":"2024-05-14T16:49:54.615488+0000","last_became_active":"2024-05-14T16:47:33.717368+0000","last_became_peered":"2024-05-14T16:47:33.717368+0000","last_unstale":"2024-05-14T16:49:54.615488+0000","last_undegraded":"2024-05-14T16:49:54.615488+0000","last_fullsized":"2024-05-14T16:49:54.615488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:07:43.990388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614203+0000","last_change":"2024-05-14T16:46:40.855458+0000","last_active":"2024-05-14T16:49:54.614203+0000","last_peered":"2024-05-14T16:49:54.614203+0000","last_clean":"2024-05-14T16:49:54.614203+0000","last_became_active":"2024-05-14T16:46:40.855122+0000","last_became_peered":"2024-05-14T16:46:40.855122+0000","last_unstale":"2024-05-14T16:49:54.614203+0000","last_undegraded":"2024-05-14T16:49:54.614203+0000","last_fullsized":"2024-05-14T16:49:54.614203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:20:43.435860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615935+0000","last_change":"2024-05-14T16:46:43.566482+0000","last_active":"2024-05-14T16:49:54.615935+0000","last_peered":"2024-05-14T16:49:54.615935+0000","last_clean":"2024-05-14T16:49:54.615935+0000","last_became_active":"2024-05-14T16:46:43.566145+0000","last_became_peered":"2024-05-14T16:46:43.566145+0000","last_unstale":"2024-05-14T16:49:54.615935+0000","last_undegraded":"2024-05-14T16:49:54.615935+0000","last_fullsized":"2024-05-14T16:49:54.615935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:14:22.731629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614710+0000","last_change":"2024-05-14T16:46:40.541655+0000","last_active":"2024-05-14T16:49:54.614710+0000","last_peered":"2024-05-14T16:49:54.614710+0000","last_clean":"2024-05-14T16:49:54.614710+0000","last_became_active":"2024-05-14T16:46:40.541331+0000","last_became_peered":"2024-05-14T16:46:40.541331+0000","last_unstale":"2024-05-14T16:49:54.614710+0000","last_undegraded":"2024-05-14T16:49:54.614710+0000","last_fullsized":"2024-05-14T16:49:54.614710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:47:27.950532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607663+0000","last_change":"2024-05-14T16:46:43.564216+0000","last_active":"2024-05-14T16:49:54.607663+0000","last_peered":"2024-05-14T16:49:54.607663+0000","last_clean":"2024-05-14T16:49:54.607663+0000","last_became_active":"2024-05-14T16:46:43.563894+0000","last_became_peered":"2024-05-14T16:46:43.563894+0000","last_unstale":"2024-05-14T16:49:54.607663+0000","last_undegraded":"2024-05-14T16:49:54.607663+0000","last_fullsized":"2024-05-14T16:49:54.607663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:16:09.256909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615063+0000","last_change":"2024-05-14T16:46:40.551034+0000","last_active":"2024-05-14T16:49:54.615063+0000","last_peered":"2024-05-14T16:49:54.615063+0000","last_clean":"2024-05-14T16:49:54.615063+0000","last_became_active":"2024-05-14T16:46:40.550211+0000","last_became_peered":"2024-05-14T16:46:40.550211+0000","last_unstale":"2024-05-14T16:49:54.615063+0000","last_undegraded":"2024-05-14T16:49:54.615063+0000","last_fullsized":"2024-05-14T16:49:54.615063+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:15.720942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614751+0000","last_change":"2024-05-14T16:46:43.558264+0000","last_active":"2024-05-14T16:49:54.614751+0000","last_peered":"2024-05-14T16:49:54.614751+0000","last_clean":"2024-05-14T16:49:54.614751+0000","last_became_active":"2024-05-14T16:46:43.557907+0000","last_became_peered":"2024-05-14T16:46:43.557907+0000","last_unstale":"2024-05-14T16:49:54.614751+0000","last_undegraded":"2024-05-14T16:49:54.614751+0000","last_fullsized":"2024-05-14T16:49:54.614751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:20:52.752482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.616215+0000","last_change":"2024-05-14T16:46:40.551013+0000","last_active":"2024-05-14T16:49:54.616215+0000","last_peered":"2024-05-14T16:49:54.616215+0000","last_clean":"2024-05-14T16:49:54.616215+0000","last_became_active":"2024-05-14T16:46:40.550658+0000","last_became_peered":"2024-05-14T16:46:40.550658+0000","last_unstale":"2024-05-14T16:49:54.616215+0000","last_undegraded":"2024-05-14T16:49:54.616215+0000","last_fullsized":"2024-05-14T16:49:54.616215+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:36.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836187+0000","last_change":"2024-05-14T16:46:43.572270+0000","last_active":"2024-05-14T16:49:54.836187+0000","last_peered":"2024-05-14T16:49:54.836187+0000","last_clean":"2024-05-14T16:49:54.836187+0000","last_became_active":"2024-05-14T16:46:43.571886+0000","last_became_peered":"2024-05-14T16:46:43.571886+0000","last_unstale":"2024-05-14T16:49:54.836187+0000","last_undegraded":"2024-05-14T16:49:54.836187+0000","last_fullsized":"2024-05-14T16:49:54.836187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:11.890881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614625+0000","last_change":"2024-05-14T16:47:33.715246+0000","last_active":"2024-05-14T16:49:54.614625+0000","last_peered":"2024-05-14T16:49:54.614625+0000","last_clean":"2024-05-14T16:49:54.614625+0000","last_became_active":"2024-05-14T16:47:33.708725+0000","last_became_peered":"2024-05-14T16:47:33.708725+0000","last_unstale":"2024-05-14T16:49:54.614625+0000","last_undegraded":"2024-05-14T16:49:54.614625+0000","last_fullsized":"2024-05-14T16:49:54.614625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:19.118454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836720+0000","last_change":"2024-05-14T16:46:43.572455+0000","last_active":"2024-05-14T16:49:54.836720+0000","last_peered":"2024-05-14T16:49:54.836720+0000","last_clean":"2024-05-14T16:49:54.836720+0000","last_became_active":"2024-05-14T16:46:43.571930+0000","last_became_peered":"2024-05-14T16:46:43.571930+0000","last_unstale":"2024-05-14T16:49:54.836720+0000","last_undegraded":"2024-05-14T16:49:54.836720+0000","last_fullsized":"2024-05-14T16:49:54.836720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:35:39.643239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'73","reported_seq":143,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:50:00.151881+0000","last_change":"2024-05-14T16:46:40.541708+0000","last_active":"2024-05-14T16:50:00.151881+0000","last_peered":"2024-05-14T16:50:00.151881+0000","last_clean":"2024-05-14T16:50:00.151881+0000","last_became_active":"2024-05-14T16:46:40.541202+0000","last_became_peered":"2024-05-14T16:46:40.541202+0000","last_unstale":"2024-05-14T16:50:00.151881+0000","last_undegraded":"2024-05-14T16:50:00.151881+0000","last_fullsized":"2024-05-14T16:50:00.151881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:50:01.821013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836382+0000","last_change":"2024-05-14T16:46:43.574039+0000","last_active":"2024-05-14T16:49:54.836382+0000","last_peered":"2024-05-14T16:49:54.836382+0000","last_clean":"2024-05-14T16:49:54.836382+0000","last_became_active":"2024-05-14T16:46:43.573237+0000","last_became_peered":"2024-05-14T16:46:43.573237+0000","last_unstale":"2024-05-14T16:49:54.836382+0000","last_undegraded":"2024-05-14T16:49:54.836382+0000","last_fullsized":"2024-05-14T16:49:54.836382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:13.234495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605951+0000","last_change":"2024-05-14T16:46:40.549519+0000","last_active":"2024-05-14T16:49:54.605951+0000","last_peered":"2024-05-14T16:49:54.605951+0000","last_clean":"2024-05-14T16:49:54.605951+0000","last_became_active":"2024-05-14T16:46:40.549169+0000","last_became_peered":"2024-05-14T16:46:40.549169+0000","last_unstale":"2024-05-14T16:49:54.605951+0000","last_undegraded":"2024-05-14T16:49:54.605951+0000","last_fullsized":"2024-05-14T16:49:54.605951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:39.519344+0000","last_clean_scrub_stamp":"2024-05-14T16:46:39.519344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:22.559724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.836719+0000","last_change":"2024-05-14T16:46:43.567674+0000","last_active":"2024-05-14T16:49:54.836719+0000","last_peered":"2024-05-14T16:49:54.836719+0000","last_clean":"2024-05-14T16:49:54.836719+0000","last_became_active":"2024-05-14T16:46:43.567337+0000","last_became_peered":"2024-05-14T16:46:43.567337+0000","last_unstale":"2024-05-14T16:49:54.836719+0000","last_undegraded":"2024-05-14T16:49:54.836719+0000","last_fullsized":"2024-05-14T16:49:54.836719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:02.321955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607773+0000","last_change":"2024-05-14T16:46:43.567627+0000","last_active":"2024-05-14T16:49:54.607773+0000","last_peered":"2024-05-14T16:49:54.607773+0000","last_clean":"2024-05-14T16:49:54.607773+0000","last_became_active":"2024-05-14T16:46:43.567313+0000","last_became_peered":"2024-05-14T16:46:43.567313+0000","last_unstale":"2024-05-14T16:49:54.607773+0000","last_undegraded":"2024-05-14T16:49:54.607773+0000","last_fullsized":"2024-05-14T16:49:54.607773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:47.396923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615757+0000","last_change":"2024-05-14T16:47:33.708354+0000","last_active":"2024-05-14T16:49:54.615757+0000","last_peered":"2024-05-14T16:49:54.615757+0000","last_clean":"2024-05-14T16:49:54.615757+0000","last_became_active":"2024-05-14T16:47:33.707842+0000","last_became_peered":"2024-05-14T16:47:33.707842+0000","last_unstale":"2024-05-14T16:49:54.615757+0000","last_undegraded":"2024-05-14T16:49:54.615757+0000","last_fullsized":"2024-05-14T16:49:54.615757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:56:26.845782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614936+0000","last_change":"2024-05-14T16:47:33.708332+0000","last_active":"2024-05-14T16:49:54.614936+0000","last_peered":"2024-05-14T16:49:54.614936+0000","last_clean":"2024-05-14T16:49:54.614936+0000","last_became_active":"2024-05-14T16:47:33.707774+0000","last_became_peered":"2024-05-14T16:47:33.707774+0000","last_unstale":"2024-05-14T16:49:54.614936+0000","last_undegraded":"2024-05-14T16:49:54.614936+0000","last_fullsized":"2024-05-14T16:49:54.614936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:44:47.699303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615706+0000","last_change":"2024-05-14T16:46:43.572168+0000","last_active":"2024-05-14T16:49:54.615706+0000","last_peered":"2024-05-14T16:49:54.615706+0000","last_clean":"2024-05-14T16:49:54.615706+0000","last_became_active":"2024-05-14T16:46:43.571822+0000","last_became_peered":"2024-05-14T16:46:43.571822+0000","last_unstale":"2024-05-14T16:49:54.615706+0000","last_undegraded":"2024-05-14T16:49:54.615706+0000","last_fullsized":"2024-05-14T16:49:54.615706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:57:57.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.605874+0000","last_change":"2024-05-14T16:46:43.570935+0000","last_active":"2024-05-14T16:49:54.605874+0000","last_peered":"2024-05-14T16:49:54.605874+0000","last_clean":"2024-05-14T16:49:54.605874+0000","last_became_active":"2024-05-14T16:46:43.570605+0000","last_became_peered":"2024-05-14T16:46:43.570605+0000","last_unstale":"2024-05-14T16:49:54.605874+0000","last_undegraded":"2024-05-14T16:49:54.605874+0000","last_fullsized":"2024-05-14T16:49:54.605874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:17:55.161195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.608013+0000","last_change":"2024-05-14T16:46:43.568442+0000","last_active":"2024-05-14T16:49:54.608013+0000","last_peered":"2024-05-14T16:49:54.608013+0000","last_clean":"2024-05-14T16:49:54.608013+0000","last_became_active":"2024-05-14T16:46:43.568118+0000","last_became_peered":"2024-05-14T16:46:43.568118+0000","last_unstale":"2024-05-14T16:49:54.608013+0000","last_undegraded":"2024-05-14T16:49:54.608013+0000","last_fullsized":"2024-05-14T16:49:54.608013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:52.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.615674+0000","last_change":"2024-05-14T16:46:43.576174+0000","last_active":"2024-05-14T16:49:54.615674+0000","last_peered":"2024-05-14T16:49:54.615674+0000","last_clean":"2024-05-14T16:49:54.615674+0000","last_became_active":"2024-05-14T16:46:43.575737+0000","last_became_peered":"2024-05-14T16:46:43.575737+0000","last_unstale":"2024-05-14T16:49:54.615674+0000","last_undegraded":"2024-05-14T16:49:54.615674+0000","last_fullsized":"2024-05-14T16:49:54.615674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:08.453070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614955+0000","last_change":"2024-05-14T16:46:43.566256+0000","last_active":"2024-05-14T16:49:54.614955+0000","last_peered":"2024-05-14T16:49:54.614955+0000","last_clean":"2024-05-14T16:49:54.614955+0000","last_became_active":"2024-05-14T16:46:43.565886+0000","last_became_peered":"2024-05-14T16:46:43.565886+0000","last_unstale":"2024-05-14T16:49:54.614955+0000","last_undegraded":"2024-05-14T16:49:54.614955+0000","last_fullsized":"2024-05-14T16:49:54.614955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:59:50.021286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614405+0000","last_change":"2024-05-14T16:46:43.569819+0000","last_active":"2024-05-14T16:49:54.614405+0000","last_peered":"2024-05-14T16:49:54.614405+0000","last_clean":"2024-05-14T16:49:54.614405+0000","last_became_active":"2024-05-14T16:46:43.569461+0000","last_became_peered":"2024-05-14T16:46:43.569461+0000","last_unstale":"2024-05-14T16:49:54.614405+0000","last_undegraded":"2024-05-14T16:49:54.614405+0000","last_fullsized":"2024-05-14T16:49:54.614405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:52.013604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614556+0000","last_change":"2024-05-14T16:46:43.567569+0000","last_active":"2024-05-14T16:49:54.614556+0000","last_peered":"2024-05-14T16:49:54.614556+0000","last_clean":"2024-05-14T16:49:54.614556+0000","last_became_active":"2024-05-14T16:46:43.564808+0000","last_became_peered":"2024-05-14T16:46:43.564808+0000","last_unstale":"2024-05-14T16:49:54.614556+0000","last_undegraded":"2024-05-14T16:49:54.614556+0000","last_fullsized":"2024-05-14T16:49:54.614556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:13.744730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.614465+0000","last_change":"2024-05-14T16:46:43.578417+0000","last_active":"2024-05-14T16:49:54.614465+0000","last_peered":"2024-05-14T16:49:54.614465+0000","last_clean":"2024-05-14T16:49:54.614465+0000","last_became_active":"2024-05-14T16:46:43.577687+0000","last_became_peered":"2024-05-14T16:46:43.577687+0000","last_unstale":"2024-05-14T16:49:54.614465+0000","last_undegraded":"2024-05-14T16:49:54.614465+0000","last_fullsized":"2024-05-14T16:49:54.614465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:46:32.060934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.606013+0000","last_change":"2024-05-14T16:46:43.564114+0000","last_active":"2024-05-14T16:49:54.606013+0000","last_peered":"2024-05-14T16:49:54.606013+0000","last_clean":"2024-05-14T16:49:54.606013+0000","last_became_active":"2024-05-14T16:46:43.563734+0000","last_became_peered":"2024-05-14T16:46:43.563734+0000","last_unstale":"2024-05-14T16:49:54.606013+0000","last_undegraded":"2024-05-14T16:49:54.606013+0000","last_fullsized":"2024-05-14T16:49:54.606013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:19.016741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:49:54.607626+0000","last_change":"2024-05-14T16:46:43.566490+0000","last_active":"2024-05-14T16:49:54.607626+0000","last_peered":"2024-05-14T16:49:54.607626+0000","last_clean":"2024-05-14T16:49:54.607626+0000","last_became_active":"2024-05-14T16:46:43.566086+0000","last_became_peered":"2024-05-14T16:46:43.566086+0000","last_unstale":"2024-05-14T16:49:54.607626+0000","last_undegraded":"2024-05-14T16:49:54.607626+0000","last_fullsized":"2024-05-14T16:49:54.607626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:42.538087+0000","last_clean_scrub_stamp":"2024-05-14T16:46:42.538087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:39:09.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107202,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":795380,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1572,"num_read_kb":1047,"num_write":479,"num_write_kb":2618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":728,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2386140,"data_compressed":103494,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710775,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":434319,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907003,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":1421988,"data_compressed":59633,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070529,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":827625,"data_compressed":27993,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":827433,"data_compressed":27993,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462986,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":1027349,"data_compressed":44431,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626513,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":433798,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":500994,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":435069,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":502119,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1155557,"data_compressed":43195,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986266,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":888832,"data_stored":435516,"data_compressed":8697,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":1354175,"data_compressed":59633,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34498,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330490,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":393842,"data_compressed":19296,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19296,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":399350,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:50:05.863 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:50:06.105 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:05 smithi069 ceph-mon[31507]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-05-14T16:50:06.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:05 smithi203 ceph-mon[35787]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-05-14T16:50:06.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:05 smithi092 ceph-mon[35763]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-05-14T16:50:07.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:06 smithi069 ceph-mon[31507]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:50:07.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:06 smithi203 ceph-mon[35787]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:50:07.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:06 smithi092 ceph-mon[35763]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:50:07.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.813+0000 7f0676f83700 1 -- 172.21.15.69:0/2241137810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06701024d0 msgr2=0x7f0670102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:07.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.813+0000 7f0676f83700 1 --2- 172.21.15.69:0/2241137810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06701024d0 0x7f0670102950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f065c009990 tx=0x7f065c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:50:07.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 -- 172.21.15.69:0/2241137810 shutdown_connections 2024-05-14T16:50:07.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 --2- 172.21.15.69:0/2241137810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0670102e90 0x7f067010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:07.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 --2- 172.21.15.69:0/2241137810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06701084d0 0x7f06701088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:07.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 --2- 172.21.15.69:0/2241137810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06701024d0 0x7f0670102950 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:07.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 -- 172.21.15.69:0/2241137810 >> 172.21.15.69:0/2241137810 conn(0x7f0670078490 msgr2=0x7f06700788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:07.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 -- 172.21.15.69:0/2241137810 shutdown_connections 2024-05-14T16:50:07.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 -- 172.21.15.69:0/2241137810 wait complete. 2024-05-14T16:50:07.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 Processor -- start 2024-05-14T16:50:07.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 -- start start 2024-05-14T16:50:07.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06701024d0 0x7f067019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:07.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0670102e90 0x7f067019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:07.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06701084d0 0x7f0670199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:07.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0670113760 con 0x7f06701024d0 2024-05-14T16:50:07.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f06701135e0 con 0x7f0670102e90 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.814+0000 7f0676f83700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06701138e0 con 0x7f06701084d0 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0674d1f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06701024d0 0x7f067019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0674d1f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06701024d0 0x7f067019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42924/0 (socket says 172.21.15.69:42924) 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0674d1f700 1 -- 172.21.15.69:0/1019816861 learned_addr learned my addr 172.21.15.69:0/1019816861 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f066ffff700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0670102e90 0x7f067019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0675520700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06701084d0 0x7f0670199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0674d1f700 1 -- 172.21.15.69:0/1019816861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06701084d0 msgr2=0x7f0670199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0674d1f700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06701084d0 0x7f0670199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0674d1f700 1 -- 172.21.15.69:0/1019816861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0670102e90 msgr2=0x7f067019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0674d1f700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0670102e90 0x7f067019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0674d1f700 1 -- 172.21.15.69:0/1019816861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067019a240 con 0x7f06701024d0 2024-05-14T16:50:07.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f066ffff700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0670102e90 0x7f067019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:07.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0674d1f700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06701024d0 0x7f067019f8d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f066400d950 tx=0x7f066400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:07.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f066dffb700 1 -- 172.21.15.69:0/1019816861 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664016070 con 0x7f06701024d0 2024-05-14T16:50:07.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f066dffb700 1 -- 172.21.15.69:0/1019816861 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0664010460 con 0x7f06701024d0 2024-05-14T16:50:07.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f066dffb700 1 -- 172.21.15.69:0/1019816861 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066400f620 con 0x7f06701024d0 2024-05-14T16:50:07.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0676f83700 1 -- 172.21.15.69:0/1019816861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067019a530 con 0x7f06701024d0 2024-05-14T16:50:07.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.815+0000 7f0676f83700 1 -- 172.21.15.69:0/1019816861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06701a67a0 con 0x7f06701024d0 2024-05-14T16:50:07.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.816+0000 7f0676f83700 1 -- 172.21.15.69:0/1019816861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067006a230 con 0x7f06701024d0 2024-05-14T16:50:07.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.817+0000 7f066dffb700 1 -- 172.21.15.69:0/1019816861 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f0664019070 con 0x7f06701024d0 2024-05-14T16:50:07.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.818+0000 7f066dffb700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06600b2b90 0x7f06600b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:07.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.819+0000 7f066ffff700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06600b2b90 0x7f06600b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:07.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.819+0000 7f066dffb700 1 -- 172.21.15.69:0/1019816861 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f06640e2460 con 0x7f06701024d0 2024-05-14T16:50:07.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.819+0000 7f066ffff700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06600b2b90 0x7f06600b5050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f065c006010 tx=0x7f065c00b540 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:07.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:07.823+0000 7f066dffb700 1 -- 172.21.15.69:0/1019816861 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06640a9c00 con 0x7f06701024d0 2024-05-14T16:50:08.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.080+0000 7f0676f83700 1 -- 172.21.15.69:0/1019816861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f067002e560 con 0x7f06701024d0 2024-05-14T16:50:08.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.081+0000 7f066dffb700 1 -- 172.21.15.69:0/1019816861 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f06640a9500 con 0x7f06701024d0 2024-05-14T16:50:08.081 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:08.081 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:49:52.473295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:50:08.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.083+0000 7f0676f83700 1 -- 172.21.15.69:0/1019816861 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06600b2b90 msgr2=0x7f06600b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:08.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.083+0000 7f0676f83700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06600b2b90 0x7f06600b5050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f065c006010 tx=0x7f065c00b540 comp rx=0 tx=0).stop 2024-05-14T16:50:08.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.083+0000 7f0676f83700 1 -- 172.21.15.69:0/1019816861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06701024d0 msgr2=0x7f067019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:08.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.083+0000 7f0676f83700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06701024d0 0x7f067019f8d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f066400d950 tx=0x7f066400dd10 comp rx=0 tx=0).stop 2024-05-14T16:50:08.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.083+0000 7f0676f83700 1 -- 172.21.15.69:0/1019816861 shutdown_connections 2024-05-14T16:50:08.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.083+0000 7f0676f83700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06600b2b90 0x7f06600b5050 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:08.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.083+0000 7f0676f83700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06701024d0 0x7f067019f8d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:08.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.083+0000 7f0676f83700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06701084d0 0x7f0670199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:08.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.083+0000 7f0676f83700 1 --2- 172.21.15.69:0/1019816861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0670102e90 0x7f067019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:08.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.083+0000 7f0676f83700 1 -- 172.21.15.69:0/1019816861 >> 172.21.15.69:0/1019816861 conn(0x7f0670078490 msgr2=0x7f0670100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:08.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.084+0000 7f0676f83700 1 -- 172.21.15.69:0/1019816861 shutdown_connections 2024-05-14T16:50:08.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:08.084+0000 7f0676f83700 1 -- 172.21.15.69:0/1019816861 wait complete. 2024-05-14T16:50:08.084 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 25 2024-05-14T16:50:08.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:07 smithi069 ceph-mon[31507]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 107 KiB/s wr, 63 op/s 2024-05-14T16:50:08.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:07 smithi203 ceph-mon[35787]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 107 KiB/s wr, 63 op/s 2024-05-14T16:50:08.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:07 smithi092 ceph-mon[35763]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 107 KiB/s wr, 63 op/s 2024-05-14T16:50:08.776 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:50:09.018 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:09.140 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:08 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1019816861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:09.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:08 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1019816861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:09.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:08 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1019816861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 -- 172.21.15.69:0/4153098941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c102370 msgr2=0x7f3e6c10e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 --2- 172.21.15.69:0/4153098941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c102370 0x7f3e6c10e830 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009cd0 tx=0x7f3e6402a8e0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 -- 172.21.15.69:0/4153098941 shutdown_connections 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 --2- 172.21.15.69:0/4153098941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c102370 0x7f3e6c10e830 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 --2- 172.21.15.69:0/4153098941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c1019b0 0x7f3e6c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 --2- 172.21.15.69:0/4153098941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e6c107970 0x7f3e6c107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 -- 172.21.15.69:0/4153098941 >> 172.21.15.69:0/4153098941 conn(0x7f3e6c078b60 msgr2=0x7f3e6c078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 -- 172.21.15.69:0/4153098941 shutdown_connections 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 -- 172.21.15.69:0/4153098941 wait complete. 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 Processor -- start 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.070+0000 7f3e723c1640 1 -- start start 2024-05-14T16:50:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e723c1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e6c1019b0 0x7f3e6c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e723c1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c102370 0x7f3e6c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e723c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c107970 0x7f3e6c1a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e723c1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c111c80 con 0x7f3e6c102370 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e723c1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c111b00 con 0x7f3e6c107970 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e723c1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c111e00 con 0x7f3e6c1019b0 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c102370 0x7f3e6c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c102370 0x7f3e6c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42930/0 (socket says 172.21.15.69:42930) 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/2384775762 learned_addr learned my addr 172.21.15.69:0/2384775762 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e6c1019b0 0x7f3e6c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e7095e640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c107970 0x7f3e6c1a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/2384775762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e6c1019b0 msgr2=0x7f3e6c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e6b7fe640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e6c1019b0 0x7f3e6c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/2384775762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c107970 msgr2=0x7f3e6c1a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e6b7fe640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c107970 0x7f3e6c1a5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/2384775762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c1a5560 con 0x7f3e6c102370 2024-05-14T16:50:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e7095e640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c107970 0x7f3e6c1a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:50:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e6b7fe640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c102370 0x7f3e6c19a4e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000d950 tx=0x7f3e6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e697fa640 1 -- 172.21.15.69:0/2384775762 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60017070 con 0x7f3e6c102370 2024-05-14T16:50:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e697fa640 1 -- 172.21.15.69:0/2384775762 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e60009c30 con 0x7f3e6c102370 2024-05-14T16:50:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e723c1640 1 -- 172.21.15.69:0/2384775762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6c1a5730 con 0x7f3e6c102370 2024-05-14T16:50:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.071+0000 7f3e697fa640 1 -- 172.21.15.69:0/2384775762 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6000f620 con 0x7f3e6c102370 2024-05-14T16:50:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.072+0000 7f3e723c1640 1 -- 172.21.15.69:0/2384775762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6c1a5c80 con 0x7f3e6c102370 2024-05-14T16:50:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.074+0000 7f3e697fa640 1 -- 172.21.15.69:0/2384775762 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f3e60010460 con 0x7f3e6c102370 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.074+0000 7f3e751bc740 1 Processor -- start 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.074+0000 7f3e751bc740 1 -- start start 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.074+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d7328f0 0x558f0d732cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.074+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.074+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 0x558f0d73fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.074+0000 7f3e751bc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558f0d6a5c30 con 0x558f0d733210 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.074+0000 7f3e751bc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558f0d6957d0 con 0x558f0d73dad0 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.074+0000 7f3e751bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558f0d699810 con 0x558f0d7328f0 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.074+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42934/0 (socket says 172.21.15.69:42934) 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e6bfff640 1 -- 172.21.15.69:0/648390258 learned_addr learned my addr 172.21.15.69:0/648390258 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e7095e640 1 --2- 172.21.15.69:0/648390258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d7328f0 0x558f0d732cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e6b7fe640 1 --2- 172.21.15.69:0/648390258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 0x558f0d73fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e6bfff640 1 -- 172.21.15.69:0/648390258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d7328f0 msgr2=0x558f0d732cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/648390258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d7328f0 0x558f0d732cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e6bfff640 1 -- 172.21.15.69:0/648390258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 msgr2=0x558f0d73fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/648390258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 0x558f0d73fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e6bfff640 1 -- 172.21.15.69:0/648390258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f0d3aa210 con 0x558f0d733210 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e7095e640 1 --2- 172.21.15.69:0/648390258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d7328f0 0x558f0d732cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e697fa640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3e540b2b30 0x7f3e540b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e7095e640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3e540b2b30 0x7f3e540b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/648390258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73d590 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00b710 tx=0x7f3e5c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12f47c50967e498 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e697fa640 1 -- 172.21.15.69:0/2384775762 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f3e600e24f0 con 0x7f3e6c102370 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e527fc640 1 -- 172.21.15.69:0/648390258 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c003bd0 con 0x558f0d733210 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e527fc640 1 -- 172.21.15.69:0/648390258 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5c015070 con 0x558f0d733210 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.075+0000 7f3e527fc640 1 -- 172.21.15.69:0/648390258 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c004de0 con 0x558f0d733210 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e751bc740 1 -- 172.21.15.69:0/648390258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 msgr2=0x558f0d73d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e7095e640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3e540b2b30 0x7f3e540b4ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009a60 tx=0x7f3e640098b0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e751bc740 1 --2- 172.21.15.69:0/648390258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73d590 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00b710 tx=0x7f3e5c004610 comp rx=0 tx=0).stop 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e751bc740 1 -- 172.21.15.69:0/648390258 shutdown_connections 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e751bc740 1 --2- 172.21.15.69:0/648390258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 0x558f0d73fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e751bc740 1 --2- 172.21.15.69:0/648390258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73d590 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e751bc740 1 --2- 172.21.15.69:0/648390258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d7328f0 0x558f0d732cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e751bc740 1 -- 172.21.15.69:0/648390258 >> 172.21.15.69:0/648390258 conn(0x558f0d72ee00 msgr2=0x558f0d72f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e751bc740 1 -- 172.21.15.69:0/648390258 shutdown_connections 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e751bc740 1 -- 172.21.15.69:0/648390258 wait complete. 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.076+0000 7f3e751bc740 1 -- start start 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e751bc740 1 Processor -- start 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d7328f0 0x558f0d73d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d73dad0 0x558f0d73add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e751bc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558f0d37b790 con 0x558f0d733210 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e751bc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558f0d651890 con 0x558f0d7328f0 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e751bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558f0d7407a0 con 0x558f0d73dad0 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e751bc740 10 client.0 _ll_register_callbacks cb 0x558f0d654d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42950/0 (socket says 172.21.15.69:42950) 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/629212878 learned_addr learned my addr 172.21.15.69:0/629212878 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e7095e640 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d73dad0 0x558f0d73add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/629212878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d73dad0 msgr2=0x558f0d73add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e6b7fe640 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d73dad0 0x558f0d73add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/629212878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d7328f0 msgr2=0x558f0d73d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e6b7fe640 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d7328f0 0x558f0d73d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/629212878 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f0d476620 con 0x558f0d733210 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e7095e640 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d73dad0 0x558f0d73add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e6b7fe640 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73a7c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3e600a6800 tx=0x7f3e600a8ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324652b784a88669 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.077+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/629212878 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60020050 con 0x558f0d733210 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/629212878 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f3e600dfdb0 con 0x558f0d733210 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e751bc740 1 -- 172.21.15.69:0/629212878 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e5c015070 con 0x558f0d733210 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/629212878 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e600a90a0 con 0x558f0d733210 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e751bc740 10 client.15291 fetch_fsmap learned FSMap version 25 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e751bc740 1 -- 172.21.15.69:0/629212878 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558f0d74c2f0 con 0x558f0d733210 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/629212878 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5c015070 con 0x558f0d733210 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/629212878 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3e600a87f0 con 0x558f0d733210 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e751bc740 10 client.15291 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e751bc740 10 client.15291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e751bc740 20 client.15291 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e751bc740 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f0d74d820 0x558f0d74fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e751bc740 4 client.15291 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:50:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e751bc740 1 -- 172.21.15.69:0/629212878 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f0d6a7b80 con 0x558f0d74d820 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.078+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f0d74d820 0x558f0d74fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.079+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f0d74d820 0x558f0d74fc10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ae3e86ec5edba5 server_cookie=bcf18a093777923f in_seq=0 out_seq=0 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.079+0000 7f3e4b7fe640 10 client.15291 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/629212878 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f0d6a7b80 con 0x558f0d74d820 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e4b7fe640 10 client.15291 handle_command_reply: tid=0 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 client.15291 shutdown 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 -- 172.21.15.69:0/629212878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 msgr2=0x558f0d73a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73a7c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3e600a6800 tx=0x7f3e600a8ee0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 -- 172.21.15.69:0/629212878 shutdown_connections 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f0d74d820 0x558f0d74fc10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d73dad0 0x558f0d73add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d73a7c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e4b7fe640 0 client.15291 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 --2- 172.21.15.69:0/629212878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d7328f0 0x558f0d73d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 -- 172.21.15.69:0/629212878 >> 172.21.15.69:0/629212878 conn(0x558f0d72ee00 msgr2=0x558f0d72e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 -- 172.21.15.69:0/629212878 shutdown_connections 2024-05-14T16:50:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 1 -- 172.21.15.69:0/629212878 wait complete. 2024-05-14T16:50:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.080+0000 7f3e751bc740 20 client.15291 trim_cache size 0 max 16384 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e751bc740 1 Processor -- start 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e751bc740 1 -- start start 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d7328f0 0x558f0d749670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d733210 0x558f0d755570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 0x558f0d755ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e751bc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1b0e10 con 0x558f0d7328f0 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e751bc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1ae6e0 con 0x558f0d73dad0 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e751bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1ac530 con 0x558f0d733210 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d7328f0 0x558f0d749670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d7328f0 0x558f0d749670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42966/0 (socket says 172.21.15.69:42966) 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e6bfff640 1 -- 172.21.15.69:0/1882961341 learned_addr learned my addr 172.21.15.69:0/1882961341 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e6bfff640 1 -- 172.21.15.69:0/1882961341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d733210 msgr2=0x558f0d755570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/1882961341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d733210 0x558f0d755570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e6bfff640 1 -- 172.21.15.69:0/1882961341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 msgr2=0x558f0d755ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/1882961341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 0x558f0d755ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e6bfff640 1 -- 172.21.15.69:0/1882961341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f0d73b140 con 0x558f0d7328f0 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.095+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/1882961341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d7328f0 0x558f0d749670 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00b710 tx=0x7f3e5c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c186ab2b5a87d26d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e51ffb640 1 -- 172.21.15.69:0/1882961341 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c02b070 con 0x558f0d7328f0 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e51ffb640 1 -- 172.21.15.69:0/1882961341 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5c015050 con 0x558f0d7328f0 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e51ffb640 1 -- 172.21.15.69:0/1882961341 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c026a80 con 0x558f0d7328f0 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e751bc740 1 -- 172.21.15.69:0/1882961341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d7328f0 msgr2=0x558f0d749670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e751bc740 1 --2- 172.21.15.69:0/1882961341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d7328f0 0x558f0d749670 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00b710 tx=0x7f3e5c00b670 comp rx=0 tx=0).stop 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e751bc740 1 -- 172.21.15.69:0/1882961341 shutdown_connections 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e751bc740 1 --2- 172.21.15.69:0/1882961341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 0x558f0d755ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e751bc740 1 --2- 172.21.15.69:0/1882961341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d733210 0x558f0d755570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e751bc740 1 --2- 172.21.15.69:0/1882961341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d7328f0 0x558f0d749670 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e751bc740 1 -- 172.21.15.69:0/1882961341 >> 172.21.15.69:0/1882961341 conn(0x558f0d72ee00 msgr2=0x558f0d739090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e751bc740 1 -- 172.21.15.69:0/1882961341 shutdown_connections 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.096+0000 7f3e751bc740 1 -- 172.21.15.69:0/1882961341 wait complete. 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e751bc740 1 -- start start 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e751bc740 1 Processor -- start 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d7328f0 0x558f0d7438c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d743e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e751bc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 0x558f0d7b7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e751bc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1acba0 con 0x558f0d733210 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e751bc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1ae010 con 0x558f0d73dad0 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e751bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1ad270 con 0x558f0d7328f0 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e751bc740 10 client.0 _ll_register_callbacks cb 0x7f3e600a9360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d743e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d743e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42974/0 (socket says 172.21.15.69:42974) 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/1031395308 learned_addr learned my addr 172.21.15.69:0/1031395308 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/1031395308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d7328f0 msgr2=0x558f0d7438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e6b7fe640 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d7328f0 0x558f0d7438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/1031395308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 msgr2=0x558f0d7b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e6b7fe640 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 0x558f0d7b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e6b7fe640 1 -- 172.21.15.69:0/1031395308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f0d74ff80 con 0x558f0d733210 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e6b7fe640 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d743e00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3e600a9240 tx=0x7f3e600a52f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6da44da8c567f69 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.097+0000 7f3e4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/1031395308 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60020070 con 0x558f0d733210 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/1031395308 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f3e600a5b80 con 0x558f0d733210 2024-05-14T16:50:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e751bc740 1 -- 172.21.15.69:0/1031395308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e5c015050 con 0x558f0d733210 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/1031395308 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e600dfa10 con 0x558f0d733210 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e751bc740 10 client.15297 fetch_fsmap learned FSMap version 25 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e751bc740 1 -- 172.21.15.69:0/1031395308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558f0d7b8570 con 0x558f0d733210 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/1031395308 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e600c3030 con 0x558f0d733210 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/1031395308 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3e6008fab0 con 0x558f0d733210 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e751bc740 10 client.15297 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e751bc740 10 client.15297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e751bc740 20 client.15297 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e751bc740 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f0d7b8cf0 0x558f0d7bb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e751bc740 4 client.15297 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e751bc740 1 -- 172.21.15.69:0/1031395308 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f0d6a1360 con 0x558f0d7b8cf0 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.098+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f0d7b8cf0 0x558f0d7bb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.099+0000 7f3e4b7fe640 10 client.15297 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.099+0000 7f3e6bfff640 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f0d7b8cf0 0x558f0d7bb130 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b60fed44c9ecc9 server_cookie=ffe9df6d26f376cb in_seq=0 out_seq=0 2024-05-14T16:50:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e4b7fe640 1 -- 172.21.15.69:0/1031395308 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558f0d6a1360 con 0x558f0d7b8cf0 2024-05-14T16:50:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e4b7fe640 10 client.15297 handle_command_reply: tid=0 2024-05-14T16:50:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 client.15297 shutdown 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 -- 172.21.15.69:0/1031395308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 msgr2=0x558f0d743e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d743e00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3e600a9240 tx=0x7f3e600a52f0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 -- 172.21.15.69:0/1031395308 shutdown_connections 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f0d7b8cf0 0x558f0d7bb130 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f0d73dad0 0x558f0d7b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e4b7fe640 0 client.15297 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f0d733210 0x558f0d743e00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 --2- 172.21.15.69:0/1031395308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f0d7328f0 0x558f0d7438c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 -- 172.21.15.69:0/1031395308 >> 172.21.15.69:0/1031395308 conn(0x558f0d72ee00 msgr2=0x558f0d748660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 -- 172.21.15.69:0/1031395308 shutdown_connections 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.110+0000 7f3e751bc740 1 -- 172.21.15.69:0/1031395308 wait complete. 2024-05-14T16:50:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e751bc740 20 client.15297 trim_cache size 0 max 16384 2024-05-14T16:50:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:50:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:50:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 -- 172.21.15.69:0/2384775762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3e540b2b30 msgr2=0x7f3e540b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3e540b2b30 0x7f3e540b4ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009a60 tx=0x7f3e640098b0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 -- 172.21.15.69:0/2384775762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c102370 msgr2=0x7f3e6c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c102370 0x7f3e6c19a4e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000d950 tx=0x7f3e6000de10 comp rx=0 tx=0).stop 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 -- 172.21.15.69:0/2384775762 shutdown_connections 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3e540b2b30 0x7f3e540b4ff0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c107970 0x7f3e6c1a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c102370 0x7f3e6c19a4e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 --2- 172.21.15.69:0/2384775762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e6c1019b0 0x7f3e6c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 -- 172.21.15.69:0/2384775762 >> 172.21.15.69:0/2384775762 conn(0x7f3e6c078b60 msgr2=0x7f3e6c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 -- 172.21.15.69:0/2384775762 shutdown_connections 2024-05-14T16:50:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:10.111+0000 7f3e723c1640 1 -- 172.21.15.69:0/2384775762 wait complete. 2024-05-14T16:50:10.152 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:09 smithi069 ceph-mon[31507]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 98 KiB/s wr, 58 op/s 2024-05-14T16:50:10.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:09 smithi203 ceph-mon[35787]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 98 KiB/s wr, 58 op/s 2024-05-14T16:50:10.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:09 smithi092 ceph-mon[35763]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 98 KiB/s wr, 58 op/s 2024-05-14T16:50:10.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.689+0000 7f9661386700 1 -- 172.21.15.69:0/1916816558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f965c06bcb0 msgr2=0x7f965c06c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.689+0000 7f9661386700 1 --2- 172.21.15.69:0/1916816558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f965c06bcb0 0x7f965c06c130 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9650009990 tx=0x7f9650009ca0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.689+0000 7f9661386700 1 -- 172.21.15.69:0/1916816558 shutdown_connections 2024-05-14T16:50:10.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.689+0000 7f9661386700 1 --2- 172.21.15.69:0/1916816558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f965c06c670 0x7f965c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.689+0000 7f9661386700 1 --2- 172.21.15.69:0/1916816558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965c1095b0 0x7f965c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.689+0000 7f9661386700 1 --2- 172.21.15.69:0/1916816558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f965c06bcb0 0x7f965c06c130 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.689+0000 7f9661386700 1 -- 172.21.15.69:0/1916816558 >> 172.21.15.69:0/1916816558 conn(0x7f965c0784b0 msgr2=0x7f965c0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:10.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.689+0000 7f9661386700 1 -- 172.21.15.69:0/1916816558 shutdown_connections 2024-05-14T16:50:10.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.689+0000 7f9661386700 1 -- 172.21.15.69:0/1916816558 wait complete. 2024-05-14T16:50:10.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.690+0000 7f9661386700 1 Processor -- start 2024-05-14T16:50:10.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.690+0000 7f9661386700 1 -- start start 2024-05-14T16:50:10.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.690+0000 7f9661386700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f965c06bcb0 0x7f965c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.690+0000 7f9661386700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965c06c670 0x7f965c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f9661386700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f965c1095b0 0x7f965c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f9661386700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f965c113810 con 0x7f965c1095b0 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f9661386700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f965c113690 con 0x7f965c06c670 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f9661386700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f965c113990 con 0x7f965c06bcb0 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f965c1095b0 0x7f965c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f965c1095b0 0x7f965c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42996/0 (socket says 172.21.15.69:42996) 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965b7fe700 1 -- 172.21.15.69:0/2974595470 learned_addr learned my addr 172.21.15.69:0/2974595470 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965affd700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f965c06bcb0 0x7f965c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965a7fc700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965c06c670 0x7f965c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965b7fe700 1 -- 172.21.15.69:0/2974595470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f965c06bcb0 msgr2=0x7f965c199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965b7fe700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f965c06bcb0 0x7f965c199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965b7fe700 1 -- 172.21.15.69:0/2974595470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965c06c670 msgr2=0x7f965c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965b7fe700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965c06c670 0x7f965c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965b7fe700 1 -- 172.21.15.69:0/2974595470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c19edd0 con 0x7f965c1095b0 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965a7fc700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965c06c670 0x7f965c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f965b7fe700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f965c1095b0 0x7f965c19e690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f965400b970 tx=0x7f965400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f9643fff700 1 -- 172.21.15.69:0/2974595470 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654019070 con 0x7f965c1095b0 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f9643fff700 1 -- 172.21.15.69:0/2974595470 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f965400cde0 con 0x7f965c1095b0 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f9661386700 1 -- 172.21.15.69:0/2974595470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965c19f0c0 con 0x7f965c1095b0 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f9643fff700 1 -- 172.21.15.69:0/2974595470 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654007370 con 0x7f965c1095b0 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.691+0000 7f9661386700 1 -- 172.21.15.69:0/2974595470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965c1a6940 con 0x7f965c1095b0 2024-05-14T16:50:10.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.692+0000 7f9661386700 1 -- 172.21.15.69:0/2974595470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965c10ba90 con 0x7f965c1095b0 2024-05-14T16:50:10.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.693+0000 7f9643fff700 1 -- 172.21.15.69:0/2974595470 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f965401c070 con 0x7f965c1095b0 2024-05-14T16:50:10.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.694+0000 7f9643fff700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f96440b2b10 0x7f96440b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:10.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.694+0000 7f965affd700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f96440b2b10 0x7f96440b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:10.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.694+0000 7f9643fff700 1 -- 172.21.15.69:0/2974595470 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f96540e2410 con 0x7f965c1095b0 2024-05-14T16:50:10.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.695+0000 7f965affd700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f96440b2b10 0x7f96440b4fd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f964c009a10 tx=0x7f964c009190 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:10.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.700+0000 7f9643fff700 1 -- 172.21.15.69:0/2974595470 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9654023070 con 0x7f965c1095b0 2024-05-14T16:50:10.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.948+0000 7f9661386700 1 -- 172.21.15.69:0/2974595470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f965c19b120 con 0x7f965c1095b0 2024-05-14T16:50:10.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.949+0000 7f9643fff700 1 -- 172.21.15.69:0/2974595470 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f96540a9cc0 con 0x7f965c1095b0 2024-05-14T16:50:10.949 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:10.949 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:49:52.473295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:50:10.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.951+0000 7f9661386700 1 -- 172.21.15.69:0/2974595470 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f96440b2b10 msgr2=0x7f96440b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.951+0000 7f9661386700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f96440b2b10 0x7f96440b4fd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f964c009a10 tx=0x7f964c009190 comp rx=0 tx=0).stop 2024-05-14T16:50:10.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.951+0000 7f9661386700 1 -- 172.21.15.69:0/2974595470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f965c1095b0 msgr2=0x7f965c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:10.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.951+0000 7f9661386700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f965c1095b0 0x7f965c19e690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f965400b970 tx=0x7f965400bc80 comp rx=0 tx=0).stop 2024-05-14T16:50:10.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.951+0000 7f9661386700 1 -- 172.21.15.69:0/2974595470 shutdown_connections 2024-05-14T16:50:10.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.951+0000 7f9661386700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f96440b2b10 0x7f96440b4fd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.951+0000 7f9661386700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f965c1095b0 0x7f965c19e690 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.951+0000 7f9661386700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f965c06bcb0 0x7f965c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.951+0000 7f9661386700 1 --2- 172.21.15.69:0/2974595470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965c06c670 0x7f965c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:10.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.951+0000 7f9661386700 1 -- 172.21.15.69:0/2974595470 >> 172.21.15.69:0/2974595470 conn(0x7f965c0784b0 msgr2=0x7f965c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:10.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.952+0000 7f9661386700 1 -- 172.21.15.69:0/2974595470 shutdown_connections 2024-05-14T16:50:10.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:10.952+0000 7f9661386700 1 -- 172.21.15.69:0/2974595470 wait complete. 2024-05-14T16:50:10.952 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 25 2024-05-14T16:50:11.652 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:50:11.891 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:11.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:11 smithi069 ceph-mon[31507]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-05-14T16:50:11.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:11 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2974595470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:12.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:11 smithi203 ceph-mon[35787]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-05-14T16:50:12.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:11 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2974595470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:12.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:11 smithi092 ceph-mon[35763]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-05-14T16:50:12.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:11 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2974595470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:13.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.575+0000 7f47792e2700 1 -- 172.21.15.69:0/4169312776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4774102440 msgr2=0x7f47741028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:13.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.575+0000 7f47792e2700 1 --2- 172.21.15.69:0/4169312776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4774102440 0x7f47741028c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4768009990 tx=0x7f4768009ca0 comp rx=0 tx=0).stop 2024-05-14T16:50:13.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.575+0000 7f47792e2700 1 -- 172.21.15.69:0/4169312776 shutdown_connections 2024-05-14T16:50:13.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.575+0000 7f47792e2700 1 --2- 172.21.15.69:0/4169312776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4774102e00 0x7f477410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:13.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.575+0000 7f47792e2700 1 --2- 172.21.15.69:0/4169312776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4774108520 0x7f4774108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:13.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.575+0000 7f47792e2700 1 --2- 172.21.15.69:0/4169312776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4774102440 0x7f47741028c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:13.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.575+0000 7f47792e2700 1 -- 172.21.15.69:0/4169312776 >> 172.21.15.69:0/4169312776 conn(0x7f47740793f0 msgr2=0x7f4774079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:13.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.575+0000 7f47792e2700 1 -- 172.21.15.69:0/4169312776 shutdown_connections 2024-05-14T16:50:13.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.575+0000 7f47792e2700 1 -- 172.21.15.69:0/4169312776 wait complete. 2024-05-14T16:50:13.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.575+0000 7f47792e2700 1 Processor -- start 2024-05-14T16:50:13.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f47792e2700 1 -- start start 2024-05-14T16:50:13.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f47792e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4774102440 0x7f477419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:13.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f47792e2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4774102e00 0x7f477419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:13.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f47792e2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4774108520 0x7f47741a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:13.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f47792e2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f47741125d0 con 0x7f4774102440 2024-05-14T16:50:13.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f47792e2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4774112450 con 0x7f4774108520 2024-05-14T16:50:13.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f47792e2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4774112750 con 0x7f4774102e00 2024-05-14T16:50:13.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f4772ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4774102440 0x7f477419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:13.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f4772ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4774102440 0x7f477419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43020/0 (socket says 172.21.15.69:43020) 2024-05-14T16:50:13.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f4772ffd700 1 -- 172.21.15.69:0/3104645809 learned_addr learned my addr 172.21.15.69:0/3104645809 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f47737fe700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4774108520 0x7f47741a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f47727fc700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4774102e00 0x7f477419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f4772ffd700 1 -- 172.21.15.69:0/3104645809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4774102e00 msgr2=0x7f477419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f4772ffd700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4774102e00 0x7f477419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f4772ffd700 1 -- 172.21.15.69:0/3104645809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4774108520 msgr2=0x7f47741a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f4772ffd700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4774108520 0x7f47741a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f4772ffd700 1 -- 172.21.15.69:0/3104645809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47741a5f50 con 0x7f4774102440 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.576+0000 7f4772ffd700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4774102440 0x7f477419a990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f476400d980 tx=0x7f476400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.577+0000 7f475bfff700 1 -- 172.21.15.69:0/3104645809 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764016070 con 0x7f4774102440 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.577+0000 7f47792e2700 1 -- 172.21.15.69:0/3104645809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47741987e0 con 0x7f4774102440 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.577+0000 7f475bfff700 1 -- 172.21.15.69:0/3104645809 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4764004830 con 0x7f4774102440 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.577+0000 7f475bfff700 1 -- 172.21.15.69:0/3104645809 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47640054c0 con 0x7f4774102440 2024-05-14T16:50:13.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.577+0000 7f47792e2700 1 -- 172.21.15.69:0/3104645809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4774198d70 con 0x7f4774102440 2024-05-14T16:50:13.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.578+0000 7f47792e2700 1 -- 172.21.15.69:0/3104645809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f477406a230 con 0x7f4774102440 2024-05-14T16:50:13.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.579+0000 7f475bfff700 1 -- 172.21.15.69:0/3104645809 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f4764004220 con 0x7f4774102440 2024-05-14T16:50:13.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.580+0000 7f475bfff700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f475c0b2b10 0x7f475c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:13.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.580+0000 7f47727fc700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f475c0b2b10 0x7f475c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:13.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.580+0000 7f475bfff700 1 -- 172.21.15.69:0/3104645809 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f47640e2b70 con 0x7f4774102440 2024-05-14T16:50:13.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.580+0000 7f47727fc700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f475c0b2b10 0x7f475c0b4fd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4768006010 tx=0x7f476801a040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:13.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.583+0000 7f475bfff700 1 -- 172.21.15.69:0/3104645809 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f476401f020 con 0x7f4774102440 2024-05-14T16:50:13.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.787+0000 7f47792e2700 1 -- 172.21.15.69:0/3104645809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f477402e560 con 0x7f4774102440 2024-05-14T16:50:13.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.788+0000 7f475bfff700 1 -- 172.21.15.69:0/3104645809 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23040 (secure 0 0 0) 0x7f47640aa310 con 0x7f4774102440 2024-05-14T16:50:13.789 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:13.789 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":81,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T16:49:54.600559+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:46:38.843325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:46:42.395401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4012740750":"2024-05-15T16:49:54.600513+0000","172.21.15.69:6800/4012740750":"2024-05-15T16:49:54.600513+0000","172.21.15.69:0/70559253":"2024-05-15T16:49:54.600513+0000","172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2944165482":"2024-05-15T16:49:54.600513+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/1931804604":"2024-05-15T16:49:54.600513+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/3176577502":"2024-05-15T16:49:54.600513+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:50:13.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 -- 172.21.15.69:0/3104645809 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f475c0b2b10 msgr2=0x7f475c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:13.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f475c0b2b10 0x7f475c0b4fd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4768006010 tx=0x7f476801a040 comp rx=0 tx=0).stop 2024-05-14T16:50:13.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 -- 172.21.15.69:0/3104645809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4774102440 msgr2=0x7f477419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:13.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4774102440 0x7f477419a990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f476400d980 tx=0x7f476400dc90 comp rx=0 tx=0).stop 2024-05-14T16:50:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 -- 172.21.15.69:0/3104645809 shutdown_connections 2024-05-14T16:50:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f475c0b2b10 0x7f475c0b4fd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4774102440 0x7f477419a990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4774108520 0x7f47741a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 --2- 172.21.15.69:0/3104645809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4774102e00 0x7f477419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 -- 172.21.15.69:0/3104645809 >> 172.21.15.69:0/3104645809 conn(0x7f47740793f0 msgr2=0x7f4774100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 -- 172.21.15.69:0/3104645809 shutdown_connections 2024-05-14T16:50:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:13.791+0000 7f47792e2700 1 -- 172.21.15.69:0/3104645809 wait complete. 2024-05-14T16:50:14.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:13 smithi069 ceph-mon[31507]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 52 KiB/s wr, 24 op/s 2024-05-14T16:50:14.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:13 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3104645809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:50:14.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:13 smithi203 ceph-mon[35787]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 52 KiB/s wr, 24 op/s 2024-05-14T16:50:14.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:13 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3104645809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:50:14.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:13 smithi092 ceph-mon[35763]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 52 KiB/s wr, 24 op/s 2024-05-14T16:50:14.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:13 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3104645809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:50:14.378 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-14T16:50:14.616 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:16.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:15 smithi069 ceph-mon[31507]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-05-14T16:50:16.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:15 smithi203 ceph-mon[35787]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-05-14T16:50:16.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:15 smithi092 ceph-mon[35763]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-05-14T16:50:16.996 INFO:teuthology.orchestra.run.smithi069.stdout:{"run_dir":"/var/run/ceph"} 2024-05-14T16:50:16.997 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-14T16:50:16.997 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 710 2024-05-14T16:50:16.997 DEBUG:tasks.quiescer:Yielding 2024-05-14T16:50:16.998 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T16:50:17.008 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T16:50:17.008 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T16:50:17.008 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-14T16:50:17.008 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:50:17.045 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:17.059 INFO:teuthology.orchestra.run.smithi069.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-14T16:50:17.061 INFO:teuthology.orchestra.run.smithi069.stdout:0@356e185e-1210-11ef-bc98-c7b262605968.cephfs=/volumes/qa/sv_1/265d528f-6ed7-4d79-b5c4-99c32193f887 340G 0 340G 0% /tmp/cephfs 2024-05-14T16:50:17.062 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-14T16:50:17.153 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-14T16:50:17.196 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-14T16:50:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:17.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:17 smithi069 ceph-mon[31507]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-14T16:50:18.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:17 smithi203 ceph-mon[35787]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-14T16:50:18.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:17 smithi092 ceph-mon[35763]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-14T16:50:19.067 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-14T16:50:19.144 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-14T16:50:19.147 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-14T16:50:19.225 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-14T16:50:19.300 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/: 2024-05-14T16:50:19.300 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:19.300 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 . 2024-05-14T16:50:19.300 INFO:teuthology.orchestra.run.smithi069.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 .. 2024-05-14T16:50:19.300 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 data 2024-05-14T16:50:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:19.300 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data: 2024-05-14T16:50:19.300 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:19.301 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:19.301 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 .. 2024-05-14T16:50:19.302 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-14T16:50:19.387 INFO:teuthology.orchestra.run.smithi069.stdout:[Service] 2024-05-14T16:50:19.387 INFO:teuthology.orchestra.run.smithi069.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-14T16:50:19.387 INFO:teuthology.orchestra.run.smithi069.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-14T16:50:19.390 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-14T16:50:19.529 INFO:teuthology.orchestra.run.smithi069.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-14T16:50:19.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.948+0000 7f1b621fa700 1 -- 172.21.15.69:0/1254241665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b5c102df0 msgr2=0x7f1b5c10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:19.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.948+0000 7f1b621fa700 1 --2- 172.21.15.69:0/1254241665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b5c102df0 0x7f1b5c10f2e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400b0a0 tx=0x7f1b5400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:50:19.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.948+0000 7f1b621fa700 1 -- 172.21.15.69:0/1254241665 shutdown_connections 2024-05-14T16:50:19.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.948+0000 7f1b621fa700 1 --2- 172.21.15.69:0/1254241665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b5c102df0 0x7f1b5c10f2e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:19.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.948+0000 7f1b621fa700 1 --2- 172.21.15.69:0/1254241665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b5c108510 0x7f1b5c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:19.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.949+0000 7f1b621fa700 1 --2- 172.21.15.69:0/1254241665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b5c102430 0x7f1b5c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:19.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.949+0000 7f1b621fa700 1 -- 172.21.15.69:0/1254241665 >> 172.21.15.69:0/1254241665 conn(0x7f1b5c0793b0 msgr2=0x7f1b5c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:19.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.949+0000 7f1b621fa700 1 -- 172.21.15.69:0/1254241665 shutdown_connections 2024-05-14T16:50:19.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.949+0000 7f1b621fa700 1 -- 172.21.15.69:0/1254241665 wait complete. 2024-05-14T16:50:19.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.949+0000 7f1b621fa700 1 Processor -- start 2024-05-14T16:50:19.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.949+0000 7f1b621fa700 1 -- start start 2024-05-14T16:50:19.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b621fa700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b5c102430 0x7f1b5c104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:19.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b621fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b5c102df0 0x7f1b5c1045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:19.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b621fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b5c108510 0x7f1b5c104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:19.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b621fa700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c112640 con 0x7f1b5c102430 2024-05-14T16:50:19.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b621fa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c1124c0 con 0x7f1b5c102df0 2024-05-14T16:50:19.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b621fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c1127c0 con 0x7f1b5c108510 2024-05-14T16:50:19.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b5c102430 0x7f1b5c104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:19.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b5c102430 0x7f1b5c104070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33388/0 (socket says 172.21.15.69:33388) 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5b7fe700 1 -- 172.21.15.69:0/3467613408 learned_addr learned my addr 172.21.15.69:0/3467613408 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5affd700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b5c102df0 0x7f1b5c1045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5bfff700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b5c108510 0x7f1b5c104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5b7fe700 1 -- 172.21.15.69:0/3467613408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b5c108510 msgr2=0x7f1b5c104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5b7fe700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b5c108510 0x7f1b5c104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5b7fe700 1 -- 172.21.15.69:0/3467613408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b5c102df0 msgr2=0x7f1b5c1045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5b7fe700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b5c102df0 0x7f1b5c1045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5b7fe700 1 -- 172.21.15.69:0/3467613408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c1a81d0 con 0x7f1b5c102430 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.950+0000 7f1b5b7fe700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b5c102430 0x7f1b5c104070 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00b530 tx=0x7f1b4c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.951+0000 7f1b58ff9700 1 -- 172.21.15.69:0/3467613408 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c015020 con 0x7f1b5c102430 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.951+0000 7f1b621fa700 1 -- 172.21.15.69:0/3467613408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c1a84c0 con 0x7f1b5c102430 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.951+0000 7f1b58ff9700 1 -- 172.21.15.69:0/3467613408 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4c011ce0 con 0x7f1b5c102430 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.951+0000 7f1b58ff9700 1 -- 172.21.15.69:0/3467613408 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c010dd0 con 0x7f1b5c102430 2024-05-14T16:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.951+0000 7f1b621fa700 1 -- 172.21.15.69:0/3467613408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c1a8a00 con 0x7f1b5c102430 2024-05-14T16:50:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.952+0000 7f1b621fa700 1 -- 172.21.15.69:0/3467613408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5c06a230 con 0x7f1b5c102430 2024-05-14T16:50:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.953+0000 7f1b58ff9700 1 -- 172.21.15.69:0/3467613408 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f1b4c018020 con 0x7f1b5c102430 2024-05-14T16:50:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.954+0000 7f1b58ff9700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b480b2b10 0x7f1b480b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.954+0000 7f1b5affd700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b480b2b10 0x7f1b480b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.954+0000 7f1b58ff9700 1 -- 172.21.15.69:0/3467613408 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f1b4c0e2410 con 0x7f1b5c102430 2024-05-14T16:50:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.954+0000 7f1b5affd700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b480b2b10 0x7f1b480b4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c079b00 tx=0x7f1b44009260 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:19.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:19.957+0000 7f1b58ff9700 1 -- 172.21.15.69:0/3467613408 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1b4c01f020 con 0x7f1b5c102430 2024-05-14T16:50:20.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:19 smithi069 ceph-mon[31507]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-14T16:50:20.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:19 smithi203 ceph-mon[35787]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-14T16:50:20.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.238+0000 7f1b621fa700 1 -- 172.21.15.69:0/3467613408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b5c1055d0 con 0x7f1b5c102430 2024-05-14T16:50:20.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.239+0000 7f1b58ff9700 1 -- 172.21.15.69:0/3467613408 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f1b5c1055d0 con 0x7f1b5c102430 2024-05-14T16:50:20.239 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:20.239 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:49:52.473295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1824212740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1824212740},{"type":"v1","addr":"172.21.15.203:6837","nonce":1824212740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 -- 172.21.15.69:0/3467613408 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b480b2b10 msgr2=0x7f1b480b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b480b2b10 0x7f1b480b4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c079b00 tx=0x7f1b44009260 comp rx=0 tx=0).stop 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 -- 172.21.15.69:0/3467613408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b5c102430 msgr2=0x7f1b5c104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b5c102430 0x7f1b5c104070 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00b530 tx=0x7f1b4c00b840 comp rx=0 tx=0).stop 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 -- 172.21.15.69:0/3467613408 shutdown_connections 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b480b2b10 0x7f1b480b4fd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b5c102430 0x7f1b5c104070 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b5c108510 0x7f1b5c104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 --2- 172.21.15.69:0/3467613408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b5c102df0 0x7f1b5c1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 -- 172.21.15.69:0/3467613408 >> 172.21.15.69:0/3467613408 conn(0x7f1b5c0793b0 msgr2=0x7f1b5c06be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:20.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 -- 172.21.15.69:0/3467613408 shutdown_connections 2024-05-14T16:50:20.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:20.241+0000 7f1b621fa700 1 -- 172.21.15.69:0/3467613408 wait complete. 2024-05-14T16:50:20.242 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 25 2024-05-14T16:50:20.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:19 smithi092 ceph-mon[35763]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-14T16:50:21.011 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T16:46:45.333110+0000', 'modified': '2024-05-14T16:49:52.473295+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24446, 'mds_1': 24433, 'mds_2': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24395': {'gid': 24395, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6835/2453543352', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 2453543352}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 2453543352}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/3334129544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 3334129544}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 3334129544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6837/1824212740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 1824212740}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 1824212740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.92:6837/1194779586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 1194779586}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1194779586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.203:6839/398835794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 398835794}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 398835794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6839/2169416341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 2169416341}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 2169416341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24446, 'qdb_cluster': [24422, 24433, 24446]} 2024-05-14T16:50:21.011 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T16:50:21.011 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-14T16:50:21.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:20 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3467613408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:21.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:20 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3467613408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:21.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:20 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3467613408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:21.261 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:22.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:21 smithi069 ceph-mon[31507]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-05-14T16:50:22.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:21 smithi203 ceph-mon[35787]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-05-14T16:50:22.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:21 smithi092 ceph-mon[35763]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-05-14T16:50:23.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.019+0000 7fa81a533700 1 -- 172.21.15.69:0/300240883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa814103560 msgr2=0x7fa81410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.019+0000 7fa81a533700 1 --2- 172.21.15.69:0/300240883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa814103560 0x7fa81410f550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00b0a0 tx=0x7fa80c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.019+0000 7fa81a533700 1 -- 172.21.15.69:0/300240883 shutdown_connections 2024-05-14T16:50:23.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.019+0000 7fa81a533700 1 --2- 172.21.15.69:0/300240883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa814103560 0x7fa81410f550 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.019+0000 7fa81a533700 1 --2- 172.21.15.69:0/300240883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa814108710 0x7fa814108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.019+0000 7fa81a533700 1 --2- 172.21.15.69:0/300240883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa814102ba0 0x7fa814103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.019+0000 7fa81a533700 1 -- 172.21.15.69:0/300240883 >> 172.21.15.69:0/300240883 conn(0x7fa814078a40 msgr2=0x7fa8140ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.019+0000 7fa81a533700 1 -- 172.21.15.69:0/300240883 shutdown_connections 2024-05-14T16:50:23.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.020+0000 7fa81a533700 1 -- 172.21.15.69:0/300240883 wait complete. 2024-05-14T16:50:23.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.020+0000 7fa81a533700 1 Processor -- start 2024-05-14T16:50:23.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.020+0000 7fa81a533700 1 -- start start 2024-05-14T16:50:23.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa81a533700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa814102ba0 0x7fa81419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa81a533700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa814103560 0x7fa81419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa81a533700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa814108710 0x7fa8141a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa81a533700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141128d0 con 0x7fa814108710 2024-05-14T16:50:23.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa81a533700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa814112750 con 0x7fa814102ba0 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa81a533700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa814112a50 con 0x7fa814103560 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa813fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa814102ba0 0x7fa81419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa818ad0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa814108710 0x7fa8141a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa813fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa814102ba0 0x7fa81419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:56418/0 (socket says 172.21.15.69:56418) 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa818ad0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa814108710 0x7fa8141a5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33396/0 (socket says 172.21.15.69:33396) 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa813fff700 1 -- 172.21.15.69:0/720177966 learned_addr learned my addr 172.21.15.69:0/720177966 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa8137fe700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa814103560 0x7fa81419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa813fff700 1 -- 172.21.15.69:0/720177966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa814103560 msgr2=0x7fa81419b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa813fff700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa814103560 0x7fa81419b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa813fff700 1 -- 172.21.15.69:0/720177966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa814108710 msgr2=0x7fa8141a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa813fff700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa814108710 0x7fa8141a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa813fff700 1 -- 172.21.15.69:0/720177966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8141a6210 con 0x7fa814102ba0 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.021+0000 7fa813fff700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa814102ba0 0x7fa81419ac50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa80400d980 tx=0x7fa80400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.022+0000 7fa8117fa700 1 -- 172.21.15.69:0/720177966 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804017070 con 0x7fa814102ba0 2024-05-14T16:50:23.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.022+0000 7fa8117fa700 1 -- 172.21.15.69:0/720177966 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa80400fd40 con 0x7fa814102ba0 2024-05-14T16:50:23.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.022+0000 7fa8117fa700 1 -- 172.21.15.69:0/720177966 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80400e3b0 con 0x7fa814102ba0 2024-05-14T16:50:23.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.022+0000 7fa81a533700 1 -- 172.21.15.69:0/720177966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8141a6440 con 0x7fa814102ba0 2024-05-14T16:50:23.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.022+0000 7fa81a533700 1 -- 172.21.15.69:0/720177966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8141a6930 con 0x7fa814102ba0 2024-05-14T16:50:23.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.024+0000 7fa8117fa700 1 -- 172.21.15.69:0/720177966 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fa80400f890 con 0x7fa814102ba0 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.025+0000 7fa8117fa700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7fc0b2b10 0x7fa7fc0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.025+0000 7fa8137fe700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7fc0b2b10 0x7fa7fc0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa8117fa700 1 -- 172.21.15.69:0/720177966 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7fa8040e23a0 con 0x7fa814102ba0 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa8137fe700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7fc0b2b10 0x7fa7fc0b4fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa8141073a0 tx=0x7fa808009260 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa824dc9b80 1 Processor -- start 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa824dc9b80 1 -- start start 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb894fa0 0x55acbb895380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb88f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb88fdc0 0x55acbb890220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55acbb7edd20 con 0x55acbb88fdc0 2024-05-14T16:50:23.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55acbb7dd8c0 con 0x55acbb8958c0 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55acbb7e1900 con 0x55acbb894fa0 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa818ad0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb88fdc0 0x55acbb890220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa8137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb88f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa8137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb88f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:56428/0 (socket says 172.21.15.69:56428) 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.026+0000 7fa8137fe700 1 -- 172.21.15.69:0/2156216995 learned_addr learned my addr 172.21.15.69:0/2156216995 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.027+0000 7fa813fff700 1 --2- 172.21.15.69:0/2156216995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb894fa0 0x55acbb895380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.027+0000 7fa813fff700 1 -- 172.21.15.69:0/2156216995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 msgr2=0x55acbb88f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.027+0000 7fa813fff700 1 --2- 172.21.15.69:0/2156216995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb88f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.027+0000 7fa813fff700 1 -- 172.21.15.69:0/2156216995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb88fdc0 msgr2=0x55acbb890220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.027+0000 7fa813fff700 1 --2- 172.21.15.69:0/2156216995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb88fdc0 0x55acbb890220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.027+0000 7fa813fff700 1 -- 172.21.15.69:0/2156216995 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acbb786f60 con 0x55acbb894fa0 2024-05-14T16:50:23.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.027+0000 7fa813fff700 1 --2- 172.21.15.69:0/2156216995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb894fa0 0x55acbb895380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa8040df920 tx=0x7fa8040a8b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcfa7c164deb37a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.027+0000 7fa7fa7fc700 1 -- 172.21.15.69:0/2156216995 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804020070 con 0x55acbb894fa0 2024-05-14T16:50:23.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.027+0000 7fa7fa7fc700 1 -- 172.21.15.69:0/2156216995 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8040a8ef0 con 0x55acbb894fa0 2024-05-14T16:50:23.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.027+0000 7fa7fa7fc700 1 -- 172.21.15.69:0/2156216995 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8040b2010 con 0x55acbb894fa0 2024-05-14T16:50:23.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.028+0000 7fa824dc9b80 1 -- 172.21.15.69:0/2156216995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb894fa0 msgr2=0x55acbb895380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.028+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/2156216995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb894fa0 0x55acbb895380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa8040df920 tx=0x7fa8040a8b50 comp rx=0 tx=0).stop 2024-05-14T16:50:23.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.028+0000 7fa824dc9b80 1 -- 172.21.15.69:0/2156216995 shutdown_connections 2024-05-14T16:50:23.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.028+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/2156216995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb88fdc0 0x55acbb890220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.028+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/2156216995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb894fa0 0x55acbb895380 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.028+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/2156216995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb88f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.028+0000 7fa824dc9b80 1 -- 172.21.15.69:0/2156216995 >> 172.21.15.69:0/2156216995 conn(0x55acbb7bcd60 msgr2=0x55acbb889e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.028+0000 7fa824dc9b80 1 -- 172.21.15.69:0/2156216995 shutdown_connections 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.028+0000 7fa824dc9b80 1 -- 172.21.15.69:0/2156216995 wait complete. 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa824dc9b80 1 -- start start 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa824dc9b80 1 Processor -- start 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb88fdc0 0x55acbb89b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 0x55acbb898a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb898f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55acbb6e3f50 con 0x55acbb894fa0 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55acbb745d50 con 0x55acbb8958c0 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55acbb89c2d0 con 0x55acbb88fdc0 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa824dc9b80 10 client.0 _ll_register_callbacks cb 0x55acbb778900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:23.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa818ad0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb898f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa818ad0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb898f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:56442/0 (socket says 172.21.15.69:56442) 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa8137fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 0x55acbb898a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.029+0000 7fa818ad0700 1 -- 172.21.15.69:0/1445961144 learned_addr learned my addr 172.21.15.69:0/1445961144 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa813fff700 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb88fdc0 0x55acbb89b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa813fff700 1 -- 172.21.15.69:0/1445961144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 msgr2=0x55acbb898f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa813fff700 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb898f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa813fff700 1 -- 172.21.15.69:0/1445961144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 msgr2=0x55acbb898a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa813fff700 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 0x55acbb898a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa813fff700 1 -- 172.21.15.69:0/1445961144 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acbb72d880 con 0x55acbb88fdc0 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa818ad0700 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb898f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa8137fe700 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 0x55acbb898a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa813fff700 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb88fdc0 0x55acbb89b670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa804091120 tx=0x7fa8040b2060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ad170c8ad4bb1c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa7f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa7f37fe700 1 -- 172.21.15.69:0/1445961144 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804020040 con 0x55acbb88fdc0 2024-05-14T16:50:23.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa7f37fe700 1 -- 172.21.15.69:0/1445961144 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7fa8040a59d0 con 0x55acbb88fdc0 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1445961144 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8040a8ef0 con 0x55acbb88fdc0 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa7f37fe700 1 -- 172.21.15.69:0/1445961144 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8040a9ae0 con 0x55acbb88fdc0 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa824dc9b80 10 client.24880 fetch_fsmap learned FSMap version 25 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.030+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1445961144 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55acbb797710 con 0x55acbb88fdc0 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.031+0000 7fa7f37fe700 1 -- 172.21.15.69:0/1445961144 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8040a8ef0 con 0x55acbb88fdc0 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.031+0000 7fa7f37fe700 1 -- 172.21.15.69:0/1445961144 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa8040b0b10 con 0x55acbb88fdc0 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.031+0000 7fa824dc9b80 10 client.24880 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.031+0000 7fa824dc9b80 10 client.24880 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.031+0000 7fa824dc9b80 20 client.24880 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.031+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55acbb8ace70 0x55acbb8ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.031+0000 7fa824dc9b80 4 client.24880 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.031+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1445961144 --> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acbb7efc70 con 0x55acbb8ace70 2024-05-14T16:50:23.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.031+0000 7fa8137fe700 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55acbb8ace70 0x55acbb8ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.032+0000 7fa7f37fe700 10 client.24880 ms_handle_connect on v2:172.21.15.92:6836/1194779586 2024-05-14T16:50:23.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.032+0000 7fa8137fe700 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55acbb8ace70 0x55acbb8ae2e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d42c468bfcef16e server_cookie=e2a7dc05ea2024b7 in_seq=0 out_seq=0 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.032+0000 7fa7f37fe700 1 -- 172.21.15.69:0/1445961144 <== mds.2 v2:172.21.15.92:6836/1194779586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55acbb7efc70 con 0x55acbb8ace70 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.032+0000 7fa7f37fe700 10 client.24880 handle_command_reply: tid=0 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.032+0000 7fa824dc9b80 1 client.24880 shutdown 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1445961144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb88fdc0 msgr2=0x55acbb89b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb88fdc0 0x55acbb89b670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa804091120 tx=0x7fa8040b2060 comp rx=0 tx=0).stop 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1445961144 shutdown_connections 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55acbb8ace70 0x55acbb8ae2e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb8958c0 0x55acbb898f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb88fdc0 0x55acbb89b670 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa7f37fe700 0 client.24880 ms_handle_reset on v2:172.21.15.92:6836/1194779586 2024-05-14T16:50:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/1445961144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 0x55acbb898a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1445961144 >> 172.21.15.69:0/1445961144 conn(0x55acbb7bcd60 msgr2=0x55acbb889630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1445961144 shutdown_connections 2024-05-14T16:50:23.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1445961144 wait complete. 2024-05-14T16:50:23.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.033+0000 7fa824dc9b80 20 client.24880 trim_cache size 0 max 16384 2024-05-14T16:50:23.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.054+0000 7fa824dc9b80 1 Processor -- start 2024-05-14T16:50:23.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.054+0000 7fa824dc9b80 1 -- start start 2024-05-14T16:50:23.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.054+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb88f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.054+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 0x55acbb889db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb8958c0 0x55acbb8942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55acbb8ac620 con 0x55acbb894fa0 2024-05-14T16:50:23.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55acbb8accf0 con 0x55acbb88fdc0 2024-05-14T16:50:23.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55acbb8ad3c0 con 0x55acbb8958c0 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa813fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb88f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa8137fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 0x55acbb889db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa813fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb88f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:56452/0 (socket says 172.21.15.69:56452) 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa813fff700 1 -- 172.21.15.69:0/1250712073 learned_addr learned my addr 172.21.15.69:0/1250712073 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa818ad0700 1 --2- 172.21.15.69:0/1250712073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb8958c0 0x55acbb8942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa8137fe700 1 -- 172.21.15.69:0/1250712073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb8958c0 msgr2=0x55acbb8942b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa8137fe700 1 --2- 172.21.15.69:0/1250712073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb8958c0 0x55acbb8942b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa8137fe700 1 -- 172.21.15.69:0/1250712073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 msgr2=0x55acbb88f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa8137fe700 1 --2- 172.21.15.69:0/1250712073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb88f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa8137fe700 1 -- 172.21.15.69:0/1250712073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acbb88bcf0 con 0x55acbb894fa0 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa8137fe700 1 --2- 172.21.15.69:0/1250712073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 0x55acbb889db0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa808005c50 tx=0x7fa808014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cd50d97dbc693a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.055+0000 7fa7f9ffb700 1 -- 172.21.15.69:0/1250712073 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808032070 con 0x55acbb894fa0 2024-05-14T16:50:23.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.056+0000 7fa7f9ffb700 1 -- 172.21.15.69:0/1250712073 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa808004520 con 0x55acbb894fa0 2024-05-14T16:50:23.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.056+0000 7fa7f9ffb700 1 -- 172.21.15.69:0/1250712073 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80801d670 con 0x55acbb894fa0 2024-05-14T16:50:23.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.056+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1250712073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 msgr2=0x55acbb889db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.056+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/1250712073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 0x55acbb889db0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa808005c50 tx=0x7fa808014000 comp rx=0 tx=0).stop 2024-05-14T16:50:23.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.056+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1250712073 shutdown_connections 2024-05-14T16:50:23.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.056+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/1250712073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb8958c0 0x55acbb8942b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.056+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/1250712073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb88f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.057+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/1250712073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb894fa0 0x55acbb889db0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.057+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1250712073 >> 172.21.15.69:0/1250712073 conn(0x55acbb7bcd60 msgr2=0x55acbb893ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.057+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1250712073 shutdown_connections 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.057+0000 7fa824dc9b80 1 -- 172.21.15.69:0/1250712073 wait complete. 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.057+0000 7fa824dc9b80 1 -- start start 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.057+0000 7fa824dc9b80 1 Processor -- start 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb8b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb8958c0 0x55acbb8b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa824dc9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb8b2710 0x55acbb8b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55acbb8ae160 con 0x55acbb8958c0 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55acbb8ada90 con 0x55acbb88fdc0 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa824dc9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55acbb8ae830 con 0x55acbb8b2710 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa824dc9b80 10 client.0 _ll_register_callbacks cb 0x55acbb778900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa813fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb8b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa813fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb8b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:56456/0 (socket says 172.21.15.69:56456) 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa813fff700 1 -- 172.21.15.69:0/245578313 learned_addr learned my addr 172.21.15.69:0/245578313 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa818ad0700 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb8b2710 0x55acbb8b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa8137fe700 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb8958c0 0x55acbb8b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa813fff700 1 -- 172.21.15.69:0/245578313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb8b2710 msgr2=0x55acbb8b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa813fff700 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb8b2710 0x55acbb8b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa813fff700 1 -- 172.21.15.69:0/245578313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb8958c0 msgr2=0x55acbb8b7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa813fff700 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb8958c0 0x55acbb8b7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa813fff700 1 -- 172.21.15.69:0/245578313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acbb8b3390 con 0x55acbb88fdc0 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa813fff700 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb8b7760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa8040b0ed0 tx=0x7fa80400ffa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7455f7e09eb347f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.058+0000 7fa7f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa7f37fe700 1 -- 172.21.15.69:0/245578313 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804020070 con 0x55acbb88fdc0 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa824dc9b80 1 -- 172.21.15.69:0/245578313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa808004520 con 0x55acbb88fdc0 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa7f37fe700 1 -- 172.21.15.69:0/245578313 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7fa8040a59d0 con 0x55acbb88fdc0 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa7f37fe700 1 -- 172.21.15.69:0/245578313 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8040a3b50 con 0x55acbb88fdc0 2024-05-14T16:50:23.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa824dc9b80 10 client.24917 fetch_fsmap learned FSMap version 25 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa824dc9b80 1 -- 172.21.15.69:0/245578313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55acbb919230 con 0x55acbb88fdc0 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa7f37fe700 1 -- 172.21.15.69:0/245578313 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa808004520 con 0x55acbb88fdc0 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa7f37fe700 1 -- 172.21.15.69:0/245578313 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa8040b0b10 con 0x55acbb88fdc0 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa824dc9b80 10 client.24917 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa824dc9b80 10 client.24917 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa824dc9b80 20 client.24917 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55acbb91db00 0x55acbb91c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa824dc9b80 4 client.24917 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa824dc9b80 1 -- 172.21.15.69:0/245578313 --> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] -- command(tid 0: {"prefix": "status"}) -- 0x55acbb7e9450 con 0x55acbb91db00 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.059+0000 7fa8137fe700 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55acbb91db00 0x55acbb91c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.060+0000 7fa7f37fe700 10 client.24917 ms_handle_connect on v2:172.21.15.92:6836/1194779586 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.060+0000 7fa8137fe700 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55acbb91db00 0x55acbb91c810 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a6c1f89f76518a5 server_cookie=ff410c631228573a in_seq=0 out_seq=0 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.060+0000 7fa7f37fe700 1 -- 172.21.15.69:0/245578313 <== mds.2 v2:172.21.15.92:6836/1194779586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55acbb7e9450 con 0x55acbb91db00 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.060+0000 7fa7f37fe700 10 client.24917 handle_command_reply: tid=0 2024-05-14T16:50:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.060+0000 7fa824dc9b80 1 client.24917 shutdown 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.060+0000 7fa824dc9b80 1 -- 172.21.15.69:0/245578313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 msgr2=0x55acbb8b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.060+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb8b7760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa8040b0ed0 tx=0x7fa80400ffa0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa824dc9b80 1 -- 172.21.15.69:0/245578313 shutdown_connections 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.92:6836/1194779586,v1:172.21.15.92:6837/1194779586] conn(0x55acbb91db00 0x55acbb91c810 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55acbb8b2710 0x55acbb8b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55acbb88fdc0 0x55acbb8b7760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa824dc9b80 1 --2- 172.21.15.69:0/245578313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acbb8958c0 0x55acbb8b7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa824dc9b80 1 -- 172.21.15.69:0/245578313 >> 172.21.15.69:0/245578313 conn(0x55acbb7bcd60 msgr2=0x55acbb8a4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa7f37fe700 0 client.24917 ms_handle_reset on v2:172.21.15.92:6836/1194779586 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa824dc9b80 1 -- 172.21.15.69:0/245578313 shutdown_connections 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa824dc9b80 1 -- 172.21.15.69:0/245578313 wait complete. 2024-05-14T16:50:23.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa824dc9b80 20 client.24917 trim_cache size 0 max 16384 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24422, 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 203.53318088, 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 25, 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 81, 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 247.334941563 2024-05-14T16:50:23.067 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:50:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa81a533700 1 -- 172.21.15.69:0/720177966 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7fc0b2b10 msgr2=0x7fa7fc0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa81a533700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7fc0b2b10 0x7fa7fc0b4fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa8141073a0 tx=0x7fa808009260 comp rx=0 tx=0).stop 2024-05-14T16:50:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa81a533700 1 -- 172.21.15.69:0/720177966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa814102ba0 msgr2=0x7fa81419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.061+0000 7fa81a533700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa814102ba0 0x7fa81419ac50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa80400d980 tx=0x7fa80400dc90 comp rx=0 tx=0).stop 2024-05-14T16:50:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.062+0000 7fa81a533700 1 -- 172.21.15.69:0/720177966 shutdown_connections 2024-05-14T16:50:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.062+0000 7fa81a533700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7fc0b2b10 0x7fa7fc0b4fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.062+0000 7fa81a533700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa814108710 0x7fa8141a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.062+0000 7fa81a533700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa814102ba0 0x7fa81419ac50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.062+0000 7fa81a533700 1 --2- 172.21.15.69:0/720177966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa814103560 0x7fa81419b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.062+0000 7fa81a533700 1 -- 172.21.15.69:0/720177966 >> 172.21.15.69:0/720177966 conn(0x7fa814078a40 msgr2=0x7fa8140fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.062+0000 7fa81a533700 1 -- 172.21.15.69:0/720177966 shutdown_connections 2024-05-14T16:50:23.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:23.062+0000 7fa81a533700 1 -- 172.21.15.69:0/720177966 wait complete. 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.202+0000 7fea99f46640 1 -- 172.21.15.69:0/3356455818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea94102340 msgr2=0x7fea9410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.202+0000 7fea99f46640 1 --2- 172.21.15.69:0/3356455818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea94102340 0x7fea9410e800 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009cd0 tx=0x7fea8c02a8e0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.202+0000 7fea99f46640 1 -- 172.21.15.69:0/3356455818 shutdown_connections 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.202+0000 7fea99f46640 1 --2- 172.21.15.69:0/3356455818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea94102340 0x7fea9410e800 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.202+0000 7fea99f46640 1 --2- 172.21.15.69:0/3356455818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea94101980 0x7fea94101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.202+0000 7fea99f46640 1 --2- 172.21.15.69:0/3356455818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea94107940 0x7fea94107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.202+0000 7fea99f46640 1 -- 172.21.15.69:0/3356455818 >> 172.21.15.69:0/3356455818 conn(0x7fea94078ad0 msgr2=0x7fea94078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.202+0000 7fea99f46640 1 -- 172.21.15.69:0/3356455818 shutdown_connections 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.202+0000 7fea99f46640 1 -- 172.21.15.69:0/3356455818 wait complete. 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea99f46640 1 Processor -- start 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea99f46640 1 -- start start 2024-05-14T16:50:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea99f46640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea94101980 0x7fea94199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea99f46640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea94102340 0x7fea941996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea99f46640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea94107940 0x7fea9419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea99f46640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea94111bd0 con 0x7fea94101980 2024-05-14T16:50:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea99f46640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fea94111a50 con 0x7fea94102340 2024-05-14T16:50:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea99f46640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea94111d50 con 0x7fea94107940 2024-05-14T16:50:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea92ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea94102340 0x7fea941996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea937fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea94101980 0x7fea94199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea92ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea94102340 0x7fea941996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:56470/0 (socket says 172.21.15.69:56470) 2024-05-14T16:50:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea92ffd640 1 -- 172.21.15.69:0/557598161 learned_addr learned my addr 172.21.15.69:0/557598161 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.203+0000 7fea937fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea94101980 0x7fea94199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33446/0 (socket says 172.21.15.69:33446) 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea93fff640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea94107940 0x7fea9419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea92ffd640 1 -- 172.21.15.69:0/557598161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea94107940 msgr2=0x7fea9419da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea92ffd640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea94107940 0x7fea9419da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea92ffd640 1 -- 172.21.15.69:0/557598161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea94101980 msgr2=0x7fea94199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea92ffd640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea94101980 0x7fea94199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea92ffd640 1 -- 172.21.15.69:0/557598161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9419e160 con 0x7fea94102340 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea92ffd640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea94102340 0x7fea941996b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c920 tx=0x7fea8800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea90ff9640 1 -- 172.21.15.69:0/557598161 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88007c70 con 0x7fea94102340 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea99f46640 1 -- 172.21.15.69:0/557598161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9419e450 con 0x7fea94102340 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea90ff9640 1 -- 172.21.15.69:0/557598161 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea88007e10 con 0x7fea94102340 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea90ff9640 1 -- 172.21.15.69:0/557598161 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88018820 con 0x7fea94102340 2024-05-14T16:50:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.204+0000 7fea99f46640 1 -- 172.21.15.69:0/557598161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea941a5ce0 con 0x7fea94102340 2024-05-14T16:50:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea9cc66740 1 Processor -- start 2024-05-14T16:50:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea9cc66740 1 -- start start 2024-05-14T16:50:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea90ff9640 1 -- 172.21.15.69:0/557598161 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fea880189c0 con 0x7fea94102340 2024-05-14T16:50:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69976cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 0x55da69981590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da69983ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea9cc66740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55da698e9c30 con 0x55da699768f0 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea9cc66740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55da698d97d0 con 0x55da69977210 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea9cc66740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55da698dd810 con 0x55da69981ad0 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea93fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69976cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea93fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69976cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33460/0 (socket says 172.21.15.69:33460) 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea93fff640 1 -- 172.21.15.69:0/2776657876 learned_addr learned my addr 172.21.15.69:0/2776657876 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea937fe640 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 0x55da69981590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.207+0000 7fea92ffd640 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da69983ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea93fff640 1 -- 172.21.15.69:0/2776657876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 msgr2=0x55da69983ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea93fff640 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da69983ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea93fff640 1 -- 172.21.15.69:0/2776657876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 msgr2=0x55da69981590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea93fff640 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 0x55da69981590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea93fff640 1 -- 172.21.15.69:0/2776657876 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da695ee210 con 0x55da699768f0 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea937fe640 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 0x55da69981590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea92ffd640 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da69983ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea93fff640 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69976cd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009cd0 tx=0x7fea8c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285f3f3168df4585 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea79ffb640 1 -- 172.21.15.69:0/2776657876 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c049070 con 0x55da699768f0 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea90ff9640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea7c0b2b30 0x7fea7c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea79ffb640 1 -- 172.21.15.69:0/2776657876 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c036040 con 0x55da699768f0 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea79ffb640 1 -- 172.21.15.69:0/2776657876 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c004260 con 0x55da699768f0 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea937fe640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea7c0b2b30 0x7fea7c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.208+0000 7fea90ff9640 1 -- 172.21.15.69:0/557598161 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7fea880e2970 con 0x7fea94102340 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 -- 172.21.15.69:0/2776657876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 msgr2=0x55da69976cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea937fe640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea7c0b2b30 0x7fea7c0b4ff0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55da69977e90 tx=0x7fea8400b410 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69976cd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009cd0 tx=0x7fea8c009a20 comp rx=0 tx=0).stop 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 -- 172.21.15.69:0/2776657876 shutdown_connections 2024-05-14T16:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da69983ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 0x55da69981590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2776657876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69976cd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 -- 172.21.15.69:0/2776657876 >> 172.21.15.69:0/2776657876 conn(0x55da69972e00 msgr2=0x55da699731e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 -- 172.21.15.69:0/2776657876 shutdown_connections 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 -- 172.21.15.69:0/2776657876 wait complete. 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 -- start start 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.209+0000 7fea9cc66740 1 Processor -- start 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69981320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 0x55da6997e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da6997eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea9cc66740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55da695bf790 con 0x55da699768f0 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea9cc66740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55da69895890 con 0x55da69977210 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea9cc66740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55da69984740 con 0x55da69981ad0 2024-05-14T16:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea9cc66740 10 client.0 _ll_register_callbacks cb 0x55da69898d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea937fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69981320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea92ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 0x55da6997e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea937fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69981320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33472/0 (socket says 172.21.15.69:33472) 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea937fe640 1 -- 172.21.15.69:0/2374882545 learned_addr learned my addr 172.21.15.69:0/2374882545 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea93fff640 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da6997eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea92ffd640 1 -- 172.21.15.69:0/2374882545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 msgr2=0x55da6997eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea92ffd640 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da6997eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea92ffd640 1 -- 172.21.15.69:0/2374882545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 msgr2=0x55da69981320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea92ffd640 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69981320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea92ffd640 1 -- 172.21.15.69:0/2374882545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da696ba620 con 0x55da69977210 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea937fe640 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69981320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea93fff640 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da6997eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea92ffd640 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 0x55da6997e610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea880dff50 tx=0x7fea880a98a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e35151705395c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.210+0000 7fea72ffd640 1 -- 172.21.15.69:0/2374882545 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea880a51a0 con 0x55da69977210 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea9cc66740 1 -- 172.21.15.69:0/2374882545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea8c036040 con 0x55da69977210 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea72ffd640 1 -- 172.21.15.69:0/2374882545 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7fea880b25f0 con 0x55da69977210 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea72ffd640 1 -- 172.21.15.69:0/2374882545 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea880b21b0 con 0x55da69977210 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea9cc66740 10 client.24929 fetch_fsmap learned FSMap version 25 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea9cc66740 1 -- 172.21.15.69:0/2374882545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55da699900a0 con 0x55da69977210 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea72ffd640 1 -- 172.21.15.69:0/2374882545 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c036040 con 0x55da69977210 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea72ffd640 1 -- 172.21.15.69:0/2374882545 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fea880e0230 con 0x55da69977210 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea9cc66740 10 client.24929 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea9cc66740 10 client.24929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea9cc66740 20 client.24929 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da69991510 0x55da69993900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea9cc66740 4 client.24929 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea9cc66740 1 -- 172.21.15.69:0/2374882545 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da698ebb80 con 0x55da69991510 2024-05-14T16:50:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.211+0000 7fea937fe640 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da69991510 0x55da69993900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.212+0000 7fea937fe640 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da69991510 0x55da69993900 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93d5e8c5321b341 server_cookie=2bf34e033540c63f in_seq=0 out_seq=0 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.212+0000 7fea72ffd640 10 client.24929 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.212+0000 7fea72ffd640 1 -- 172.21.15.69:0/2374882545 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da698ebb80 con 0x55da69991510 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.212+0000 7fea72ffd640 10 client.24929 handle_command_reply: tid=0 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.212+0000 7fea9cc66740 1 client.24929 shutdown 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 1 -- 172.21.15.69:0/2374882545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 msgr2=0x55da6997e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 0x55da6997e610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea880dff50 tx=0x7fea880a98a0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 1 -- 172.21.15.69:0/2374882545 shutdown_connections 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da69991510 0x55da69993900 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da6997eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69977210 0x55da6997e610 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea72ffd640 0 client.24929 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2374882545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69981320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 1 -- 172.21.15.69:0/2374882545 >> 172.21.15.69:0/2374882545 conn(0x55da69972e00 msgr2=0x55da69972100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 1 -- 172.21.15.69:0/2374882545 shutdown_connections 2024-05-14T16:50:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 1 -- 172.21.15.69:0/2374882545 wait complete. 2024-05-14T16:50:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.213+0000 7fea9cc66740 20 client.24929 trim_cache size 0 max 16384 2024-05-14T16:50:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea9cc66740 1 Processor -- start 2024-05-14T16:50:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea9cc66740 1 -- start start 2024-05-14T16:50:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da699768f0 0x55da69978b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da69977210 0x55da69980e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da69999150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea9cc66740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea941b0d50 con 0x55da69977210 2024-05-14T16:50:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea9cc66740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fea941ae620 con 0x55da699768f0 2024-05-14T16:50:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea9cc66740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea941ac470 con 0x55da69981ad0 2024-05-14T16:50:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea92ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da69977210 0x55da69980e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea92ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da69977210 0x55da69980e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33484/0 (socket says 172.21.15.69:33484) 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea92ffd640 1 -- 172.21.15.69:0/2316866106 learned_addr learned my addr 172.21.15.69:0/2316866106 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea937fe640 1 --2- 172.21.15.69:0/2316866106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da699768f0 0x55da69978b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea93fff640 1 --2- 172.21.15.69:0/2316866106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da69999150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea92ffd640 1 -- 172.21.15.69:0/2316866106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 msgr2=0x55da69999150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea92ffd640 1 --2- 172.21.15.69:0/2316866106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da69999150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea92ffd640 1 -- 172.21.15.69:0/2316866106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da699768f0 msgr2=0x55da69978b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea92ffd640 1 --2- 172.21.15.69:0/2316866106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da699768f0 0x55da69978b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.228+0000 7fea92ffd640 1 -- 172.21.15.69:0/2316866106 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da69993ec0 con 0x55da69977210 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.229+0000 7fea92ffd640 1 --2- 172.21.15.69:0/2316866106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da69977210 0x55da69980e50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fea880b2a30 tx=0x7fea880a9910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d40530a5f1a575 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.229+0000 7fea797fa640 1 -- 172.21.15.69:0/2316866106 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea880bee60 con 0x55da69977210 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.229+0000 7fea797fa640 1 -- 172.21.15.69:0/2316866106 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea880a9a10 con 0x55da69977210 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.229+0000 7fea797fa640 1 -- 172.21.15.69:0/2316866106 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea880aa110 con 0x55da69977210 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.229+0000 7fea9cc66740 1 -- 172.21.15.69:0/2316866106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da69977210 msgr2=0x55da69980e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.229+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2316866106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da69977210 0x55da69980e50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fea880b2a30 tx=0x7fea880a9910 comp rx=0 tx=0).stop 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 -- 172.21.15.69:0/2316866106 shutdown_connections 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2316866106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69981ad0 0x55da69999150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2316866106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da69977210 0x55da69980e50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 --2- 172.21.15.69:0/2316866106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da699768f0 0x55da69978b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 -- 172.21.15.69:0/2316866106 >> 172.21.15.69:0/2316866106 conn(0x55da69972e00 msgr2=0x55da69979ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 -- 172.21.15.69:0/2316866106 shutdown_connections 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 -- 172.21.15.69:0/2316866106 wait complete. 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 -- start start 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 Processor -- start 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69985b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69977210 0x55da699860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69981ad0 0x55da699fad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea941acae0 con 0x55da699768f0 2024-05-14T16:50:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fea941adf50 con 0x55da69981ad0 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea941ad1b0 con 0x55da69977210 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.230+0000 7fea9cc66740 10 client.0 _ll_register_callbacks cb 0x55da69898d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea937fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69985b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea937fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69985b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33500/0 (socket says 172.21.15.69:33500) 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea937fe640 1 -- 172.21.15.69:0/515537896 learned_addr learned my addr 172.21.15.69:0/515537896 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea92ffd640 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69977210 0x55da699860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea93fff640 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69981ad0 0x55da699fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea937fe640 1 -- 172.21.15.69:0/515537896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69977210 msgr2=0x55da699860b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea937fe640 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69977210 0x55da699860b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea937fe640 1 -- 172.21.15.69:0/515537896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69981ad0 msgr2=0x55da699fad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea937fe640 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69981ad0 0x55da699fad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea937fe640 1 -- 172.21.15.69:0/515537896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da69972c30 con 0x55da699768f0 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea93fff640 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69981ad0 0x55da699fad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea937fe640 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69985b70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fea840205d0 tx=0x7fea84015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6527b82990ba2880 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea72ffd640 1 -- 172.21.15.69:0/515537896 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea840224e0 con 0x55da699768f0 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea72ffd640 1 -- 172.21.15.69:0/515537896 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7fea84020680 con 0x55da699768f0 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea9cc66740 1 -- 172.21.15.69:0/515537896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da698e5360 con 0x55da699768f0 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea72ffd640 1 -- 172.21.15.69:0/515537896 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea84004020 con 0x55da699768f0 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea9cc66740 10 client.15351 fetch_fsmap learned FSMap version 25 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.231+0000 7fea9cc66740 1 -- 172.21.15.69:0/515537896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55da699fc2e0 con 0x55da699768f0 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea72ffd640 1 -- 172.21.15.69:0/515537896 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55da698e5360 con 0x55da699768f0 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea72ffd640 1 -- 172.21.15.69:0/515537896 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fea84028ce0 con 0x55da699768f0 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea9cc66740 10 client.15351 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea9cc66740 10 client.15351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea9cc66740 20 client.15351 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea9cc66740 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da699fc960 0x55da699fed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea9cc66740 4 client.15351 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea9cc66740 1 -- 172.21.15.69:0/515537896 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da6985f5f0 con 0x55da699fc960 2024-05-14T16:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea92ffd640 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da699fc960 0x55da699fed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea92ffd640 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da699fc960 0x55da699fed80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7b39c9e034739a server_cookie=1480cd9fd674a6ef in_seq=0 out_seq=0 2024-05-14T16:50:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.232+0000 7fea72ffd640 10 client.15351 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.453+0000 7fea72ffd640 1 -- 172.21.15.69:0/515537896 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55da6985f5f0 con 0x55da699fc960 2024-05-14T16:50:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.453+0000 7fea72ffd640 10 client.15351 handle_command_reply: tid=0 2024-05-14T16:50:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.453+0000 7fea9cc66740 1 client.15351 shutdown 2024-05-14T16:50:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 1 -- 172.21.15.69:0/515537896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 msgr2=0x55da69985b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69985b70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fea840205d0 tx=0x7fea84015000 comp rx=0 tx=0).stop 2024-05-14T16:50:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 1 -- 172.21.15.69:0/515537896 shutdown_connections 2024-05-14T16:50:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da699fc960 0x55da699fed80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da69981ad0 0x55da699fad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da69977210 0x55da699860b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea72ffd640 0 client.15351 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 1 --2- 172.21.15.69:0/515537896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da699768f0 0x55da69985b70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 1 -- 172.21.15.69:0/515537896 >> 172.21.15.69:0/515537896 conn(0x55da69972e00 msgr2=0x55da69972100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 1 -- 172.21.15.69:0/515537896 shutdown_connections 2024-05-14T16:50:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 1 -- 172.21.15.69:0/515537896 wait complete. 2024-05-14T16:50:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea9cc66740 20 client.15351 trim_cache size 0 max 16384 2024-05-14T16:50:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:50:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:50:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:50:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea99f46640 1 -- 172.21.15.69:0/557598161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea7c0b2b30 msgr2=0x7fea7c0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea99f46640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea7c0b2b30 0x7fea7c0b4ff0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55da69977e90 tx=0x7fea8400b410 comp rx=0 tx=0).stop 2024-05-14T16:50:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea99f46640 1 -- 172.21.15.69:0/557598161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea94102340 msgr2=0x7fea941996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea99f46640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea94102340 0x7fea941996b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c920 tx=0x7fea8800cde0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.454+0000 7fea99f46640 1 -- 172.21.15.69:0/557598161 shutdown_connections 2024-05-14T16:50:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.455+0000 7fea99f46640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea7c0b2b30 0x7fea7c0b4ff0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.455+0000 7fea99f46640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea94107940 0x7fea9419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.455+0000 7fea99f46640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea94102340 0x7fea941996b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.455+0000 7fea99f46640 1 --2- 172.21.15.69:0/557598161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea94101980 0x7fea94199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.455+0000 7fea99f46640 1 -- 172.21.15.69:0/557598161 >> 172.21.15.69:0/557598161 conn(0x7fea94078ad0 msgr2=0x7fea9406a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.455+0000 7fea99f46640 1 -- 172.21.15.69:0/557598161 shutdown_connections 2024-05-14T16:50:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:23.455+0000 7fea99f46640 1 -- 172.21.15.69:0/557598161 wait complete. 2024-05-14T16:50:23.839 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-14T16:50:24.039 INFO:teuthology.orchestra.run.smithi069.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-14T16:50:24.040 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-14T16:50:24.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:23 smithi069 ceph-mon[31507]: pgmap v17: 97 pgs: 97 active+clean; 2.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 70 KiB/s wr, 14 op/s 2024-05-14T16:50:24.092 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/: 2024-05-14T16:50:24.092 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.092 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 . 2024-05-14T16:50:24.092 INFO:teuthology.orchestra.run.smithi069.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 .. 2024-05-14T16:50:24.092 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 data 2024-05-14T16:50:24.092 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data: 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:total 35 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 . 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 .. 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 base 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 May 14 16:50 global 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 log 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_commit_ts 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_dynshmem 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 14 16:50 pg_hba.conf 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 14 16:50 pg_ident.conf 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 pg_logical 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 pg_multixact 2024-05-14T16:50:24.093 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_notify 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_replslot 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_serial 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_snapshots 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_stat 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_stat_tmp 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 pg_subtrans 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_tblspc 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_twophase 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 PG_VERSION 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 pg_wal 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 pg_xact 2024-05-14T16:50:24.094 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 14 16:50 postgresql.auto.conf 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 14 16:50 postgresql.conf 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base: 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 . 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:50 1 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:50 13493 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:50 13494 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:total 7777 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:50 . 2024-05-14T16:50:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 .. 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 112 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 113 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 1247 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1247_fsm 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1247_vm 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:50 1249 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1249_fsm 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1249_vm 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:50 1255 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1255_fsm 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1255_vm 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 16:50 1259 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1259_fsm 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1259_vm 2024-05-14T16:50:24.096 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:50 13344 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13344_fsm 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13344_vm 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13346 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13348 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13349_fsm 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349_vm 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13351 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13353 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13354_fsm 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354_vm 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13356 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13358 2024-05-14T16:50:24.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13359_fsm 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359_vm 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13361 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13363 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1417 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1418 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 174 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 175 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2187 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2224 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2328 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2336 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2337 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2579 2024-05-14T16:50:24.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2600 2024-05-14T16:50:24.099 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2600_fsm 2024-05-14T16:50:24.099 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2600_vm 2024-05-14T16:50:24.099 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2601_fsm 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601_vm 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2602 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2602_fsm 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2602_vm 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2603 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2603_fsm 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2603_vm 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2604 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2605 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2605_fsm 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2605_vm 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606 2024-05-14T16:50:24.100 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2606_fsm 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606_vm 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2607 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2607_fsm 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2607_vm 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:50 2608 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2608_fsm 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2608_vm 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:50 2609 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2609_fsm 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2609_vm 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2610 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2610_fsm 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2610_vm 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2611 2024-05-14T16:50:24.101 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2612_fsm 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612_vm 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2613 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2615_fsm 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615_vm 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2616 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2616_fsm 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2616_vm 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:50 2617 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2617_fsm 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2617_vm 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:50 2618 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2618_fsm 2024-05-14T16:50:24.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2618_vm 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 14 16:50 2619 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2619_fsm 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2619_vm 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2620 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2650 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2651 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2652 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2653 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2654 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2655 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2656 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2657 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:50 2658 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 2659 2024-05-14T16:50:24.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2660 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2661 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2662 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2663 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2664 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2665 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2666 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2667 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2668 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2669 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2670 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:50 2673 2024-05-14T16:50:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:50 2674 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:50 2675 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2678 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2679 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2680 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2681 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2682 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2683 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2684 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2685 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2686 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2687 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2688 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2689 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:50 2690 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:50 2691 2024-05-14T16:50:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2692 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2693 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2696 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2699 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2701 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2702 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2703 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2704 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2753 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2753_fsm 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2753_vm 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2754 2024-05-14T16:50:24.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2755 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2756 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2757 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2830 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2831 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2832 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2833 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2834 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2835 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2836 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2837 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:50 2838 2024-05-14T16:50:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2838_fsm 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2838_vm 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2839 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840_fsm 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2840_vm 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2841 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2995 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2996 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3079_fsm 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079_vm 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3080 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3081 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3085 2024-05-14T16:50:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3118 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3119 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:50 3164 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3256 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3257 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3258 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3350 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3351 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3379 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3380 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3381 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3394 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3394_fsm 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3394_vm 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3395 2024-05-14T16:50:24.109 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3429 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3430 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3431 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3433 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3439 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3440 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3455 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:50 3456 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3456_fsm 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3456_vm 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3466 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3467 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3468 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3501 2024-05-14T16:50:24.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3502 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3503 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3534 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3541_fsm 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541_vm 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3542 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3574 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3575 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3576 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3596 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3597 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3598 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3599 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3600_fsm 2024-05-14T16:50:24.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600_vm 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3601_fsm 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601_vm 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3602_fsm 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602_vm 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603_fsm 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3603_vm 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3604 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3605 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3606 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3607 2024-05-14T16:50:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3608 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 3609 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3712 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3764_fsm 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764_vm 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3766 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3767 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3997 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4143 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4144 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4145 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4146 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4147 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4148 2024-05-14T16:50:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4149 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4150 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4151 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4152 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4153 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4154 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4155 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4156 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4157 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4158 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4159 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4160 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4161 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4162 2024-05-14T16:50:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4163 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4164 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4165 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4166 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4167 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4168 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4169 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4170 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4171 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4172 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4173 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4174 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 5002 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 548 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 549 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6102 2024-05-14T16:50:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6104 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6106 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6110 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6111 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6112 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6113 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6117 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 826 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 827 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 828 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:50 pg_filenode.map 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 PG_VERSION 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:total 7777 2024-05-14T16:50:24.116 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:50 . 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 .. 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 112 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 113 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 1247 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1247_fsm 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1247_vm 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:50 1249 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1249_fsm 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1249_vm 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:50 1255 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1255_fsm 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1255_vm 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 16:50 1259 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1259_fsm 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1259_vm 2024-05-14T16:50:24.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:50 13344 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13344_fsm 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13344_vm 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13346 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13348 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13349_fsm 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349_vm 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13351 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13353 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13354_fsm 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354_vm 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13356 2024-05-14T16:50:24.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13358 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13359_fsm 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359_vm 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13361 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13363 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1417 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1418 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 174 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 175 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2187 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2224 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2328 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2336 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2337 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2579 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2600 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2600_fsm 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2600_vm 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2601_fsm 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601_vm 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2602 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2602_fsm 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2602_vm 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2603 2024-05-14T16:50:24.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2603_fsm 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2603_vm 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2604 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2605 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2605_fsm 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2605_vm 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2606_fsm 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606_vm 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2607 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2607_fsm 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2607_vm 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:50 2608 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2608_fsm 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2608_vm 2024-05-14T16:50:24.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:50 2609 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2609_fsm 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2609_vm 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2610 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2610_fsm 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2610_vm 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2611 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2612_fsm 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612_vm 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2613 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2615_fsm 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615_vm 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2616 2024-05-14T16:50:24.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2616_fsm 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2616_vm 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:50 2617 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2617_fsm 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2617_vm 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:50 2618 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2618_fsm 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2618_vm 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 14 16:50 2619 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2619_fsm 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2619_vm 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2620 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2650 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2651 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2652 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2653 2024-05-14T16:50:24.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2654 2024-05-14T16:50:24.125 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2655 2024-05-14T16:50:24.125 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2656 2024-05-14T16:50:24.125 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2657 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:50 2658 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 2659 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2660 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2661 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2662 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2663 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2664 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2665 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2666 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2667 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2668 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2669 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2670 2024-05-14T16:50:24.126 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:50 2673 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:50 2674 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:50 2675 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2678 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2679 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2680 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2681 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2682 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2683 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2684 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2685 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2686 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2687 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2688 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2689 2024-05-14T16:50:24.127 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:50 2690 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:50 2691 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2692 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2693 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2696 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2699 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2701 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2702 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2703 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2704 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2753 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2753_fsm 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2753_vm 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2754 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2755 2024-05-14T16:50:24.128 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2756 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2757 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2830 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2831 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2832 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2833 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2834 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2835 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2836 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2837 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:50 2838 2024-05-14T16:50:24.129 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2838_fsm 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2838_vm 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2839 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840_fsm 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2840_vm 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2841 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2995 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2996 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3079_fsm 2024-05-14T16:50:24.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079_vm 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3080 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3081 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3085 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3118 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3119 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:50 3164 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3256 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3257 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3258 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3350 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3351 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3379 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3380 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3381 2024-05-14T16:50:24.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3394 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3394_fsm 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3394_vm 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3395 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3429 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3430 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3431 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3433 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3439 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3440 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3455 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:50 3456 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3456_fsm 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3456_vm 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3466 2024-05-14T16:50:24.132 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3467 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3468 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3501 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3502 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3503 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3534 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3541_fsm 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541_vm 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3542 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3574 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3575 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3576 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3596 2024-05-14T16:50:24.133 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3597 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3598 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3599 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3600_fsm 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600_vm 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3601_fsm 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601_vm 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3602_fsm 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602_vm 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603 2024-05-14T16:50:24.134 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603_fsm 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3603_vm 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3604 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3605 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3606 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3607 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3608 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 3609 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3712 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3764_fsm 2024-05-14T16:50:24.135 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764_vm 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3766 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3767 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3997 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4143 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4144 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4145 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4146 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4147 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4148 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4149 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4150 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4151 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4152 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4153 2024-05-14T16:50:24.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4154 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4155 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4156 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4157 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4158 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4159 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4160 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4161 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4162 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4163 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4164 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4165 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4166 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4167 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4168 2024-05-14T16:50:24.137 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4169 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4170 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4171 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4172 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4173 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4174 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 5002 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 548 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 549 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6102 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6104 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6106 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6110 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6111 2024-05-14T16:50:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6112 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6113 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6117 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 826 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 827 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 828 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:50 pg_filenode.map 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 PG_VERSION 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:total 7777 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:50 . 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 .. 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 112 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 113 2024-05-14T16:50:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 1247 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1247_fsm 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1247_vm 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:50 1249 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1249_fsm 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1249_vm 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:50 1255 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1255_fsm 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1255_vm 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 16:50 1259 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1259_fsm 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1259_vm 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:50 13344 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13344_fsm 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13344_vm 2024-05-14T16:50:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13346 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13348 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13349_fsm 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349_vm 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13351 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13353 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13354_fsm 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354_vm 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13356 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13358 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13359_fsm 2024-05-14T16:50:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359_vm 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13361 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13363 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1417 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1418 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 174 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 175 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2187 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2224 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2328 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2336 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2337 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2579 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2600 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2600_fsm 2024-05-14T16:50:24.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2600_vm 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2601_fsm 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601_vm 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2602 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2602_fsm 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2602_vm 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2603 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2603_fsm 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2603_vm 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2604 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2605 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2605_fsm 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2605_vm 2024-05-14T16:50:24.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2606_fsm 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606_vm 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2607 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2607_fsm 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2607_vm 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:50 2608 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2608_fsm 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2608_vm 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:50 2609 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2609_fsm 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2609_vm 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2610 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2610_fsm 2024-05-14T16:50:24.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2610_vm 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2611 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2612_fsm 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612_vm 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2613 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2615_fsm 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615_vm 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2616 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2616_fsm 2024-05-14T16:50:24.145 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2616_vm 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:50 2617 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2617_fsm 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2617_vm 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:50 2618 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2618_fsm 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2618_vm 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 14 16:50 2619 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2619_fsm 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2619_vm 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2620 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2650 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2651 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2652 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2653 2024-05-14T16:50:24.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2654 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2655 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2656 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2657 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:50 2658 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 2659 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2660 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2661 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2662 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2663 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2664 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2665 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2666 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2667 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2668 2024-05-14T16:50:24.147 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2669 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2670 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:50 2673 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:50 2674 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:50 2675 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2678 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2679 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2680 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2681 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2682 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2683 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2684 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2685 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2686 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2687 2024-05-14T16:50:24.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2688 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2689 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:50 2690 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:50 2691 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2692 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2693 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2696 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2699 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2701 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2702 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2703 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2704 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2753 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2753_fsm 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2753_vm 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2754 2024-05-14T16:50:24.149 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2755 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2756 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2757 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2830 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2831 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2832 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2833 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2834 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2835 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2836 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2837 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:50 2838 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2838_fsm 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2838_vm 2024-05-14T16:50:24.150 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2839 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840_fsm 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2840_vm 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2841 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2995 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2996 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3079_fsm 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079_vm 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3080 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3081 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3085 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3118 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3119 2024-05-14T16:50:24.151 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:50 3164 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3256 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3257 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3258 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3350 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3351 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3379 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3380 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3381 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3394 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3394_fsm 2024-05-14T16:50:24.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3394_vm 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3395 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3429 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3430 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3431 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3433 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3439 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3440 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3455 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:50 3456 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3456_fsm 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3456_vm 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3466 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3467 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3468 2024-05-14T16:50:24.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3501 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3502 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3503 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3534 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3541_fsm 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541_vm 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3542 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3574 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3575 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3576 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3596 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3597 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3598 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3599 2024-05-14T16:50:24.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3600_fsm 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600_vm 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3601_fsm 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601_vm 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3602_fsm 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602_vm 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603_fsm 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3603_vm 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3604 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3605 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3606 2024-05-14T16:50:24.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3607 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3608 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 3609 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3712 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3764_fsm 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764_vm 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3766 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3767 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3997 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4143 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4144 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4145 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4146 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4147 2024-05-14T16:50:24.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4148 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4149 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4150 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4151 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4152 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4153 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4154 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4155 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4156 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4157 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4158 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4159 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4160 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4161 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4162 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4163 2024-05-14T16:50:24.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4164 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4165 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4166 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4167 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4168 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4169 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4170 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4171 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4172 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4173 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4174 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 5002 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 548 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 549 2024-05-14T16:50:24.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6102 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6104 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6106 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6110 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6111 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6112 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6113 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6117 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 826 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 827 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 828 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:50 pg_filenode.map 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 PG_VERSION 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/global: 2024-05-14T16:50:24.159 INFO:teuthology.orchestra.run.smithi069.stdout:total 537 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 May 14 16:50 . 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1213 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1213_fsm 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1213_vm 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1214 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1214_fsm 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1214_vm 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 1232 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 1233 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1260 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1260_fsm 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1260_vm 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1261 2024-05-14T16:50:24.160 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1261_fsm 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1261_vm 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1262 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1262_fsm 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1262_vm 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2396 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2396_fsm 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2396_vm 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2397 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2671 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2672 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2676 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2677 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2694 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2695 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2697 2024-05-14T16:50:24.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2698 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2846 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2847 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2964 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2965 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2966 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2967 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3592 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3593 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4060 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4061 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4175 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4176 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4177 2024-05-14T16:50:24.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4178 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4181 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4182 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4183 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4184 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4185 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4186 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6000 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6001 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6002 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6100 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6114 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6115 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 pg_control 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:50 pg_filenode.map 2024-05-14T16:50:24.163 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/log: 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:total 1 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 . 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 mappings 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 14 16:50 replorigin_checkpoint 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 snapshots 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 .. 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-14T16:50:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 .. 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 . 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 members 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 offsets 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 . 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 .. 2024-05-14T16:50:24.166 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 0000 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 . 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 .. 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 0000 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.167 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.168 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 . 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 0000 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.169 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:total 16384 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 . 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:50 000000010000000000000001 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 archive_status 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.170 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-14T16:50:24.171 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T16:50:24.171 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T16:50:24.171 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 .. 2024-05-14T16:50:24.171 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:24.171 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-14T16:50:24.171 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-05-14T16:50:24.171 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 . 2024-05-14T16:50:24.171 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:50 .. 2024-05-14T16:50:24.171 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 0000 2024-05-14T16:50:24.171 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:24.172 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-14T16:50:24.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:23 smithi203 ceph-mon[35787]: pgmap v17: 97 pgs: 97 active+clean; 2.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 70 KiB/s wr, 14 op/s 2024-05-14T16:50:24.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:23 smithi092 ceph-mon[35763]: pgmap v17: 97 pgs: 97 active+clean; 2.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 70 KiB/s wr, 14 op/s 2024-05-14T16:50:24.362 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-14T16:50:24.413 INFO:teuthology.orchestra.run.smithi069.stderr:dropping old tables... 2024-05-14T16:50:24.413 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-14T16:50:24.413 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-14T16:50:24.413 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-14T16:50:24.413 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-14T16:50:24.414 INFO:teuthology.orchestra.run.smithi069.stderr:creating tables... 2024-05-14T16:50:24.424 INFO:teuthology.orchestra.run.smithi069.stderr:generating data (client-side)... 2024-05-14T16:50:24.497 INFO:teuthology.orchestra.run.smithi069.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.78 s) 2024-05-14T16:50:24.654 INFO:teuthology.orchestra.run.smithi069.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.21 s, remaining 3.21 s) 2024-05-14T16:50:24.743 INFO:teuthology.orchestra.run.smithi069.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.30 s, remaining 2.94 s) 2024-05-14T16:50:24.878 INFO:teuthology.orchestra.run.smithi069.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.44 s, remaining 3.07 s) 2024-05-14T16:50:25.011 INFO:teuthology.orchestra.run.smithi069.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.57 s, remaining 3.09 s) 2024-05-14T16:50:25.089 INFO:teuthology.orchestra.run.smithi069.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.65 s, remaining 2.81 s) 2024-05-14T16:50:25.233 INFO:teuthology.orchestra.run.smithi069.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.79 s, remaining 2.83 s) 2024-05-14T16:50:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.97 s, remaining 2.91 s) 2024-05-14T16:50:25.445 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:25 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[66273]: *** Caught signal (Segmentation fault) ** 2024-05-14T16:50:25.445 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:25 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[66273]: in thread 7f9677443700 thread_name:MR_Finisher 2024-05-14T16:50:25.445 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:25 smithi203 podman[70077]: 2024-05-14 16:50:25.148447712 +0000 UTC m=+0.025889531 container died 37afd030daf7762b90a58fb8f92d12f60d0d9b0992490c8e2f2ba980e9423d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:50:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.07 s, remaining 2.75 s) 2024-05-14T16:50:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.23 s, remaining 2.70 s) 2024-05-14T16:50:25.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.817+0000 7fc998609700 1 -- 172.21.15.69:0/3238430001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc990108550 msgr2=0x7fc990108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:25.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.817+0000 7fc998609700 1 --2- 172.21.15.69:0/3238430001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc990108550 0x7fc990108930 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0099c0 tx=0x7fc98c009cd0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.817+0000 7fc998609700 1 -- 172.21.15.69:0/3238430001 shutdown_connections 2024-05-14T16:50:25.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.817+0000 7fc998609700 1 --2- 172.21.15.69:0/3238430001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc990108550 0x7fc990108930 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.817+0000 7fc998609700 1 --2- 172.21.15.69:0/3238430001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc990102e30 0x7fc99010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.817+0000 7fc998609700 1 --2- 172.21.15.69:0/3238430001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc990102470 0x7fc9901028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.817+0000 7fc998609700 1 -- 172.21.15.69:0/3238430001 >> 172.21.15.69:0/3238430001 conn(0x7fc990079480 msgr2=0x7fc990079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.817+0000 7fc998609700 1 -- 172.21.15.69:0/3238430001 shutdown_connections 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.817+0000 7fc998609700 1 -- 172.21.15.69:0/3238430001 wait complete. 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc998609700 1 Processor -- start 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc998609700 1 -- start start 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc998609700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc990102470 0x7fc99019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc998609700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc990102e30 0x7fc99019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc998609700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc990108550 0x7fc990199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc998609700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc990112620 con 0x7fc990108550 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc998609700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901124a0 con 0x7fc990102470 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc998609700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901127a0 con 0x7fc990102e30 2024-05-14T16:50:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc9963a5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc990102470 0x7fc99019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc996ba6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc990108550 0x7fc990199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc9963a5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc990102470 0x7fc99019f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:53362/0 (socket says 172.21.15.69:53362) 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc9963a5700 1 -- 172.21.15.69:0/2768633603 learned_addr learned my addr 172.21.15.69:0/2768633603 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc995ba4700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc990102e30 0x7fc99019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc9963a5700 1 -- 172.21.15.69:0/2768633603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc990102e30 msgr2=0x7fc99019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc9963a5700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc990102e30 0x7fc99019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc9963a5700 1 -- 172.21.15.69:0/2768633603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc990108550 msgr2=0x7fc990199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc9963a5700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc990108550 0x7fc990199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc9963a5700 1 -- 172.21.15.69:0/2768633603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99019a210 con 0x7fc990102470 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc996ba6700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc990108550 0x7fc990199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.818+0000 7fc995ba4700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc990102e30 0x7fc99019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:25.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.819+0000 7fc9963a5700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc990102470 0x7fc99019f8a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc98c006010 tx=0x7fc98c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:25.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.819+0000 7fc9877fe700 1 -- 172.21.15.69:0/2768633603 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c026070 con 0x7fc990102470 2024-05-14T16:50:25.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.819+0000 7fc998609700 1 -- 172.21.15.69:0/2768633603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99019a4a0 con 0x7fc990102470 2024-05-14T16:50:25.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.819+0000 7fc9877fe700 1 -- 172.21.15.69:0/2768633603 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98c00f460 con 0x7fc990102470 2024-05-14T16:50:25.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.819+0000 7fc9877fe700 1 -- 172.21.15.69:0/2768633603 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c009e60 con 0x7fc990102470 2024-05-14T16:50:25.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.819+0000 7fc998609700 1 -- 172.21.15.69:0/2768633603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a6770 con 0x7fc990102470 2024-05-14T16:50:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.822+0000 7fc9877fe700 1 -- 172.21.15.69:0/2768633603 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fc98c00f8c0 con 0x7fc990102470 2024-05-14T16:50:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.823+0000 7fc9877fe700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc97c0b2b10 0x7fc97c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.823+0000 7fc995ba4700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc97c0b2b10 0x7fc97c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.823+0000 7fc9877fe700 1 -- 172.21.15.69:0/2768633603 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7fc98c0e3190 con 0x7fc990102470 2024-05-14T16:50:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.824+0000 7fc995ba4700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc97c0b2b10 0x7fc97c0b4fd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc980005950 tx=0x7fc9800058e0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.824+0000 7fc9a2e9fb80 1 Processor -- start 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.824+0000 7fc9a2e9fb80 1 -- start start 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 0x55c7a62bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62bc910 0x55c7a62bcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c7a6216c40 con 0x55c7a62b82f0 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c7a62067e0 con 0x55c7a62bc910 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c7a620a820 con 0x55c7a62b8c10 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9963a5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc995ba4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 0x55c7a62bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9963a5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62b86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55384/0 (socket says 172.21.15.69:55384) 2024-05-14T16:50:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9963a5700 1 -- 172.21.15.69:0/4046586701 learned_addr learned my addr 172.21.15.69:0/4046586701 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc995ba4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 0x55c7a62bc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:58478/0 (socket says 172.21.15.69:58478) 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc996ba6700 1 --2- 172.21.15.69:0/4046586701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62bc910 0x55c7a62bcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9963a5700 1 -- 172.21.15.69:0/4046586701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 msgr2=0x55c7a62bc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9963a5700 1 --2- 172.21.15.69:0/4046586701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 0x55c7a62bc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9963a5700 1 -- 172.21.15.69:0/4046586701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62bc910 msgr2=0x55c7a62bcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9963a5700 1 --2- 172.21.15.69:0/4046586701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62bc910 0x55c7a62bcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9963a5700 1 -- 172.21.15.69:0/4046586701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7a6156880 con 0x55c7a62b82f0 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc996ba6700 1 --2- 172.21.15.69:0/4046586701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62bc910 0x55c7a62bcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc995ba4700 1 --2- 172.21.15.69:0/4046586701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 0x55c7a62bc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc9963a5700 1 --2- 172.21.15.69:0/4046586701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62b86d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0059d0 tx=0x7fc98c0a9db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d88979b8f111774 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.825+0000 7fc984ff9700 1 -- 172.21.15.69:0/4046586701 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0aaa90 con 0x55c7a62b82f0 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.826+0000 7fc984ff9700 1 -- 172.21.15.69:0/4046586701 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98c0aa390 con 0x55c7a62b82f0 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.826+0000 7fc984ff9700 1 -- 172.21.15.69:0/4046586701 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0b2cd0 con 0x55c7a62b82f0 2024-05-14T16:50:25.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.826+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/4046586701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 msgr2=0x55c7a62b86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.826+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/4046586701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62b86d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0059d0 tx=0x7fc98c0a9db0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.826+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/4046586701 shutdown_connections 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.826+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/4046586701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62b86d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.826+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/4046586701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62bc910 0x55c7a62bcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.826+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/4046586701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 0x55c7a62bc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.826+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/4046586701 >> 172.21.15.69:0/4046586701 conn(0x55c7a61e3200 msgr2=0x55c7a62b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.826+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/4046586701 shutdown_connections 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/4046586701 wait complete. 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9a2e9fb80 1 -- start start 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9a2e9fb80 1 Processor -- start 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b82f0 0x55c7a62c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 0x55c7a62c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 0x55c7a62c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c7a610cf50 con 0x55c7a62bc910 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c7a61a8770 con 0x55c7a62b82f0 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c7a62c4930 con 0x55c7a62b8c10 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9a2e9fb80 10 client.0 _ll_register_callbacks cb 0x55c7a619d530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc996ba6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 0x55c7a62c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc996ba6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 0x55c7a62c16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55386/0 (socket says 172.21.15.69:55386) 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc996ba6700 1 -- 172.21.15.69:0/2049556264 learned_addr learned my addr 172.21.15.69:0/2049556264 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.827+0000 7fc9963a5700 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b82f0 0x55c7a62c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc995ba4700 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 0x55c7a62c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc996ba6700 1 -- 172.21.15.69:0/2049556264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 msgr2=0x55c7a62c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc996ba6700 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 0x55c7a62c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc996ba6700 1 -- 172.21.15.69:0/2049556264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b82f0 msgr2=0x55c7a62c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc996ba6700 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b82f0 0x55c7a62c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc996ba6700 1 -- 172.21.15.69:0/2049556264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7a61a5640 con 0x55c7a62bc910 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc9963a5700 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b82f0 0x55c7a62c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc996ba6700 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 0x55c7a62c16d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc9880042c0 tx=0x7fc9880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4163a440e67749 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc979ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc979ffb700 1 -- 172.21.15.69:0/2049556264 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98801a070 con 0x55c7a62bc910 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc979ffb700 1 -- 172.21.15.69:0/2049556264 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7fc988007700 con 0x55c7a62bc910 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/2049556264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98c0aa390 con 0x55c7a62bc910 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc979ffb700 1 -- 172.21.15.69:0/2049556264 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc988026050 con 0x55c7a62bc910 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc9a2e9fb80 10 client.15363 fetch_fsmap learned FSMap version 25 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.828+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/2049556264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7a62cf000 con 0x55c7a62bc910 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.829+0000 7fc979ffb700 1 -- 172.21.15.69:0/2049556264 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98c0aa390 con 0x55c7a62bc910 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.829+0000 7fc979ffb700 1 -- 172.21.15.69:0/2049556264 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc98800eac0 con 0x55c7a62bc910 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.829+0000 7fc9a2e9fb80 10 client.15363 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.829+0000 7fc9a2e9fb80 10 client.15363 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.829+0000 7fc9a2e9fb80 20 client.15363 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:25.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.829+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55c7a62d1670 0x55c7a62d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.829+0000 7fc9a2e9fb80 4 client.15363 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:50:25.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.829+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/2049556264 --> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7a6284310 con 0x55c7a62d1670 2024-05-14T16:50:25.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.829+0000 7fc9963a5700 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55c7a62d1670 0x55c7a62d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.830+0000 7fc979ffb700 10 client.15363 ms_handle_connect on v2:172.21.15.203:6838/398835794 2024-05-14T16:50:25.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.830+0000 7fc9963a5700 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55c7a62d1670 0x55c7a62d2ae0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d97508a23f431605 server_cookie=c50e631852e1478e in_seq=0 out_seq=0 2024-05-14T16:50:25.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.830+0000 7fc979ffb700 1 -- 172.21.15.69:0/2049556264 <== mds.1 v2:172.21.15.203:6838/398835794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c7a6284310 con 0x55c7a62d1670 2024-05-14T16:50:25.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.830+0000 7fc979ffb700 10 client.15363 handle_command_reply: tid=0 2024-05-14T16:50:25.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.830+0000 7fc9a2e9fb80 1 client.15363 shutdown 2024-05-14T16:50:25.835 INFO:teuthology.orchestra.run.smithi069.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.39 s, remaining 2.65 s) 2024-05-14T16:50:25.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.830+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/2049556264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 msgr2=0x55c7a62c16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.830+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 0x55c7a62c16d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc9880042c0 tx=0x7fc9880042f0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.831+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/2049556264 shutdown_connections 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.831+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55c7a62d1670 0x55c7a62d2ae0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.831+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 0x55c7a62c16d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.831+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b82f0 0x55c7a62c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.831+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/2049556264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b8c10 0x55c7a62c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.831+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/2049556264 >> 172.21.15.69:0/2049556264 conn(0x55c7a61e3200 msgr2=0x55c7a62bdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.831+0000 7fc979ffb700 0 client.15363 ms_handle_reset on v2:172.21.15.203:6838/398835794 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.831+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/2049556264 shutdown_connections 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.831+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/2049556264 wait complete. 2024-05-14T16:50:25.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.831+0000 7fc9a2e9fb80 20 client.15363 trim_cache size 0 max 16384 2024-05-14T16:50:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.852+0000 7fc9a2e9fb80 1 Processor -- start 2024-05-14T16:50:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.852+0000 7fc9a2e9fb80 1 -- start start 2024-05-14T16:50:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.852+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62bfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.852+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 0x55c7a62bffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.852+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62bc910 0x55c7a62be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.852+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c7a62d0e20 con 0x55c7a62b82f0 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.852+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c7a62d14f0 con 0x55c7a62b8c10 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.852+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c7a62d1bc0 con 0x55c7a62bc910 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc9963a5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62bfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc9963a5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62bfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55390/0 (socket says 172.21.15.69:55390) 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc9963a5700 1 -- 172.21.15.69:0/1514744643 learned_addr learned my addr 172.21.15.69:0/1514744643 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc995ba4700 1 --2- 172.21.15.69:0/1514744643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 0x55c7a62bffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc9963a5700 1 -- 172.21.15.69:0/1514744643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62bc910 msgr2=0x55c7a62be480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc9963a5700 1 --2- 172.21.15.69:0/1514744643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62bc910 0x55c7a62be480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc9963a5700 1 -- 172.21.15.69:0/1514744643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 msgr2=0x55c7a62bffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc9963a5700 1 --2- 172.21.15.69:0/1514744643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 0x55c7a62bffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc9963a5700 1 -- 172.21.15.69:0/1514744643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7a62bf700 con 0x55c7a62b82f0 2024-05-14T16:50:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc9963a5700 1 --2- 172.21.15.69:0/1514744643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62bfa80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0a7600 tx=0x7fc98c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c257a4e606bab452 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc97bfff700 1 -- 172.21.15.69:0/1514744643 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0c4070 con 0x55c7a62b82f0 2024-05-14T16:50:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc97bfff700 1 -- 172.21.15.69:0/1514744643 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98c0ae740 con 0x55c7a62b82f0 2024-05-14T16:50:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.853+0000 7fc97bfff700 1 -- 172.21.15.69:0/1514744643 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0b2f30 con 0x55c7a62b82f0 2024-05-14T16:50:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.854+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/1514744643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 msgr2=0x55c7a62bfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.854+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/1514744643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62bfa80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0a7600 tx=0x7fc98c00fc60 comp rx=0 tx=0).stop 2024-05-14T16:50:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.854+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/1514744643 shutdown_connections 2024-05-14T16:50:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.854+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/1514744643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62b82f0 0x55c7a62bfa80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.854+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/1514744643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62bc910 0x55c7a62be480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.854+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/1514744643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 0x55c7a62bffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.854+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/1514744643 >> 172.21.15.69:0/1514744643 conn(0x55c7a61e3200 msgr2=0x55c7a62c1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.854+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/1514744643 shutdown_connections 2024-05-14T16:50:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.854+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/1514744643 wait complete. 2024-05-14T16:50:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.854+0000 7fc9a2e9fb80 1 -- start start 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc9a2e9fb80 1 Processor -- start 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b82f0 0x55c7a62ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 0x55c7a62ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc9a2e9fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 0x55c7a62bab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c7a62d2960 con 0x55c7a62bc910 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c7a62d2290 con 0x55c7a62b8c10 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc9a2e9fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c7a62d3030 con 0x55c7a62b82f0 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc9a2e9fb80 10 client.0 _ll_register_callbacks cb 0x55c7a619d530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc995ba4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 0x55c7a62ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc996ba6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 0x55c7a62bab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc9963a5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b82f0 0x55c7a62ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc995ba4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 0x55c7a62ba5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:53386/0 (socket says 172.21.15.69:53386) 2024-05-14T16:50:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc995ba4700 1 -- 172.21.15.69:0/990338633 learned_addr learned my addr 172.21.15.69:0/990338633 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc995ba4700 1 -- 172.21.15.69:0/990338633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b82f0 msgr2=0x55c7a62ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc995ba4700 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b82f0 0x55c7a62ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc995ba4700 1 -- 172.21.15.69:0/990338633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 msgr2=0x55c7a62bab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc995ba4700 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 0x55c7a62bab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc995ba4700 1 -- 172.21.15.69:0/990338633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7a62d8710 con 0x55c7a62b8c10 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc9963a5700 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b82f0 0x55c7a62ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc995ba4700 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 0x55c7a62ba5d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc98000b5d0 tx=0x7fc980004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42f197497a41a303 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.855+0000 7fc979ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc979ffb700 1 -- 172.21.15.69:0/990338633 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980003950 con 0x55c7a62b8c10 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc979ffb700 1 -- 172.21.15.69:0/990338633 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7fc980003af0 con 0x55c7a62b8c10 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/990338633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c7a6218b90 con 0x55c7a62b8c10 2024-05-14T16:50:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc979ffb700 1 -- 172.21.15.69:0/990338633 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc980003cf0 con 0x55c7a62b8c10 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc9a2e9fb80 10 client.24953 fetch_fsmap learned FSMap version 25 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/990338633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7a633d050 con 0x55c7a62b8c10 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc979ffb700 1 -- 172.21.15.69:0/990338633 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c7a6218b90 con 0x55c7a62b8c10 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc979ffb700 1 -- 172.21.15.69:0/990338633 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc9800114d0 con 0x55c7a62b8c10 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc9a2e9fb80 10 client.24953 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc9a2e9fb80 10 client.24953 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc9a2e9fb80 20 client.24953 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55c7a6341c00 0x55c7a6340910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc9a2e9fb80 4 client.24953 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.856+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/990338633 --> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] -- command(tid 0: {"prefix": "status"}) -- 0x55c7a6212370 con 0x55c7a6341c00 2024-05-14T16:50:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9963a5700 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55c7a6341c00 0x55c7a6340910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9963a5700 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55c7a6341c00 0x55c7a6340910 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3e786ecdafabb2d server_cookie=26a0e96781f7f22 in_seq=0 out_seq=0 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc979ffb700 10 client.24953 ms_handle_connect on v2:172.21.15.203:6838/398835794 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc979ffb700 1 -- 172.21.15.69:0/990338633 <== mds.1 v2:172.21.15.203:6838/398835794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c7a6212370 con 0x55c7a6341c00 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc979ffb700 10 client.24953 handle_command_reply: tid=0 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 client.24953 shutdown 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/990338633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 msgr2=0x55c7a62ba5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 0x55c7a62ba5d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc98000b5d0 tx=0x7fc980004750 comp rx=0 tx=0).stop 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/990338633 shutdown_connections 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.203:6838/398835794,v1:172.21.15.203:6839/398835794] conn(0x55c7a6341c00 0x55c7a6340910 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7a62bc910 0x55c7a62bab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc979ffb700 0 client.24953 ms_handle_reset on v2:172.21.15.203:6838/398835794 2024-05-14T16:50:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c7a62b82f0 0x55c7a62ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 --2- 172.21.15.69:0/990338633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7a62b8c10 0x55c7a62ba5d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/990338633 >> 172.21.15.69:0/990338633 conn(0x55c7a61e3200 msgr2=0x55c7a62b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/990338633 shutdown_connections 2024-05-14T16:50:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.857+0000 7fc9a2e9fb80 1 -- 172.21.15.69:0/990338633 wait complete. 2024-05-14T16:50:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc9a2e9fb80 20 client.24953 trim_cache size 0 max 16384 2024-05-14T16:50:25.865 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:50:25.865 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24433, 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 207.335895827, 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 25, 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 81, 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 247.16196353500001 2024-05-14T16:50:25.866 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:50:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 -- 172.21.15.69:0/2768633603 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc97c0b2b10 msgr2=0x7fc97c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc97c0b2b10 0x7fc97c0b4fd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc980005950 tx=0x7fc9800058e0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 -- 172.21.15.69:0/2768633603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc990102470 msgr2=0x7fc99019f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc990102470 0x7fc99019f8a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc98c006010 tx=0x7fc98c0045e0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 -- 172.21.15.69:0/2768633603 shutdown_connections 2024-05-14T16:50:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc97c0b2b10 0x7fc97c0b4fd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc990108550 0x7fc990199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc990102470 0x7fc99019f8a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 --2- 172.21.15.69:0/2768633603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc990102e30 0x7fc99019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 -- 172.21.15.69:0/2768633603 >> 172.21.15.69:0/2768633603 conn(0x7fc990079480 msgr2=0x7fc9901063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 -- 172.21.15.69:0/2768633603 shutdown_connections 2024-05-14T16:50:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:25.858+0000 7fc998609700 1 -- 172.21.15.69:0/2768633603 wait complete. 2024-05-14T16:50:25.925 INFO:teuthology.orchestra.run.smithi069.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.49 s, remaining 2.48 s) 2024-05-14T16:50:26.062 INFO:teuthology.orchestra.run.smithi069.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.62 s, remaining 2.37 s) 2024-05-14T16:50:26.101 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:25 smithi203 podman[70077]: 2024-05-14 16:50:25.856475502 +0000 UTC m=+0.733917310 container remove 37afd030daf7762b90a58fb8f92d12f60d0d9b0992490c8e2f2ba980e9423d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-14T16:50:26.102 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:25 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-05-14T16:50:26.102 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:25 smithi203 ceph-mon[35787]: pgmap v18: 97 pgs: 97 active+clean; 2.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 69 KiB/s wr, 8 op/s 2024-05-14T16:50:26.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:25 smithi069 ceph-mon[31507]: pgmap v18: 97 pgs: 97 active+clean; 2.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 69 KiB/s wr, 8 op/s 2024-05-14T16:50:26.224 INFO:teuthology.orchestra.run.smithi069.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.78 s, remaining 2.29 s) 2024-05-14T16:50:26.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:25 smithi092 ceph-mon[35763]: pgmap v18: 97 pgs: 97 active+clean; 2.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 69 KiB/s wr, 8 op/s 2024-05-14T16:50:26.311 INFO:teuthology.orchestra.run.smithi069.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.87 s, remaining 2.12 s) 2024-05-14T16:50:26.443 INFO:teuthology.orchestra.run.smithi069.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.00 s, remaining 2.00 s) 2024-05-14T16:50:26.459 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:26 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-14T16:50:26.459 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:26 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Consumed 1.241s CPU time. 2024-05-14T16:50:26.527 INFO:teuthology.orchestra.run.smithi069.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.09 s, remaining 1.84 s) 2024-05-14T16:50:26.637 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T16:50:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.25 s, remaining 1.75 s) 2024-05-14T16:50:26.847 INFO:teuthology.orchestra.run.smithi069.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.41 s, remaining 1.65 s) 2024-05-14T16:50:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:27.010 INFO:teuthology.orchestra.run.smithi069.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.57 s, remaining 1.54 s) 2024-05-14T16:50:27.088 INFO:teuthology.orchestra.run.smithi069.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.65 s, remaining 1.39 s) 2024-05-14T16:50:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.80 s, remaining 1.27 s) 2024-05-14T16:50:27.317 INFO:teuthology.orchestra.run.smithi069.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.88 s, remaining 1.13 s) 2024-05-14T16:50:27.455 INFO:teuthology.orchestra.run.smithi069.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.02 s, remaining 1.01 s) 2024-05-14T16:50:27.621 INFO:teuthology.orchestra.run.smithi069.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.18 s, remaining 0.89 s) 2024-05-14T16:50:27.725 INFO:teuthology.orchestra.run.smithi069.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.29 s, remaining 0.76 s) 2024-05-14T16:50:27.852 INFO:teuthology.orchestra.run.smithi069.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.41 s, remaining 0.63 s) 2024-05-14T16:50:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.51 s, remaining 0.50 s) 2024-05-14T16:50:28.017 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:27 smithi069 ceph-mon[31507]: pgmap v19: 97 pgs: 97 active+clean; 141 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 19 MiB/s wr, 271 op/s 2024-05-14T16:50:28.103 INFO:teuthology.orchestra.run.smithi069.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.66 s, remaining 0.38 s) 2024-05-14T16:50:28.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:27 smithi203 ceph-mon[35787]: pgmap v19: 97 pgs: 97 active+clean; 141 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 19 MiB/s wr, 271 op/s 2024-05-14T16:50:28.249 INFO:teuthology.orchestra.run.smithi069.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.81 s, remaining 0.25 s) 2024-05-14T16:50:28.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:27 smithi092 ceph-mon[35763]: pgmap v19: 97 pgs: 97 active+clean; 141 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 19 MiB/s wr, 271 op/s 2024-05-14T16:50:28.352 INFO:teuthology.orchestra.run.smithi069.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.91 s, remaining 0.13 s) 2024-05-14T16:50:28.516 INFO:teuthology.orchestra.run.smithi069.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.08 s, remaining 0.00 s) 2024-05-14T16:50:28.528 INFO:teuthology.orchestra.run.smithi069.stderr:vacuuming... 2024-05-14T16:50:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.645+0000 7f762247d700 1 -- 172.21.15.69:0/1305876077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f761c103fa0 msgr2=0x7f761c110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.645+0000 7f762247d700 1 --2- 172.21.15.69:0/1305876077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f761c103fa0 0x7f761c110450 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f761400b0d0 tx=0x7f761400b3e0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.645+0000 7f762247d700 1 -- 172.21.15.69:0/1305876077 shutdown_connections 2024-05-14T16:50:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.645+0000 7f762247d700 1 --2- 172.21.15.69:0/1305876077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f761c103fa0 0x7f761c110450 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.645+0000 7f762247d700 1 --2- 172.21.15.69:0/1305876077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f761c109680 0x7f761c109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.645+0000 7f762247d700 1 --2- 172.21.15.69:0/1305876077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c1035e0 0x7f761c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.645+0000 7f762247d700 1 -- 172.21.15.69:0/1305876077 >> 172.21.15.69:0/1305876077 conn(0x7f761c07a6f0 msgr2=0x7f761c07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.646+0000 7f762247d700 1 -- 172.21.15.69:0/1305876077 shutdown_connections 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.646+0000 7f762247d700 1 -- 172.21.15.69:0/1305876077 wait complete. 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.646+0000 7f762247d700 1 Processor -- start 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f762247d700 1 -- start start 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f762247d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f761c1035e0 0x7f761c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f762247d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c103fa0 0x7f761c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f762247d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f761c109680 0x7f761c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f762247d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1137a0 con 0x7f761c1035e0 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f762247d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f761c113620 con 0x7f761c109680 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f762247d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f761c113920 con 0x7f761c103fa0 2024-05-14T16:50:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f761bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f761c1035e0 0x7f761c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f761bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f761c1035e0 0x7f761c19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55426/0 (socket says 172.21.15.69:55426) 2024-05-14T16:50:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f761bfff700 1 -- 172.21.15.69:0/2904080367 learned_addr learned my addr 172.21.15.69:0/2904080367 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f7620a1a700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f761c109680 0x7f761c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.647+0000 7f761b7fe700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c103fa0 0x7f761c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f761bfff700 1 -- 172.21.15.69:0/2904080367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c103fa0 msgr2=0x7f761c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f761bfff700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c103fa0 0x7f761c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f761bfff700 1 -- 172.21.15.69:0/2904080367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f761c109680 msgr2=0x7f761c199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f761bfff700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f761c109680 0x7f761c199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f761bfff700 1 -- 172.21.15.69:0/2904080367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c19a0e0 con 0x7f761c1035e0 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f7620a1a700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f761c109680 0x7f761c199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f761bfff700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f761c1035e0 0x7f761c19f770 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f760c00c9b0 tx=0x7f760c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f76197fa700 1 -- 172.21.15.69:0/2904080367 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c007b00 con 0x7f761c1035e0 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f762247d700 1 -- 172.21.15.69:0/2904080367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c19a3d0 con 0x7f761c1035e0 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f76197fa700 1 -- 172.21.15.69:0/2904080367 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760c00ce50 con 0x7f761c1035e0 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f76197fa700 1 -- 172.21.15.69:0/2904080367 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c018660 con 0x7f761c1035e0 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.648+0000 7f762247d700 1 -- 172.21.15.69:0/2904080367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c1a6640 con 0x7f761c1035e0 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.651+0000 7f76197fa700 1 -- 172.21.15.69:0/2904080367 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f760c004830 con 0x7f761c1035e0 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f762cd13b80 1 Processor -- start 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f76197fa700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76040b2b10 0x7f76040b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f762cd13b80 1 -- start start 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f7620a1a700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76040b2b10 0x7f76040b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f76197fa700 1 -- 172.21.15.69:0/2904080367 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f760c0e2860 con 0x7f761c1035e0 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d0771fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 0x55d0772003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077200910 0x55d077200d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f762cd13b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d07715ac40 con 0x55d0771fc2f0 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f762cd13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d07714a7e0 con 0x55d077200910 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f762cd13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d07714e820 con 0x55d0771fcc10 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f761bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d0771fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f761bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d0771fc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55432/0 (socket says 172.21.15.69:55432) 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.652+0000 7f761bfff700 1 -- 172.21.15.69:0/2307063981 learned_addr learned my addr 172.21.15.69:0/2307063981 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f7620a1a700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76040b2b10 0x7f76040b4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7614009fd0 tx=0x7f761400b040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f7620a1a700 1 --2- 172.21.15.69:0/2307063981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077200910 0x55d077200d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f761b7fe700 1 --2- 172.21.15.69:0/2307063981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 0x55d0772003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f761bfff700 1 -- 172.21.15.69:0/2307063981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 msgr2=0x55d0772003d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f761bfff700 1 --2- 172.21.15.69:0/2307063981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 0x55d0772003d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f761bfff700 1 -- 172.21.15.69:0/2307063981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077200910 msgr2=0x55d077200d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f761bfff700 1 --2- 172.21.15.69:0/2307063981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077200910 0x55d077200d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f761bfff700 1 -- 172.21.15.69:0/2307063981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d07709a880 con 0x55d0771fc2f0 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f761bfff700 1 --2- 172.21.15.69:0/2307063981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d0771fc6d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f760c0a9830 tx=0x7f760c0a9270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7cafff541afe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f76027fc700 1 -- 172.21.15.69:0/2307063981 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0c3070 con 0x55d0771fc2f0 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f76027fc700 1 -- 172.21.15.69:0/2307063981 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760c0a93b0 con 0x55d0771fc2f0 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.653+0000 7f76027fc700 1 -- 172.21.15.69:0/2307063981 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0a9c70 con 0x55d0771fc2f0 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.654+0000 7f762cd13b80 1 -- 172.21.15.69:0/2307063981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 msgr2=0x55d0771fc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.654+0000 7f762cd13b80 1 --2- 172.21.15.69:0/2307063981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d0771fc6d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f760c0a9830 tx=0x7f760c0a9270 comp rx=0 tx=0).stop 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.654+0000 7f762cd13b80 1 -- 172.21.15.69:0/2307063981 shutdown_connections 2024-05-14T16:50:29.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.654+0000 7f762cd13b80 1 --2- 172.21.15.69:0/2307063981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d0771fc6d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.654+0000 7f762cd13b80 1 --2- 172.21.15.69:0/2307063981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077200910 0x55d077200d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.654+0000 7f762cd13b80 1 --2- 172.21.15.69:0/2307063981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 0x55d0772003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.654+0000 7f762cd13b80 1 -- 172.21.15.69:0/2307063981 >> 172.21.15.69:0/2307063981 conn(0x55d077127200 msgr2=0x55d0771f6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.654+0000 7f762cd13b80 1 -- 172.21.15.69:0/2307063981 shutdown_connections 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.654+0000 7f762cd13b80 1 -- 172.21.15.69:0/2307063981 wait complete. 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.654+0000 7f762cd13b80 1 -- start start 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f762cd13b80 1 Processor -- start 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d077207d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 0x55d077205130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077200910 0x55d077205670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f762cd13b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d077050f50 con 0x55d0771fc2f0 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f762cd13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d0770ec770 con 0x55d077200910 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f762cd13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d077208b70 con 0x55d0771fcc10 2024-05-14T16:50:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f762cd13b80 10 client.0 _ll_register_callbacks cb 0x55d0770e1530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f761bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d077207d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f761bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d077207d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55444/0 (socket says 172.21.15.69:55444) 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f761bfff700 1 -- 172.21.15.69:0/760168811 learned_addr learned my addr 172.21.15.69:0/760168811 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f7620a1a700 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077200910 0x55d077205670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f761b7fe700 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 0x55d077205130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f761bfff700 1 -- 172.21.15.69:0/760168811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 msgr2=0x55d077205130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f761bfff700 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 0x55d077205130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f761bfff700 1 -- 172.21.15.69:0/760168811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077200910 msgr2=0x55d077205670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f761bfff700 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077200910 0x55d077205670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f761bfff700 1 -- 172.21.15.69:0/760168811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0770e9640 con 0x55d0771fc2f0 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.655+0000 7f761b7fe700 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 0x55d077205130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f761bfff700 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d077207d90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f760c0a51a0 tx=0x7f760c0a9d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dbe5a898175d58 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:29.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f75fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:50:29.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f75fb7fe700 1 -- 172.21.15.69:0/760168811 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0c3070 con 0x55d0771fc2f0 2024-05-14T16:50:29.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f75fb7fe700 1 -- 172.21.15.69:0/760168811 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f760c0a5e90 con 0x55d0771fc2f0 2024-05-14T16:50:29.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f762cd13b80 1 -- 172.21.15.69:0/760168811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f760c0a93b0 con 0x55d0771fc2f0 2024-05-14T16:50:29.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f75fb7fe700 1 -- 172.21.15.69:0/760168811 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f760c088050 con 0x55d0771fc2f0 2024-05-14T16:50:29.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f762cd13b80 10 client.15381 fetch_fsmap learned FSMap version 25 2024-05-14T16:50:29.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f762cd13b80 1 -- 172.21.15.69:0/760168811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d077213230 con 0x55d0771fc2f0 2024-05-14T16:50:29.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f75fb7fe700 1 -- 172.21.15.69:0/760168811 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760c0a93b0 con 0x55d0771fc2f0 2024-05-14T16:50:29.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f75fb7fe700 1 -- 172.21.15.69:0/760168811 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f760c0b2270 con 0x55d0771fc2f0 2024-05-14T16:50:29.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f762cd13b80 10 client.15381 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:50:29.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f762cd13b80 10 client.15381 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:50:29.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f762cd13b80 20 client.15381 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:29.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f762cd13b80 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d077214580 0x55d077216780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f762cd13b80 4 client.15381 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:50:29.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.656+0000 7f762cd13b80 1 -- 172.21.15.69:0/760168811 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0771c8310 con 0x55d077214580 2024-05-14T16:50:29.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.657+0000 7f761b7fe700 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d077214580 0x55d077216780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.657+0000 7f761b7fe700 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d077214580 0x55d077216780 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1882a27c6ee90848 server_cookie=e88fa913bd0f9ad2 in_seq=0 out_seq=0 2024-05-14T16:50:29.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.657+0000 7f75fb7fe700 10 client.15381 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:29.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f75fb7fe700 1 -- 172.21.15.69:0/760168811 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0771c8310 con 0x55d077214580 2024-05-14T16:50:29.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f75fb7fe700 10 client.15381 handle_command_reply: tid=0 2024-05-14T16:50:29.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 client.15381 shutdown 2024-05-14T16:50:29.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 -- 172.21.15.69:0/760168811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 msgr2=0x55d077207d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d077207d90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f760c0a51a0 tx=0x7f760c0a9d30 comp rx=0 tx=0).stop 2024-05-14T16:50:29.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 -- 172.21.15.69:0/760168811 shutdown_connections 2024-05-14T16:50:29.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d077214580 0x55d077216780 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fc2f0 0x55d077207d90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077200910 0x55d077205670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 --2- 172.21.15.69:0/760168811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fcc10 0x55d077205130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 -- 172.21.15.69:0/760168811 >> 172.21.15.69:0/760168811 conn(0x55d077127200 msgr2=0x55d077201de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:29.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f75fb7fe700 0 client.15381 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:29.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 -- 172.21.15.69:0/760168811 shutdown_connections 2024-05-14T16:50:29.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 1 -- 172.21.15.69:0/760168811 wait complete. 2024-05-14T16:50:29.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.658+0000 7f762cd13b80 20 client.15381 trim_cache size 0 max 16384 2024-05-14T16:50:29.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f762cd13b80 1 Processor -- start 2024-05-14T16:50:29.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f762cd13b80 1 -- start start 2024-05-14T16:50:29.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077127200 0x55d0771fb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fc2f0 0x55d0771fe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d0771fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f762cd13b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d077214ac0 con 0x55d0771fcc10 2024-05-14T16:50:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f762cd13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d077215190 con 0x55d077127200 2024-05-14T16:50:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f762cd13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d077215860 con 0x55d0771fc2f0 2024-05-14T16:50:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f7620a1a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d0771fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f7620a1a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d0771fe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55454/0 (socket says 172.21.15.69:55454) 2024-05-14T16:50:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.679+0000 7f7620a1a700 1 -- 172.21.15.69:0/3870231755 learned_addr learned my addr 172.21.15.69:0/3870231755 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f761bfff700 1 --2- 172.21.15.69:0/3870231755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077127200 0x55d0771fb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f761b7fe700 1 --2- 172.21.15.69:0/3870231755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fc2f0 0x55d0771fe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f7620a1a700 1 -- 172.21.15.69:0/3870231755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fc2f0 msgr2=0x55d0771fe340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f7620a1a700 1 --2- 172.21.15.69:0/3870231755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fc2f0 0x55d0771fe340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f7620a1a700 1 -- 172.21.15.69:0/3870231755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077127200 msgr2=0x55d0771fb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f7620a1a700 1 --2- 172.21.15.69:0/3870231755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077127200 0x55d0771fb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f7620a1a700 1 -- 172.21.15.69:0/3870231755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d077214fb0 con 0x55d0771fcc10 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f761b7fe700 1 --2- 172.21.15.69:0/3870231755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fc2f0 0x55d0771fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f7620a1a700 1 --2- 172.21.15.69:0/3870231755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d0771fe880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7614007770 tx=0x7f761401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ce468831bc3b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f7601ffb700 1 -- 172.21.15.69:0/3870231755 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614007c60 con 0x55d0771fcc10 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f7601ffb700 1 -- 172.21.15.69:0/3870231755 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76140298d0 con 0x55d0771fcc10 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.680+0000 7f7601ffb700 1 -- 172.21.15.69:0/3870231755 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761401c650 con 0x55d0771fcc10 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 -- 172.21.15.69:0/3870231755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 msgr2=0x55d0771fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 --2- 172.21.15.69:0/3870231755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d0771fe880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7614007770 tx=0x7f761401e000 comp rx=0 tx=0).stop 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 -- 172.21.15.69:0/3870231755 shutdown_connections 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 --2- 172.21.15.69:0/3870231755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d0771fe880 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 --2- 172.21.15.69:0/3870231755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d077127200 0x55d0771fb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 --2- 172.21.15.69:0/3870231755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0771fc2f0 0x55d0771fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 -- 172.21.15.69:0/3870231755 >> 172.21.15.69:0/3870231755 conn(0x55d077206020 msgr2=0x55d0771fb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 -- 172.21.15.69:0/3870231755 shutdown_connections 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 -- 172.21.15.69:0/3870231755 wait complete. 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 -- start start 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.681+0000 7f762cd13b80 1 Processor -- start 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d077127200 0x55d07721fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0771fc2f0 0x55d07721abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f762cd13b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d07721b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f762cd13b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d077216600 con 0x55d0771fcc10 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f762cd13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d077215f30 con 0x55d0771fc2f0 2024-05-14T16:50:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f762cd13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d077216cd0 con 0x55d077127200 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f762cd13b80 10 client.0 _ll_register_callbacks cb 0x55d0770e1530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f7620a1a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d07721b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f7620a1a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d07721b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55470/0 (socket says 172.21.15.69:55470) 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f7620a1a700 1 -- 172.21.15.69:0/1853266562 learned_addr learned my addr 172.21.15.69:0/1853266562 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f761bfff700 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d077127200 0x55d07721fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f761b7fe700 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0771fc2f0 0x55d07721abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f7620a1a700 1 -- 172.21.15.69:0/1853266562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d077127200 msgr2=0x55d07721fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f7620a1a700 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d077127200 0x55d07721fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f7620a1a700 1 -- 172.21.15.69:0/1853266562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0771fc2f0 msgr2=0x55d07721abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f7620a1a700 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0771fc2f0 0x55d07721abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f7620a1a700 1 -- 172.21.15.69:0/1853266562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d07721b6b0 con 0x55d0771fcc10 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f7620a1a700 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d07721b0f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7614034040 tx=0x7f7614029790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cf18624757cbff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.682+0000 7f75fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f75fb7fe700 1 -- 172.21.15.69:0/1853266562 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614040060 con 0x55d0771fcc10 2024-05-14T16:50:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f762cd13b80 1 -- 172.21.15.69:0/1853266562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76140298d0 con 0x55d0771fcc10 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f75fb7fe700 1 -- 172.21.15.69:0/1853266562 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f761402f040 con 0x55d0771fcc10 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f75fb7fe700 1 -- 172.21.15.69:0/1853266562 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7614029d90 con 0x55d0771fcc10 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f762cd13b80 10 client.15387 fetch_fsmap learned FSMap version 25 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f762cd13b80 1 -- 172.21.15.69:0/1853266562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d077281530 con 0x55d0771fcc10 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f75fb7fe700 1 -- 172.21.15.69:0/1853266562 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76140298d0 con 0x55d0771fcc10 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f75fb7fe700 1 -- 172.21.15.69:0/1853266562 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7614011070 con 0x55d0771fcc10 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f762cd13b80 10 client.15387 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f762cd13b80 10 client.15387 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f762cd13b80 20 client.15387 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f762cd13b80 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d077200910 0x55d077284870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f762cd13b80 4 client.15387 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f762cd13b80 1 -- 172.21.15.69:0/1853266562 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "status"}) -- 0x55d07715cb90 con 0x55d077200910 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.683+0000 7f761bfff700 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d077200910 0x55d077284870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f75fb7fe700 10 client.15387 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f761bfff700 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d077200910 0x55d077284870 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52f7590c8e44805 server_cookie=17fb98732dbd8624 in_seq=0 out_seq=0 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f75fb7fe700 1 -- 172.21.15.69:0/1853266562 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d07715cb90 con 0x55d077200910 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f75fb7fe700 10 client.15387 handle_command_reply: tid=0 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 client.15387 shutdown 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 -- 172.21.15.69:0/1853266562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 msgr2=0x55d07721b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d07721b0f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7614034040 tx=0x7f7614029790 comp rx=0 tx=0).stop 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 -- 172.21.15.69:0/1853266562 shutdown_connections 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d077200910 0x55d077284870 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0771fcc10 0x55d07721b0f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d077127200 0x55d07721fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f75fb7fe700 0 client.15387 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 --2- 172.21.15.69:0/1853266562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0771fc2f0 0x55d07721abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 -- 172.21.15.69:0/1853266562 >> 172.21.15.69:0/1853266562 conn(0x55d077206020 msgr2=0x55d0771fb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 -- 172.21.15.69:0/1853266562 shutdown_connections 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 1 -- 172.21.15.69:0/1853266562 wait complete. 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.684+0000 7f762cd13b80 20 client.15387 trim_cache size 0 max 16384 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "356e185e-1210-11ef-bc98-c7b262605968", 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24446, 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 223.121982422, 2024-05-14T16:50:29.087 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 25, 2024-05-14T16:50:29.088 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 81, 2024-05-14T16:50:29.088 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T16:50:29.088 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 240.08690028800001 2024-05-14T16:50:29.088 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:50:29.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 -- 172.21.15.69:0/2904080367 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76040b2b10 msgr2=0x7f76040b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76040b2b10 0x7f76040b4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7614009fd0 tx=0x7f761400b040 comp rx=0 tx=0).stop 2024-05-14T16:50:29.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 -- 172.21.15.69:0/2904080367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f761c1035e0 msgr2=0x7f761c19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f761c1035e0 0x7f761c19f770 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f760c00c9b0 tx=0x7f760c00ccc0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 -- 172.21.15.69:0/2904080367 shutdown_connections 2024-05-14T16:50:29.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76040b2b10 0x7f76040b4fd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f761c1035e0 0x7f761c19f770 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f761c109680 0x7f761c199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 --2- 172.21.15.69:0/2904080367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f761c103fa0 0x7f761c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 -- 172.21.15.69:0/2904080367 >> 172.21.15.69:0/2904080367 conn(0x7f761c07a6f0 msgr2=0x7f761c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:29.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 -- 172.21.15.69:0/2904080367 shutdown_connections 2024-05-14T16:50:29.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:28.685+0000 7f762247d700 1 -- 172.21.15.69:0/2904080367 wait complete. 2024-05-14T16:50:29.128 INFO:teuthology.orchestra.run.smithi069.stderr:creating primary keys... 2024-05-14T16:50:29.837 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-14T16:50:30.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:29 smithi069 ceph-mon[31507]: pgmap v20: 97 pgs: 97 active+clean; 141 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 19 MiB/s wr, 269 op/s 2024-05-14T16:50:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:29 smithi203 ceph-mon[35787]: pgmap v20: 97 pgs: 97 active+clean; 141 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 19 MiB/s wr, 269 op/s 2024-05-14T16:50:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:29 smithi092 ceph-mon[35763]: pgmap v20: 97 pgs: 97 active+clean; 141 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 19 MiB/s wr, 269 op/s 2024-05-14T16:50:30.335 INFO:teuthology.orchestra.run.smithi069.stderr:done in 5.92 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.10 s, vacuum 0.60 s, primary keys 1.21 s). 2024-05-14T16:50:30.338 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-14T16:50:30.394 INFO:teuthology.orchestra.run.smithi069.stderr:starting vacuum...end. 2024-05-14T16:50:32.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:31 smithi069 ceph-mon[31507]: pgmap v21: 97 pgs: 97 active+clean; 309 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 45 MiB/s wr, 284 op/s 2024-05-14T16:50:32.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:31 smithi203 ceph-mon[35787]: pgmap v21: 97 pgs: 97 active+clean; 309 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 45 MiB/s wr, 284 op/s 2024-05-14T16:50:32.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:31 smithi092 ceph-mon[35763]: pgmap v21: 97 pgs: 97 active+clean; 309 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 45 MiB/s wr, 284 op/s 2024-05-14T16:50:34.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:33 smithi069 ceph-mon[31507]: pgmap v22: 97 pgs: 97 active+clean; 498 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 74 MiB/s wr, 307 op/s 2024-05-14T16:50:34.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:33 smithi203 ceph-mon[35787]: pgmap v22: 97 pgs: 97 active+clean; 498 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 74 MiB/s wr, 307 op/s 2024-05-14T16:50:34.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:33 smithi092 ceph-mon[35763]: pgmap v22: 97 pgs: 97 active+clean; 498 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 74 MiB/s wr, 307 op/s 2024-05-14T16:50:35.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:34 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:50:35.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:34 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:50:35.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:34 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:50:35.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 5.0 s, 3494.4 tps, lat 28.145 ms stddev 28.912 2024-05-14T16:50:36.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:35 smithi069 ceph-mon[31507]: pgmap v23: 97 pgs: 97 active+clean; 498 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 74 MiB/s wr, 302 op/s 2024-05-14T16:50:36.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:35 smithi203 ceph-mon[35787]: pgmap v23: 97 pgs: 97 active+clean; 498 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 74 MiB/s wr, 302 op/s 2024-05-14T16:50:36.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:35 smithi092 ceph-mon[35763]: pgmap v23: 97 pgs: 97 active+clean; 498 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 74 MiB/s wr, 302 op/s 2024-05-14T16:50:36.473 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:36 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-05-14T16:50:36.473 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:36 smithi203 systemd[1]: Stopped Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:50:36.473 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:36 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Consumed 1.241s CPU time. 2024-05-14T16:50:36.473 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:36 smithi203 systemd[1]: Starting Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:50:36.473 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:36 smithi203 podman[70169]: 2024-05-14 16:50:36.353352121 +0000 UTC m=+0.029255915 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:50:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.573+0000 7f64a499d640 1 -- 172.21.15.69:0/2682066891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64a0102a30 msgr2=0x7f64a0102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.573+0000 7f64a499d640 1 --2- 172.21.15.69:0/2682066891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64a0102a30 0x7f64a0102eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6488009930 tx=0x7f648802a440 comp rx=0 tx=0).stop 2024-05-14T16:50:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.573+0000 7f64a499d640 1 -- 172.21.15.69:0/2682066891 shutdown_connections 2024-05-14T16:50:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.573+0000 7f64a499d640 1 --2- 172.21.15.69:0/2682066891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a01033f0 0x7f64a010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.573+0000 7f64a499d640 1 --2- 172.21.15.69:0/2682066891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64a0102a30 0x7f64a0102eb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.573+0000 7f64a499d640 1 --2- 172.21.15.69:0/2682066891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64a0108a40 0x7f64a0108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.573+0000 7f64a499d640 1 -- 172.21.15.69:0/2682066891 >> 172.21.15.69:0/2682066891 conn(0x7f64a00fe500 msgr2=0x7f64a0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.573+0000 7f64a499d640 1 -- 172.21.15.69:0/2682066891 shutdown_connections 2024-05-14T16:50:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.573+0000 7f64a499d640 1 -- 172.21.15.69:0/2682066891 wait complete. 2024-05-14T16:50:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.575+0000 7f64a499d640 1 Processor -- start 2024-05-14T16:50:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.575+0000 7f64a499d640 1 -- start start 2024-05-14T16:50:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.575+0000 7f64a499d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a0102a30 0x7f64a0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.575+0000 7f64a499d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64a01033f0 0x7f64a0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.575+0000 7f64a499d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64a0108a40 0x7f64a019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.575+0000 7f64a499d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0112cd0 con 0x7f64a01033f0 2024-05-14T16:50:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.575+0000 7f64a499d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0112b50 con 0x7f64a0102a30 2024-05-14T16:50:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.575+0000 7f64a499d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0112e50 con 0x7f64a0108a40 2024-05-14T16:50:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f649e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a0102a30 0x7f64a0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f6497fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64a01033f0 0x7f64a0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f6497fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64a01033f0 0x7f64a0199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46282/0 (socket says 172.21.15.69:46282) 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f649e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a0102a30 0x7f64a0199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:46298/0 (socket says 172.21.15.69:46298) 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f6497fff640 1 -- 172.21.15.69:0/2123015728 learned_addr learned my addr 172.21.15.69:0/2123015728 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f649ed9d640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64a0108a40 0x7f64a019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f6497fff640 1 -- 172.21.15.69:0/2123015728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64a0108a40 msgr2=0x7f64a019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f6497fff640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64a0108a40 0x7f64a019daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f6497fff640 1 -- 172.21.15.69:0/2123015728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a0102a30 msgr2=0x7f64a0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f6497fff640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a0102a30 0x7f64a0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f6497fff640 1 -- 172.21.15.69:0/2123015728 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a019e220 con 0x7f64a01033f0 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f6497fff640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64a01033f0 0x7f64a0199770 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6488004770 tx=0x7f64880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f64977fe640 1 -- 172.21.15.69:0/2123015728 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648803f070 con 0x7f64a01033f0 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f64977fe640 1 -- 172.21.15.69:0/2123015728 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f648803b460 con 0x7f64a01033f0 2024-05-14T16:50:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f64a499d640 1 -- 172.21.15.69:0/2123015728 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a019e4b0 con 0x7f64a01033f0 2024-05-14T16:50:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f64977fe640 1 -- 172.21.15.69:0/2123015728 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488035070 con 0x7f64a01033f0 2024-05-14T16:50:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.576+0000 7f64a499d640 1 -- 172.21.15.69:0/2123015728 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a01a5da0 con 0x7f64a01033f0 2024-05-14T16:50:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.579+0000 7f64977fe640 1 -- 172.21.15.69:0/2123015728 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f648803a620 con 0x7f64a01033f0 2024-05-14T16:50:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.579+0000 7f64a778c740 1 Processor -- start 2024-05-14T16:50:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.580+0000 7f64a778c740 1 -- start start 2024-05-14T16:50:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.580+0000 7f64a778c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 0x559b33f21cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.580+0000 7f64a778c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f22210 0x559b33f2c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.580+0000 7f64977fe640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f648c0b2b30 0x7f648c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 0x559b33f21cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 0x559b33f21cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:46312/0 (socket says 172.21.15.69:46312) 2024-05-14T16:50:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.580+0000 7f64a778c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 0x559b33f2eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649ed9d640 1 -- 172.21.15.69:0/188659605 learned_addr learned my addr 172.21.15.69:0/188659605 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649e59c640 1 --2- 172.21.15.69:0/188659605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f22210 0x559b33f2c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f64a778c740 1 -- 172.21.15.69:0/188659605 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559b33e94c30 con 0x559b33f22210 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f64a778c740 1 -- 172.21.15.69:0/188659605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559b33e847d0 con 0x559b33f218f0 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f64a778c740 1 -- 172.21.15.69:0/188659605 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559b33e88810 con 0x559b33f2cad0 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649ed9d640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f648c0b2b30 0x7f648c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f64977fe640 1 -- 172.21.15.69:0/2123015728 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f6488102af0 con 0x7f64a01033f0 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f6497fff640 1 --2- 172.21.15.69:0/188659605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 0x559b33f2eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649e59c640 1 -- 172.21.15.69:0/188659605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 msgr2=0x559b33f2eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649e59c640 1 --2- 172.21.15.69:0/188659605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 0x559b33f2eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649e59c640 1 -- 172.21.15.69:0/188659605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 msgr2=0x559b33f21cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649e59c640 1 --2- 172.21.15.69:0/188659605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 0x559b33f21cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649e59c640 1 -- 172.21.15.69:0/188659605 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b33b99210 con 0x559b33f22210 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649e59c640 1 --2- 172.21.15.69:0/188659605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f22210 0x559b33f2c590 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f649000bb70 tx=0x7f64900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a2ee1154ef43a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f649ed9d640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f648c0b2b30 0x7f648c0b4ff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6498009270 tx=0x7f64980096a0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f6494ff9640 1 -- 172.21.15.69:0/188659605 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490004030 con 0x559b33f22210 2024-05-14T16:50:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f6494ff9640 1 -- 172.21.15.69:0/188659605 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64900035f0 con 0x559b33f22210 2024-05-14T16:50:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.581+0000 7f6494ff9640 1 -- 172.21.15.69:0/188659605 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490011680 con 0x559b33f22210 2024-05-14T16:50:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.582+0000 7f64a778c740 1 -- 172.21.15.69:0/188659605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f22210 msgr2=0x559b33f2c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.582+0000 7f64a778c740 1 --2- 172.21.15.69:0/188659605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f22210 0x559b33f2c590 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f649000bb70 tx=0x7f64900107a0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.582+0000 7f64a778c740 1 -- 172.21.15.69:0/188659605 shutdown_connections 2024-05-14T16:50:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.582+0000 7f64a778c740 1 --2- 172.21.15.69:0/188659605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 0x559b33f2eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.582+0000 7f64a778c740 1 --2- 172.21.15.69:0/188659605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f22210 0x559b33f2c590 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.582+0000 7f64a778c740 1 --2- 172.21.15.69:0/188659605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 0x559b33f21cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.582+0000 7f64a778c740 1 -- 172.21.15.69:0/188659605 >> 172.21.15.69:0/188659605 conn(0x559b33f1de00 msgr2=0x559b33f1e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.582+0000 7f64a778c740 1 -- 172.21.15.69:0/188659605 shutdown_connections 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.582+0000 7f64a778c740 1 -- 172.21.15.69:0/188659605 wait complete. 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.583+0000 7f64a778c740 1 -- start start 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.583+0000 7f64a778c740 1 Processor -- start 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.583+0000 7f64a778c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 0x559b33f2c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.583+0000 7f64a778c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 0x559b33f296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.583+0000 7f64a778c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f2cad0 0x559b33f29c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.583+0000 7f649ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f2cad0 0x559b33f29c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.583+0000 7f6497fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 0x559b33f296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.583+0000 7f6497fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 0x559b33f296f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:59340/0 (socket says 172.21.15.69:59340) 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.583+0000 7f6497fff640 1 -- 172.21.15.69:0/2239263215 learned_addr learned my addr 172.21.15.69:0/2239263215 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.584+0000 7f649e59c640 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 0x559b33f2c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.584+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559b33b6a790 con 0x559b33f2cad0 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.584+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559b33e40890 con 0x559b33f218f0 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.584+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559b33f33bb0 con 0x559b33f22210 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.584+0000 7f64a778c740 10 client.0 _ll_register_callbacks cb 0x559b33e43d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f649ed9d640 1 -- 172.21.15.69:0/2239263215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 msgr2=0x559b33f296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f649ed9d640 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 0x559b33f296f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f649ed9d640 1 -- 172.21.15.69:0/2239263215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 msgr2=0x559b33f2c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f649ed9d640 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 0x559b33f2c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f649ed9d640 1 -- 172.21.15.69:0/2239263215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b33c65620 con 0x559b33f2cad0 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f649ed9d640 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f2cad0 0x559b33f29c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6498013000 tx=0x7f6498004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7faf19f38339fd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f6485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f6497fff640 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 0x559b33f296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f6485ffb640 1 -- 172.21.15.69:0/2239263215 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498025070 con 0x559b33f2cad0 2024-05-14T16:50:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64900035f0 con 0x559b33f2cad0 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.585+0000 7f6485ffb640 1 -- 172.21.15.69:0/2239263215 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f6498004b20 con 0x559b33f2cad0 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f6485ffb640 1 -- 172.21.15.69:0/2239263215 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6498004320 con 0x559b33f2cad0 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f64a778c740 10 client.15399 fetch_fsmap learned FSMap version 26 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=26}) -- 0x559b33f3f5a0 con 0x559b33f2cad0 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f6485ffb640 1 -- 172.21.15.69:0/2239263215 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 26) ==== 8342+0+0 (secure 0 0 0) 0x7f649801e040 con 0x559b33f2cad0 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f64a778c740 10 client.15399 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f64a778c740 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f64a778c740 20 client.15399 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f64a778c740 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b33f40950 0x559b33f42d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f64a778c740 4 client.15399 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b33e96b80 con 0x559b33f40950 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.586+0000 7f649e59c640 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b33f40950 0x559b33f42d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.587+0000 7f649e59c640 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b33f40950 0x559b33f42d60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222f055ff1581277 server_cookie=2bbd46c07feba54f in_seq=0 out_seq=0 2024-05-14T16:50:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.587+0000 7f6485ffb640 10 client.15399 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.587+0000 7f6485ffb640 1 -- 172.21.15.69:0/2239263215 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b33e96b80 con 0x559b33f40950 2024-05-14T16:50:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.587+0000 7f6485ffb640 10 client.15399 handle_command_reply: tid=0 2024-05-14T16:50:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.587+0000 7f64a778c740 1 client.15399 shutdown 2024-05-14T16:50:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.587+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f2cad0 msgr2=0x559b33f29c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.587+0000 7f64a778c740 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f2cad0 0x559b33f29c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6498013000 tx=0x7f6498004a40 comp rx=0 tx=0).stop 2024-05-14T16:50:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.587+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 shutdown_connections 2024-05-14T16:50:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.587+0000 7f64a778c740 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b33f40950 0x559b33f42d60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.588+0000 7f64a778c740 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f2cad0 0x559b33f29c30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.588+0000 7f6485ffb640 0 client.15399 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.588+0000 7f64a778c740 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 0x559b33f296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.588+0000 7f64a778c740 1 --2- 172.21.15.69:0/2239263215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f218f0 0x559b33f2c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.588+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 >> 172.21.15.69:0/2239263215 conn(0x559b33f1de00 msgr2=0x559b33f1d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.588+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 shutdown_connections 2024-05-14T16:50:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.588+0000 7f64a778c740 1 -- 172.21.15.69:0/2239263215 wait complete. 2024-05-14T16:50:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.588+0000 7f64a778c740 20 client.15399 trim_cache size 0 max 16384 2024-05-14T16:50:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.609+0000 7f64a778c740 1 Processor -- start 2024-05-14T16:50:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.609+0000 7f64a778c740 1 -- start start 2024-05-14T16:50:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f64a778c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f3ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f64a778c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 0x559b33f48670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f64a778c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f2cad0 0x559b33f48bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f64a778c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01aec80 con 0x559b33f218f0 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f64a778c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01ac550 con 0x559b33f2cad0 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f64a778c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01aa3a0 con 0x559b33f22210 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f3ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f2cad0 0x559b33f48bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f2cad0 0x559b33f48bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:46326/0 (socket says 172.21.15.69:46326) 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f3ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46312/0 (socket says 172.21.15.69:46312) 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649ed9d640 1 -- 172.21.15.69:0/1156098516 learned_addr learned my addr 172.21.15.69:0/1156098516 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f6497fff640 1 --2- 172.21.15.69:0/1156098516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 0x559b33f48670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649ed9d640 1 -- 172.21.15.69:0/1156098516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 msgr2=0x559b33f48670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649ed9d640 1 --2- 172.21.15.69:0/1156098516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 0x559b33f48670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649ed9d640 1 -- 172.21.15.69:0/1156098516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 msgr2=0x559b33f3ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649ed9d640 1 --2- 172.21.15.69:0/1156098516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f3ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649ed9d640 1 -- 172.21.15.69:0/1156098516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b33f1dc30 con 0x559b33f2cad0 2024-05-14T16:50:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649e59c640 1 --2- 172.21.15.69:0/1156098516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f3ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f649ed9d640 1 --2- 172.21.15.69:0/1156098516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f2cad0 0x559b33f48bb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6498009240 tx=0x7f6498022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f60851d2604a2cda server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f6487fff640 1 -- 172.21.15.69:0/1156098516 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498025070 con 0x559b33f2cad0 2024-05-14T16:50:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f6487fff640 1 -- 172.21.15.69:0/1156098516 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64900035f0 con 0x559b33f2cad0 2024-05-14T16:50:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.610+0000 7f6487fff640 1 -- 172.21.15.69:0/1156098516 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64980109d0 con 0x559b33f2cad0 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.611+0000 7f64a778c740 1 -- 172.21.15.69:0/1156098516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f2cad0 msgr2=0x559b33f48bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.611+0000 7f64a778c740 1 --2- 172.21.15.69:0/1156098516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f2cad0 0x559b33f48bb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6498009240 tx=0x7f6498022000 comp rx=0 tx=0).stop 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.612+0000 7f64a778c740 1 -- 172.21.15.69:0/1156098516 shutdown_connections 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.612+0000 7f64a778c740 1 --2- 172.21.15.69:0/1156098516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f2cad0 0x559b33f48bb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.612+0000 7f64a778c740 1 --2- 172.21.15.69:0/1156098516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f22210 0x559b33f48670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.612+0000 7f64a778c740 1 --2- 172.21.15.69:0/1156098516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f3ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.612+0000 7f64a778c740 1 -- 172.21.15.69:0/1156098516 >> 172.21.15.69:0/1156098516 conn(0x559b33f1de00 msgr2=0x559b33f29810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.612+0000 7f64a778c740 1 -- 172.21.15.69:0/1156098516 shutdown_connections 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.612+0000 7f64a778c740 1 -- 172.21.15.69:0/1156098516 wait complete. 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f64a778c740 1 -- start start 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f64a778c740 1 Processor -- start 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f64a778c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f38d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f64a778c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f22210 0x559b33f39240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f64a778c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 0x559b33f36250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f64a778c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01aaa10 con 0x559b33f218f0 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f64a778c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01abe80 con 0x559b33f22210 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f64a778c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01ab0e0 con 0x559b33f2cad0 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f64a778c740 10 client.0 _ll_register_callbacks cb 0x7f64980129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f649e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f38d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f6497fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f22210 0x559b33f39240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f649ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 0x559b33f36250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f649e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f38d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46326/0 (socket says 172.21.15.69:46326) 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.613+0000 7f649e59c640 1 -- 172.21.15.69:0/186588834 learned_addr learned my addr 172.21.15.69:0/186588834 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f6497fff640 1 -- 172.21.15.69:0/186588834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 msgr2=0x559b33f36250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f6497fff640 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 0x559b33f36250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f6497fff640 1 -- 172.21.15.69:0/186588834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 msgr2=0x559b33f38d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f6497fff640 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f38d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f6497fff640 1 -- 172.21.15.69:0/186588834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b33f36940 con 0x559b33f22210 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f649ed9d640 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 0x559b33f36250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f649e59c640 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f38d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f6497fff640 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f22210 0x559b33f39240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64881001e0 tx=0x7f648803b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac1d741cd2f0a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f6485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f6485ffb640 1 -- 172.21.15.69:0/186588834 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488048040 con 0x559b33f22210 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f64a778c740 1 -- 172.21.15.69:0/186588834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64900035f0 con 0x559b33f22210 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.614+0000 7f6485ffb640 1 -- 172.21.15.69:0/186588834 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f648803b8d0 con 0x559b33f22210 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f6485ffb640 1 -- 172.21.15.69:0/186588834 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6488042730 con 0x559b33f22210 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f64a778c740 10 client.24986 fetch_fsmap learned FSMap version 26 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f64a778c740 1 -- 172.21.15.69:0/186588834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b33faadf0 con 0x559b33f22210 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f6485ffb640 1 -- 172.21.15.69:0/186588834 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64900035f0 con 0x559b33f22210 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f6485ffb640 1 -- 172.21.15.69:0/186588834 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8342+0+0 (secure 0 0 0) 0x7f64880e4070 con 0x559b33f22210 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f64a778c740 10 client.24986 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T16:50:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f64a778c740 10 client.24986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:50:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f64a778c740 20 client.24986 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f64a778c740 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b33f40950 0x559b33fad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f64a778c740 4 client.24986 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:50:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.615+0000 7f64a778c740 1 -- 172.21.15.69:0/186588834 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b33e90360 con 0x559b33f40950 2024-05-14T16:50:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.616+0000 7f649e59c640 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b33f40950 0x559b33fad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.616+0000 7f6485ffb640 10 client.24986 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.616+0000 7f649e59c640 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b33f40950 0x559b33fad630 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f30386290caf8e42 server_cookie=445a34c06b12cc35 in_seq=0 out_seq=0 2024-05-14T16:50:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f6485ffb640 1 -- 172.21.15.69:0/186588834 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559b33e90360 con 0x559b33f40950 2024-05-14T16:50:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f6485ffb640 10 client.24986 handle_command_reply: tid=0 2024-05-14T16:50:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 client.24986 shutdown 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 -- 172.21.15.69:0/186588834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f22210 msgr2=0x559b33f39240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f22210 0x559b33f39240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64881001e0 tx=0x7f648803b840 comp rx=0 tx=0).stop 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 -- 172.21.15.69:0/186588834 shutdown_connections 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b33f40950 0x559b33fad630 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b33f2cad0 0x559b33f36250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f6485ffb640 0 client.24986 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b33f22210 0x559b33f39240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 --2- 172.21.15.69:0/186588834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b33f218f0 0x559b33f38d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 -- 172.21.15.69:0/186588834 >> 172.21.15.69:0/186588834 conn(0x559b33f1de00 msgr2=0x559b33f3ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 -- 172.21.15.69:0/186588834 shutdown_connections 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 1 -- 172.21.15.69:0/186588834 wait complete. 2024-05-14T16:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.674+0000 7f64a778c740 20 client.24986 trim_cache size 0 max 16384 2024-05-14T16:50:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:50:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:50:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 -- 172.21.15.69:0/2123015728 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f648c0b2b30 msgr2=0x7f648c0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f648c0b2b30 0x7f648c0b4ff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6498009270 tx=0x7f64980096a0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 -- 172.21.15.69:0/2123015728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64a01033f0 msgr2=0x7f64a0199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64a01033f0 0x7f64a0199770 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6488004770 tx=0x7f64880042f0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 -- 172.21.15.69:0/2123015728 shutdown_connections 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f648c0b2b30 0x7f648c0b4ff0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64a0108a40 0x7f64a019daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64a01033f0 0x7f64a0199770 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 --2- 172.21.15.69:0/2123015728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a0102a30 0x7f64a0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 -- 172.21.15.69:0/2123015728 >> 172.21.15.69:0/2123015728 conn(0x7f64a00fe500 msgr2=0x7f64a00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 -- 172.21.15.69:0/2123015728 shutdown_connections 2024-05-14T16:50:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:36.675+0000 7f64a499d640 1 -- 172.21.15.69:0/2123015728 wait complete. 2024-05-14T16:50:36.959 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:36 smithi203 podman[70169]: 2024-05-14 16:50:36.474576874 +0000 UTC m=+0.150480649 container create b1b76cada8d68d563bc76b784912d9d52a347648c9e3ac16a997a4a24cbd23a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:50:37.283 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:36 smithi203 podman[70169]: 2024-05-14 16:50:36.962283305 +0000 UTC m=+0.638187104 container init b1b76cada8d68d563bc76b784912d9d52a347648c9e3ac16a997a4a24cbd23a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-14T16:50:37.284 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:36 smithi203 podman[70169]: 2024-05-14 16:50:36.966699524 +0000 UTC m=+0.642603318 container start b1b76cada8d68d563bc76b784912d9d52a347648c9e3ac16a997a4a24cbd23a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-14T16:50:37.284 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:36 smithi203 ceph-mds[70187]: -- 172.21.15.203:0/284952703 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557115741a40 con 0x557115714c00 2024-05-14T16:50:37.284 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:36 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: starting mds.c at 2024-05-14T16:50:37.709 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:37 smithi203 bash[70169]: b1b76cada8d68d563bc76b784912d9d52a347648c9e3ac16a997a4a24cbd23a5 2024-05-14T16:50:37.709 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:50:37 smithi203 systemd[1]: Started Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:50:38.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:37 smithi069 ceph-mon[31507]: pgmap v24: 97 pgs: 97 active+clean; 869 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 78 MiB/s wr, 508 op/s 2024-05-14T16:50:38.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:37 smithi069 ceph-mon[31507]: daemon mds.c restarted 2024-05-14T16:50:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:37 smithi203 ceph-mon[35787]: pgmap v24: 97 pgs: 97 active+clean; 869 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 78 MiB/s wr, 508 op/s 2024-05-14T16:50:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:37 smithi203 ceph-mon[35787]: daemon mds.c restarted 2024-05-14T16:50:38.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:37 smithi092 ceph-mon[35763]: pgmap v24: 97 pgs: 97 active+clean; 869 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 78 MiB/s wr, 508 op/s 2024-05-14T16:50:38.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:37 smithi092 ceph-mon[35763]: daemon mds.c restarted 2024-05-14T16:50:39.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:38 smithi069 ceph-mon[31507]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T16:50:39.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:38 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6836/2802264018,v1:172.21.15.203:6837/2802264018] up:boot 2024-05-14T16:50:39.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:38 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T16:50:39.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:38 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:50:39.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:38 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:50:39.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:38 smithi203 ceph-mon[35787]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T16:50:39.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:38 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6836/2802264018,v1:172.21.15.203:6837/2802264018] up:boot 2024-05-14T16:50:39.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:38 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T16:50:39.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:38 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:50:39.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:38 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:50:39.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:38 smithi092 ceph-mon[35763]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T16:50:39.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:38 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6836/2802264018,v1:172.21.15.203:6837/2802264018] up:boot 2024-05-14T16:50:39.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:38 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T16:50:39.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:38 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:50:39.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:38 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:50:40.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:39 smithi069 ceph-mon[31507]: pgmap v26: 97 pgs: 97 active+clean; 869 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 MiB/s wr, 294 op/s 2024-05-14T16:50:40.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:39 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:50:40.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:39 smithi203 ceph-mon[35787]: pgmap v26: 97 pgs: 97 active+clean; 869 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 MiB/s wr, 294 op/s 2024-05-14T16:50:40.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:39 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:50:40.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:39 smithi092 ceph-mon[35763]: pgmap v26: 97 pgs: 97 active+clean; 869 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 MiB/s wr, 294 op/s 2024-05-14T16:50:40.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:39 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:50:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 10.0 s, 3428.6 tps, lat 29.116 ms stddev 28.750 2024-05-14T16:50:42.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:41 smithi069 ceph-mon[31507]: pgmap v27: 97 pgs: 97 active+clean; 912 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 45 MiB/s wr, 662 op/s 2024-05-14T16:50:42.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:41 smithi203 ceph-mon[35787]: pgmap v27: 97 pgs: 97 active+clean; 912 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 45 MiB/s wr, 662 op/s 2024-05-14T16:50:42.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:41 smithi092 ceph-mon[35763]: pgmap v27: 97 pgs: 97 active+clean; 912 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 45 MiB/s wr, 662 op/s 2024-05-14T16:50:44.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:43 smithi069 ceph-mon[31507]: pgmap v28: 97 pgs: 97 active+clean; 926 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 10 MiB/s wr, 673 op/s 2024-05-14T16:50:44.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:43 smithi203 ceph-mon[35787]: pgmap v28: 97 pgs: 97 active+clean; 926 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 10 MiB/s wr, 673 op/s 2024-05-14T16:50:44.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:43 smithi092 ceph-mon[35763]: pgmap v28: 97 pgs: 97 active+clean; 926 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 10 MiB/s wr, 673 op/s 2024-05-14T16:50:44.838 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.2 2024-05-14T16:50:44.838 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:50:45.186 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:45.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 15.0 s, 3602.2 tps, lat 27.779 ms stddev 27.988 2024-05-14T16:50:46.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:45 smithi069 ceph-mon[31507]: pgmap v29: 97 pgs: 97 active+clean; 926 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 10 MiB/s wr, 673 op/s 2024-05-14T16:50:46.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:45 smithi203 ceph-mon[35787]: pgmap v29: 97 pgs: 97 active+clean; 926 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 10 MiB/s wr, 673 op/s 2024-05-14T16:50:46.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:45 smithi092 ceph-mon[35763]: pgmap v29: 97 pgs: 97 active+clean; 926 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 10 MiB/s wr, 673 op/s 2024-05-14T16:50:47.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.085+0000 7f10ae69e700 1 -- 172.21.15.69:0/219762688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a8109680 msgr2=0x7f10a8109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:47.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.085+0000 7f10ae69e700 1 --2- 172.21.15.69:0/219762688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a8109680 0x7f10a8109a60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f10980098f0 tx=0x7f1098009c00 comp rx=0 tx=0).stop 2024-05-14T16:50:47.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.085+0000 7f10ae69e700 1 -- 172.21.15.69:0/219762688 shutdown_connections 2024-05-14T16:50:47.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.085+0000 7f10ae69e700 1 --2- 172.21.15.69:0/219762688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10a8103fa0 0x7f10a8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:47.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.085+0000 7f10ae69e700 1 --2- 172.21.15.69:0/219762688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a8109680 0x7f10a8109a60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.085+0000 7f10ae69e700 1 --2- 172.21.15.69:0/219762688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a81035e0 0x7f10a8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.085+0000 7f10ae69e700 1 -- 172.21.15.69:0/219762688 >> 172.21.15.69:0/219762688 conn(0x7f10a807a6f0 msgr2=0x7f10a807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.086+0000 7f10ae69e700 1 -- 172.21.15.69:0/219762688 shutdown_connections 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.086+0000 7f10ae69e700 1 -- 172.21.15.69:0/219762688 wait complete. 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.086+0000 7f10ae69e700 1 Processor -- start 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.086+0000 7f10ae69e700 1 -- start start 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10ae69e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a81035e0 0x7f10a8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10ae69e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10a8103fa0 0x7f10a8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10ae69e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a8109680 0x7f10a80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10ae69e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8113820 con 0x7f10a8103fa0 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10ae69e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81136a0 con 0x7f10a81035e0 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10ae69e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81139a0 con 0x7f10a8109680 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10acc3b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a8109680 0x7f10a80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:47.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10a77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10a8103fa0 0x7f10a8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10acc3b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a8109680 0x7f10a80762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:50946/0 (socket says 172.21.15.69:50946) 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10acc3b700 1 -- 172.21.15.69:0/298617556 learned_addr learned my addr 172.21.15.69:0/298617556 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10a7fff700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a81035e0 0x7f10a8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10acc3b700 1 -- 172.21.15.69:0/298617556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a81035e0 msgr2=0x7f10a8075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10acc3b700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a81035e0 0x7f10a8075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10acc3b700 1 -- 172.21.15.69:0/298617556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10a8103fa0 msgr2=0x7f10a8075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10acc3b700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10a8103fa0 0x7f10a8075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10acc3b700 1 -- 172.21.15.69:0/298617556 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a81a81c0 con 0x7f10a8109680 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10a77fe700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10a8103fa0 0x7f10a8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10acc3b700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a8109680 0x7f10a80762c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10a0007f50 tx=0x7f10a000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:47.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.087+0000 7f10a57fa700 1 -- 172.21.15.69:0/298617556 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0004bb0 con 0x7f10a8109680 2024-05-14T16:50:47.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.088+0000 7f10a57fa700 1 -- 172.21.15.69:0/298617556 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a0004d50 con 0x7f10a8109680 2024-05-14T16:50:47.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.088+0000 7f10a57fa700 1 -- 172.21.15.69:0/298617556 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0005940 con 0x7f10a8109680 2024-05-14T16:50:47.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.088+0000 7f10ae69e700 1 -- 172.21.15.69:0/298617556 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a81a84b0 con 0x7f10a8109680 2024-05-14T16:50:47.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.088+0000 7f10ae69e700 1 -- 172.21.15.69:0/298617556 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a81a89a0 con 0x7f10a8109680 2024-05-14T16:50:47.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.090+0000 7f10ae69e700 1 -- 172.21.15.69:0/298617556 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a810bb10 con 0x7f10a8109680 2024-05-14T16:50:47.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.091+0000 7f10a57fa700 1 -- 172.21.15.69:0/298617556 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f10a001a020 con 0x7f10a8109680 2024-05-14T16:50:47.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.092+0000 7f10a57fa700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10900b2bd0 0x7f10900b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:47.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.092+0000 7f10a7fff700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10900b2bd0 0x7f10900b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:47.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.092+0000 7f10a57fa700 1 -- 172.21.15.69:0/298617556 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f10a00e1d70 con 0x7f10a8109680 2024-05-14T16:50:47.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.092+0000 7f10a7fff700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10900b2bd0 0x7f10900b5090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1098000c00 tx=0x7f1098019040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:47.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.097+0000 7f10a57fa700 1 -- 172.21.15.69:0/298617556 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10a0010070 con 0x7f10a8109680 2024-05-14T16:50:47.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.462+0000 7f10ae69e700 1 -- 172.21.15.69:0/298617556 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10a810b4f0 con 0x7f10a8109680 2024-05-14T16:50:47.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.463+0000 7f10a57fa700 1 -- 172.21.15.69:0/298617556 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f10a00a9510 con 0x7f10a8109680 2024-05-14T16:50:47.463 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:50:47.463 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:50:38.704223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24958":{"gid":24958,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/2802264018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2802264018},{"type":"v1","addr":"172.21.15.203:6837","nonce":2802264018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24446,24422,24433]},"id":1}]} 2024-05-14T16:50:47.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.466+0000 7f10ae69e700 1 -- 172.21.15.69:0/298617556 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10900b2bd0 msgr2=0x7f10900b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:47.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.466+0000 7f10ae69e700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10900b2bd0 0x7f10900b5090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1098000c00 tx=0x7f1098019040 comp rx=0 tx=0).stop 2024-05-14T16:50:47.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.466+0000 7f10ae69e700 1 -- 172.21.15.69:0/298617556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a8109680 msgr2=0x7f10a80762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.466+0000 7f10ae69e700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a8109680 0x7f10a80762c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10a0007f50 tx=0x7f10a000d3b0 comp rx=0 tx=0).stop 2024-05-14T16:50:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.467+0000 7f10ae69e700 1 -- 172.21.15.69:0/298617556 shutdown_connections 2024-05-14T16:50:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.467+0000 7f10ae69e700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a8109680 0x7f10a80762c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.467+0000 7f10ae69e700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a81035e0 0x7f10a8075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.467+0000 7f10ae69e700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10900b2bd0 0x7f10900b5090 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.467+0000 7f10ae69e700 1 --2- 172.21.15.69:0/298617556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10a8103fa0 0x7f10a8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.467+0000 7f10ae69e700 1 -- 172.21.15.69:0/298617556 >> 172.21.15.69:0/298617556 conn(0x7f10a807a6f0 msgr2=0x7f10a8102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.467+0000 7f10ae69e700 1 -- 172.21.15.69:0/298617556 shutdown_connections 2024-05-14T16:50:47.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:47.467+0000 7f10ae69e700 1 -- 172.21.15.69:0/298617556 wait complete. 2024-05-14T16:50:47.469 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 29 2024-05-14T16:50:48.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:47 smithi069 ceph-mon[31507]: pgmap v30: 97 pgs: 97 active+clean; 932 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.3 MiB/s wr, 801 op/s 2024-05-14T16:50:48.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:47 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/298617556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:48.149 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 135.2 --await-for 45.0' 2024-05-14T16:50:48.149 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 135.2 --await-for 45.0 2024-05-14T16:50:48.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:47 smithi203 ceph-mon[35787]: pgmap v30: 97 pgs: 97 active+clean; 932 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.3 MiB/s wr, 801 op/s 2024-05-14T16:50:48.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:47 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/298617556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:48.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:47 smithi092 ceph-mon[35763]: pgmap v30: 97 pgs: 97 active+clean; 932 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.3 MiB/s wr, 801 op/s 2024-05-14T16:50:48.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:47 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/298617556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:48.486 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.805+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/1108524286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4810a410 msgr2=0x7f9f48116360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.805+0000 7f9f4d5f0640 1 --2- 172.21.15.69:0/1108524286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4810a410 0x7f9f48116360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b450 tx=0x7f9f4002aad0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.806+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/1108524286 shutdown_connections 2024-05-14T16:50:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.806+0000 7f9f4d5f0640 1 --2- 172.21.15.69:0/1108524286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4810a410 0x7f9f48116360 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.806+0000 7f9f4d5f0640 1 --2- 172.21.15.69:0/1108524286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f48109a50 0x7f9f48109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.806+0000 7f9f4d5f0640 1 --2- 172.21.15.69:0/1108524286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f4810f5e0 0x7f9f4810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.806+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/1108524286 >> 172.21.15.69:0/1108524286 conn(0x7f9f4806dfb0 msgr2=0x7f9f4806e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.807+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/1108524286 shutdown_connections 2024-05-14T16:50:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.807+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/1108524286 wait complete. 2024-05-14T16:50:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.807+0000 7f9f4d5f0640 1 Processor -- start 2024-05-14T16:50:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f4d5f0640 1 -- start start 2024-05-14T16:50:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f4d5f0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48109a50 0x7f9f481a2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f4d5f0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f4810a410 0x7f9f481a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f4d5f0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f4810f5e0 0x7f9f481ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f4d5f0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48119300 con 0x7f9f4810f5e0 2024-05-14T16:50:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f4d5f0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48119180 con 0x7f9f48109a50 2024-05-14T16:50:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f4d5f0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48119480 con 0x7f9f4810a410 2024-05-14T16:50:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f4810f5e0 0x7f9f481ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f467fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f4810a410 0x7f9f481a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f4810f5e0 0x7f9f481ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56068/0 (socket says 172.21.15.69:56068) 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f477fe640 1 -- 172.21.15.69:0/3186009458 learned_addr learned my addr 172.21.15.69:0/3186009458 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48109a50 0x7f9f481a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f467fc640 1 -- 172.21.15.69:0/3186009458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48109a50 msgr2=0x7f9f481a2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f467fc640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48109a50 0x7f9f481a2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f467fc640 1 -- 172.21.15.69:0/3186009458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f4810f5e0 msgr2=0x7f9f481ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f467fc640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f4810f5e0 0x7f9f481ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f467fc640 1 -- 172.21.15.69:0/3186009458 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f481add10 con 0x7f9f4810a410 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.808+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48109a50 0x7f9f481a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.809+0000 7f9f467fc640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f4810a410 0x7f9f481a2c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00e9f0 tx=0x7f9f3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.809+0000 7f9f2ffff640 1 -- 172.21.15.69:0/3186009458 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c00ccc0 con 0x7f9f4810a410 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.809+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/3186009458 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f481adee0 con 0x7f9f4810a410 2024-05-14T16:50:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.809+0000 7f9f2ffff640 1 -- 172.21.15.69:0/3186009458 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f3c005020 con 0x7f9f4810a410 2024-05-14T16:50:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.809+0000 7f9f2ffff640 1 -- 172.21.15.69:0/3186009458 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c004a00 con 0x7f9f4810a410 2024-05-14T16:50:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.809+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/3186009458 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f481ae3a0 con 0x7f9f4810a410 2024-05-14T16:50:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f2ffff640 1 -- 172.21.15.69:0/3186009458 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f9f3c004ba0 con 0x7f9f4810a410 2024-05-14T16:50:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f503e9740 1 Processor -- start 2024-05-14T16:50:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f503e9740 1 -- start start 2024-05-14T16:50:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed58f0 0x558033ed5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed6210 0x558033ee0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ee0ad0 0x558033ee2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f503e9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558033e48c30 con 0x558033ed58f0 2024-05-14T16:50:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f503e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558033e387d0 con 0x558033ee0ad0 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f503e9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558033e3c810 con 0x558033ed6210 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed58f0 0x558033ed5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed58f0 0x558033ed5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56076/0 (socket says 172.21.15.69:56076) 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f46ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed6210 0x558033ee0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f477fe640 1 -- 172.21.15.69:0/1212486280 learned_addr learned my addr 172.21.15.69:0/1212486280 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f467fc640 1 --2- 172.21.15.69:0/1212486280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ee0ad0 0x558033ee2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f46ffd640 1 -- 172.21.15.69:0/1212486280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ee0ad0 msgr2=0x558033ee2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/1212486280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ee0ad0 0x558033ee2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f46ffd640 1 -- 172.21.15.69:0/1212486280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed58f0 msgr2=0x558033ed5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/1212486280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed58f0 0x558033ed5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f46ffd640 1 -- 172.21.15.69:0/1212486280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558033b4d210 con 0x558033ed6210 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.813+0000 7f9f467fc640 1 --2- 172.21.15.69:0/1212486280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ee0ad0 0x558033ee2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.814+0000 7f9f2ffff640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9f300b2b30 0x7f9f300b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.814+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/1212486280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed6210 0x558033ee0590 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9f38005fd0 tx=0x7f9f380107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75dcccf18eb49373 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.814+0000 7f9f477fe640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9f300b2b30 0x7f9f300b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.814+0000 7f9f2ffff640 1 -- 172.21.15.69:0/3186009458 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f9f3c0e28d0 con 0x7f9f4810a410 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.814+0000 7f9f2d7fa640 1 -- 172.21.15.69:0/1212486280 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38004020 con 0x558033ed6210 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.814+0000 7f9f2d7fa640 1 -- 172.21.15.69:0/1212486280 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f380041c0 con 0x558033ed6210 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.814+0000 7f9f2d7fa640 1 -- 172.21.15.69:0/1212486280 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f380116a0 con 0x558033ed6210 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.815+0000 7f9f477fe640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9f300b2b30 0x7f9f300b4ff0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b290 tx=0x7f9f40039040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.815+0000 7f9f503e9740 1 -- 172.21.15.69:0/1212486280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed6210 msgr2=0x558033ee0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.815+0000 7f9f503e9740 1 --2- 172.21.15.69:0/1212486280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed6210 0x558033ee0590 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9f38005fd0 tx=0x7f9f380107a0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.815+0000 7f9f503e9740 1 -- 172.21.15.69:0/1212486280 shutdown_connections 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.815+0000 7f9f503e9740 1 --2- 172.21.15.69:0/1212486280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ee0ad0 0x558033ee2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.815+0000 7f9f503e9740 1 --2- 172.21.15.69:0/1212486280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed6210 0x558033ee0590 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.815+0000 7f9f503e9740 1 --2- 172.21.15.69:0/1212486280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed58f0 0x558033ed5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.815+0000 7f9f503e9740 1 -- 172.21.15.69:0/1212486280 >> 172.21.15.69:0/1212486280 conn(0x558033ed1e00 msgr2=0x558033ed21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.816+0000 7f9f503e9740 1 -- 172.21.15.69:0/1212486280 shutdown_connections 2024-05-14T16:50:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.816+0000 7f9f503e9740 1 -- 172.21.15.69:0/1212486280 wait complete. 2024-05-14T16:50:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.817+0000 7f9f503e9740 1 -- start start 2024-05-14T16:50:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.817+0000 7f9f503e9740 1 Processor -- start 2024-05-14T16:50:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed58f0 0x558033ee0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ee0ad0 0x558033eddc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f503e9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558033b1e790 con 0x558033ed6210 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f503e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558033df4890 con 0x558033ed58f0 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f503e9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558033ee37a0 con 0x558033ee0ad0 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f467fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f503e9740 10 client.0 _ll_register_callbacks cb 0x558033df7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ee0ad0 0x558033eddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f467fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56084/0 (socket says 172.21.15.69:56084) 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f467fc640 1 -- 172.21.15.69:0/3651589902 learned_addr learned my addr 172.21.15.69:0/3651589902 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed58f0 0x558033ee0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f467fc640 1 -- 172.21.15.69:0/3651589902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ee0ad0 msgr2=0x558033eddc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f467fc640 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ee0ad0 0x558033eddc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f467fc640 1 -- 172.21.15.69:0/3651589902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed58f0 msgr2=0x558033ee0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f467fc640 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed58f0 0x558033ee0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f467fc640 1 -- 172.21.15.69:0/3651589902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558033c19620 con 0x558033ed6210 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f477fe640 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ee0ad0 0x558033eddc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f467fc640 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edd6f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0a6be0 tx=0x7f9f3c0a9740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5787421d8426349 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.818+0000 7f9f2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.819+0000 7f9f2aff5640 1 -- 172.21.15.69:0/3651589902 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c01c070 con 0x558033ed6210 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.819+0000 7f9f2aff5640 1 -- 172.21.15.69:0/3651589902 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f9f3c0a9bd0 con 0x558033ed6210 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.819+0000 7f9f503e9740 1 -- 172.21.15.69:0/3651589902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f380041c0 con 0x558033ed6210 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.819+0000 7f9f2aff5640 1 -- 172.21.15.69:0/3651589902 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f3c0a5100 con 0x558033ed6210 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.819+0000 7f9f503e9740 10 client.15429 fetch_fsmap learned FSMap version 29 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.819+0000 7f9f503e9740 1 -- 172.21.15.69:0/3651589902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558033ef79b0 con 0x558033ed6210 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.819+0000 7f9f2aff5640 1 -- 172.21.15.69:0/3651589902 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f380041c0 con 0x558033ed6210 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.819+0000 7f9f2aff5640 1 -- 172.21.15.69:0/3651589902 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f9f3c0b2340 con 0x558033ed6210 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.820+0000 7f9f503e9740 10 client.15429 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.820+0000 7f9f503e9740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:50:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.820+0000 7f9f503e9740 20 client.15429 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.820+0000 7f9f503e9740 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558033ef8e70 0x558033efb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.820+0000 7f9f503e9740 4 client.15429 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.820+0000 7f9f503e9740 1 -- 172.21.15.69:0/3651589902 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558033e4ab80 con 0x558033ef8e70 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.820+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558033ef8e70 0x558033efb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.821+0000 7f9f2aff5640 10 client.15429 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.821+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558033ef8e70 0x558033efb2a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28f2e0021f0b478 server_cookie=363ccd46ca9f6675 in_seq=0 out_seq=0 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f2aff5640 1 -- 172.21.15.69:0/3651589902 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558033e4ab80 con 0x558033ef8e70 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f2aff5640 10 client.15429 handle_command_reply: tid=0 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 client.15429 shutdown 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 -- 172.21.15.69:0/3651589902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 msgr2=0x558033edd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edd6f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0a6be0 tx=0x7f9f3c0a9740 comp rx=0 tx=0).stop 2024-05-14T16:50:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 -- 172.21.15.69:0/3651589902 shutdown_connections 2024-05-14T16:50:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558033ef8e70 0x558033efb2a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ee0ad0 0x558033eddc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edd6f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 --2- 172.21.15.69:0/3651589902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed58f0 0x558033ee0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 -- 172.21.15.69:0/3651589902 >> 172.21.15.69:0/3651589902 conn(0x558033ed1e00 msgr2=0x558033ed1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f2aff5640 0 client.15429 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 -- 172.21.15.69:0/3651589902 shutdown_connections 2024-05-14T16:50:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 1 -- 172.21.15.69:0/3651589902 wait complete. 2024-05-14T16:50:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.822+0000 7f9f503e9740 20 client.15429 trim_cache size 0 max 16384 2024-05-14T16:50:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.852+0000 7f9f503e9740 1 Processor -- start 2024-05-14T16:50:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.852+0000 7f9f503e9740 1 -- start start 2024-05-14T16:50:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.852+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed1e00 0x558033edd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.852+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed58f0 0x558033ef4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.852+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.853+0000 7f9f503e9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b7550 con 0x558033ed6210 2024-05-14T16:50:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.853+0000 7f9f503e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b4d90 con 0x558033ed1e00 2024-05-14T16:50:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.853+0000 7f9f503e9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b2be0 con 0x558033ed58f0 2024-05-14T16:50:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.853+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.853+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56094/0 (socket says 172.21.15.69:56094) 2024-05-14T16:50:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.853+0000 7f9f477fe640 1 -- 172.21.15.69:0/243054913 learned_addr learned my addr 172.21.15.69:0/243054913 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.854+0000 7f9f477fe640 1 -- 172.21.15.69:0/243054913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed58f0 msgr2=0x558033ef4ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.854+0000 7f9f477fe640 1 --2- 172.21.15.69:0/243054913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed58f0 0x558033ef4ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.854+0000 7f9f477fe640 1 -- 172.21.15.69:0/243054913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed1e00 msgr2=0x558033edd6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.854+0000 7f9f477fe640 1 --2- 172.21.15.69:0/243054913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed1e00 0x558033edd6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.854+0000 7f9f477fe640 1 -- 172.21.15.69:0/243054913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558033ed1c30 con 0x558033ed6210 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.854+0000 7f9f477fe640 1 --2- 172.21.15.69:0/243054913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edff30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009f80 tx=0x7f9f4003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bb6acc27d637c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.857+0000 7f9f2cff9640 1 -- 172.21.15.69:0/243054913 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40049040 con 0x558033ed6210 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.857+0000 7f9f2cff9640 1 -- 172.21.15.69:0/243054913 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f40054070 con 0x558033ed6210 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.857+0000 7f9f2cff9640 1 -- 172.21.15.69:0/243054913 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40032040 con 0x558033ed6210 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.858+0000 7f9f503e9740 1 -- 172.21.15.69:0/243054913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 msgr2=0x558033edff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.858+0000 7f9f503e9740 1 --2- 172.21.15.69:0/243054913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edff30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009f80 tx=0x7f9f4003f000 comp rx=0 tx=0).stop 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.858+0000 7f9f503e9740 1 -- 172.21.15.69:0/243054913 shutdown_connections 2024-05-14T16:50:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.858+0000 7f9f503e9740 1 --2- 172.21.15.69:0/243054913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033edff30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.858+0000 7f9f503e9740 1 --2- 172.21.15.69:0/243054913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed58f0 0x558033ef4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.858+0000 7f9f503e9740 1 --2- 172.21.15.69:0/243054913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed1e00 0x558033edd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.858+0000 7f9f503e9740 1 -- 172.21.15.69:0/243054913 >> 172.21.15.69:0/243054913 conn(0x558033eddd90 msgr2=0x558033ed7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.860+0000 7f9f503e9740 1 -- 172.21.15.69:0/243054913 shutdown_connections 2024-05-14T16:50:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.860+0000 7f9f503e9740 1 -- 172.21.15.69:0/243054913 wait complete. 2024-05-14T16:50:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.861+0000 7f9f503e9740 1 -- start start 2024-05-14T16:50:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.861+0000 7f9f503e9740 1 Processor -- start 2024-05-14T16:50:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.861+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed1e00 0x558033ee9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.861+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed58f0 0x558033ee9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.862+0000 7f9f503e9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033ef0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.862+0000 7f9f503e9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b3250 con 0x558033ed6210 2024-05-14T16:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.862+0000 7f9f503e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b46c0 con 0x558033ed58f0 2024-05-14T16:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.862+0000 7f9f503e9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b3920 con 0x558033ed1e00 2024-05-14T16:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.862+0000 7f9f503e9740 10 client.0 _ll_register_callbacks cb 0x7f9f3c0a5830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.862+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033ef0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.862+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033ef0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56096/0 (socket says 172.21.15.69:56096) 2024-05-14T16:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.862+0000 7f9f477fe640 1 -- 172.21.15.69:0/2871701317 learned_addr learned my addr 172.21.15.69:0/2871701317 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.863+0000 7f9f467fc640 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed58f0 0x558033ee9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.863+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed1e00 0x558033ee9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.863+0000 7f9f477fe640 1 -- 172.21.15.69:0/2871701317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed1e00 msgr2=0x558033ee9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.863+0000 7f9f477fe640 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed1e00 0x558033ee9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.863+0000 7f9f477fe640 1 -- 172.21.15.69:0/2871701317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed58f0 msgr2=0x558033ee9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.863+0000 7f9f477fe640 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed58f0 0x558033ee9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.863+0000 7f9f477fe640 1 -- 172.21.15.69:0/2871701317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558033ef1770 con 0x558033ed6210 2024-05-14T16:50:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.864+0000 7f9f477fe640 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033ef0f40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9f4004ff60 tx=0x7f9f40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65ada80f41698a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.864+0000 7f9f2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:50:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.865+0000 7f9f2aff5640 1 -- 172.21.15.69:0/2871701317 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4005e070 con 0x558033ed6210 2024-05-14T16:50:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.865+0000 7f9f503e9740 1 -- 172.21.15.69:0/2871701317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f40054070 con 0x558033ed6210 2024-05-14T16:50:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.865+0000 7f9f2aff5640 1 -- 172.21.15.69:0/2871701317 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f9f40003b70 con 0x558033ed6210 2024-05-14T16:50:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.865+0000 7f9f2aff5640 1 -- 172.21.15.69:0/2871701317 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f40069050 con 0x558033ed6210 2024-05-14T16:50:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.866+0000 7f9f503e9740 10 client.15435 fetch_fsmap learned FSMap version 29 2024-05-14T16:50:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.866+0000 7f9f503e9740 1 -- 172.21.15.69:0/2871701317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558033ee5240 con 0x558033ed6210 2024-05-14T16:50:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.866+0000 7f9f2aff5640 1 -- 172.21.15.69:0/2871701317 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f40054070 con 0x558033ed6210 2024-05-14T16:50:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.866+0000 7f9f2aff5640 1 -- 172.21.15.69:0/2871701317 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f9f4005a860 con 0x558033ed6210 2024-05-14T16:50:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.866+0000 7f9f503e9740 10 client.15435 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T16:50:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.866+0000 7f9f503e9740 10 client.15435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:50:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.866+0000 7f9f503e9740 20 client.15435 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.866+0000 7f9f503e9740 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558033ee0ad0 0x558033f6eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.866+0000 7f9f503e9740 4 client.15435 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:50:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.866+0000 7f9f503e9740 1 -- 172.21.15.69:0/2871701317 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558033e44360 con 0x558033ee0ad0 2024-05-14T16:50:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.867+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558033ee0ad0 0x558033f6eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.868+0000 7f9f46ffd640 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558033ee0ad0 0x558033f6eaa0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f5e5c84fab6127 server_cookie=94c14a69e8f76d1 in_seq=0 out_seq=0 2024-05-14T16:50:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.868+0000 7f9f2aff5640 10 client.15435 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f2aff5640 1 -- 172.21.15.69:0/2871701317 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558033e44360 con 0x558033ee0ad0 2024-05-14T16:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f2aff5640 10 client.15435 handle_command_reply: tid=0 2024-05-14T16:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 client.15435 shutdown 2024-05-14T16:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 -- 172.21.15.69:0/2871701317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 msgr2=0x558033ef0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033ef0f40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9f4004ff60 tx=0x7f9f40004a40 comp rx=0 tx=0).stop 2024-05-14T16:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 -- 172.21.15.69:0/2871701317 shutdown_connections 2024-05-14T16:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558033ee0ad0 0x558033f6eaa0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558033ed6210 0x558033ef0f40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558033ed58f0 0x558033ee9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 --2- 172.21.15.69:0/2871701317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558033ed1e00 0x558033ee9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 -- 172.21.15.69:0/2871701317 >> 172.21.15.69:0/2871701317 conn(0x558033eddd90 msgr2=0x558033ed8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f2aff5640 0 client.15435 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 -- 172.21.15.69:0/2871701317 shutdown_connections 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 1 -- 172.21.15.69:0/2871701317 wait complete. 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.880+0000 7f9f503e9740 20 client.15435 trim_cache size 0 max 16384 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:50:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:50:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.881+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/3186009458 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9f300b2b30 msgr2=0x7f9f300b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.881+0000 7f9f4d5f0640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9f300b2b30 0x7f9f300b4ff0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b290 tx=0x7f9f40039040 comp rx=0 tx=0).stop 2024-05-14T16:50:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.881+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/3186009458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f4810a410 msgr2=0x7f9f481a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.881+0000 7f9f4d5f0640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f4810a410 0x7f9f481a2c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00e9f0 tx=0x7f9f3c00eeb0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.882+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/3186009458 shutdown_connections 2024-05-14T16:50:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.882+0000 7f9f4d5f0640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9f300b2b30 0x7f9f300b4ff0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.882+0000 7f9f4d5f0640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f4810f5e0 0x7f9f481ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.882+0000 7f9f4d5f0640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f4810a410 0x7f9f481a2c90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.882+0000 7f9f4d5f0640 1 --2- 172.21.15.69:0/3186009458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48109a50 0x7f9f481a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.882+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/3186009458 >> 172.21.15.69:0/3186009458 conn(0x7f9f4806dfb0 msgr2=0x7f9f4810b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.882+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/3186009458 shutdown_connections 2024-05-14T16:50:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:50:49.882+0000 7f9f4d5f0640 1 -- 172.21.15.69:0/3186009458 wait complete. 2024-05-14T16:50:50.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:49 smithi069 ceph-mon[31507]: pgmap v31: 97 pgs: 97 active+clean; 932 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.7 MiB/s wr, 748 op/s 2024-05-14T16:50:50.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:49 smithi203 ceph-mon[35787]: pgmap v31: 97 pgs: 97 active+clean; 932 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.7 MiB/s wr, 748 op/s 2024-05-14T16:50:50.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:49 smithi092 ceph-mon[35763]: pgmap v31: 97 pgs: 97 active+clean; 932 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.7 MiB/s wr, 748 op/s 2024-05-14T16:50:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 20.0 s, 3936.8 tps, lat 25.383 ms stddev 26.081 2024-05-14T16:50:50.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.842+0000 7fac6f14d700 1 -- 172.21.15.69:0/1828862872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac68102740 msgr2=0x7fac68113a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:50.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.842+0000 7fac6f14d700 1 --2- 172.21.15.69:0/1828862872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac68102740 0x7fac68113a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac540098f0 tx=0x7fac54009c00 comp rx=0 tx=0).stop 2024-05-14T16:50:50.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.842+0000 7fac6f14d700 1 -- 172.21.15.69:0/1828862872 shutdown_connections 2024-05-14T16:50:50.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.842+0000 7fac6f14d700 1 --2- 172.21.15.69:0/1828862872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac68114140 0x7fac681165e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.842+0000 7fac6f14d700 1 --2- 172.21.15.69:0/1828862872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac68101d90 0x7fac68102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.842+0000 7fac6f14d700 1 --2- 172.21.15.69:0/1828862872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac68102740 0x7fac68113a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.842+0000 7fac6f14d700 1 -- 172.21.15.69:0/1828862872 >> 172.21.15.69:0/1828862872 conn(0x7fac68078ae0 msgr2=0x7fac680ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:50.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.842+0000 7fac6f14d700 1 -- 172.21.15.69:0/1828862872 shutdown_connections 2024-05-14T16:50:50.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.842+0000 7fac6f14d700 1 -- 172.21.15.69:0/1828862872 wait complete. 2024-05-14T16:50:50.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.843+0000 7fac6f14d700 1 Processor -- start 2024-05-14T16:50:50.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.844+0000 7fac6f14d700 1 -- start start 2024-05-14T16:50:50.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac6f14d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac68101d90 0x7fac681115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac6f14d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac68102740 0x7fac6810a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac6f14d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac68114140 0x7fac6810abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac6f14d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac68119d90 con 0x7fac68101d90 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac6f14d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fac68119c10 con 0x7fac68114140 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac6f14d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fac68119f10 con 0x7fac68102740 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac6cee9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac68101d90 0x7fac681115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac67fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac68102740 0x7fac6810a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac67fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac68102740 0x7fac6810a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:51010/0 (socket says 172.21.15.69:51010) 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac6cee9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac68101d90 0x7fac681115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56120/0 (socket says 172.21.15.69:56120) 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac67fff700 1 -- 172.21.15.69:0/2869972887 learned_addr learned my addr 172.21.15.69:0/2869972887 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac6d6ea700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac68114140 0x7fac6810abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac67fff700 1 -- 172.21.15.69:0/2869972887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac68114140 msgr2=0x7fac6810abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac67fff700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac68114140 0x7fac6810abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac67fff700 1 -- 172.21.15.69:0/2869972887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac68101d90 msgr2=0x7fac681115e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac67fff700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac68101d90 0x7fac681115e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.845+0000 7fac67fff700 1 -- 172.21.15.69:0/2869972887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6810b220 con 0x7fac68102740 2024-05-14T16:50:50.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.847+0000 7fac67fff700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac68102740 0x7fac6810a670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c010 tx=0x7fac5400fa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:50.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.847+0000 7fac65ffb700 1 -- 172.21.15.69:0/2869972887 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54027070 con 0x7fac68102740 2024-05-14T16:50:50.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.847+0000 7fac65ffb700 1 -- 172.21.15.69:0/2869972887 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5400b560 con 0x7fac68102740 2024-05-14T16:50:50.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.847+0000 7fac65ffb700 1 -- 172.21.15.69:0/2869972887 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac540049a0 con 0x7fac68102740 2024-05-14T16:50:50.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.847+0000 7fac6f14d700 1 -- 172.21.15.69:0/2869972887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6810b4b0 con 0x7fac68102740 2024-05-14T16:50:50.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.847+0000 7fac6f14d700 1 -- 172.21.15.69:0/2869972887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6810f340 con 0x7fac68102740 2024-05-14T16:50:50.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.852+0000 7fac65ffb700 1 -- 172.21.15.69:0/2869972887 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fac5402a070 con 0x7fac68102740 2024-05-14T16:50:50.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.852+0000 7fac799e3b80 1 Processor -- start 2024-05-14T16:50:50.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.853+0000 7fac799e3b80 1 -- start start 2024-05-14T16:50:50.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.853+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.853+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd64540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.853+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 0x55a3bfd64fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.853+0000 7fac799e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfcc2e20 con 0x55a3bfd69e30 2024-05-14T16:50:50.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.853+0000 7fac799e3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfcb29c0 con 0x55a3bfd64b60 2024-05-14T16:50:50.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.853+0000 7fac799e3b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfcb6a00 con 0x55a3bfd6a750 2024-05-14T16:50:50.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.854+0000 7fac6d6ea700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.854+0000 7fac6d6ea700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56126/0 (socket says 172.21.15.69:56126) 2024-05-14T16:50:50.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.854+0000 7fac6d6ea700 1 -- 172.21.15.69:0/4097227570 learned_addr learned my addr 172.21.15.69:0/4097227570 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:50.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.854+0000 7fac6cee9700 1 --2- 172.21.15.69:0/4097227570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd64540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.854+0000 7fac67fff700 1 --2- 172.21.15.69:0/4097227570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 0x55a3bfd64fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.854+0000 7fac6d6ea700 1 -- 172.21.15.69:0/4097227570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 msgr2=0x55a3bfd64540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:50.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac6d6ea700 1 --2- 172.21.15.69:0/4097227570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd64540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac6d6ea700 1 -- 172.21.15.69:0/4097227570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 msgr2=0x55a3bfd64fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:50.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac6d6ea700 1 --2- 172.21.15.69:0/4097227570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 0x55a3bfd64fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac6d6ea700 1 -- 172.21.15.69:0/4097227570 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3bfc02a60 con 0x55a3bfd69e30 2024-05-14T16:50:50.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac6cee9700 1 --2- 172.21.15.69:0/4097227570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd64540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:50:50.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac65ffb700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fac580b2c60 0x7fac580b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac6d6ea700 1 --2- 172.21.15.69:0/4097227570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6a210 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fac60009370 tx=0x7fac600099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f8b26835329466 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:50.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac65ffb700 1 -- 172.21.15.69:0/2869972887 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7fac540e32f0 con 0x7fac68102740 2024-05-14T16:50:50.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac6cee9700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fac580b2c60 0x7fac580b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac52ffd700 1 -- 172.21.15.69:0/4097227570 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60004db0 con 0x55a3bfd69e30 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.855+0000 7fac52ffd700 1 -- 172.21.15.69:0/4097227570 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac600158a0 con 0x55a3bfd69e30 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.856+0000 7fac799e3b80 1 -- 172.21.15.69:0/4097227570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 msgr2=0x55a3bfd6a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.856+0000 7fac799e3b80 1 --2- 172.21.15.69:0/4097227570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6a210 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fac60009370 tx=0x7fac600099e0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.857+0000 7fac6cee9700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fac580b2c60 0x7fac580b5120 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x55a3bfd6ab50 tx=0x7fac5c00b410 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.857+0000 7fac799e3b80 1 -- 172.21.15.69:0/4097227570 shutdown_connections 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.857+0000 7fac799e3b80 1 --2- 172.21.15.69:0/4097227570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6a210 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.857+0000 7fac799e3b80 1 --2- 172.21.15.69:0/4097227570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 0x55a3bfd64fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.857+0000 7fac799e3b80 1 --2- 172.21.15.69:0/4097227570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd64540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.857+0000 7fac799e3b80 1 -- 172.21.15.69:0/4097227570 >> 172.21.15.69:0/4097227570 conn(0x55a3bfc8f3e0 msgr2=0x55a3bfd5ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:50.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.857+0000 7fac799e3b80 1 -- 172.21.15.69:0/4097227570 shutdown_connections 2024-05-14T16:50:50.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.858+0000 7fac799e3b80 1 -- 172.21.15.69:0/4097227570 wait complete. 2024-05-14T16:50:50.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.858+0000 7fac799e3b80 1 -- start start 2024-05-14T16:50:50.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.858+0000 7fac799e3b80 1 Processor -- start 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.859+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd64b60 0x55a3bfd70350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.859+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.859+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd6dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.859+0000 7fac799e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfbb9130 con 0x55a3bfd64b60 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.859+0000 7fac799e3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfc54950 con 0x55a3bfd69e30 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.859+0000 7fac799e3b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfd71230 con 0x55a3bfd6a750 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.859+0000 7fac799e3b80 10 client.0 _ll_register_callbacks cb 0x55a3bfc49710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.860+0000 7fac6cee9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd64b60 0x55a3bfd70350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.860+0000 7fac6cee9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd64b60 0x55a3bfd70350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56128/0 (socket says 172.21.15.69:56128) 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.860+0000 7fac6cee9700 1 -- 172.21.15.69:0/1770738810 learned_addr learned my addr 172.21.15.69:0/1770738810 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.860+0000 7fac67fff700 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.860+0000 7fac6d6ea700 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd6dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.860+0000 7fac67fff700 1 -- 172.21.15.69:0/1770738810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 msgr2=0x55a3bfd6dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:50.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.860+0000 7fac67fff700 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd6dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.860+0000 7fac67fff700 1 -- 172.21.15.69:0/1770738810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd64b60 msgr2=0x55a3bfd70350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:50.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.860+0000 7fac67fff700 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd64b60 0x55a3bfd70350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.860+0000 7fac67fff700 1 -- 172.21.15.69:0/1770738810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3bfc51820 con 0x55a3bfd69e30 2024-05-14T16:50:50.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.861+0000 7fac67fff700 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6d740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fac540e08d0 tx=0x7fac540a6930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c593c68fbb937a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:50.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.861+0000 7fac47fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:50:50.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.862+0000 7fac47fff700 1 -- 172.21.15.69:0/1770738810 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac540a62e0 con 0x55a3bfd69e30 2024-05-14T16:50:50.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.862+0000 7fac47fff700 1 -- 172.21.15.69:0/1770738810 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7fac540b2f70 con 0x55a3bfd69e30 2024-05-14T16:50:50.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.862+0000 7fac799e3b80 1 -- 172.21.15.69:0/1770738810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac600158a0 con 0x55a3bfd69e30 2024-05-14T16:50:50.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.862+0000 7fac47fff700 1 -- 172.21.15.69:0/1770738810 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac540a9b90 con 0x55a3bfd69e30 2024-05-14T16:50:50.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.862+0000 7fac799e3b80 10 client.25019 fetch_fsmap learned FSMap version 29 2024-05-14T16:50:50.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.862+0000 7fac799e3b80 1 -- 172.21.15.69:0/1770738810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a3bfd7b7f0 con 0x55a3bfd69e30 2024-05-14T16:50:50.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.865+0000 7fac47fff700 1 -- 172.21.15.69:0/1770738810 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac600158a0 con 0x55a3bfd69e30 2024-05-14T16:50:50.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.865+0000 7fac47fff700 1 -- 172.21.15.69:0/1770738810 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fac540e0bb0 con 0x55a3bfd69e30 2024-05-14T16:50:50.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.865+0000 7fac799e3b80 10 client.25019 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T16:50:50.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.865+0000 7fac799e3b80 10 client.25019 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:50:50.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.865+0000 7fac799e3b80 20 client.25019 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:50.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.865+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a3bfd7dcc0 0x55a3bfd80120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.865+0000 7fac799e3b80 4 client.25019 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:50:50.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.865+0000 7fac799e3b80 1 -- 172.21.15.69:0/1770738810 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3bfd304f0 con 0x55a3bfd7dcc0 2024-05-14T16:50:50.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.865+0000 7fac6cee9700 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a3bfd7dcc0 0x55a3bfd80120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.867+0000 7fac6cee9700 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a3bfd7dcc0 0x55a3bfd80120 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06912e1e30c2f89 server_cookie=e9c51c58707f3d02 in_seq=0 out_seq=0 2024-05-14T16:50:50.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.867+0000 7fac47fff700 10 client.25019 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:50.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.868+0000 7fac47fff700 1 -- 172.21.15.69:0/1770738810 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a3bfd304f0 con 0x55a3bfd7dcc0 2024-05-14T16:50:50.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.868+0000 7fac47fff700 10 client.25019 handle_command_reply: tid=0 2024-05-14T16:50:50.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.870+0000 7fac799e3b80 1 client.25019 shutdown 2024-05-14T16:50:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.870+0000 7fac799e3b80 1 -- 172.21.15.69:0/1770738810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd69e30 msgr2=0x55a3bfd6d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.870+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6d740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fac540e08d0 tx=0x7fac540a6930 comp rx=0 tx=0).stop 2024-05-14T16:50:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.870+0000 7fac799e3b80 1 -- 172.21.15.69:0/1770738810 shutdown_connections 2024-05-14T16:50:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.870+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a3bfd7dcc0 0x55a3bfd80120 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.870+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd64b60 0x55a3bfd70350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.870+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd6dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.870+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1770738810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd69e30 0x55a3bfd6d740 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.870+0000 7fac799e3b80 1 -- 172.21.15.69:0/1770738810 >> 172.21.15.69:0/1770738810 conn(0x55a3bfc8f3e0 msgr2=0x55a3bfd59860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.870+0000 7fac47fff700 0 client.25019 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:50.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.872+0000 7fac799e3b80 1 -- 172.21.15.69:0/1770738810 shutdown_connections 2024-05-14T16:50:50.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.872+0000 7fac799e3b80 1 -- 172.21.15.69:0/1770738810 wait complete. 2024-05-14T16:50:50.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.872+0000 7fac799e3b80 20 client.25019 trim_cache size 0 max 16384 2024-05-14T16:50:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.910+0000 7fac799e3b80 1 Processor -- start 2024-05-14T16:50:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.910+0000 7fac799e3b80 1 -- start start 2024-05-14T16:50:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.910+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 0x55a3bfd7a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.910+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd64540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.910+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd59970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.910+0000 7fac799e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac681bf3b0 con 0x55a3bfd69e30 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.910+0000 7fac799e3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fac681bcbb0 con 0x55a3bfd64b60 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.910+0000 7fac799e3b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fac681baa00 con 0x55a3bfd6a750 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.910+0000 7fac67fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd64540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac67fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd64540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56134/0 (socket says 172.21.15.69:56134) 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac67fff700 1 -- 172.21.15.69:0/1115978094 learned_addr learned my addr 172.21.15.69:0/1115978094 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac6d6ea700 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd59970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac6cee9700 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 0x55a3bfd7a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac67fff700 1 -- 172.21.15.69:0/1115978094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 msgr2=0x55a3bfd59970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac67fff700 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd59970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac67fff700 1 -- 172.21.15.69:0/1115978094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 msgr2=0x55a3bfd7a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac67fff700 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 0x55a3bfd7a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac67fff700 1 -- 172.21.15.69:0/1115978094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3bfd6ff90 con 0x55a3bfd69e30 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac6d6ea700 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd59970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac6cee9700 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 0x55a3bfd7a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:50.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac67fff700 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd64540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fac54009680 tx=0x7fac5400fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46587e9f1e6145b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:50.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac527fc700 1 -- 172.21.15.69:0/1115978094 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac540a9d70 con 0x55a3bfd69e30 2024-05-14T16:50:50.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac527fc700 1 -- 172.21.15.69:0/1115978094 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac540a9f30 con 0x55a3bfd69e30 2024-05-14T16:50:50.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.911+0000 7fac527fc700 1 -- 172.21.15.69:0/1115978094 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac540aa1f0 con 0x55a3bfd69e30 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.912+0000 7fac799e3b80 1 -- 172.21.15.69:0/1115978094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 msgr2=0x55a3bfd64540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.913+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd64540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fac54009680 tx=0x7fac5400fb00 comp rx=0 tx=0).stop 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.913+0000 7fac799e3b80 1 -- 172.21.15.69:0/1115978094 shutdown_connections 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.913+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd6a750 0x55a3bfd59970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.913+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd64b60 0x55a3bfd7a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.913+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1115978094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd64540 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.914+0000 7fac799e3b80 1 -- 172.21.15.69:0/1115978094 >> 172.21.15.69:0/1115978094 conn(0x55a3bfc8f3e0 msgr2=0x55a3bfd6c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.914+0000 7fac799e3b80 1 -- 172.21.15.69:0/1115978094 shutdown_connections 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.914+0000 7fac799e3b80 1 -- 172.21.15.69:0/1115978094 wait complete. 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.914+0000 7fac799e3b80 1 -- start start 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.914+0000 7fac799e3b80 1 Processor -- start 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd64b60 0x55a3bfd738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd73e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac799e3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd6a750 0x55a3bfde25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac799e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac681bb070 con 0x55a3bfd69e30 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac799e3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fac681bc4e0 con 0x55a3bfd6a750 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac799e3b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fac681bb740 con 0x55a3bfd64b60 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac799e3b80 10 client.0 _ll_register_callbacks cb 0x55a3bfc49710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac6cee9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd64b60 0x55a3bfd738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac67fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd73e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac6cee9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd64b60 0x55a3bfd738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:51040/0 (socket says 172.21.15.69:51040) 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac6cee9700 1 -- 172.21.15.69:0/1553523730 learned_addr learned my addr 172.21.15.69:0/1553523730 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac6d6ea700 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd6a750 0x55a3bfde25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac6cee9700 1 -- 172.21.15.69:0/1553523730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd6a750 msgr2=0x55a3bfde25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac6cee9700 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd6a750 0x55a3bfde25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac6cee9700 1 -- 172.21.15.69:0/1553523730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 msgr2=0x55a3bfd73e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:50:50.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac6cee9700 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd73e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.915+0000 7fac6cee9700 1 -- 172.21.15.69:0/1553523730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3bfde2bc0 con 0x55a3bfd64b60 2024-05-14T16:50:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.916+0000 7fac6cee9700 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd64b60 0x55a3bfd738e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00bfd0 tx=0x7fac5c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ba3ee04206da89 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.916+0000 7fac47fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:50:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.916+0000 7fac47fff700 1 -- 172.21.15.69:0/1553523730 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c012d40 con 0x55a3bfd64b60 2024-05-14T16:50:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.916+0000 7fac47fff700 1 -- 172.21.15.69:0/1553523730 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7fac5c027250 con 0x55a3bfd64b60 2024-05-14T16:50:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.916+0000 7fac799e3b80 1 -- 172.21.15.69:0/1553523730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac540a9f30 con 0x55a3bfd64b60 2024-05-14T16:50:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.916+0000 7fac47fff700 1 -- 172.21.15.69:0/1553523730 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac5c037050 con 0x55a3bfd64b60 2024-05-14T16:50:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.916+0000 7fac799e3b80 10 client.24994 fetch_fsmap learned FSMap version 29 2024-05-14T16:50:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.916+0000 7fac799e3b80 1 -- 172.21.15.69:0/1553523730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a3bfde9f80 con 0x55a3bfd64b60 2024-05-14T16:50:50.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.918+0000 7fac47fff700 1 -- 172.21.15.69:0/1553523730 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac540a9f30 con 0x55a3bfd64b60 2024-05-14T16:50:50.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.918+0000 7fac47fff700 1 -- 172.21.15.69:0/1553523730 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fac5c027ce0 con 0x55a3bfd64b60 2024-05-14T16:50:50.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.918+0000 7fac799e3b80 10 client.24994 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T16:50:50.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.918+0000 7fac799e3b80 10 client.24994 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:50:50.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.918+0000 7fac799e3b80 20 client.24994 populate_metadata read hostname 'smithi069' 2024-05-14T16:50:50.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.918+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a3bfdea790 0x55a3bfdecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:50.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.918+0000 7fac799e3b80 4 client.24994 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.2, "await_for": 45.0}] 2024-05-14T16:50:50.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.918+0000 7fac799e3b80 1 -- 172.21.15.69:0/1553523730 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.2, "await_for": 45.0}) -- 0x55a3bfcc4d70 con 0x55a3bfdea790 2024-05-14T16:50:50.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.918+0000 7fac67fff700 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a3bfdea790 0x55a3bfdecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:50.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.919+0000 7fac67fff700 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a3bfdea790 0x55a3bfdecbb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=827d03e2312216b8 server_cookie=cef97c6afecc1cb9 in_seq=0 out_seq=0 2024-05-14T16:50:50.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:50.919+0000 7fac47fff700 10 client.24994 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:52.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac47fff700 1 -- 172.21.15.69:0/1553523730 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a3bfcc4d70 con 0x55a3bfdea790 2024-05-14T16:50:52.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac47fff700 10 client.24994 handle_command_reply: tid=0 2024-05-14T16:50:52.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 client.24994 shutdown 2024-05-14T16:50:52.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 -- 172.21.15.69:0/1553523730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd64b60 msgr2=0x55a3bfd738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd64b60 0x55a3bfd738e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00bfd0 tx=0x7fac5c004750 comp rx=0 tx=0).stop 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 -- 172.21.15.69:0/1553523730 shutdown_connections 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a3bfdea790 0x55a3bfdecbb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3bfd6a750 0x55a3bfde25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a3bfd64b60 0x55a3bfd738e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac47fff700 0 client.24994 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 --2- 172.21.15.69:0/1553523730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3bfd69e30 0x55a3bfd73e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 -- 172.21.15.69:0/1553523730 >> 172.21.15.69:0/1553523730 conn(0x55a3bfc8f3e0 msgr2=0x55a3bfd6c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 -- 172.21.15.69:0/1553523730 shutdown_connections 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 1 -- 172.21.15.69:0/1553523730 wait complete. 2024-05-14T16:50:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.028+0000 7fac799e3b80 20 client.24994 trim_cache size 0 max 16384 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 29, 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 2, 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "44cae421": { 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 2, 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:50:52.031 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 135.2, 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:50:52.032 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:50:52.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 -- 172.21.15.69:0/2869972887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fac580b2c60 msgr2=0x7fac580b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:52.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fac580b2c60 0x7fac580b5120 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x55a3bfd6ab50 tx=0x7fac5c00b410 comp rx=0 tx=0).stop 2024-05-14T16:50:52.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 -- 172.21.15.69:0/2869972887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac68102740 msgr2=0x7fac6810a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:52.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac68102740 0x7fac6810a670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c010 tx=0x7fac5400fa90 comp rx=0 tx=0).stop 2024-05-14T16:50:52.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 -- 172.21.15.69:0/2869972887 shutdown_connections 2024-05-14T16:50:52.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fac580b2c60 0x7fac580b5120 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:52.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac68101d90 0x7fac681115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:52.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac68114140 0x7fac6810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:52.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 --2- 172.21.15.69:0/2869972887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac68102740 0x7fac6810a670 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:52.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 -- 172.21.15.69:0/2869972887 >> 172.21.15.69:0/2869972887 conn(0x7fac68078ae0 msgr2=0x7fac680fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:52.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 -- 172.21.15.69:0/2869972887 shutdown_connections 2024-05-14T16:50:52.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:50:52.029+0000 7fac6f14d700 1 -- 172.21.15.69:0/2869972887 wait complete. 2024-05-14T16:50:52.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:51 smithi069 ceph-mon[31507]: pgmap v32: 97 pgs: 97 active+clean; 950 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-05-14T16:50:52.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:51 smithi203 ceph-mon[35787]: pgmap v32: 97 pgs: 97 active+clean; 950 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-05-14T16:50:52.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:51 smithi092 ceph-mon[35763]: pgmap v32: 97 pgs: 97 active+clean; 950 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-05-14T16:50:52.827 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '44cae421', quiesce took 8.0 seconds. Will release after: 7.199999999999999 2024-05-14T16:50:54.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:53 smithi069 ceph-mon[31507]: pgmap v33: 97 pgs: 97 active+clean; 954 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 679 op/s 2024-05-14T16:50:54.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:53 smithi203 ceph-mon[35787]: pgmap v33: 97 pgs: 97 active+clean; 954 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 679 op/s 2024-05-14T16:50:54.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:53 smithi092 ceph-mon[35763]: pgmap v33: 97 pgs: 97 active+clean; 954 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 679 op/s 2024-05-14T16:50:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 25.0 s, 432.4 tps, lat 25.739 ms stddev 26.795 2024-05-14T16:50:56.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:55 smithi069 ceph-mon[31507]: pgmap v34: 97 pgs: 97 active+clean; 954 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 645 op/s 2024-05-14T16:50:56.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:55 smithi203 ceph-mon[35787]: pgmap v34: 97 pgs: 97 active+clean; 954 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 645 op/s 2024-05-14T16:50:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:55 smithi092 ceph-mon[35763]: pgmap v34: 97 pgs: 97 active+clean; 954 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 645 op/s 2024-05-14T16:50:58.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:57 smithi069 ceph-mon[31507]: pgmap v35: 97 pgs: 97 active+clean; 956 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 8.3 MiB/s wr, 759 op/s 2024-05-14T16:50:58.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:57 smithi203 ceph-mon[35787]: pgmap v35: 97 pgs: 97 active+clean; 956 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 8.3 MiB/s wr, 759 op/s 2024-05-14T16:50:58.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:57 smithi092 ceph-mon[35763]: pgmap v35: 97 pgs: 97 active+clean; 956 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 8.3 MiB/s wr, 759 op/s 2024-05-14T16:51:00.028 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:51:00.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:59 smithi069 ceph-mon[31507]: pgmap v36: 97 pgs: 97 active+clean; 956 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.1 MiB/s wr, 446 op/s 2024-05-14T16:51:00.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:50:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:51:00.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:59 smithi203 ceph-mon[35787]: pgmap v36: 97 pgs: 97 active+clean; 956 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.1 MiB/s wr, 446 op/s 2024-05-14T16:51:00.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:50:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:51:00.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:59 smithi092 ceph-mon[35763]: pgmap v36: 97 pgs: 97 active+clean; 956 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.1 MiB/s wr, 446 op/s 2024-05-14T16:51:00.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:50:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:51:00.295 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:00.398 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 30.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:51:01.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:01 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:51:01.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:01 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:51:01.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:01 smithi069 ceph-mon[31507]: pgmap v37: 97 pgs: 97 active+clean; 1020 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 18 MiB/s wr, 629 op/s 2024-05-14T16:51:01.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:01 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:01.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:01 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:51:01.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:01 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:51:01.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:01 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:51:01.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:01 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:51:01.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:01 smithi203 ceph-mon[35787]: pgmap v37: 97 pgs: 97 active+clean; 1020 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 18 MiB/s wr, 629 op/s 2024-05-14T16:51:01.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:01 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:01.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:01 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:51:01.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:01 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:51:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:01 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:51:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:01 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:51:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:01 smithi092 ceph-mon[35763]: pgmap v37: 97 pgs: 97 active+clean; 1020 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 18 MiB/s wr, 629 op/s 2024-05-14T16:51:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:01 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:01 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:51:01.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:01 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:51:02.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.399+0000 7f7042e8c700 1 -- 172.21.15.69:0/1712419524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f703c102df0 msgr2=0x7f703c10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:02.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.399+0000 7f7042e8c700 1 --2- 172.21.15.69:0/1712419524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f703c102df0 0x7f703c10f2e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7024009990 tx=0x7f7024009ca0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.399+0000 7f7042e8c700 1 -- 172.21.15.69:0/1712419524 shutdown_connections 2024-05-14T16:51:02.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.399+0000 7f7042e8c700 1 --2- 172.21.15.69:0/1712419524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f703c102df0 0x7f703c10f2e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.399+0000 7f7042e8c700 1 --2- 172.21.15.69:0/1712419524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f703c108510 0x7f703c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.399+0000 7f7042e8c700 1 --2- 172.21.15.69:0/1712419524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c102430 0x7f703c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.399+0000 7f7042e8c700 1 -- 172.21.15.69:0/1712419524 >> 172.21.15.69:0/1712419524 conn(0x7f703c0793b0 msgr2=0x7f703c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:02.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.399+0000 7f7042e8c700 1 -- 172.21.15.69:0/1712419524 shutdown_connections 2024-05-14T16:51:02.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.399+0000 7f7042e8c700 1 -- 172.21.15.69:0/1712419524 wait complete. 2024-05-14T16:51:02.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.400+0000 7f7042e8c700 1 Processor -- start 2024-05-14T16:51:02.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.400+0000 7f7042e8c700 1 -- start start 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.400+0000 7f7042e8c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c102430 0x7f703c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7042e8c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f703c102df0 0x7f703c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7042e8c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f703c108510 0x7f703c1a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7042e8c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f703c112640 con 0x7f703c108510 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7042e8c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1124c0 con 0x7f703c102430 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7042e8c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1127c0 con 0x7f703c102df0 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7041429700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f703c108510 0x7f703c1a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7041429700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f703c108510 0x7f703c1a5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36456/0 (socket says 172.21.15.69:36456) 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7041429700 1 -- 172.21.15.69:0/1356036077 learned_addr learned my addr 172.21.15.69:0/1356036077 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f703bfff700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f703c102df0 0x7f703c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7040c28700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c102430 0x7f703c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7041429700 1 -- 172.21.15.69:0/1356036077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f703c102df0 msgr2=0x7f703c19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7041429700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f703c102df0 0x7f703c19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7041429700 1 -- 172.21.15.69:0/1356036077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c102430 msgr2=0x7f703c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7041429700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c102430 0x7f703c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7041429700 1 -- 172.21.15.69:0/1356036077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c1a5fc0 con 0x7f703c108510 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7040c28700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c102430 0x7f703c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7041429700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f703c108510 0x7f703c1a5a80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f702400c010 tx=0x7f7024009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7039ffb700 1 -- 172.21.15.69:0/1356036077 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024027070 con 0x7f703c108510 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7039ffb700 1 -- 172.21.15.69:0/1356036077 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7024003fa0 con 0x7f703c108510 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7042e8c700 1 -- 172.21.15.69:0/1356036077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c1a6190 con 0x7f703c108510 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.401+0000 7f7039ffb700 1 -- 172.21.15.69:0/1356036077 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024022d70 con 0x7f703c108510 2024-05-14T16:51:02.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.402+0000 7f7042e8c700 1 -- 172.21.15.69:0/1356036077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c1a6640 con 0x7f703c108510 2024-05-14T16:51:02.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.403+0000 7f7042e8c700 1 -- 172.21.15.69:0/1356036077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c06a230 con 0x7f703c108510 2024-05-14T16:51:02.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.404+0000 7f7039ffb700 1 -- 172.21.15.69:0/1356036077 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f702402a070 con 0x7f703c108510 2024-05-14T16:51:02.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.405+0000 7f7039ffb700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70280b2b90 0x7f70280b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.405+0000 7f7040c28700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70280b2b90 0x7f70280b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.405+0000 7f7039ffb700 1 -- 172.21.15.69:0/1356036077 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f70240e4070 con 0x7f703c108510 2024-05-14T16:51:02.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.405+0000 7f7040c28700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70280b2b90 0x7f70280b5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7034005fd0 tx=0x7f7034005e20 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:02.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.409+0000 7f7039ffb700 1 -- 172.21.15.69:0/1356036077 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7024031020 con 0x7f703c108510 2024-05-14T16:51:02.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.658+0000 7f7042e8c700 1 -- 172.21.15.69:0/1356036077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f703c02e720 con 0x7f703c108510 2024-05-14T16:51:02.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.658+0000 7f7039ffb700 1 -- 172.21.15.69:0/1356036077 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f7024013070 con 0x7f703c108510 2024-05-14T16:51:02.658 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:51:02.659 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:50:38.704223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24958":{"gid":24958,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/2802264018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2802264018},{"type":"v1","addr":"172.21.15.203:6837","nonce":2802264018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24446,24422,24433]},"id":1}]} 2024-05-14T16:51:02.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 -- 172.21.15.69:0/1356036077 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70280b2b90 msgr2=0x7f70280b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:02.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70280b2b90 0x7f70280b5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7034005fd0 tx=0x7f7034005e20 comp rx=0 tx=0).stop 2024-05-14T16:51:02.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 -- 172.21.15.69:0/1356036077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f703c108510 msgr2=0x7f703c1a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:02.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f703c108510 0x7f703c1a5a80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f702400c010 tx=0x7f7024009ee0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 -- 172.21.15.69:0/1356036077 shutdown_connections 2024-05-14T16:51:02.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70280b2b90 0x7f70280b5050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f703c108510 0x7f703c1a5a80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f703c102430 0x7f703c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 --2- 172.21.15.69:0/1356036077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f703c102df0 0x7f703c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 -- 172.21.15.69:0/1356036077 >> 172.21.15.69:0/1356036077 conn(0x7f703c0793b0 msgr2=0x7f703c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:02.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 -- 172.21.15.69:0/1356036077 shutdown_connections 2024-05-14T16:51:02.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:02.661+0000 7f7042e8c700 1 -- 172.21.15.69:0/1356036077 wait complete. 2024-05-14T16:51:02.662 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 29 2024-05-14T16:51:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.978+0000 7f365022a640 1 -- 172.21.15.69:0/2690920172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3648102340 msgr2=0x7f364810e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.978+0000 7f365022a640 1 --2- 172.21.15.69:0/2690920172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3648102340 0x7f364810e800 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3640009cd0 tx=0x7f364002a8e0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.978+0000 7f365022a640 1 -- 172.21.15.69:0/2690920172 shutdown_connections 2024-05-14T16:51:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.978+0000 7f365022a640 1 --2- 172.21.15.69:0/2690920172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3648102340 0x7f364810e800 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.978+0000 7f365022a640 1 --2- 172.21.15.69:0/2690920172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3648101980 0x7f3648101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.978+0000 7f365022a640 1 --2- 172.21.15.69:0/2690920172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3648107940 0x7f3648107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.978+0000 7f365022a640 1 -- 172.21.15.69:0/2690920172 >> 172.21.15.69:0/2690920172 conn(0x7f3648078ad0 msgr2=0x7f3648078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.978+0000 7f365022a640 1 -- 172.21.15.69:0/2690920172 shutdown_connections 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.978+0000 7f365022a640 1 -- 172.21.15.69:0/2690920172 wait complete. 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f365022a640 1 Processor -- start 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f365022a640 1 -- start start 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f365022a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3648101980 0x7f3648199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f365022a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3648102340 0x7f364819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f365022a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3648107940 0x7f36481a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f365022a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3648111c00 con 0x7f3648101980 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f365022a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3648111a80 con 0x7f3648102340 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f365022a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3648111d80 con 0x7f3648107940 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364dfc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3648101980 0x7f3648199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364dfc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3648101980 0x7f3648199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36482/0 (socket says 172.21.15.69:36482) 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364dfc6640 1 -- 172.21.15.69:0/3699445507 learned_addr learned my addr 172.21.15.69:0/3699445507 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364d7c5640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3648102340 0x7f364819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364dfc6640 1 -- 172.21.15.69:0/3699445507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3648107940 msgr2=0x7f36481a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364dfc6640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3648107940 0x7f36481a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364dfc6640 1 -- 172.21.15.69:0/3699445507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3648102340 msgr2=0x7f364819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364dfc6640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3648102340 0x7f364819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364dfc6640 1 -- 172.21.15.69:0/3699445507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36481a5540 con 0x7f3648101980 2024-05-14T16:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364d7c5640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3648102340 0x7f364819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f364dfc6640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3648101980 0x7f3648199f80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f364400d770 tx=0x7f364400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.979+0000 7f363effd640 1 -- 172.21.15.69:0/3699445507 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644017070 con 0x7f3648101980 2024-05-14T16:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.980+0000 7f363effd640 1 -- 172.21.15.69:0/3699445507 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36440054d0 con 0x7f3648101980 2024-05-14T16:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.980+0000 7f365022a640 1 -- 172.21.15.69:0/3699445507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36481a5710 con 0x7f3648101980 2024-05-14T16:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.980+0000 7f363effd640 1 -- 172.21.15.69:0/3699445507 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644004c30 con 0x7f3648101980 2024-05-14T16:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.980+0000 7f365022a640 1 -- 172.21.15.69:0/3699445507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36481a5be0 con 0x7f3648101980 2024-05-14T16:51:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.982+0000 7f363effd640 1 -- 172.21.15.69:0/3699445507 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f3644004e10 con 0x7f3648101980 2024-05-14T16:51:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.982+0000 7f3653021740 1 Processor -- start 2024-05-14T16:51:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.982+0000 7f3653021740 1 -- start start 2024-05-14T16:51:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.982+0000 7f3653021740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbde4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f3653021740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f3653021740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 0x562bdbdf1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f3653021740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562bdbd57c30 con 0x562bdbde5210 2024-05-14T16:51:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f3653021740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562bdbd477d0 con 0x562bdbdefad0 2024-05-14T16:51:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f3653021740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562bdbd4b810 con 0x562bdbde48f0 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364dfc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364dfc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdef590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36484/0 (socket says 172.21.15.69:36484) 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364dfc6640 1 -- 172.21.15.69:0/3997851612 learned_addr learned my addr 172.21.15.69:0/3997851612 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364e7c7640 1 --2- 172.21.15.69:0/3997851612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbde4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364d7c5640 1 --2- 172.21.15.69:0/3997851612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 0x562bdbdf1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364dfc6640 1 -- 172.21.15.69:0/3997851612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 msgr2=0x562bdbde4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364dfc6640 1 --2- 172.21.15.69:0/3997851612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbde4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364dfc6640 1 -- 172.21.15.69:0/3997851612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 msgr2=0x562bdbdf1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364dfc6640 1 --2- 172.21.15.69:0/3997851612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 0x562bdbdf1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364dfc6640 1 -- 172.21.15.69:0/3997851612 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bdba5c210 con 0x562bdbde5210 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364d7c5640 1 --2- 172.21.15.69:0/3997851612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 0x562bdbdf1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f363effd640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f36340b2b30 0x7f36340b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364e7c7640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f36340b2b30 0x7f36340b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f364dfc6640 1 --2- 172.21.15.69:0/3997851612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdef590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f36440dfab0 tx=0x7f36440eaa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d76bfa4c8a3438 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f363effd640 1 -- 172.21.15.69:0/3699445507 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f36440e23c0 con 0x7f3648101980 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.983+0000 7f3633fff640 1 -- 172.21.15.69:0/3997851612 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644020030 con 0x562bdbde5210 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3633fff640 1 -- 172.21.15.69:0/3997851612 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36440a8f70 con 0x562bdbde5210 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3633fff640 1 -- 172.21.15.69:0/3997851612 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36440a9230 con 0x562bdbde5210 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3653021740 1 -- 172.21.15.69:0/3997851612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 msgr2=0x562bdbdef590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f364e7c7640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f36340b2b30 0x7f36340b4ff0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3640009ca0 tx=0x7f3640005f30 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3653021740 1 --2- 172.21.15.69:0/3997851612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdef590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f36440dfab0 tx=0x7f36440eaa30 comp rx=0 tx=0).stop 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3653021740 1 -- 172.21.15.69:0/3997851612 shutdown_connections 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3653021740 1 --2- 172.21.15.69:0/3997851612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 0x562bdbdf1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3653021740 1 --2- 172.21.15.69:0/3997851612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdef590 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3653021740 1 --2- 172.21.15.69:0/3997851612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbde4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3653021740 1 -- 172.21.15.69:0/3997851612 >> 172.21.15.69:0/3997851612 conn(0x562bdbde0e00 msgr2=0x562bdbde11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3653021740 1 -- 172.21.15.69:0/3997851612 shutdown_connections 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.984+0000 7f3653021740 1 -- 172.21.15.69:0/3997851612 wait complete. 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f3653021740 1 -- start start 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f3653021740 1 Processor -- start 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f3653021740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbde48f0 0x562bdbdef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f3653021740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde5210 0x562bdbdec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f3653021740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbdecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f3653021740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562bdba2d790 con 0x562bdbdefad0 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f3653021740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562bdbd03890 con 0x562bdbde48f0 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f3653021740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562bdbdf2740 con 0x562bdbde5210 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f3653021740 10 client.0 _ll_register_callbacks cb 0x562bdbd06d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f364e7c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbdecf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f364e7c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbdecf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36490/0 (socket says 172.21.15.69:36490) 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.985+0000 7f364e7c7640 1 -- 172.21.15.69:0/2375926520 learned_addr learned my addr 172.21.15.69:0/2375926520 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f364dfc6640 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbde48f0 0x562bdbdef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f364d7c5640 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde5210 0x562bdbdec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f364e7c7640 1 -- 172.21.15.69:0/2375926520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde5210 msgr2=0x562bdbdec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f364e7c7640 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde5210 0x562bdbdec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f364e7c7640 1 -- 172.21.15.69:0/2375926520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbde48f0 msgr2=0x562bdbdef500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f364e7c7640 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbde48f0 0x562bdbdef500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f364e7c7640 1 -- 172.21.15.69:0/2375926520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bdbb28620 con 0x562bdbdefad0 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f364d7c5640 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde5210 0x562bdbdec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f364e7c7640 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbdecf30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3640031c50 tx=0x7f36400095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d900978407dd37f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f36317fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f36317fa640 1 -- 172.21.15.69:0/2375926520 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640004240 con 0x562bdbdefad0 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f36317fa640 1 -- 172.21.15.69:0/2375926520 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f36400043e0 con 0x562bdbdefad0 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f3653021740 1 -- 172.21.15.69:0/2375926520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36440a8f70 con 0x562bdbdefad0 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f36317fa640 1 -- 172.21.15.69:0/2375926520 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f36400045e0 con 0x562bdbdefad0 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f3653021740 10 client.15471 fetch_fsmap learned FSMap version 29 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f3653021740 1 -- 172.21.15.69:0/2375926520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562bdbdfe180 con 0x562bdbdefad0 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.986+0000 7f36317fa640 1 -- 172.21.15.69:0/2375926520 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36440a8f70 con 0x562bdbdefad0 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.987+0000 7f36317fa640 1 -- 172.21.15.69:0/2375926520 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f3640030070 con 0x562bdbdefad0 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.987+0000 7f3653021740 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.987+0000 7f3653021740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.987+0000 7f3653021740 20 client.15471 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.987+0000 7f3653021740 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562bdbdff6b0 0x562bdbe01ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.987+0000 7f3653021740 4 client.15471 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.987+0000 7f3653021740 1 -- 172.21.15.69:0/2375926520 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bdbd59b80 con 0x562bdbdff6b0 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.987+0000 7f364dfc6640 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562bdbdff6b0 0x562bdbe01ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.987+0000 7f36317fa640 10 client.15471 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.987+0000 7f364dfc6640 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562bdbdff6b0 0x562bdbe01ae0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a9ec261523f8 server_cookie=b25a2f33b43bc385 in_seq=0 out_seq=0 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f36317fa640 1 -- 172.21.15.69:0/2375926520 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562bdbd59b80 con 0x562bdbdff6b0 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f36317fa640 10 client.15471 handle_command_reply: tid=0 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 client.15471 shutdown 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 -- 172.21.15.69:0/2375926520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 msgr2=0x562bdbdecf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbdecf30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3640031c50 tx=0x7f36400095c0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 -- 172.21.15.69:0/2375926520 shutdown_connections 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562bdbdff6b0 0x562bdbe01ae0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbdecf30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde5210 0x562bdbdec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f36317fa640 0 client.15471 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 --2- 172.21.15.69:0/2375926520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbde48f0 0x562bdbdef500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 -- 172.21.15.69:0/2375926520 >> 172.21.15.69:0/2375926520 conn(0x562bdbde0e00 msgr2=0x562bdbde0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 -- 172.21.15.69:0/2375926520 shutdown_connections 2024-05-14T16:51:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 1 -- 172.21.15.69:0/2375926520 wait complete. 2024-05-14T16:51:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:02.988+0000 7f3653021740 20 client.15471 trim_cache size 0 max 16384 2024-05-14T16:51:03.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:02 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1356036077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.003+0000 7f3653021740 1 Processor -- start 2024-05-14T16:51:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.003+0000 7f3653021740 1 -- start start 2024-05-14T16:51:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.003+0000 7f3653021740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbdfb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f3653021740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbde5210 0x562bdbe07440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f3653021740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbe07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f3653021740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f36481b0f70 con 0x562bdbdefad0 2024-05-14T16:51:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f3653021740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f36481ae7b0 con 0x562bdbde5210 2024-05-14T16:51:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f3653021740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f36481ac600 con 0x562bdbde48f0 2024-05-14T16:51:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364e7c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbe07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364e7c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbe07980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36502/0 (socket says 172.21.15.69:36502) 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364e7c7640 1 -- 172.21.15.69:0/788312913 learned_addr learned my addr 172.21.15.69:0/788312913 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364d7c5640 1 --2- 172.21.15.69:0/788312913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbde5210 0x562bdbe07440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364dfc6640 1 --2- 172.21.15.69:0/788312913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbdfb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364e7c7640 1 -- 172.21.15.69:0/788312913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 msgr2=0x562bdbdfb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364e7c7640 1 --2- 172.21.15.69:0/788312913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbdfb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364e7c7640 1 -- 172.21.15.69:0/788312913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbde5210 msgr2=0x562bdbe07440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364e7c7640 1 --2- 172.21.15.69:0/788312913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbde5210 0x562bdbe07440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364e7c7640 1 -- 172.21.15.69:0/788312913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bdbde19d0 con 0x562bdbdefad0 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f364e7c7640 1 --2- 172.21.15.69:0/788312913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbe07980 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3640030360 tx=0x7f3640005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2e98f47bbe07df server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f36337fe640 1 -- 172.21.15.69:0/788312913 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640004000 con 0x562bdbdefad0 2024-05-14T16:51:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f36337fe640 1 -- 172.21.15.69:0/788312913 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f364005e070 con 0x562bdbdefad0 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.004+0000 7f36337fe640 1 -- 172.21.15.69:0/788312913 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36400305a0 con 0x562bdbdefad0 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 -- 172.21.15.69:0/788312913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 msgr2=0x562bdbe07980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 --2- 172.21.15.69:0/788312913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbe07980 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3640030360 tx=0x7f3640005d80 comp rx=0 tx=0).stop 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 -- 172.21.15.69:0/788312913 shutdown_connections 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 --2- 172.21.15.69:0/788312913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbdefad0 0x562bdbe07980 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 --2- 172.21.15.69:0/788312913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbde5210 0x562bdbe07440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 --2- 172.21.15.69:0/788312913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbdfb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 -- 172.21.15.69:0/788312913 >> 172.21.15.69:0/788312913 conn(0x562bdbde0e00 msgr2=0x562bdbdeb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 -- 172.21.15.69:0/788312913 shutdown_connections 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 -- 172.21.15.69:0/788312913 wait complete. 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 -- start start 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.005+0000 7f3653021740 1 Processor -- start 2024-05-14T16:51:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f3653021740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbdf5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f3653021740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdf5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f3653021740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 0x562bdbdf61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f3653021740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f36481acc70 con 0x562bdbde5210 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f3653021740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f36481ae0e0 con 0x562bdbdefad0 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f3653021740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f36481ad340 con 0x562bdbde48f0 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364d7c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdf5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f3653021740 10 client.0 _ll_register_callbacks cb 0x7f3640031920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364d7c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdf5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36514/0 (socket says 172.21.15.69:36514) 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364d7c5640 1 -- 172.21.15.69:0/26976829 learned_addr learned my addr 172.21.15.69:0/26976829 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364e7c7640 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 0x562bdbdf61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364dfc6640 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbdf5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364d7c5640 1 -- 172.21.15.69:0/26976829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 msgr2=0x562bdbdf5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364d7c5640 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbdf5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364d7c5640 1 -- 172.21.15.69:0/26976829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 msgr2=0x562bdbdf61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364d7c5640 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 0x562bdbdf61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364d7c5640 1 -- 172.21.15.69:0/26976829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bdbe69200 con 0x562bdbde5210 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f364d7c5640 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdf5c70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x562bdbde1f90 tx=0x7f3638013b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f809b6fcab762cb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f36317fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f36317fa640 1 -- 172.21.15.69:0/26976829 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638020060 con 0x562bdbde5210 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f36317fa640 1 -- 172.21.15.69:0/26976829 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f3638011da0 con 0x562bdbde5210 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.006+0000 7f3653021740 1 -- 172.21.15.69:0/26976829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364005e070 con 0x562bdbde5210 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f36317fa640 1 -- 172.21.15.69:0/26976829 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f363800f450 con 0x562bdbde5210 2024-05-14T16:51:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f3653021740 10 client.15477 fetch_fsmap learned FSMap version 29 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f3653021740 1 -- 172.21.15.69:0/26976829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562bdbe6a560 con 0x562bdbde5210 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f36317fa640 1 -- 172.21.15.69:0/26976829 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f364005e070 con 0x562bdbde5210 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f36317fa640 1 -- 172.21.15.69:0/26976829 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f363801a030 con 0x562bdbde5210 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f3653021740 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f3653021740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f3653021740 20 client.15477 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f3653021740 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562bdbe6ada0 0x562bdbe6d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f3653021740 4 client.15477 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f3653021740 1 -- 172.21.15.69:0/26976829 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562bdbd53360 con 0x562bdbe6ada0 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f364dfc6640 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562bdbe6ada0 0x562bdbe6d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f36317fa640 10 client.15477 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.007+0000 7f364dfc6640 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562bdbe6ada0 0x562bdbe6d1c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35124d8db54427 server_cookie=9d9b613811ac710b in_seq=0 out_seq=0 2024-05-14T16:51:03.063 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:02 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1356036077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:03.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:02 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1356036077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:03.381 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id 44cae421 --release --await' 2024-05-14T16:51:03.382 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id 44cae421 --release --await 2024-05-14T16:51:03.633 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f36317fa640 1 -- 172.21.15.69:0/26976829 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562bdbd53360 con 0x562bdbe6ada0 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f36317fa640 10 client.15477 handle_command_reply: tid=0 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 client.15477 shutdown 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 -- 172.21.15.69:0/26976829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 msgr2=0x562bdbdf5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdf5c70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x562bdbde1f90 tx=0x7f3638013b30 comp rx=0 tx=0).stop 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 -- 172.21.15.69:0/26976829 shutdown_connections 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562bdbe6ada0 0x562bdbe6d1c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bdbdefad0 0x562bdbdf61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bdbde5210 0x562bdbdf5c70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f36317fa640 0 client.15477 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 --2- 172.21.15.69:0/26976829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bdbde48f0 0x562bdbdf5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 -- 172.21.15.69:0/26976829 >> 172.21.15.69:0/26976829 conn(0x562bdbde0e00 msgr2=0x562bdbdfa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 -- 172.21.15.69:0/26976829 shutdown_connections 2024-05-14T16:51:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 1 -- 172.21.15.69:0/26976829 wait complete. 2024-05-14T16:51:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.642+0000 7f3653021740 20 client.15477 trim_cache size 0 max 16384 2024-05-14T16:51:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:51:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:51:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:51:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 -- 172.21.15.69:0/3699445507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f36340b2b30 msgr2=0x7f36340b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f36340b2b30 0x7f36340b4ff0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3640009ca0 tx=0x7f3640005f30 comp rx=0 tx=0).stop 2024-05-14T16:51:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 -- 172.21.15.69:0/3699445507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3648101980 msgr2=0x7f3648199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3648101980 0x7f3648199f80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f364400d770 tx=0x7f364400dc30 comp rx=0 tx=0).stop 2024-05-14T16:51:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 -- 172.21.15.69:0/3699445507 shutdown_connections 2024-05-14T16:51:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f36340b2b30 0x7f36340b4ff0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3648107940 0x7f36481a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3648102340 0x7f364819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 --2- 172.21.15.69:0/3699445507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3648101980 0x7f3648199f80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 -- 172.21.15.69:0/3699445507 >> 172.21.15.69:0/3699445507 conn(0x7f3648078ad0 msgr2=0x7f3648104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 -- 172.21.15.69:0/3699445507 shutdown_connections 2024-05-14T16:51:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:03.643+0000 7f365022a640 1 -- 172.21.15.69:0/3699445507 wait complete. 2024-05-14T16:51:03.908 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:03 smithi069 ceph-mon[31507]: pgmap v38: 97 pgs: 97 active+clean; 1020 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 15 MiB/s wr, 368 op/s 2024-05-14T16:51:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:03 smithi092 ceph-mon[35763]: pgmap v38: 97 pgs: 97 active+clean; 1020 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 15 MiB/s wr, 368 op/s 2024-05-14T16:51:04.047 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:03 smithi203 ceph-mon[35787]: pgmap v38: 97 pgs: 97 active+clean; 1020 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 15 MiB/s wr, 368 op/s 2024-05-14T16:51:04.460 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: *** Caught signal (Segmentation fault) ** 2024-05-14T16:51:04.460 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: in thread 7f7fbbc78700 thread_name:md_log_replay 2024-05-14T16:51:04.460 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T16:51:04.460 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7fcd03ad20] 2024-05-14T16:51:04.460 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x5571131d5c1f] 2024-05-14T16:51:04.460 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5571131d93b3] 2024-05-14T16:51:04.460 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 4: (MDLog::_replay_thread()+0x617) [0x557113234197] 2024-05-14T16:51:04.460 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55711323da11] 2024-05-14T16:51:04.460 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f7fcd0301ca] 2024-05-14T16:51:04.460 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 7: clone() 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2024-05-14T16:51:04.047+0000 7f7fbbc78700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: in thread 7f7fbbc78700 thread_name:md_log_replay 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7fcd03ad20] 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x5571131d5c1f] 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5571131d93b3] 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 4: (MDLog::_replay_thread()+0x617) [0x557113234197] 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55711323da11] 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f7fcd0301ca] 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 7: clone() 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T16:51:04.461 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 0> 2024-05-14T16:51:04.047+0000 7f7fbbc78700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: in thread 7f7fbbc78700 thread_name:md_log_replay 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7fcd03ad20] 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x5571131d5c1f] 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5571131d93b3] 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 4: (MDLog::_replay_thread()+0x617) [0x557113234197] 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55711323da11] 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f7fcd0301ca] 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 7: clone() 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T16:51:04.462 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 0> 2024-05-14T16:51:04.047+0000 7f7fbbc78700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: in thread 7f7fbbc78700 thread_name:md_log_replay 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7fcd03ad20] 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x5571131d5c1f] 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5571131d93b3] 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 4: (MDLog::_replay_thread()+0x617) [0x557113234197] 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55711323da11] 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f7fcd0301ca] 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 7: clone() 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70183]: 2024-05-14T16:51:04.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 podman[70456]: 2024-05-14 16:51:04.110239291 +0000 UTC m=+0.019953041 container died b1b76cada8d68d563bc76b784912d9d52a347648c9e3ac16a997a4a24cbd23a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-14T16:51:05.177 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 podman[70456]: 2024-05-14 16:51:04.919479505 +0000 UTC m=+0.829193251 container remove b1b76cada8d68d563bc76b784912d9d52a347648c9e3ac16a997a4a24cbd23a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T16:51:05.177 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:04 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-05-14T16:51:05.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.359+0000 7f1a5a948700 1 -- 172.21.15.69:0/936859458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a541095e0 msgr2=0x7f1a541099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:05.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.359+0000 7f1a5a948700 1 --2- 172.21.15.69:0/936859458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a541095e0 0x7f1a541099c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009990 tx=0x7f1a44009ca0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.359+0000 7f1a5a948700 1 -- 172.21.15.69:0/936859458 shutdown_connections 2024-05-14T16:51:05.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.359+0000 7f1a5a948700 1 --2- 172.21.15.69:0/936859458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a541095e0 0x7f1a541099c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.359+0000 7f1a5a948700 1 --2- 172.21.15.69:0/936859458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a5406c5d0 0x7f1a54110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.359+0000 7f1a5a948700 1 --2- 172.21.15.69:0/936859458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5406bc10 0x7f1a5406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.359+0000 7f1a5a948700 1 -- 172.21.15.69:0/936859458 >> 172.21.15.69:0/936859458 conn(0x7f1a54079480 msgr2=0x7f1a54079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:05.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.359+0000 7f1a5a948700 1 -- 172.21.15.69:0/936859458 shutdown_connections 2024-05-14T16:51:05.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.359+0000 7f1a5a948700 1 -- 172.21.15.69:0/936859458 wait complete. 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.360+0000 7f1a5a948700 1 Processor -- start 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.360+0000 7f1a5a948700 1 -- start start 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.360+0000 7f1a5a948700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5406bc10 0x7f1a5419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.360+0000 7f1a5a948700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a5406c5d0 0x7f1a5419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.360+0000 7f1a5a948700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a541095e0 0x7f1a54199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.360+0000 7f1a5a948700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541136f0 con 0x7f1a541095e0 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.360+0000 7f1a5a948700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54113570 con 0x7f1a5406c5d0 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.360+0000 7f1a5a948700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54113870 con 0x7f1a5406bc10 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a58ee5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a541095e0 0x7f1a54199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a58ee5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a541095e0 0x7f1a54199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50034/0 (socket says 172.21.15.69:50034) 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a58ee5700 1 -- 172.21.15.69:0/3941765486 learned_addr learned my addr 172.21.15.69:0/3941765486 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a4bfff700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a5406c5d0 0x7f1a5419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a53fff700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5406bc10 0x7f1a5419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a58ee5700 1 -- 172.21.15.69:0/3941765486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5406bc10 msgr2=0x7f1a5419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a58ee5700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5406bc10 0x7f1a5419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a58ee5700 1 -- 172.21.15.69:0/3941765486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a5406c5d0 msgr2=0x7f1a5419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a58ee5700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a5406c5d0 0x7f1a5419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a58ee5700 1 -- 172.21.15.69:0/3941765486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5419a190 con 0x7f1a541095e0 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a58ee5700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a541095e0 0x7f1a54199a30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c007ee0 tx=0x7f1a4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a51ffb700 1 -- 172.21.15.69:0/3941765486 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c004bb0 con 0x7f1a541095e0 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a5a948700 1 -- 172.21.15.69:0/3941765486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5419a480 con 0x7f1a541095e0 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a51ffb700 1 -- 172.21.15.69:0/3941765486 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4c004d50 con 0x7f1a541095e0 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a51ffb700 1 -- 172.21.15.69:0/3941765486 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c005920 con 0x7f1a541095e0 2024-05-14T16:51:05.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.361+0000 7f1a5a948700 1 -- 172.21.15.69:0/3941765486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a541a6740 con 0x7f1a541095e0 2024-05-14T16:51:05.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.364+0000 7f1a51ffb700 1 -- 172.21.15.69:0/3941765486 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f1a4c01a020 con 0x7f1a541095e0 2024-05-14T16:51:05.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a51ffb700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a400b2b10 0x7f1a400b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a53fff700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a400b2b10 0x7f1a400b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a51ffb700 1 -- 172.21.15.69:0/3941765486 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f1a4c0e18c0 con 0x7f1a541095e0 2024-05-14T16:51:05.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a651deb80 1 Processor -- start 2024-05-14T16:51:05.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a651deb80 1 -- start start 2024-05-14T16:51:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a53fff700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a400b2b10 0x7f1a400b4fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400b2e0 tx=0x7f1a4401a040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942420 0x55e486942800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942d40 0x55e48693dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e48693e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a651deb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e48689be70 con 0x55e48693e2c0 2024-05-14T16:51:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a651deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e48688ba10 con 0x55e486942420 2024-05-14T16:51:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a651deb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e48688fa50 con 0x55e486942d40 2024-05-14T16:51:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a58ee5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e48693e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a53fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942420 0x55e486942800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a58ee5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e48693e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50044/0 (socket says 172.21.15.69:50044) 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.365+0000 7f1a58ee5700 1 -- 172.21.15.69:0/755406723 learned_addr learned my addr 172.21.15.69:0/755406723 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a4bfff700 1 --2- 172.21.15.69:0/755406723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942d40 0x55e48693dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a53fff700 1 -- 172.21.15.69:0/755406723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942d40 msgr2=0x55e48693dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a53fff700 1 --2- 172.21.15.69:0/755406723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942d40 0x55e48693dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a53fff700 1 -- 172.21.15.69:0/755406723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 msgr2=0x55e48693e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a53fff700 1 --2- 172.21.15.69:0/755406723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e48693e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a53fff700 1 -- 172.21.15.69:0/755406723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4868350b0 con 0x55e486942420 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a58ee5700 1 --2- 172.21.15.69:0/755406723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e48693e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a53fff700 1 --2- 172.21.15.69:0/755406723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942420 0x55e486942800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a440059d0 tx=0x7f1a4401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf13da9e849fb237 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a4a7fc700 1 -- 172.21.15.69:0/755406723 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400bb00 con 0x55e486942420 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a4a7fc700 1 -- 172.21.15.69:0/755406723 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4400bca0 con 0x55e486942420 2024-05-14T16:51:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.366+0000 7f1a4a7fc700 1 -- 172.21.15.69:0/755406723 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400f7e0 con 0x55e486942420 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.367+0000 7f1a651deb80 1 -- 172.21.15.69:0/755406723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942420 msgr2=0x55e486942800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.367+0000 7f1a651deb80 1 --2- 172.21.15.69:0/755406723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942420 0x55e486942800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a440059d0 tx=0x7f1a4401f000 comp rx=0 tx=0).stop 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.367+0000 7f1a651deb80 1 -- 172.21.15.69:0/755406723 shutdown_connections 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.367+0000 7f1a651deb80 1 --2- 172.21.15.69:0/755406723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e48693e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.367+0000 7f1a651deb80 1 --2- 172.21.15.69:0/755406723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942420 0x55e486942800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.367+0000 7f1a651deb80 1 --2- 172.21.15.69:0/755406723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942d40 0x55e48693dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.367+0000 7f1a651deb80 1 -- 172.21.15.69:0/755406723 >> 172.21.15.69:0/755406723 conn(0x55e48686aeb0 msgr2=0x55e4869384a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.367+0000 7f1a651deb80 1 -- 172.21.15.69:0/755406723 shutdown_connections 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.367+0000 7f1a651deb80 1 -- 172.21.15.69:0/755406723 wait complete. 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.367+0000 7f1a651deb80 1 -- start start 2024-05-14T16:51:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a651deb80 1 Processor -- start 2024-05-14T16:51:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e48693e2c0 0x55e486949b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942420 0x55e486946ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942d40 0x55e4869473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a651deb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e4867920a0 con 0x55e486942d40 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a651deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e4867f3ea0 con 0x55e48693e2c0 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a651deb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e48694a7c0 con 0x55e486942420 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a651deb80 10 client.0 _ll_register_callbacks cb 0x55e486826a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a58ee5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942d40 0x55e4869473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a58ee5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942d40 0x55e4869473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50052/0 (socket says 172.21.15.69:50052) 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a58ee5700 1 -- 172.21.15.69:0/2448957321 learned_addr learned my addr 172.21.15.69:0/2448957321 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a53fff700 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e48693e2c0 0x55e486949b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a4bfff700 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942420 0x55e486946ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a58ee5700 1 -- 172.21.15.69:0/2448957321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942420 msgr2=0x55e486946ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a58ee5700 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942420 0x55e486946ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a58ee5700 1 -- 172.21.15.69:0/2448957321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e48693e2c0 msgr2=0x55e486949b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a58ee5700 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e48693e2c0 0x55e486949b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a58ee5700 1 -- 172.21.15.69:0/2448957321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4867db9d0 con 0x55e486942d40 2024-05-14T16:51:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a58ee5700 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942d40 0x55e4869473e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0defb0 tx=0x7f1a4c0b15b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded76c3026a58c97 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.368+0000 7f1a337fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a337fe700 1 -- 172.21.15.69:0/2448957321 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0a40f0 con 0x55e486942d40 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a337fe700 1 -- 172.21.15.69:0/2448957321 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f1a4c004f00 con 0x55e486942d40 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a651deb80 1 -- 172.21.15.69:0/2448957321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a4400bca0 con 0x55e486942d40 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a337fe700 1 -- 172.21.15.69:0/2448957321 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a4c0a8730 con 0x55e486942d40 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a651deb80 10 client.15489 fetch_fsmap learned FSMap version 29 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a651deb80 1 -- 172.21.15.69:0/2448957321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e486845860 con 0x55e486942d40 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a337fe700 1 -- 172.21.15.69:0/2448957321 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4400bca0 con 0x55e486942d40 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a337fe700 1 -- 172.21.15.69:0/2448957321 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f1a4c06e150 con 0x55e486942d40 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a651deb80 10 client.15489 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a651deb80 10 client.15489 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a651deb80 20 client.15489 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e486957300 0x55e4869596f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a651deb80 4 client.15489 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a651deb80 1 -- 172.21.15.69:0/2448957321 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e48689ddc0 con 0x55e486957300 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.369+0000 7f1a53fff700 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e486957300 0x55e4869596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.370+0000 7f1a53fff700 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e486957300 0x55e4869596f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ede796cc473486b server_cookie=f7046dbb2b727a37 in_seq=0 out_seq=0 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.370+0000 7f1a337fe700 10 client.15489 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.370+0000 7f1a337fe700 1 -- 172.21.15.69:0/2448957321 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e48689ddc0 con 0x55e486957300 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.370+0000 7f1a337fe700 10 client.15489 handle_command_reply: tid=0 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.370+0000 7f1a651deb80 1 client.15489 shutdown 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 1 -- 172.21.15.69:0/2448957321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942d40 msgr2=0x55e4869473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942d40 0x55e4869473e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0defb0 tx=0x7f1a4c0b15b0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 1 -- 172.21.15.69:0/2448957321 shutdown_connections 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e486957300 0x55e4869596f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942d40 0x55e4869473e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e48693e2c0 0x55e486949b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2448957321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942420 0x55e486946ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 1 -- 172.21.15.69:0/2448957321 >> 172.21.15.69:0/2448957321 conn(0x55e48686aeb0 msgr2=0x55e486932fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:05.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a337fe700 0 client.15489 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:05.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 1 -- 172.21.15.69:0/2448957321 shutdown_connections 2024-05-14T16:51:05.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 1 -- 172.21.15.69:0/2448957321 wait complete. 2024-05-14T16:51:05.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.371+0000 7f1a651deb80 20 client.15489 trim_cache size 0 max 16384 2024-05-14T16:51:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.390+0000 7f1a651deb80 1 Processor -- start 2024-05-14T16:51:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.390+0000 7f1a651deb80 1 -- start start 2024-05-14T16:51:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e48693e2c0 0x55e48693d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942420 0x55e486946040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 0x55e4869330d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a651deb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541b06c0 con 0x55e486942420 2024-05-14T16:51:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a651deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541adec0 con 0x55e486942d40 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a651deb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541abd10 con 0x55e48693e2c0 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a4bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942420 0x55e486946040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a4bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942420 0x55e486946040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50064/0 (socket says 172.21.15.69:50064) 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a4bfff700 1 -- 172.21.15.69:0/43029538 learned_addr learned my addr 172.21.15.69:0/43029538 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a53fff700 1 --2- 172.21.15.69:0/43029538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e48693e2c0 0x55e48693d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a58ee5700 1 --2- 172.21.15.69:0/43029538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 0x55e4869330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a4bfff700 1 -- 172.21.15.69:0/43029538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e48693e2c0 msgr2=0x55e48693d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a4bfff700 1 --2- 172.21.15.69:0/43029538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e48693e2c0 0x55e48693d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a4bfff700 1 -- 172.21.15.69:0/43029538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 msgr2=0x55e4869330d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a4bfff700 1 --2- 172.21.15.69:0/43029538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 0x55e4869330d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a4bfff700 1 -- 172.21.15.69:0/43029538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e486947780 con 0x55e486942420 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a4bfff700 1 --2- 172.21.15.69:0/43029538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942420 0x55e486946040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c80 tx=0x7f1a3c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b853b91b1fef5eb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a49ffb700 1 -- 172.21.15.69:0/43029538 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c022070 con 0x55e486942420 2024-05-14T16:51:05.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a49ffb700 1 -- 172.21.15.69:0/43029538 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a3c004050 con 0x55e486942420 2024-05-14T16:51:05.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.391+0000 7f1a49ffb700 1 -- 172.21.15.69:0/43029538 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c003be0 con 0x55e486942420 2024-05-14T16:51:05.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.392+0000 7f1a651deb80 1 -- 172.21.15.69:0/43029538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942420 msgr2=0x55e486946040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.392+0000 7f1a651deb80 1 --2- 172.21.15.69:0/43029538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942420 0x55e486946040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c80 tx=0x7f1a3c009260 comp rx=0 tx=0).stop 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.392+0000 7f1a651deb80 1 -- 172.21.15.69:0/43029538 shutdown_connections 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.392+0000 7f1a651deb80 1 --2- 172.21.15.69:0/43029538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 0x55e4869330d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.392+0000 7f1a651deb80 1 --2- 172.21.15.69:0/43029538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e48693e2c0 0x55e48693d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.392+0000 7f1a651deb80 1 --2- 172.21.15.69:0/43029538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e486942420 0x55e486946040 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.392+0000 7f1a651deb80 1 -- 172.21.15.69:0/43029538 >> 172.21.15.69:0/43029538 conn(0x55e48686aeb0 msgr2=0x55e4869410e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.392+0000 7f1a651deb80 1 -- 172.21.15.69:0/43029538 shutdown_connections 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.392+0000 7f1a651deb80 1 -- 172.21.15.69:0/43029538 wait complete. 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.392+0000 7f1a651deb80 1 -- start start 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a651deb80 1 Processor -- start 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e486944bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942420 0x55e4869431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a651deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 0x55e486943910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a651deb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541ac380 con 0x55e48693e2c0 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a651deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541ad7f0 con 0x55e486942d40 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a651deb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541aca50 con 0x55e486942420 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a651deb80 10 client.0 _ll_register_callbacks cb 0x55e486826a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a53fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e486944bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a53fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e486944bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50066/0 (socket says 172.21.15.69:50066) 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a53fff700 1 -- 172.21.15.69:0/2509953346 learned_addr learned my addr 172.21.15.69:0/2509953346 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a58ee5700 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 0x55e486943910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a4bfff700 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942420 0x55e4869431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a53fff700 1 -- 172.21.15.69:0/2509953346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942420 msgr2=0x55e4869431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a53fff700 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942420 0x55e4869431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a53fff700 1 -- 172.21.15.69:0/2509953346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 msgr2=0x55e486943910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a53fff700 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 0x55e486943910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a53fff700 1 -- 172.21.15.69:0/2509953346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4869c5bf0 con 0x55e48693e2c0 2024-05-14T16:51:05.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.393+0000 7f1a58ee5700 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 0x55e486943910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a53fff700 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e486944bf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009990 tx=0x7f1a44004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb39e1f7fd94781 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a337fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a337fe700 1 -- 172.21.15.69:0/2509953346 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44003890 con 0x55e48693e2c0 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a337fe700 1 -- 172.21.15.69:0/2509953346 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8498+0+0 (secure 0 0 0) 0x7f1a44003a30 con 0x55e48693e2c0 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a651deb80 1 -- 172.21.15.69:0/2509953346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a3c004050 con 0x55e48693e2c0 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a337fe700 1 -- 172.21.15.69:0/2509953346 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a4404c050 con 0x55e48693e2c0 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a651deb80 10 client.15495 fetch_fsmap learned FSMap version 29 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a651deb80 1 -- 172.21.15.69:0/2509953346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4869c6e50 con 0x55e48693e2c0 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a337fe700 1 -- 172.21.15.69:0/2509953346 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a3c004050 con 0x55e48693e2c0 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a337fe700 1 -- 172.21.15.69:0/2509953346 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f1a4403e920 con 0x55e48693e2c0 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a651deb80 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a651deb80 10 client.15495 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a651deb80 20 client.15495 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e4869c70d0 0x55e4869c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:05.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a651deb80 4 client.15495 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "44cae421", "release": true, "await": true}] 2024-05-14T16:51:05.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.394+0000 7f1a651deb80 1 -- 172.21.15.69:0/2509953346 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "44cae421", "release": true, "await": true}) -- 0x55e4868975a0 con 0x55e4869c70d0 2024-05-14T16:51:05.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.395+0000 7f1a4bfff700 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e4869c70d0 0x55e4869c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:05.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.395+0000 7f1a4bfff700 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e4869c70d0 0x55e4869c94c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7024e12449e07d54 server_cookie=243fff25ba1099d2 in_seq=0 out_seq=0 2024-05-14T16:51:05.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.395+0000 7f1a337fe700 10 client.15495 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:05.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 35.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:51:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a337fe700 1 -- 172.21.15.69:0/2509953346 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e4868975a0 con 0x55e4869c70d0 2024-05-14T16:51:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a337fe700 10 client.15495 handle_command_reply: tid=0 2024-05-14T16:51:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 client.15495 shutdown 2024-05-14T16:51:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 -- 172.21.15.69:0/2509953346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 msgr2=0x55e486944bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e486944bf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009990 tx=0x7f1a44004770 comp rx=0 tx=0).stop 2024-05-14T16:51:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 -- 172.21.15.69:0/2509953346 shutdown_connections 2024-05-14T16:51:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e4869c70d0 0x55e4869c94c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e48693e2c0 0x55e486944bf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a337fe700 0 client.15495 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e486942d40 0x55e486943910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 --2- 172.21.15.69:0/2509953346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e486942420 0x55e4869431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 -- 172.21.15.69:0/2509953346 >> 172.21.15.69:0/2509953346 conn(0x55e48686aeb0 msgr2=0x55e486941970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:05.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 -- 172.21.15.69:0/2509953346 shutdown_connections 2024-05-14T16:51:05.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 1 -- 172.21.15.69:0/2509953346 wait complete. 2024-05-14T16:51:05.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.434+0000 7f1a651deb80 20 client.15495 trim_cache size 0 max 16384 2024-05-14T16:51:05.435 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:51:05.435 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 29, 2024-05-14T16:51:05.435 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 4, 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "44cae421": { 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 4, 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 135.2, 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:51:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:51:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:51:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:51:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:51:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:51:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:51:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:51:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:51:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:51:05.437 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 -- 172.21.15.69:0/3941765486 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a400b2b10 msgr2=0x7f1a400b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a400b2b10 0x7f1a400b4fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400b2e0 tx=0x7f1a4401a040 comp rx=0 tx=0).stop 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 -- 172.21.15.69:0/3941765486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a541095e0 msgr2=0x7f1a54199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a541095e0 0x7f1a54199a30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c007ee0 tx=0x7f1a4c00d3b0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 -- 172.21.15.69:0/3941765486 shutdown_connections 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a400b2b10 0x7f1a400b4fd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a541095e0 0x7f1a54199a30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5406bc10 0x7f1a5419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 --2- 172.21.15.69:0/3941765486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a5406c5d0 0x7f1a5419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 -- 172.21.15.69:0/3941765486 >> 172.21.15.69:0/3941765486 conn(0x7f1a54079480 msgr2=0x7f1a54101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:05.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 -- 172.21.15.69:0/3941765486 shutdown_connections 2024-05-14T16:51:05.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:05.436+0000 7f1a5a948700 1 -- 172.21.15.69:0/3941765486 wait complete. 2024-05-14T16:51:05.459 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:05 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-14T16:51:06.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:05 smithi092 ceph-mon[35763]: pgmap v39: 97 pgs: 97 active+clean; 1020 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 14 MiB/s wr, 296 op/s 2024-05-14T16:51:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:05 smithi069 ceph-mon[31507]: pgmap v39: 97 pgs: 97 active+clean; 1020 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 14 MiB/s wr, 296 op/s 2024-05-14T16:51:06.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:05 smithi203 ceph-mon[35787]: pgmap v39: 97 pgs: 97 active+clean; 1020 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 14 MiB/s wr, 296 op/s 2024-05-14T16:51:06.438 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '44cae421', total seconds elapsed: 21.6 2024-05-14T16:51:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:07 smithi092 ceph-mon[35763]: pgmap v40: 97 pgs: 97 active+clean; 958 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 15 MiB/s wr, 346 op/s 2024-05-14T16:51:08.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:07 smithi069 ceph-mon[31507]: pgmap v40: 97 pgs: 97 active+clean; 958 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 15 MiB/s wr, 346 op/s 2024-05-14T16:51:08.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:07 smithi203 ceph-mon[35787]: pgmap v40: 97 pgs: 97 active+clean; 958 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 15 MiB/s wr, 346 op/s 2024-05-14T16:51:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:09 smithi092 ceph-mon[35763]: pgmap v41: 97 pgs: 97 active+clean; 958 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 233 op/s 2024-05-14T16:51:10.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:09 smithi069 ceph-mon[31507]: pgmap v41: 97 pgs: 97 active+clean; 958 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 233 op/s 2024-05-14T16:51:10.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:09 smithi203 ceph-mon[35787]: pgmap v41: 97 pgs: 97 active+clean; 958 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 233 op/s 2024-05-14T16:51:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 40.0 s, 2961.3 tps, lat 131.478 ms stddev 1187.255 2024-05-14T16:51:12.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:11 smithi092 ceph-mon[35763]: pgmap v42: 97 pgs: 97 active+clean; 988 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 853 op/s 2024-05-14T16:51:12.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:11 smithi069 ceph-mon[31507]: pgmap v42: 97 pgs: 97 active+clean; 988 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 853 op/s 2024-05-14T16:51:12.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:11 smithi203 ceph-mon[35787]: pgmap v42: 97 pgs: 97 active+clean; 988 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 853 op/s 2024-05-14T16:51:14.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:13 smithi092 ceph-mon[35763]: pgmap v43: 97 pgs: 97 active+clean; 988 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 670 op/s 2024-05-14T16:51:14.009 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:13 smithi203 ceph-mon[35787]: pgmap v43: 97 pgs: 97 active+clean; 988 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 670 op/s 2024-05-14T16:51:14.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:13 smithi069 ceph-mon[31507]: pgmap v43: 97 pgs: 97 active+clean; 988 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 670 op/s 2024-05-14T16:51:15.317 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 2. 2024-05-14T16:51:15.317 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 systemd[1]: Stopped Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:51:15.317 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 systemd[1]: Starting Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T16:51:15.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 45.0 s, 3734.2 tps, lat 26.711 ms stddev 26.457 2024-05-14T16:51:15.576 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 podman[70551]: 2024-05-14 16:51:15.340898369 +0000 UTC m=+0.018681817 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:51:15.577 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 podman[70551]: 2024-05-14 16:51:15.454599366 +0000 UTC m=+0.132382802 container create d649402271e5a1d7485b2a2f5894f22907eda364d08f4c4f70fd0d756625e4cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-14T16:51:15.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:15 smithi203 ceph-mon[35787]: pgmap v44: 97 pgs: 97 active+clean; 988 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 670 op/s 2024-05-14T16:51:15.959 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 podman[70551]: 2024-05-14 16:51:15.708056465 +0000 UTC m=+0.385839906 container init d649402271e5a1d7485b2a2f5894f22907eda364d08f4c4f70fd0d756625e4cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-14T16:51:15.959 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 podman[70551]: 2024-05-14 16:51:15.710918551 +0000 UTC m=+0.388701990 container start d649402271e5a1d7485b2a2f5894f22907eda364d08f4c4f70fd0d756625e4cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-14T16:51:15.959 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 ceph-mds[70577]: -- 172.21.15.203:0/410613677 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56355d0efa40 con 0x56355d0c3000 2024-05-14T16:51:15.960 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70573]: starting mds.c at 2024-05-14T16:51:15.960 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 bash[70551]: d649402271e5a1d7485b2a2f5894f22907eda364d08f4c4f70fd0d756625e4cc 2024-05-14T16:51:15.960 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 16:51:15 smithi203 systemd[1]: Started Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T16:51:16.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:15 smithi092 ceph-mon[35763]: pgmap v44: 97 pgs: 97 active+clean; 988 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 670 op/s 2024-05-14T16:51:16.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:15 smithi069 ceph-mon[31507]: pgmap v44: 97 pgs: 97 active+clean; 988 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 670 op/s 2024-05-14T16:51:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.765+0000 7f4808b9d640 1 -- 172.21.15.69:0/3349137032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48041011c0 msgr2=0x7f48041015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.765+0000 7f4808b9d640 1 --2- 172.21.15.69:0/3349137032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48041011c0 0x7f48041015a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009900 tx=0x7f47f402a410 comp rx=0 tx=0).stop 2024-05-14T16:51:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.765+0000 7f4808b9d640 1 -- 172.21.15.69:0/3349137032 shutdown_connections 2024-05-14T16:51:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.765+0000 7f4808b9d640 1 --2- 172.21.15.69:0/3349137032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4804107e30 0x7f480410f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.765+0000 7f4808b9d640 1 --2- 172.21.15.69:0/3349137032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4804101b70 0x7f4804107700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.765+0000 7f4808b9d640 1 --2- 172.21.15.69:0/3349137032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48041011c0 0x7f48041015a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.765+0000 7f4808b9d640 1 -- 172.21.15.69:0/3349137032 >> 172.21.15.69:0/3349137032 conn(0x7f4804077ee0 msgr2=0x7f48040ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.765+0000 7f4808b9d640 1 -- 172.21.15.69:0/3349137032 shutdown_connections 2024-05-14T16:51:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.765+0000 7f4808b9d640 1 -- 172.21.15.69:0/3349137032 wait complete. 2024-05-14T16:51:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.766+0000 7f4808b9d640 1 Processor -- start 2024-05-14T16:51:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.766+0000 7f4808b9d640 1 -- start start 2024-05-14T16:51:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.766+0000 7f4808b9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48041011c0 0x7f4804199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.766+0000 7f4808b9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4804107e30 0x7f4804199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.766+0000 7f4808b9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f480419dad0 0x7f480419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.766+0000 7f4808b9d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4804112d90 con 0x7f480419dad0 2024-05-14T16:51:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.766+0000 7f4808b9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4804112c10 con 0x7f48041011c0 2024-05-14T16:51:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.766+0000 7f4808b9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4804112f10 con 0x7f4804107e30 2024-05-14T16:51:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f4802d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f480419dad0 0x7f480419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f480259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48041011c0 0x7f4804199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f4802d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f480419dad0 0x7f480419df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34418/0 (socket says 172.21.15.69:34418) 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f4802d9d640 1 -- 172.21.15.69:0/3877491054 learned_addr learned my addr 172.21.15.69:0/3877491054 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f47fbfff640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4804107e30 0x7f4804199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f480259c640 1 -- 172.21.15.69:0/3877491054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4804107e30 msgr2=0x7f4804199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f480259c640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4804107e30 0x7f4804199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f480259c640 1 -- 172.21.15.69:0/3877491054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f480419dad0 msgr2=0x7f480419df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f480259c640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f480419dad0 0x7f480419df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f480259c640 1 -- 172.21.15.69:0/3877491054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480419e5c0 con 0x7f48041011c0 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f4802d9d640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f480419dad0 0x7f480419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f47fbfff640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4804107e30 0x7f4804199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:51:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.767+0000 7f480259c640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48041011c0 0x7f4804199210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47f402aa00 tx=0x7f47f4003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.769+0000 7f47fb7fe640 1 -- 172.21.15.69:0/3877491054 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f403f070 con 0x7f48041011c0 2024-05-14T16:51:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.769+0000 7f47fb7fe640 1 -- 172.21.15.69:0/3877491054 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f4031b50 con 0x7f48041011c0 2024-05-14T16:51:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.769+0000 7f47fb7fe640 1 -- 172.21.15.69:0/3877491054 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4035030 con 0x7f48041011c0 2024-05-14T16:51:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.769+0000 7f4808b9d640 1 -- 172.21.15.69:0/3877491054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480419e850 con 0x7f48041011c0 2024-05-14T16:51:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.769+0000 7f4808b9d640 1 -- 172.21.15.69:0/3877491054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48041a5f60 con 0x7f48041011c0 2024-05-14T16:51:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.772+0000 7f480b98d740 1 Processor -- start 2024-05-14T16:51:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.773+0000 7f47fb7fe640 1 -- 172.21.15.69:0/3877491054 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f47f4042070 con 0x7f48041011c0 2024-05-14T16:51:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.773+0000 7f480b98d740 1 -- start start 2024-05-14T16:51:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.773+0000 7f480b98d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 0x55f9b7a8abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.773+0000 7f480b98d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7a999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.773+0000 7f480b98d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7a9c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.773+0000 7f480b98d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f9b79fdc30 con 0x55f9b7a99ee0 2024-05-14T16:51:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.773+0000 7f480b98d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f9b79ed7d0 con 0x55f9b7a8a810 2024-05-14T16:51:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.773+0000 7f480b98d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f9b79f1810 con 0x55f9b7a8b130 2024-05-14T16:51:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f4802d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 0x55f9b7a8abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f4802d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 0x55f9b7a8abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:56376/0 (socket says 172.21.15.69:56376) 2024-05-14T16:51:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f4802d9d640 1 -- 172.21.15.69:0/2294451167 learned_addr learned my addr 172.21.15.69:0/2294451167 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f4802d9d640 1 -- 172.21.15.69:0/2294451167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 msgr2=0x55f9b7a999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f4802d9d640 1 --2- 172.21.15.69:0/2294451167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7a999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f4802d9d640 1 -- 172.21.15.69:0/2294451167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 msgr2=0x55f9b7a9c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f4802d9d640 1 --2- 172.21.15.69:0/2294451167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7a9c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f4802d9d640 1 -- 172.21.15.69:0/2294451167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b7702210 con 0x55f9b7a8a810 2024-05-14T16:51:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f4802d9d640 1 --2- 172.21.15.69:0/2294451167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 0x55f9b7a8abf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47fc011850 tx=0x7f47fc011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b62b0194aa407c1f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f47f8ff9640 1 -- 172.21.15.69:0/2294451167 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc011330 con 0x55f9b7a8a810 2024-05-14T16:51:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f47f8ff9640 1 -- 172.21.15.69:0/2294451167 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47fc00fe50 con 0x55f9b7a8a810 2024-05-14T16:51:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.774+0000 7f47f8ff9640 1 -- 172.21.15.69:0/2294451167 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0193e0 con 0x55f9b7a8a810 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.775+0000 7f47fb7fe640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47f00b2d40 0x7f47f00b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.775+0000 7f47fb7fe640 1 -- 172.21.15.69:0/3877491054 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f47f4102d20 con 0x7f48041011c0 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.775+0000 7f480b98d740 1 -- 172.21.15.69:0/2294451167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 msgr2=0x55f9b7a8abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.775+0000 7f480b98d740 1 --2- 172.21.15.69:0/2294451167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 0x55f9b7a8abf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47fc011850 tx=0x7f47fc011d10 comp rx=0 tx=0).stop 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.776+0000 7f480b98d740 1 -- 172.21.15.69:0/2294451167 shutdown_connections 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.776+0000 7f480b98d740 1 --2- 172.21.15.69:0/2294451167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7a9c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.776+0000 7f480b98d740 1 --2- 172.21.15.69:0/2294451167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7a999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.776+0000 7f480b98d740 1 --2- 172.21.15.69:0/2294451167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 0x55f9b7a8abf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.776+0000 7f480b98d740 1 -- 172.21.15.69:0/2294451167 >> 172.21.15.69:0/2294451167 conn(0x55f9b7a86e00 msgr2=0x55f9b7a871e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.776+0000 7f480259c640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47f00b2d40 0x7f47f00b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.776+0000 7f480b98d740 1 -- 172.21.15.69:0/2294451167 shutdown_connections 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.776+0000 7f480b98d740 1 -- 172.21.15.69:0/2294451167 wait complete. 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.777+0000 7f480b98d740 1 -- start start 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.777+0000 7f480259c640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47f00b2d40 0x7f47f00b5200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f47f4100300 tx=0x7f47f4031f90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.777+0000 7f480b98d740 1 Processor -- start 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.777+0000 7f480b98d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 0x55f9b7a99840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.777+0000 7f480b98d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7a96b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.777+0000 7f480b98d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7a97090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.777+0000 7f480b98d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f9b76d3790 con 0x55f9b7a99ee0 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.777+0000 7f480b98d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f9b79a9890 con 0x55f9b7a8a810 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.777+0000 7f480b98d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f9b7a9cbb0 con 0x55f9b7a8b130 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.777+0000 7f480b98d740 10 client.0 _ll_register_callbacks cb 0x55f9b79acd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.778+0000 7f4802d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7a97090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.778+0000 7f4802d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7a97090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34434/0 (socket says 172.21.15.69:34434) 2024-05-14T16:51:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.778+0000 7f4802d9d640 1 -- 172.21.15.69:0/4273806958 learned_addr learned my addr 172.21.15.69:0/4273806958 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.778+0000 7f47fbfff640 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7a96b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.778+0000 7f4802d9d640 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 0x55f9b7a99840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.778+0000 7f4802d9d640 1 -- 172.21.15.69:0/4273806958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 msgr2=0x55f9b7a96b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.778+0000 7f4802d9d640 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7a96b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.778+0000 7f4802d9d640 1 -- 172.21.15.69:0/4273806958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 msgr2=0x55f9b7a99840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.778+0000 7f4802d9d640 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 0x55f9b7a99840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.778+0000 7f4802d9d640 1 -- 172.21.15.69:0/4273806958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b77ce620 con 0x55f9b7a99ee0 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.779+0000 7f4802d9d640 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7a97090 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f47fc011660 tx=0x7f47fc009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a100f99a0dc1af server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.779+0000 7f47e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.779+0000 7f47e9ffb640 1 -- 172.21.15.69:0/4273806958 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc02d060 con 0x55f9b7a99ee0 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.779+0000 7f480b98d740 1 -- 172.21.15.69:0/4273806958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47fc00fe50 con 0x55f9b7a99ee0 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.779+0000 7f47e9ffb640 1 -- 172.21.15.69:0/4273806958 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f47fc010870 con 0x55f9b7a99ee0 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.780+0000 7f47e9ffb640 1 -- 172.21.15.69:0/4273806958 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f47fc031450 con 0x55f9b7a99ee0 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.780+0000 7f480b98d740 10 client.15510 fetch_fsmap learned FSMap version 31 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.780+0000 7f480b98d740 1 -- 172.21.15.69:0/4273806958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f9b7aa85d0 con 0x55f9b7a99ee0 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.781+0000 7f47e9ffb640 1 -- 172.21.15.69:0/4273806958 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47fc00fe50 con 0x55f9b7a99ee0 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.781+0000 7f47e9ffb640 1 -- 172.21.15.69:0/4273806958 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f47fc018070 con 0x55f9b7a99ee0 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.781+0000 7f480b98d740 10 client.15510 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.781+0000 7f480b98d740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.781+0000 7f480b98d740 20 client.15510 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.781+0000 7f480b98d740 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f9b7aa9a30 0x55f9b7aabe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.781+0000 7f480b98d740 4 client.15510 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.781+0000 7f480b98d740 1 -- 172.21.15.69:0/4273806958 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b79ffb80 con 0x55f9b7aa9a30 2024-05-14T16:51:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.782+0000 7f47fbfff640 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f9b7aa9a30 0x55f9b7aabe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.782+0000 7f47fbfff640 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f9b7aa9a30 0x55f9b7aabe20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e090564745e2fe01 server_cookie=6996a96e9e4239fe in_seq=0 out_seq=0 2024-05-14T16:51:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.782+0000 7f47e9ffb640 10 client.15510 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f47e9ffb640 1 -- 172.21.15.69:0/4273806958 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9b79ffb80 con 0x55f9b7aa9a30 2024-05-14T16:51:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f47e9ffb640 10 client.15510 handle_command_reply: tid=0 2024-05-14T16:51:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f480b98d740 1 client.15510 shutdown 2024-05-14T16:51:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f480b98d740 1 -- 172.21.15.69:0/4273806958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 msgr2=0x55f9b7a97090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f480b98d740 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7a97090 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f47fc011660 tx=0x7f47fc009a30 comp rx=0 tx=0).stop 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f480b98d740 1 -- 172.21.15.69:0/4273806958 shutdown_connections 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f480b98d740 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f9b7aa9a30 0x55f9b7aabe20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f47e9ffb640 0 client.15510 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f480b98d740 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7a97090 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f480b98d740 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7a96b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f480b98d740 1 --2- 172.21.15.69:0/4273806958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8a810 0x55f9b7a99840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f480b98d740 1 -- 172.21.15.69:0/4273806958 >> 172.21.15.69:0/4273806958 conn(0x55f9b7a86e00 msgr2=0x55f9b7a8cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.783+0000 7f480b98d740 1 -- 172.21.15.69:0/4273806958 shutdown_connections 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.784+0000 7f480b98d740 1 -- 172.21.15.69:0/4273806958 wait complete. 2024-05-14T16:51:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.784+0000 7f480b98d740 20 client.15510 trim_cache size 0 max 16384 2024-05-14T16:51:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.806+0000 7f480b98d740 1 Processor -- start 2024-05-14T16:51:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.806+0000 7f480b98d740 1 -- start start 2024-05-14T16:51:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f480b98d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a8a810 0x55f9b7a99370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f480b98d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7ab15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f480b98d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a99ee0 0x55f9b7ab1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f480b98d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f48041aed60 con 0x55f9b7a8a810 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f480b98d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f48041ac5a0 con 0x55f9b7a99ee0 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f480b98d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f48041aa410 con 0x55f9b7a8b130 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f4802d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a8a810 0x55f9b7a99370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f47fbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7ab15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f4802d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a8a810 0x55f9b7a99370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34442/0 (socket says 172.21.15.69:34442) 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f4802d9d640 1 -- 172.21.15.69:0/829201676 learned_addr learned my addr 172.21.15.69:0/829201676 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f4802d9d640 1 --2- 172.21.15.69:0/829201676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a99ee0 0x55f9b7ab1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f47fbfff640 1 -- 172.21.15.69:0/829201676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a99ee0 msgr2=0x55f9b7ab1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f47fbfff640 1 --2- 172.21.15.69:0/829201676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a99ee0 0x55f9b7ab1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f47fbfff640 1 -- 172.21.15.69:0/829201676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a8a810 msgr2=0x55f9b7a99370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f47fbfff640 1 --2- 172.21.15.69:0/829201676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a8a810 0x55f9b7a99370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f47fbfff640 1 -- 172.21.15.69:0/829201676 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b7a86c30 con 0x55f9b7a8b130 2024-05-14T16:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.807+0000 7f4802d9d640 1 --2- 172.21.15.69:0/829201676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a99ee0 0x55f9b7ab1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:51:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.808+0000 7f47fbfff640 1 --2- 172.21.15.69:0/829201676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7ab15e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47ec004700 tx=0x7f47ec0129c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f372a3c608fdeb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.808+0000 7f47ebfff640 1 -- 172.21.15.69:0/829201676 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec01f430 con 0x55f9b7a8b130 2024-05-14T16:51:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.808+0000 7f47ebfff640 1 -- 172.21.15.69:0/829201676 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ec024070 con 0x55f9b7a8b130 2024-05-14T16:51:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.808+0000 7f47ebfff640 1 -- 172.21.15.69:0/829201676 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec003bc0 con 0x55f9b7a8b130 2024-05-14T16:51:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.809+0000 7f480b98d740 1 -- 172.21.15.69:0/829201676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 msgr2=0x55f9b7ab15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.809+0000 7f480b98d740 1 --2- 172.21.15.69:0/829201676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7ab15e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47ec004700 tx=0x7f47ec0129c0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.809+0000 7f480b98d740 1 -- 172.21.15.69:0/829201676 shutdown_connections 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.809+0000 7f480b98d740 1 --2- 172.21.15.69:0/829201676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a99ee0 0x55f9b7ab1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.809+0000 7f480b98d740 1 --2- 172.21.15.69:0/829201676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8b130 0x55f9b7ab15e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.809+0000 7f480b98d740 1 --2- 172.21.15.69:0/829201676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a8a810 0x55f9b7a99370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.809+0000 7f480b98d740 1 -- 172.21.15.69:0/829201676 >> 172.21.15.69:0/829201676 conn(0x55f9b7a86e00 msgr2=0x55f9b7a921d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.809+0000 7f480b98d740 1 -- 172.21.15.69:0/829201676 shutdown_connections 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.809+0000 7f480b98d740 1 -- 172.21.15.69:0/829201676 wait complete. 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.809+0000 7f480b98d740 1 -- start start 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f480b98d740 1 Processor -- start 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f480b98d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8a810 0x55f9b7a9f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f480b98d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8b130 0x55f9b7a9fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f480b98d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7aa0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f480b98d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f48041aaa60 con 0x55f9b7a99ee0 2024-05-14T16:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f480b98d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f48041abed0 con 0x55f9b7a8b130 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f480b98d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f48041ab130 con 0x55f9b7a8a810 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f480b98d740 10 client.0 _ll_register_callbacks cb 0x7f47fc00fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f4802d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7aa0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f4802d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7aa0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34452/0 (socket says 172.21.15.69:34452) 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f4802d9d640 1 -- 172.21.15.69:0/310776453 learned_addr learned my addr 172.21.15.69:0/310776453 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f4802d9d640 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8a810 0x55f9b7a9f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f47fbfff640 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8b130 0x55f9b7a9fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f4802d9d640 1 -- 172.21.15.69:0/310776453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8a810 msgr2=0x55f9b7a9f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f4802d9d640 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8a810 0x55f9b7a9f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f4802d9d640 1 -- 172.21.15.69:0/310776453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8b130 msgr2=0x55f9b7a9fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f4802d9d640 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8b130 0x55f9b7a9fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.810+0000 7f4802d9d640 1 -- 172.21.15.69:0/310776453 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b7a91310 con 0x55f9b7a99ee0 2024-05-14T16:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f4802d9d640 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7aa0350 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009600 tx=0x7f47fc011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae458c70f6fb2b9d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f47e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f47e9ffb640 1 -- 172.21.15.69:0/310776453 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc02d040 con 0x55f9b7a99ee0 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f47e9ffb640 1 -- 172.21.15.69:0/310776453 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f47fc010c00 con 0x55f9b7a99ee0 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f480b98d740 1 -- 172.21.15.69:0/310776453 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47ec024070 con 0x55f9b7a99ee0 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f47e9ffb640 1 -- 172.21.15.69:0/310776453 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f47fc0049d0 con 0x55f9b7a99ee0 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f480b98d740 10 client.15516 fetch_fsmap learned FSMap version 31 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f480b98d740 1 -- 172.21.15.69:0/310776453 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f9b7a92620 con 0x55f9b7a99ee0 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f47e9ffb640 1 -- 172.21.15.69:0/310776453 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ec024070 con 0x55f9b7a99ee0 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f47e9ffb640 1 -- 172.21.15.69:0/310776453 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f47fc018040 con 0x55f9b7a99ee0 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f480b98d740 10 client.15516 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f480b98d740 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f480b98d740 20 client.15516 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f480b98d740 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f9b7aa9a30 0x55f9b7b1ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f480b98d740 4 client.15516 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:51:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.811+0000 7f480b98d740 1 -- 172.21.15.69:0/310776453 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9b79f9360 con 0x55f9b7aa9a30 2024-05-14T16:51:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.812+0000 7f47fbfff640 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f9b7aa9a30 0x55f9b7b1ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.812+0000 7f47e9ffb640 10 client.15516 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.812+0000 7f47fbfff640 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f9b7aa9a30 0x55f9b7b1ade0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded8f78c7167e930 server_cookie=a1d0fd9cb832d65 in_seq=0 out_seq=0 2024-05-14T16:51:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.841+0000 7f47e9ffb640 1 -- 172.21.15.69:0/310776453 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f9b79f9360 con 0x55f9b7aa9a30 2024-05-14T16:51:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.841+0000 7f47e9ffb640 10 client.15516 handle_command_reply: tid=0 2024-05-14T16:51:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.842+0000 7f480b98d740 1 client.15516 shutdown 2024-05-14T16:51:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.843+0000 7f480b98d740 1 -- 172.21.15.69:0/310776453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 msgr2=0x55f9b7aa0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.843+0000 7f480b98d740 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7aa0350 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009600 tx=0x7f47fc011d10 comp rx=0 tx=0).stop 2024-05-14T16:51:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.844+0000 7f480b98d740 1 -- 172.21.15.69:0/310776453 shutdown_connections 2024-05-14T16:51:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.844+0000 7f480b98d740 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f9b7aa9a30 0x55f9b7b1ade0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.844+0000 7f480b98d740 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9b7a99ee0 0x55f9b7aa0350 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.844+0000 7f480b98d740 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9b7a8b130 0x55f9b7a9fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.844+0000 7f480b98d740 1 --2- 172.21.15.69:0/310776453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9b7a8a810 0x55f9b7a9f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.844+0000 7f480b98d740 1 -- 172.21.15.69:0/310776453 >> 172.21.15.69:0/310776453 conn(0x55f9b7a86e00 msgr2=0x55f9b7aa4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.845+0000 7f47e9ffb640 0 client.15516 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.845+0000 7f480b98d740 1 -- 172.21.15.69:0/310776453 shutdown_connections 2024-05-14T16:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.845+0000 7f480b98d740 1 -- 172.21.15.69:0/310776453 wait complete. 2024-05-14T16:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.845+0000 7f480b98d740 20 client.15516 trim_cache size 0 max 16384 2024-05-14T16:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:51:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:51:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:51:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.846+0000 7f4808b9d640 1 -- 172.21.15.69:0/3877491054 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47f00b2d40 msgr2=0x7f47f00b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.846+0000 7f4808b9d640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47f00b2d40 0x7f47f00b5200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f47f4100300 tx=0x7f47f4031f90 comp rx=0 tx=0).stop 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.846+0000 7f4808b9d640 1 -- 172.21.15.69:0/3877491054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48041011c0 msgr2=0x7f4804199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.846+0000 7f4808b9d640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48041011c0 0x7f4804199210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47f402aa00 tx=0x7f47f4003b20 comp rx=0 tx=0).stop 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.846+0000 7f4808b9d640 1 -- 172.21.15.69:0/3877491054 shutdown_connections 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.847+0000 7f4808b9d640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47f00b2d40 0x7f47f00b5200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.847+0000 7f4808b9d640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f480419dad0 0x7f480419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.847+0000 7f4808b9d640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4804107e30 0x7f4804199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.847+0000 7f4808b9d640 1 --2- 172.21.15.69:0/3877491054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48041011c0 0x7f4804199210 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.847+0000 7f4808b9d640 1 -- 172.21.15.69:0/3877491054 >> 172.21.15.69:0/3877491054 conn(0x7f4804077ee0 msgr2=0x7f4804105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.847+0000 7f4808b9d640 1 -- 172.21.15.69:0/3877491054 shutdown_connections 2024-05-14T16:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:16.847+0000 7f4808b9d640 1 -- 172.21.15.69:0/3877491054 wait complete. 2024-05-14T16:51:17.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:16 smithi092 ceph-mon[35763]: daemon mds.c restarted 2024-05-14T16:51:17.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:16 smithi069 ceph-mon[31507]: daemon mds.c restarted 2024-05-14T16:51:17.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:16 smithi203 ceph-mon[35787]: daemon mds.c restarted 2024-05-14T16:51:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:17 smithi092 ceph-mon[35763]: pgmap v45: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 MiB/s wr, 996 op/s 2024-05-14T16:51:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:17 smithi092 ceph-mon[35763]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T16:51:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:17 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6836/1947390593,v1:172.21.15.203:6837/1947390593] up:boot 2024-05-14T16:51:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:17 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T16:51:18.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:51:18.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:17 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:51:18.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:17 smithi069 ceph-mon[31507]: pgmap v45: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 MiB/s wr, 996 op/s 2024-05-14T16:51:18.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:17 smithi069 ceph-mon[31507]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T16:51:18.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:17 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6836/1947390593,v1:172.21.15.203:6837/1947390593] up:boot 2024-05-14T16:51:18.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:17 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T16:51:18.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:51:18.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:17 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:51:18.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:17 smithi203 ceph-mon[35787]: pgmap v45: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 MiB/s wr, 996 op/s 2024-05-14T16:51:18.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:17 smithi203 ceph-mon[35787]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T16:51:18.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:17 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6836/1947390593,v1:172.21.15.203:6837/1947390593] up:boot 2024-05-14T16:51:18.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:17 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T16:51:18.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:51:18.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:17 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:51:20.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:19 smithi092 ceph-mon[35763]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 42 MiB/s wr, 1.14k op/s 2024-05-14T16:51:20.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:19 smithi069 ceph-mon[31507]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 42 MiB/s wr, 1.14k op/s 2024-05-14T16:51:20.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:19 smithi203 ceph-mon[35787]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 42 MiB/s wr, 1.14k op/s 2024-05-14T16:51:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 50.0 s, 3669.4 tps, lat 27.304 ms stddev 28.912 2024-05-14T16:51:22.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:21 smithi092 ceph-mon[35763]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-14T16:51:22.005 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:21 smithi069 ceph-mon[31507]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-14T16:51:22.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:21 smithi203 ceph-mon[35787]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-14T16:51:24.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:23 smithi092 ceph-mon[35763]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-14T16:51:24.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:23 smithi069 ceph-mon[31507]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-14T16:51:24.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:23 smithi203 ceph-mon[35787]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-14T16:51:25.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 55.0 s, 3224.6 tps, lat 30.950 ms stddev 32.479 2024-05-14T16:51:26.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:25 smithi092 ceph-mon[35763]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-14T16:51:26.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:25 smithi069 ceph-mon[31507]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-14T16:51:26.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:25 smithi203 ceph-mon[35787]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-14T16:51:28.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:27 smithi069 ceph-mon[31507]: pgmap v51: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 9.0 MiB/s wr, 761 op/s 2024-05-14T16:51:28.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:27 smithi069 ceph-mon[31507]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-14T16:51:28.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:27 smithi203 ceph-mon[35787]: pgmap v51: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 9.0 MiB/s wr, 761 op/s 2024-05-14T16:51:28.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:27 smithi203 ceph-mon[35787]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-14T16:51:28.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:27 smithi092 ceph-mon[35763]: pgmap v51: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 9.0 MiB/s wr, 761 op/s 2024-05-14T16:51:28.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:27 smithi092 ceph-mon[35763]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-14T16:51:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.983+0000 7f430534e640 1 -- 172.21.15.69:0/598345368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4300107940 msgr2=0x7f4300107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.983+0000 7f430534e640 1 --2- 172.21.15.69:0/598345368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4300107940 0x7f4300107d20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009930 tx=0x7f42f002a420 comp rx=0 tx=0).stop 2024-05-14T16:51:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.983+0000 7f430534e640 1 -- 172.21.15.69:0/598345368 shutdown_connections 2024-05-14T16:51:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.983+0000 7f430534e640 1 --2- 172.21.15.69:0/598345368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4300102340 0x7f430010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.983+0000 7f430534e640 1 --2- 172.21.15.69:0/598345368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4300101980 0x7f4300101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.983+0000 7f430534e640 1 --2- 172.21.15.69:0/598345368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4300107940 0x7f4300107d20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.983+0000 7f430534e640 1 -- 172.21.15.69:0/598345368 >> 172.21.15.69:0/598345368 conn(0x7f4300078ad0 msgr2=0x7f4300078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.983+0000 7f430534e640 1 -- 172.21.15.69:0/598345368 shutdown_connections 2024-05-14T16:51:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.983+0000 7f430534e640 1 -- 172.21.15.69:0/598345368 wait complete. 2024-05-14T16:51:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.985+0000 7f430534e640 1 Processor -- start 2024-05-14T16:51:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.985+0000 7f430534e640 1 -- start start 2024-05-14T16:51:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.985+0000 7f430534e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4300101980 0x7f430010e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f430534e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4300102340 0x7f430010a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42feffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4300101980 0x7f430010e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42feffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4300101980 0x7f430010e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:32860/0 (socket says 172.21.15.69:32860) 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4300102340 0x7f430010a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4300102340 0x7f430010a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44514/0 (socket says 172.21.15.69:44514) 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f430534e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4300107940 0x7f430010abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f430534e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4300111bd0 con 0x7f4300102340 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f430534e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4300111a50 con 0x7f4300101980 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f430534e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4300111d50 con 0x7f4300107940 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42feffd640 1 -- 172.21.15.69:0/4280087846 learned_addr learned my addr 172.21.15.69:0/4280087846 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4300107940 0x7f430010abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42f7fff640 1 -- 172.21.15.69:0/4280087846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4300107940 msgr2=0x7f430010abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42f7fff640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4300107940 0x7f430010abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42f7fff640 1 -- 172.21.15.69:0/4280087846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4300101980 msgr2=0x7f430010e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42f7fff640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4300101980 0x7f430010e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42f7fff640 1 -- 172.21.15.69:0/4280087846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430010b4b0 con 0x7f4300102340 2024-05-14T16:51:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.986+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4300107940 0x7f430010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:51:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.987+0000 7f42f7fff640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4300102340 0x7f430010a6b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f42e800e9f0 tx=0x7f42e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.987+0000 7f42fcff9640 1 -- 172.21.15.69:0/4280087846 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e800cd80 con 0x7f4300102340 2024-05-14T16:51:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.987+0000 7f430534e640 1 -- 172.21.15.69:0/4280087846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43001abd90 con 0x7f4300102340 2024-05-14T16:51:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.987+0000 7f42fcff9640 1 -- 172.21.15.69:0/4280087846 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42e80035f0 con 0x7f4300102340 2024-05-14T16:51:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.987+0000 7f430534e640 1 -- 172.21.15.69:0/4280087846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43001ac2b0 con 0x7f4300102340 2024-05-14T16:51:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.987+0000 7f42fcff9640 1 -- 172.21.15.69:0/4280087846 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8010690 con 0x7f4300102340 2024-05-14T16:51:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.990+0000 7f430805f740 1 Processor -- start 2024-05-14T16:51:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.990+0000 7f430805f740 1 -- start start 2024-05-14T16:51:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f42fcff9640 1 -- 172.21.15.69:0/4280087846 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f42e80108f0 con 0x7f4300102340 2024-05-14T16:51:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f430805f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 0x559f94d4fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f430805f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d50210 0x559f94d5a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f430805f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d5aad0 0x559f94d5cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f430805f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f94cc2c30 con 0x559f94d5aad0 2024-05-14T16:51:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f430805f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559f94cb27d0 con 0x559f94d4f8f0 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f430805f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559f94cb6810 con 0x559f94d50210 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f42feffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d50210 0x559f94d5a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f42feffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d50210 0x559f94d5a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:47012/0 (socket says 172.21.15.69:47012) 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f42feffd640 1 -- 172.21.15.69:0/209127705 learned_addr learned my addr 172.21.15.69:0/209127705 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f42f7fff640 1 --2- 172.21.15.69:0/209127705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d5aad0 0x559f94d5cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/209127705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 0x559f94d4fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f42feffd640 1 -- 172.21.15.69:0/209127705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 msgr2=0x559f94d4fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.991+0000 7f42feffd640 1 --2- 172.21.15.69:0/209127705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 0x559f94d4fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.992+0000 7f42feffd640 1 -- 172.21.15.69:0/209127705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d5aad0 msgr2=0x559f94d5cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.992+0000 7f42feffd640 1 --2- 172.21.15.69:0/209127705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d5aad0 0x559f94d5cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.992+0000 7f42feffd640 1 -- 172.21.15.69:0/209127705 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f949c7210 con 0x559f94d50210 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.992+0000 7f42feffd640 1 --2- 172.21.15.69:0/209127705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d50210 0x559f94d5a590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009230 tx=0x7f42f0031bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cddfc79d05db81e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.992+0000 7f42f57fa640 1 -- 172.21.15.69:0/209127705 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0040070 con 0x559f94d50210 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.992+0000 7f42f57fa640 1 -- 172.21.15.69:0/209127705 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f00316a0 con 0x559f94d50210 2024-05-14T16:51:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.992+0000 7f42f57fa640 1 -- 172.21.15.69:0/209127705 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0035040 con 0x559f94d50210 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.992+0000 7f42fcff9640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f42ec0b2a90 0x7f42ec0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.992+0000 7f42fcff9640 1 -- 172.21.15.69:0/4280087846 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f42e80e2700 con 0x7f4300102340 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f42ec0b2a90 0x7f42ec0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f42ec0b2a90 0x7f42ec0b4f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f43000793d0 tx=0x7f42f80096a0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f430805f740 1 -- 172.21.15.69:0/209127705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d50210 msgr2=0x559f94d5a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f430805f740 1 --2- 172.21.15.69:0/209127705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d50210 0x559f94d5a590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009230 tx=0x7f42f0031bc0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f430805f740 1 -- 172.21.15.69:0/209127705 shutdown_connections 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f430805f740 1 --2- 172.21.15.69:0/209127705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d5aad0 0x559f94d5cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f430805f740 1 --2- 172.21.15.69:0/209127705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d50210 0x559f94d5a590 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f430805f740 1 --2- 172.21.15.69:0/209127705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 0x559f94d4fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f430805f740 1 -- 172.21.15.69:0/209127705 >> 172.21.15.69:0/209127705 conn(0x559f94d4be00 msgr2=0x559f94d4c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f430805f740 1 -- 172.21.15.69:0/209127705 shutdown_connections 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f430805f740 1 -- 172.21.15.69:0/209127705 wait complete. 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.993+0000 7f430805f740 1 -- start start 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f430805f740 1 Processor -- start 2024-05-14T16:51:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f430805f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94d5a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f430805f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d50210 0x559f94d57640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f430805f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d5aad0 0x559f94d57b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f430805f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f94998790 con 0x559f94d5aad0 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f430805f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559f94c6e890 con 0x559f94d50210 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f430805f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559f94d5d740 con 0x559f94d4f8f0 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f430805f740 10 client.0 _ll_register_callbacks cb 0x559f94c71d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f42feffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94d5a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f42feffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94d5a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:47028/0 (socket says 172.21.15.69:47028) 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f42feffd640 1 -- 172.21.15.69:0/130771837 learned_addr learned my addr 172.21.15.69:0/130771837 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d5aad0 0x559f94d57b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f42f7fff640 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d50210 0x559f94d57640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f42feffd640 1 -- 172.21.15.69:0/130771837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d50210 msgr2=0x559f94d57640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f42feffd640 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d50210 0x559f94d57640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f42feffd640 1 -- 172.21.15.69:0/130771837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d5aad0 msgr2=0x559f94d57b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f42feffd640 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d5aad0 0x559f94d57b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.994+0000 7f42feffd640 1 -- 172.21.15.69:0/130771837 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f94a93620 con 0x559f94d4f8f0 2024-05-14T16:51:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.996+0000 7f42feffd640 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94d5a350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f42f0004300 tx=0x7f42f002a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa1dd04f6e39a12b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.996+0000 7f42e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:51:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.996+0000 7f42e67fc640 1 -- 172.21.15.69:0/130771837 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f004c020 con 0x559f94d4f8f0 2024-05-14T16:51:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.996+0000 7f42e67fc640 1 -- 172.21.15.69:0/130771837 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f42f0005370 con 0x559f94d4f8f0 2024-05-14T16:51:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.996+0000 7f430805f740 1 -- 172.21.15.69:0/130771837 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f00316a0 con 0x559f94d4f8f0 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.996+0000 7f42e67fc640 1 -- 172.21.15.69:0/130771837 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f42f0040070 con 0x559f94d4f8f0 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f430805f740 10 client.25054 fetch_fsmap learned FSMap version 31 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f430805f740 1 -- 172.21.15.69:0/130771837 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559f94d68ea0 con 0x559f94d4f8f0 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f42e67fc640 1 -- 172.21.15.69:0/130771837 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f0009520 con 0x559f94d4f8f0 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f42e67fc640 1 -- 172.21.15.69:0/130771837 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f42f0003710 con 0x559f94d4f8f0 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f430805f740 10 client.25054 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f430805f740 10 client.25054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f430805f740 20 client.25054 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f430805f740 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559f94d6a1f0 0x559f94d6c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f430805f740 4 client.25054 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f430805f740 1 -- 172.21.15.69:0/130771837 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f42f00316a0 con 0x559f94d6a1f0 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.997+0000 7f42f7fff640 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559f94d6a1f0 0x559f94d6c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.998+0000 7f42e67fc640 10 client.25054 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.998+0000 7f42f7fff640 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559f94d6a1f0 0x559f94d6c5e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40846ad4691fc83 server_cookie=9f3440bac23cd5e7 in_seq=0 out_seq=0 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.998+0000 7f42e67fc640 1 -- 172.21.15.69:0/130771837 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f42f00316a0 con 0x559f94d6a1f0 2024-05-14T16:51:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.998+0000 7f42e67fc640 10 client.25054 handle_command_reply: tid=0 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.998+0000 7f430805f740 1 client.25054 shutdown 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 1 -- 172.21.15.69:0/130771837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 msgr2=0x559f94d5a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94d5a350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f42f0004300 tx=0x7f42f002a420 comp rx=0 tx=0).stop 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 1 -- 172.21.15.69:0/130771837 shutdown_connections 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559f94d6a1f0 0x559f94d6c5e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d5aad0 0x559f94d57b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d50210 0x559f94d57640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f42e67fc640 0 client.25054 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 1 --2- 172.21.15.69:0/130771837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94d5a350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 1 -- 172.21.15.69:0/130771837 >> 172.21.15.69:0/130771837 conn(0x559f94d4be00 msgr2=0x559f94d4b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 1 -- 172.21.15.69:0/130771837 shutdown_connections 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 1 -- 172.21.15.69:0/130771837 wait complete. 2024-05-14T16:51:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:29.999+0000 7f430805f740 20 client.25054 trim_cache size 0 max 16384 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.019+0000 7f430805f740 1 Processor -- start 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.020+0000 7f430805f740 1 -- start start 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.020+0000 7f430805f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 0x559f94d6f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.020+0000 7f430805f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 0x559f94d6fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.020+0000 7f430805f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d5aad0 0x559f94d548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.020+0000 7f430805f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42e0011160 con 0x559f94d50210 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.020+0000 7f430805f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f42e000e9a0 con 0x559f94d4f8f0 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.020+0000 7f430805f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42e000c7f0 con 0x559f94d5aad0 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42feffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 0x559f94d6f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42ff7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d5aad0 0x559f94d548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42ff7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d5aad0 0x559f94d548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:47044/0 (socket says 172.21.15.69:47044) 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 0x559f94d6fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42ff7fe640 1 -- 172.21.15.69:0/3014702861 learned_addr learned my addr 172.21.15.69:0/3014702861 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42feffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 0x559f94d6f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:32880/0 (socket says 172.21.15.69:32880) 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42ff7fe640 1 -- 172.21.15.69:0/3014702861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 msgr2=0x559f94d6f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/3014702861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 0x559f94d6f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42ff7fe640 1 -- 172.21.15.69:0/3014702861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 msgr2=0x559f94d6fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/3014702861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 0x559f94d6fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42ff7fe640 1 -- 172.21.15.69:0/3014702861 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f94d70090 con 0x559f94d5aad0 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42feffd640 1 --2- 172.21.15.69:0/3014702861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 0x559f94d6f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/3014702861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d5aad0 0x559f94d548c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f42f8010d20 tx=0x7f42f8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe2086b61a7e11e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42f4ff9640 1 -- 172.21.15.69:0/3014702861 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8010d50 con 0x559f94d5aad0 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42f4ff9640 1 -- 172.21.15.69:0/3014702861 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f8029930 con 0x559f94d5aad0 2024-05-14T16:51:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.021+0000 7f42f4ff9640 1 -- 172.21.15.69:0/3014702861 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f801c810 con 0x559f94d5aad0 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.022+0000 7f430805f740 1 -- 172.21.15.69:0/3014702861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d5aad0 msgr2=0x559f94d548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.022+0000 7f430805f740 1 --2- 172.21.15.69:0/3014702861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d5aad0 0x559f94d548c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f42f8010d20 tx=0x7f42f8019000 comp rx=0 tx=0).stop 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.022+0000 7f430805f740 1 -- 172.21.15.69:0/3014702861 shutdown_connections 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.022+0000 7f430805f740 1 --2- 172.21.15.69:0/3014702861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d5aad0 0x559f94d548c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.022+0000 7f430805f740 1 --2- 172.21.15.69:0/3014702861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 0x559f94d6fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.022+0000 7f430805f740 1 --2- 172.21.15.69:0/3014702861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d4f8f0 0x559f94d6f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.022+0000 7f430805f740 1 -- 172.21.15.69:0/3014702861 >> 172.21.15.69:0/3014702861 conn(0x559f94d4be00 msgr2=0x559f94d4b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.022+0000 7f430805f740 1 -- 172.21.15.69:0/3014702861 shutdown_connections 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.022+0000 7f430805f740 1 -- 172.21.15.69:0/3014702861 wait complete. 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.022+0000 7f430805f740 1 -- start start 2024-05-14T16:51:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f430805f740 1 Processor -- start 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f430805f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94dcc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f430805f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 0x559f94dccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f430805f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d5aad0 0x559f94dcd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f430805f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42e000ce60 con 0x559f94d50210 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f430805f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f42e000e2d0 con 0x559f94d5aad0 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f430805f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42e000d530 con 0x559f94d4f8f0 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 0x559f94dccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42feffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94dcc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f430805f740 10 client.0 _ll_register_callbacks cb 0x7f42f0031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 0x559f94dccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44548/0 (socket says 172.21.15.69:44548) 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42f7fff640 1 -- 172.21.15.69:0/1894132604 learned_addr learned my addr 172.21.15.69:0/1894132604 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d5aad0 0x559f94dcd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42feffd640 1 -- 172.21.15.69:0/1894132604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d5aad0 msgr2=0x559f94dcd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42feffd640 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d5aad0 0x559f94dcd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42feffd640 1 -- 172.21.15.69:0/1894132604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 msgr2=0x559f94dccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42feffd640 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 0x559f94dccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42feffd640 1 -- 172.21.15.69:0/1894132604 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f94dcde50 con 0x559f94d4f8f0 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.023+0000 7f42ff7fe640 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d5aad0 0x559f94dcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.024+0000 7f42feffd640 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94dcc9b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42f00042d0 tx=0x7f42f00045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fede95b512f6fe89 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.024+0000 7f42e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.024+0000 7f42e67fc640 1 -- 172.21.15.69:0/1894132604 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f005f060 con 0x559f94d4f8f0 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.024+0000 7f42e67fc640 1 -- 172.21.15.69:0/1894132604 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f42f0005570 con 0x559f94d4f8f0 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.024+0000 7f430805f740 1 -- 172.21.15.69:0/1894132604 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f8029930 con 0x559f94d4f8f0 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.024+0000 7f42e67fc640 1 -- 172.21.15.69:0/1894132604 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f42f0035040 con 0x559f94d4f8f0 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f430805f740 10 client.25060 fetch_fsmap learned FSMap version 31 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f430805f740 1 -- 172.21.15.69:0/1894132604 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559f94dd5250 con 0x559f94d4f8f0 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f42e67fc640 1 -- 172.21.15.69:0/1894132604 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f0004770 con 0x559f94d4f8f0 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f42e67fc640 1 -- 172.21.15.69:0/1894132604 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f42f005a450 con 0x559f94d4f8f0 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f430805f740 10 client.25060 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:51:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f430805f740 10 client.25060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:51:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f430805f740 20 client.25060 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f430805f740 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559f94dd5890 0x559f94dd7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f430805f740 4 client.25060 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:51:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f430805f740 1 -- 172.21.15.69:0/1894132604 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f42f8029930 con 0x559f94dd5890 2024-05-14T16:51:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.025+0000 7f42f7fff640 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559f94dd5890 0x559f94dd7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.026+0000 7f42e67fc640 10 client.25060 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.026+0000 7f42f7fff640 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559f94dd5890 0x559f94dd7cb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ed30a3947414fb server_cookie=7f4403b71791c455 in_seq=0 out_seq=0 2024-05-14T16:51:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.037+0000 7f42e67fc640 1 -- 172.21.15.69:0/1894132604 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f42f8029930 con 0x559f94dd5890 2024-05-14T16:51:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.037+0000 7f42e67fc640 10 client.25060 handle_command_reply: tid=0 2024-05-14T16:51:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.037+0000 7f430805f740 1 client.25060 shutdown 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.037+0000 7f430805f740 1 -- 172.21.15.69:0/1894132604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 msgr2=0x559f94dcc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.037+0000 7f430805f740 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94dcc9b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42f00042d0 tx=0x7f42f00045e0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.037+0000 7f430805f740 1 -- 172.21.15.69:0/1894132604 shutdown_connections 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.037+0000 7f430805f740 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559f94dd5890 0x559f94dd7cb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f42e67fc640 0 client.25060 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430805f740 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f94d5aad0 0x559f94dcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430805f740 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f94d50210 0x559f94dccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430805f740 1 --2- 172.21.15.69:0/1894132604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f94d4f8f0 0x559f94dcc9b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430805f740 1 -- 172.21.15.69:0/1894132604 >> 172.21.15.69:0/1894132604 conn(0x559f94d4be00 msgr2=0x559f94d65990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430805f740 1 -- 172.21.15.69:0/1894132604 shutdown_connections 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430805f740 1 -- 172.21.15.69:0/1894132604 wait complete. 2024-05-14T16:51:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430805f740 20 client.25060 trim_cache size 0 max 16384 2024-05-14T16:51:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:51:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:51:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:51:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430534e640 1 -- 172.21.15.69:0/4280087846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f42ec0b2a90 msgr2=0x7f42ec0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430534e640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f42ec0b2a90 0x7f42ec0b4f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f43000793d0 tx=0x7f42f80096a0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430534e640 1 -- 172.21.15.69:0/4280087846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4300102340 msgr2=0x7f430010a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430534e640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4300102340 0x7f430010a6b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f42e800e9f0 tx=0x7f42e800eeb0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430534e640 1 -- 172.21.15.69:0/4280087846 shutdown_connections 2024-05-14T16:51:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430534e640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f42ec0b2a90 0x7f42ec0b4f50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430534e640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4300107940 0x7f430010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430534e640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4300102340 0x7f430010a6b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.038+0000 7f430534e640 1 --2- 172.21.15.69:0/4280087846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4300101980 0x7f430010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.039+0000 7f430534e640 1 -- 172.21.15.69:0/4280087846 >> 172.21.15.69:0/4280087846 conn(0x7f4300078ad0 msgr2=0x7f4300100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.039+0000 7f430534e640 1 -- 172.21.15.69:0/4280087846 shutdown_connections 2024-05-14T16:51:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:30.039+0000 7f430534e640 1 -- 172.21.15.69:0/4280087846 wait complete. 2024-05-14T16:51:30.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:29 smithi069 ceph-mon[31507]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-05-14T16:51:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:29 smithi203 ceph-mon[35787]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-05-14T16:51:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:29 smithi092 ceph-mon[35763]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-05-14T16:51:30.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 60.0 s, 3641.6 tps, lat 27.451 ms stddev 28.011 2024-05-14T16:51:32.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:31 smithi069 ceph-mon[31507]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 11 MiB/s wr, 988 op/s 2024-05-14T16:51:32.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:31 smithi203 ceph-mon[35787]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 11 MiB/s wr, 988 op/s 2024-05-14T16:51:32.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:31 smithi092 ceph-mon[35763]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 11 MiB/s wr, 988 op/s 2024-05-14T16:51:34.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:33 smithi069 ceph-mon[31507]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.2 MiB/s wr, 663 op/s 2024-05-14T16:51:34.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:33 smithi203 ceph-mon[35787]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.2 MiB/s wr, 663 op/s 2024-05-14T16:51:34.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:33 smithi092 ceph-mon[35763]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.2 MiB/s wr, 663 op/s 2024-05-14T16:51:34.668 INFO:tasks.quiescer.fs.[cephfs]:Total duration (53.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-14T16:51:34.668 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.7 2024-05-14T16:51:34.668 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:51:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:35.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 65.0 s, 3435.8 tps, lat 29.134 ms stddev 30.878 2024-05-14T16:51:36.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:35 smithi069 ceph-mon[31507]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.2 MiB/s wr, 663 op/s 2024-05-14T16:51:36.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:35 smithi203 ceph-mon[35787]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.2 MiB/s wr, 663 op/s 2024-05-14T16:51:36.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:35 smithi092 ceph-mon[35763]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.2 MiB/s wr, 663 op/s 2024-05-14T16:51:37.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.006+0000 7f282145d700 1 -- 172.21.15.69:0/4180437937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c108500 msgr2=0x7f281c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:37.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.006+0000 7f282145d700 1 --2- 172.21.15.69:0/4180437937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c108500 0x7f281c1088e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f280c009990 tx=0x7f280c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.006+0000 7f282145d700 1 -- 172.21.15.69:0/4180437937 shutdown_connections 2024-05-14T16:51:37.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.006+0000 7f282145d700 1 --2- 172.21.15.69:0/4180437937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281c103f90 0x7f281c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.006+0000 7f282145d700 1 --2- 172.21.15.69:0/4180437937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c108500 0x7f281c1088e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.006+0000 7f282145d700 1 --2- 172.21.15.69:0/4180437937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f281c1035f0 0x7f281c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.006+0000 7f282145d700 1 -- 172.21.15.69:0/4180437937 >> 172.21.15.69:0/4180437937 conn(0x7f281c07a6a0 msgr2=0x7f281c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.006+0000 7f282145d700 1 -- 172.21.15.69:0/4180437937 shutdown_connections 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.006+0000 7f282145d700 1 -- 172.21.15.69:0/4180437937 wait complete. 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.006+0000 7f282145d700 1 Processor -- start 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f282145d700 1 -- start start 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f282145d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1035f0 0x7f281c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f282145d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281c103f90 0x7f281c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f282145d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f281c108500 0x7f281c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f282145d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f281c113840 con 0x7f281c108500 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f282145d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f281c1136c0 con 0x7f281c1035f0 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f282145d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f281c1139c0 con 0x7f281c103f90 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f281a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281c103f90 0x7f281c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:37.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f281a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281c103f90 0x7f281c19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:52326/0 (socket says 172.21.15.69:52326) 2024-05-14T16:51:37.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f281a7fc700 1 -- 172.21.15.69:0/4228707091 learned_addr learned my addr 172.21.15.69:0/4228707091 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:37.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f281affd700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1035f0 0x7f281c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:37.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f281b7fe700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f281c108500 0x7f281c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:37.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f281a7fc700 1 -- 172.21.15.69:0/4228707091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1035f0 msgr2=0x7f281c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:37.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f281a7fc700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1035f0 0x7f281c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f281a7fc700 1 -- 172.21.15.69:0/4228707091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f281c108500 msgr2=0x7f281c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:37.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f281a7fc700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f281c108500 0x7f281c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.007+0000 7f281a7fc700 1 -- 172.21.15.69:0/4228707091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c19ed60 con 0x7f281c103f90 2024-05-14T16:51:37.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.009+0000 7f281a7fc700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281c103f90 0x7f281c19a290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f281000b530 tx=0x7f281000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:37.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.009+0000 7f2803fff700 1 -- 172.21.15.69:0/4228707091 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810015020 con 0x7f281c103f90 2024-05-14T16:51:37.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.009+0000 7f2803fff700 1 -- 172.21.15.69:0/4228707091 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2810011ce0 con 0x7f281c103f90 2024-05-14T16:51:37.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.009+0000 7f2803fff700 1 -- 172.21.15.69:0/4228707091 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810010dd0 con 0x7f281c103f90 2024-05-14T16:51:37.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.009+0000 7f282145d700 1 -- 172.21.15.69:0/4228707091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c19f050 con 0x7f281c103f90 2024-05-14T16:51:37.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.010+0000 7f282145d700 1 -- 172.21.15.69:0/4228707091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c075770 con 0x7f281c103f90 2024-05-14T16:51:37.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.012+0000 7f282145d700 1 -- 172.21.15.69:0/4228707091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281c06a230 con 0x7f281c103f90 2024-05-14T16:51:37.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.013+0000 7f2803fff700 1 -- 172.21.15.69:0/4228707091 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2810018020 con 0x7f281c103f90 2024-05-14T16:51:37.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.013+0000 7f2803fff700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f28040b6e00 0x7f28040b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:37.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.014+0000 7f281affd700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f28040b6e00 0x7f28040b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:37.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.014+0000 7f2803fff700 1 -- 172.21.15.69:0/4228707091 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f28100e1bc0 con 0x7f281c103f90 2024-05-14T16:51:37.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.014+0000 7f281affd700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f28040b6e00 0x7f28040b92c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f280c00b5c0 tx=0x7f280c005fd0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:37.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.018+0000 7f2803fff700 1 -- 172.21.15.69:0/4228707091 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f281001f020 con 0x7f281c103f90 2024-05-14T16:51:37.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.436+0000 7f282145d700 1 -- 172.21.15.69:0/4228707091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f281c19b0b0 con 0x7f281c103f90 2024-05-14T16:51:37.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.437+0000 7f2803fff700 1 -- 172.21.15.69:0/4228707091 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11863 (secure 0 0 0) 0x7f28100a9070 con 0x7f281c103f90 2024-05-14T16:51:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:51:37.437 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:51:16.740826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:51:37.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.440+0000 7f282145d700 1 -- 172.21.15.69:0/4228707091 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f28040b6e00 msgr2=0x7f28040b92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:37.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.440+0000 7f282145d700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f28040b6e00 0x7f28040b92c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f280c00b5c0 tx=0x7f280c005fd0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.440+0000 7f282145d700 1 -- 172.21.15.69:0/4228707091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281c103f90 msgr2=0x7f281c19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:37.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.440+0000 7f282145d700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281c103f90 0x7f281c19a290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f281000b530 tx=0x7f281000b840 comp rx=0 tx=0).stop 2024-05-14T16:51:37.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.440+0000 7f282145d700 1 -- 172.21.15.69:0/4228707091 shutdown_connections 2024-05-14T16:51:37.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.440+0000 7f282145d700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f28040b6e00 0x7f28040b92c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.440+0000 7f282145d700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f281c108500 0x7f281c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.440+0000 7f282145d700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1035f0 0x7f281c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.440+0000 7f282145d700 1 --2- 172.21.15.69:0/4228707091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281c103f90 0x7f281c19a290 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.440+0000 7f282145d700 1 -- 172.21.15.69:0/4228707091 >> 172.21.15.69:0/4228707091 conn(0x7f281c07a6a0 msgr2=0x7f281c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:37.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.442+0000 7f282145d700 1 -- 172.21.15.69:0/4228707091 shutdown_connections 2024-05-14T16:51:37.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:37.442+0000 7f282145d700 1 -- 172.21.15.69:0/4228707091 wait complete. 2024-05-14T16:51:37.443 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 31 2024-05-14T16:51:38.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:37 smithi069 ceph-mon[31507]: pgmap v56: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 14 MiB/s wr, 994 op/s 2024-05-14T16:51:38.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:37 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/4228707091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:37 smithi203 ceph-mon[35787]: pgmap v56: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 14 MiB/s wr, 994 op/s 2024-05-14T16:51:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:37 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/4228707091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:38.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:37 smithi092 ceph-mon[35763]: pgmap v56: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 14 MiB/s wr, 994 op/s 2024-05-14T16:51:38.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:37 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/4228707091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:40.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:39 smithi069 ceph-mon[31507]: pgmap v57: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 684 op/s 2024-05-14T16:51:40.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:39 smithi203 ceph-mon[35787]: pgmap v57: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 684 op/s 2024-05-14T16:51:40.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:39 smithi092 ceph-mon[35763]: pgmap v57: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 684 op/s 2024-05-14T16:51:40.309 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0' 2024-05-14T16:51:40.309 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0 2024-05-14T16:51:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 70.0 s, 3323.0 tps, lat 30.144 ms stddev 33.893 2024-05-14T16:51:40.643 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:42.172 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:41 smithi069 ceph-mon[31507]: pgmap v58: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 1.08k op/s 2024-05-14T16:51:42.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:41 smithi203 ceph-mon[35787]: pgmap v58: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 1.08k op/s 2024-05-14T16:51:42.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:41 smithi092 ceph-mon[35763]: pgmap v58: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 1.08k op/s 2024-05-14T16:51:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.656+0000 7fc132b72700 1 -- 172.21.15.69:0/3080341657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc12c109610 msgr2=0x7fc12c1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.656+0000 7fc132b72700 1 --2- 172.21.15.69:0/3080341657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc12c109610 0x7fc12c1099f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc120009990 tx=0x7fc120009ca0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.656+0000 7fc132b72700 1 -- 172.21.15.69:0/3080341657 shutdown_connections 2024-05-14T16:51:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.656+0000 7fc132b72700 1 --2- 172.21.15.69:0/3080341657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c06c670 0x7fc12c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.656+0000 7fc132b72700 1 --2- 172.21.15.69:0/3080341657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc12c109610 0x7fc12c1099f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.656+0000 7fc132b72700 1 --2- 172.21.15.69:0/3080341657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc12c06bcb0 0x7fc12c06c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.656+0000 7fc132b72700 1 -- 172.21.15.69:0/3080341657 >> 172.21.15.69:0/3080341657 conn(0x7fc12c079460 msgr2=0x7fc12c079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:42.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.657+0000 7fc132b72700 1 -- 172.21.15.69:0/3080341657 shutdown_connections 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.657+0000 7fc132b72700 1 -- 172.21.15.69:0/3080341657 wait complete. 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.658+0000 7fc132b72700 1 Processor -- start 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.658+0000 7fc132b72700 1 -- start start 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.659+0000 7fc132b72700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c06bcb0 0x7fc12c10d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.659+0000 7fc132b72700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc12c06c670 0x7fc12c10d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.659+0000 7fc132b72700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc12c109610 0x7fc12c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.659+0000 7fc132b72700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c103a70 con 0x7fc12c109610 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.659+0000 7fc132b72700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1038f0 con 0x7fc12c06bcb0 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.659+0000 7fc132b72700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c103bf0 con 0x7fc12c06c670 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.659+0000 7fc12bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc12c06c670 0x7fc12c10d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.659+0000 7fc12bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc12c06c670 0x7fc12c10d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:52336/0 (socket says 172.21.15.69:52336) 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.659+0000 7fc12bfff700 1 -- 172.21.15.69:0/1973602492 learned_addr learned my addr 172.21.15.69:0/1973602492 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:42.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.659+0000 7fc13090e700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c06bcb0 0x7fc12c10d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.660+0000 7fc13110f700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc12c109610 0x7fc12c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.660+0000 7fc12bfff700 1 -- 172.21.15.69:0/1973602492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c06bcb0 msgr2=0x7fc12c10d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.660+0000 7fc12bfff700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c06bcb0 0x7fc12c10d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.660+0000 7fc12bfff700 1 -- 172.21.15.69:0/1973602492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc12c109610 msgr2=0x7fc12c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.660+0000 7fc12bfff700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc12c109610 0x7fc12c06df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.660+0000 7fc12bfff700 1 -- 172.21.15.69:0/1973602492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c06e6a0 con 0x7fc12c06c670 2024-05-14T16:51:42.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.660+0000 7fc12bfff700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc12c06c670 0x7fc12c10d5b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc11800b530 tx=0x7fc11800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:42.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.661+0000 7fc129ffb700 1 -- 172.21.15.69:0/1973602492 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118015020 con 0x7fc12c06c670 2024-05-14T16:51:42.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.661+0000 7fc129ffb700 1 -- 172.21.15.69:0/1973602492 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc118011ce0 con 0x7fc12c06c670 2024-05-14T16:51:42.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.661+0000 7fc129ffb700 1 -- 172.21.15.69:0/1973602492 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118010d20 con 0x7fc12c06c670 2024-05-14T16:51:42.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.661+0000 7fc132b72700 1 -- 172.21.15.69:0/1973602492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c06e930 con 0x7fc12c06c670 2024-05-14T16:51:42.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.661+0000 7fc132b72700 1 -- 172.21.15.69:0/1973602492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c074230 con 0x7fc12c06c670 2024-05-14T16:51:42.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.664+0000 7fc129ffb700 1 -- 172.21.15.69:0/1973602492 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc118018020 con 0x7fc12c06c670 2024-05-14T16:51:42.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.665+0000 7fc129ffb700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc11c0b2be0 0x7fc11c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.665+0000 7fc129ffb700 1 -- 172.21.15.69:0/1973602492 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc1180e1c90 con 0x7fc12c06c670 2024-05-14T16:51:42.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.665+0000 7fc13090e700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc11c0b2be0 0x7fc11c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.666+0000 7fc13d408b80 1 Processor -- start 2024-05-14T16:51:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.666+0000 7fc13090e700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc11c0b2be0 0x7fc11c0b50a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc12000b790 tx=0x7fc12001a040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.666+0000 7fc13d408b80 1 -- start start 2024-05-14T16:51:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.666+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ab9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.666+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ab4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.666+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab5350 0x55ef08ab57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.666+0000 7fc13d408b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ef08a12f00 con 0x55ef08ab94b0 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.666+0000 7fc13d408b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ef08a02aa0 con 0x55ef08ab5350 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.666+0000 7fc13d408b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ef08a06ae0 con 0x55ef08ab9dd0 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.666+0000 7fc12bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ab4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.667+0000 7fc12bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ab4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:52346/0 (socket says 172.21.15.69:52346) 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.667+0000 7fc12bfff700 1 -- 172.21.15.69:0/50192789 learned_addr learned my addr 172.21.15.69:0/50192789 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.667+0000 7fc13110f700 1 --2- 172.21.15.69:0/50192789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab5350 0x55ef08ab57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.667+0000 7fc12bfff700 1 -- 172.21.15.69:0/50192789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab5350 msgr2=0x55ef08ab57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.667+0000 7fc12bfff700 1 --2- 172.21.15.69:0/50192789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab5350 0x55ef08ab57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.667+0000 7fc12bfff700 1 -- 172.21.15.69:0/50192789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 msgr2=0x55ef08ab9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.667+0000 7fc12bfff700 1 --2- 172.21.15.69:0/50192789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ab9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.667+0000 7fc12bfff700 1 -- 172.21.15.69:0/50192789 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef089ac140 con 0x55ef08ab9dd0 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.667+0000 7fc12bfff700 1 --2- 172.21.15.69:0/50192789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ab4d30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc1180a9020 tx=0x7fc1180a8880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3229ffc17e04a930 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.668+0000 7fc116ffd700 1 -- 172.21.15.69:0/50192789 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180c3070 con 0x55ef08ab9dd0 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.668+0000 7fc116ffd700 1 -- 172.21.15.69:0/50192789 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1180ad440 con 0x55ef08ab9dd0 2024-05-14T16:51:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.668+0000 7fc116ffd700 1 -- 172.21.15.69:0/50192789 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180b1a60 con 0x55ef08ab9dd0 2024-05-14T16:51:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.668+0000 7fc13d408b80 1 -- 172.21.15.69:0/50192789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 msgr2=0x55ef08ab4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.668+0000 7fc13d408b80 1 --2- 172.21.15.69:0/50192789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ab4d30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc1180a9020 tx=0x7fc1180a8880 comp rx=0 tx=0).stop 2024-05-14T16:51:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 -- 172.21.15.69:0/50192789 shutdown_connections 2024-05-14T16:51:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 --2- 172.21.15.69:0/50192789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ab9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 --2- 172.21.15.69:0/50192789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab5350 0x55ef08ab57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 --2- 172.21.15.69:0/50192789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ab4d30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 -- 172.21.15.69:0/50192789 >> 172.21.15.69:0/50192789 conn(0x55ef089e1f40 msgr2=0x55ef08aaf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 -- 172.21.15.69:0/50192789 shutdown_connections 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 -- 172.21.15.69:0/50192789 wait complete. 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 -- start start 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 Processor -- start 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08ac0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08abe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 0x55ef08abe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ef08909130 con 0x55ef08ab94b0 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ef0896af30 con 0x55ef08ab9dd0 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.669+0000 7fc13d408b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ef08ac1910 con 0x55ef08ab5350 2024-05-14T16:51:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13d408b80 10 client.0 _ll_register_callbacks cb 0x55ef0899dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13090e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08ac0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc12bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08abe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13090e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08ac0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:52350/0 (socket says 172.21.15.69:52350) 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc12bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08abe060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60342/0 (socket says 172.21.15.69:60342) 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13090e700 1 -- 172.21.15.69:0/861464031 learned_addr learned my addr 172.21.15.69:0/861464031 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13110f700 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 0x55ef08abe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13090e700 1 -- 172.21.15.69:0/861464031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 msgr2=0x55ef08abe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13090e700 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 0x55ef08abe5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13090e700 1 -- 172.21.15.69:0/861464031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 msgr2=0x55ef08abe060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13090e700 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08abe060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13090e700 1 -- 172.21.15.69:0/861464031 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef08952a60 con 0x55ef08ab5350 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13090e700 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08ac0cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc120015000 tx=0x7fc120005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c72f2c40b74d77d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc10bfff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:51:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc10bfff700 1 -- 172.21.15.69:0/861464031 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120004c30 con 0x55ef08ab5350 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc10bfff700 1 -- 172.21.15.69:0/861464031 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc120004dd0 con 0x55ef08ab5350 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.670+0000 7fc13d408b80 1 -- 172.21.15.69:0/861464031 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1180ad440 con 0x55ef08ab5350 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc10bfff700 1 -- 172.21.15.69:0/861464031 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc12000f460 con 0x55ef08ab5350 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc13d408b80 10 client.25075 fetch_fsmap learned FSMap version 31 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc13d408b80 1 -- 172.21.15.69:0/861464031 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ef089bc8f0 con 0x55ef08ab5350 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc10bfff700 1 -- 172.21.15.69:0/861464031 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1180ad440 con 0x55ef08ab5350 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc10bfff700 1 -- 172.21.15.69:0/861464031 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fc120022070 con 0x55ef08ab5350 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc13d408b80 10 client.25075 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc13d408b80 10 client.25075 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc13d408b80 20 client.25075 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc13d408b80 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ef08ace5b0 0x55ef08ad09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc13d408b80 4 client.25075 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.671+0000 7fc13d408b80 1 -- 172.21.15.69:0/861464031 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef08a14e50 con 0x55ef08ace5b0 2024-05-14T16:51:42.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.672+0000 7fc12bfff700 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ef08ace5b0 0x55ef08ad09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.673+0000 7fc10bfff700 10 client.25075 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.673+0000 7fc12bfff700 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ef08ace5b0 0x55ef08ad09e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b41459a47b058d server_cookie=1d47c28daf4d46bc in_seq=0 out_seq=0 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.673+0000 7fc10bfff700 1 -- 172.21.15.69:0/861464031 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef08a14e50 con 0x55ef08ace5b0 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.673+0000 7fc10bfff700 10 client.25075 handle_command_reply: tid=0 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.673+0000 7fc13d408b80 1 client.25075 shutdown 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc13d408b80 1 -- 172.21.15.69:0/861464031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 msgr2=0x55ef08ac0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc13d408b80 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08ac0cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc120015000 tx=0x7fc120005fd0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc13d408b80 1 -- 172.21.15.69:0/861464031 shutdown_connections 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc13d408b80 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ef08ace5b0 0x55ef08ad09e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc13d408b80 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 0x55ef08abe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc13d408b80 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08ac0cc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc13d408b80 1 --2- 172.21.15.69:0/861464031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08abe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc13d408b80 1 -- 172.21.15.69:0/861464031 >> 172.21.15.69:0/861464031 conn(0x55ef089e1f40 msgr2=0x55ef08aaa050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc10bfff700 0 client.25075 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:42.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc13d408b80 1 -- 172.21.15.69:0/861464031 shutdown_connections 2024-05-14T16:51:42.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.674+0000 7fc13d408b80 1 -- 172.21.15.69:0/861464031 wait complete. 2024-05-14T16:51:42.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.675+0000 7fc13d408b80 20 client.25075 trim_cache size 0 max 16384 2024-05-14T16:51:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.701+0000 7fc13d408b80 1 Processor -- start 2024-05-14T16:51:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.701+0000 7fc13d408b80 1 -- start start 2024-05-14T16:51:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.701+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08aaa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.701+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ad62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.701+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 0x55ef08ad6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.701+0000 7fc13d408b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1bf440 con 0x55ef08ab94b0 2024-05-14T16:51:42.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.701+0000 7fc13d408b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1bcc40 con 0x55ef08ab9dd0 2024-05-14T16:51:42.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.701+0000 7fc13d408b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1bab00 con 0x55ef08ab5350 2024-05-14T16:51:42.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc13090e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08aaa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc13090e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08aaa160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:52360/0 (socket says 172.21.15.69:52360) 2024-05-14T16:51:42.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc13090e700 1 -- 172.21.15.69:0/2850826629 learned_addr learned my addr 172.21.15.69:0/2850826629 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:42.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc13110f700 1 --2- 172.21.15.69:0/2850826629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 0x55ef08ad6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc12bfff700 1 --2- 172.21.15.69:0/2850826629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ad62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc13110f700 1 -- 172.21.15.69:0/2850826629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 msgr2=0x55ef08aaa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:42.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc13110f700 1 --2- 172.21.15.69:0/2850826629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08aaa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc13110f700 1 -- 172.21.15.69:0/2850826629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 msgr2=0x55ef08ad62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:42.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc13110f700 1 --2- 172.21.15.69:0/2850826629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ad62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc13110f700 1 -- 172.21.15.69:0/2850826629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef08ac17f0 con 0x55ef08ab9dd0 2024-05-14T16:51:42.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.702+0000 7fc12bfff700 1 --2- 172.21.15.69:0/2850826629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ad62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:51:42.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.703+0000 7fc13110f700 1 --2- 172.21.15.69:0/2850826629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 0x55ef08ad6830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc124009400 tx=0x7fc12400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d813117037730a26 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:42.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.703+0000 7fc1167fc700 1 -- 172.21.15.69:0/2850826629 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12401b020 con 0x55ef08ab9dd0 2024-05-14T16:51:42.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.703+0000 7fc1167fc700 1 -- 172.21.15.69:0/2850826629 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc124017920 con 0x55ef08ab9dd0 2024-05-14T16:51:42.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.703+0000 7fc1167fc700 1 -- 172.21.15.69:0/2850826629 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124016830 con 0x55ef08ab9dd0 2024-05-14T16:51:42.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.704+0000 7fc13d408b80 1 -- 172.21.15.69:0/2850826629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 msgr2=0x55ef08ad6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.704+0000 7fc13d408b80 1 --2- 172.21.15.69:0/2850826629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 0x55ef08ad6830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc124009400 tx=0x7fc12400b040 comp rx=0 tx=0).stop 2024-05-14T16:51:42.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.704+0000 7fc13d408b80 1 -- 172.21.15.69:0/2850826629 shutdown_connections 2024-05-14T16:51:42.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.704+0000 7fc13d408b80 1 --2- 172.21.15.69:0/2850826629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab9dd0 0x55ef08ad6830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.704+0000 7fc13d408b80 1 --2- 172.21.15.69:0/2850826629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab5350 0x55ef08aaa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.704+0000 7fc13d408b80 1 --2- 172.21.15.69:0/2850826629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ad62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.704+0000 7fc13d408b80 1 -- 172.21.15.69:0/2850826629 >> 172.21.15.69:0/2850826629 conn(0x55ef089e1f40 msgr2=0x55ef08ab7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.704+0000 7fc13d408b80 1 -- 172.21.15.69:0/2850826629 shutdown_connections 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.704+0000 7fc13d408b80 1 -- 172.21.15.69:0/2850826629 wait complete. 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13d408b80 1 -- start start 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13d408b80 1 Processor -- start 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab5350 0x55ef08ac6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ac7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13d408b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ac3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13d408b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1bb100 con 0x55ef08ab94b0 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13d408b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1bc570 con 0x55ef08ab5350 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13d408b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c1bb7d0 con 0x55ef08ab9dd0 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13d408b80 10 client.0 _ll_register_callbacks cb 0x55ef0899dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc12bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ac7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc12bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ac7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60352/0 (socket says 172.21.15.69:60352) 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc12bfff700 1 -- 172.21.15.69:0/988833738 learned_addr learned my addr 172.21.15.69:0/988833738 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13090e700 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab5350 0x55ef08ac6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.705+0000 7fc13110f700 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ac3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc12bfff700 1 -- 172.21.15.69:0/988833738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 msgr2=0x55ef08ac3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc12bfff700 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ac3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc12bfff700 1 -- 172.21.15.69:0/988833738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab5350 msgr2=0x55ef08ac6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc12bfff700 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab5350 0x55ef08ac6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc12bfff700 1 -- 172.21.15.69:0/988833738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef08ac3ab0 con 0x55ef08ab94b0 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc13110f700 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ac3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc12bfff700 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ac7270 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc118010660 tx=0x7fc1180107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803442714d1d56b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc10bfff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc10bfff700 1 -- 172.21.15.69:0/988833738 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180c3040 con 0x55ef08ab94b0 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc10bfff700 1 -- 172.21.15.69:0/988833738 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc1180be200 con 0x55ef08ab94b0 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc13d408b80 1 -- 172.21.15.69:0/988833738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc124017920 con 0x55ef08ab94b0 2024-05-14T16:51:42.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc10bfff700 1 -- 172.21.15.69:0/988833738 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc1180ad440 con 0x55ef08ab94b0 2024-05-14T16:51:42.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc13d408b80 10 client.15549 fetch_fsmap learned FSMap version 31 2024-05-14T16:51:42.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.706+0000 7fc13d408b80 1 -- 172.21.15.69:0/988833738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ef08b39b10 con 0x55ef08ab94b0 2024-05-14T16:51:42.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.707+0000 7fc10bfff700 1 -- 172.21.15.69:0/988833738 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc124017920 con 0x55ef08ab94b0 2024-05-14T16:51:42.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.707+0000 7fc10bfff700 1 -- 172.21.15.69:0/988833738 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fc1180b1b70 con 0x55ef08ab94b0 2024-05-14T16:51:42.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.707+0000 7fc13d408b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:51:42.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.707+0000 7fc13d408b80 10 client.15549 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:51:42.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.707+0000 7fc13d408b80 20 client.15549 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:42.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.707+0000 7fc13d408b80 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ef08b39e60 0x55ef08b3c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.707+0000 7fc13d408b80 4 client.15549 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}] 2024-05-14T16:51:42.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.707+0000 7fc13d408b80 1 -- 172.21.15.69:0/988833738 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}) -- 0x55ef08a0e630 con 0x55ef08b39e60 2024-05-14T16:51:42.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.707+0000 7fc13090e700 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ef08b39e60 0x55ef08b3c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.708+0000 7fc10bfff700 10 client.15549 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:42.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.708+0000 7fc13090e700 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ef08b39e60 0x55ef08b3c250 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515ea99ff62b4250 server_cookie=a288fed296021442 in_seq=0 out_seq=0 2024-05-14T16:51:42.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc10bfff700 1 -- 172.21.15.69:0/988833738 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55ef08a0e630 con 0x55ef08b39e60 2024-05-14T16:51:42.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc10bfff700 10 client.15549 handle_command_reply: tid=0 2024-05-14T16:51:42.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 client.15549 shutdown 2024-05-14T16:51:42.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 -- 172.21.15.69:0/988833738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 msgr2=0x55ef08ac7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ac7270 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc118010660 tx=0x7fc1180107c0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 -- 172.21.15.69:0/988833738 shutdown_connections 2024-05-14T16:51:42.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ef08b39e60 0x55ef08b3c250 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc10bfff700 0 client.15549 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:42.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef08ab9dd0 0x55ef08ac3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef08ab5350 0x55ef08ac6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 --2- 172.21.15.69:0/988833738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef08ab94b0 0x55ef08ac7270 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 -- 172.21.15.69:0/988833738 >> 172.21.15.69:0/988833738 conn(0x55ef089e1f40 msgr2=0x55ef08ab87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 -- 172.21.15.69:0/988833738 shutdown_connections 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 1 -- 172.21.15.69:0/988833738 wait complete. 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.901+0000 7fc13d408b80 20 client.15549 trim_cache size 0 max 16384 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 31, 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 6, 2024-05-14T16:51:42.902 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "e3c2299d": { 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 6, 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 146.7, 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:51:42.903 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:51:42.904 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:51:42.904 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:51:42.904 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:51:42.904 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:51:42.904 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:51:42.904 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:51:42.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 -- 172.21.15.69:0/1973602492 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc11c0b2be0 msgr2=0x7fc11c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc11c0b2be0 0x7fc11c0b50a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc12000b790 tx=0x7fc12001a040 comp rx=0 tx=0).stop 2024-05-14T16:51:42.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 -- 172.21.15.69:0/1973602492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc12c06c670 msgr2=0x7fc12c10d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc12c06c670 0x7fc12c10d5b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc11800b530 tx=0x7fc11800b8f0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 -- 172.21.15.69:0/1973602492 shutdown_connections 2024-05-14T16:51:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc11c0b2be0 0x7fc11c0b50a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc12c109610 0x7fc12c06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc12c06bcb0 0x7fc12c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 --2- 172.21.15.69:0/1973602492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc12c06c670 0x7fc12c10d5b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 -- 172.21.15.69:0/1973602492 >> 172.21.15.69:0/1973602492 conn(0x7fc12c079460 msgr2=0x7fc12c112120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 -- 172.21.15.69:0/1973602492 shutdown_connections 2024-05-14T16:51:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:51:42.902+0000 7fc132b72700 1 -- 172.21.15.69:0/1973602492 wait complete. 2024-05-14T16:51:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.133+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/508988354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4b4106e90 msgr2=0x7ff4b4107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.133+0000 7ff4bbb5f640 1 --2- 172.21.15.69:0/508988354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4b4106e90 0x7ff4b4107340 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009d00 tx=0x7ff4ac02a9a0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.133+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/508988354 shutdown_connections 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.133+0000 7ff4bbb5f640 1 --2- 172.21.15.69:0/508988354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4b4106e90 0x7ff4b4107340 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.133+0000 7ff4bbb5f640 1 --2- 172.21.15.69:0/508988354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b41023b0 0x7ff4b4106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.133+0000 7ff4bbb5f640 1 --2- 172.21.15.69:0/508988354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b4101a90 0x7ff4b4101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.133+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/508988354 >> 172.21.15.69:0/508988354 conn(0x7ff4b4078ad0 msgr2=0x7ff4b4078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/508988354 shutdown_connections 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/508988354 wait complete. 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4bbb5f640 1 Processor -- start 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4bbb5f640 1 -- start start 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4bbb5f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b4101a90 0x7ff4b41a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4bbb5f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4b41023b0 0x7ff4b41a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4bbb5f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b4106e90 0x7ff4b419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4bbb5f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41192e0 con 0x7ff4b41023b0 2024-05-14T16:51:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4bbb5f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4119160 con 0x7ff4b4106e90 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4bbb5f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4119460 con 0x7ff4b4101a90 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4b90fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4b41023b0 0x7ff4b41a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4b90fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4b41023b0 0x7ff4b41a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60376/0 (socket says 172.21.15.69:60376) 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.134+0000 7ff4b90fa640 1 -- 172.21.15.69:0/2671552707 learned_addr learned my addr 172.21.15.69:0/2671552707 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4ba0fc640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b4106e90 0x7ff4b419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b4101a90 0x7ff4b41a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4b90fa640 1 -- 172.21.15.69:0/2671552707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b4101a90 msgr2=0x7ff4b41a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4b90fa640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b4101a90 0x7ff4b41a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4b90fa640 1 -- 172.21.15.69:0/2671552707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b4106e90 msgr2=0x7ff4b419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4b90fa640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b4106e90 0x7ff4b419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4b90fa640 1 -- 172.21.15.69:0/2671552707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b419fd80 con 0x7ff4b41023b0 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b4101a90 0x7ff4b41a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4b90fa640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4b41023b0 0x7ff4b41a5940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400d950 tx=0x7ff4a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4aaffd640 1 -- 172.21.15.69:0/2671552707 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4017070 con 0x7ff4b41023b0 2024-05-14T16:51:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4aaffd640 1 -- 172.21.15.69:0/2671552707 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a4009c30 con 0x7ff4b41023b0 2024-05-14T16:51:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/2671552707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b41a0070 con 0x7ff4b41023b0 2024-05-14T16:51:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4aaffd640 1 -- 172.21.15.69:0/2671552707 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400f620 con 0x7ff4b41023b0 2024-05-14T16:51:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.135+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/2671552707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b41ac2c0 con 0x7ff4b41023b0 2024-05-14T16:51:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.138+0000 7ff4be878740 1 Processor -- start 2024-05-14T16:51:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.138+0000 7ff4aaffd640 1 -- 172.21.15.69:0/2671552707 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff4a4010460 con 0x7ff4b41023b0 2024-05-14T16:51:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4be878740 1 -- start start 2024-05-14T16:51:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4be878740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fa0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4be878740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783fab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4be878740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783fadec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4be878740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563783f13c30 con 0x563783fa08f0 2024-05-14T16:51:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4be878740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563783f037d0 con 0x563783fabad0 2024-05-14T16:51:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4be878740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563783f07810 con 0x563783fa1210 2024-05-14T16:51:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4ba0fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fa0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4ba0fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fa0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60384/0 (socket says 172.21.15.69:60384) 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4ba0fc640 1 -- 172.21.15.69:0/2529782030 learned_addr learned my addr 172.21.15.69:0/2529782030 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/2529782030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783fab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4b90fa640 1 --2- 172.21.15.69:0/2529782030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783fadec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4ba0fc640 1 -- 172.21.15.69:0/2529782030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 msgr2=0x563783fab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4ba0fc640 1 --2- 172.21.15.69:0/2529782030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783fab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4ba0fc640 1 -- 172.21.15.69:0/2529782030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 msgr2=0x563783fadec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4ba0fc640 1 --2- 172.21.15.69:0/2529782030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783fadec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4ba0fc640 1 -- 172.21.15.69:0/2529782030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563783c18210 con 0x563783fa08f0 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4aaffd640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff4a00b2a10 0x7ff4a00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.139+0000 7ff4ba0fc640 1 --2- 172.21.15.69:0/2529782030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fa0cd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac035000 tx=0x7ff4ac02f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f26fa8653a3ab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.140+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff4a00b2a10 0x7ff4a00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.140+0000 7ff4aaffd640 1 -- 172.21.15.69:0/2671552707 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff4a40e21b0 con 0x7ff4b41023b0 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.140+0000 7ff49ffff640 1 -- 172.21.15.69:0/2529782030 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac048070 con 0x563783fa08f0 2024-05-14T16:51:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.140+0000 7ff49ffff640 1 -- 172.21.15.69:0/2529782030 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4ac02f4d0 con 0x563783fa08f0 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.140+0000 7ff49ffff640 1 -- 172.21.15.69:0/2529782030 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac034040 con 0x563783fa08f0 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 -- 172.21.15.69:0/2529782030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 msgr2=0x563783fa0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff4a00b2a10 0x7ff4a00b4ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0005fd0 tx=0x7ff4b00094a0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 --2- 172.21.15.69:0/2529782030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fa0cd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac035000 tx=0x7ff4ac02f3b0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 -- 172.21.15.69:0/2529782030 shutdown_connections 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 --2- 172.21.15.69:0/2529782030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783fadec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 --2- 172.21.15.69:0/2529782030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783fab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 --2- 172.21.15.69:0/2529782030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fa0cd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 -- 172.21.15.69:0/2529782030 >> 172.21.15.69:0/2529782030 conn(0x563783f9ce00 msgr2=0x563783f9d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 -- 172.21.15.69:0/2529782030 shutdown_connections 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 -- 172.21.15.69:0/2529782030 wait complete. 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 -- start start 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.141+0000 7ff4be878740 1 Processor -- start 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4be878740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4be878740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783fa8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4be878740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783fa8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4be878740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563783be9790 con 0x563783fa08f0 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4be878740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563783ebf890 con 0x563783fabad0 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4be878740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563783fae740 con 0x563783fa1210 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4be878740 10 client.0 _ll_register_callbacks cb 0x563783ec2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4b98fb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4b98fb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60392/0 (socket says 172.21.15.69:60392) 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4b98fb640 1 -- 172.21.15.69:0/3327948263 learned_addr learned my addr 172.21.15.69:0/3327948263 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4ba0fc640 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783fa8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4b98fb640 1 -- 172.21.15.69:0/3327948263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 msgr2=0x563783fa8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4b90fa640 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783fa8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783fa8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4b98fb640 1 -- 172.21.15.69:0/3327948263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 msgr2=0x563783fa8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783fa8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4b98fb640 1 -- 172.21.15.69:0/3327948263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563783ce4620 con 0x563783fa08f0 2024-05-14T16:51:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4ba0fc640 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783fa8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fab440 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00126b0 tx=0x7ff4b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec89c70f60b43dbf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff49d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff49d7fa640 1 -- 172.21.15.69:0/3327948263 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b001f8f0 con 0x563783fa08f0 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff49d7fa640 1 -- 172.21.15.69:0/3327948263 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff4b001fa90 con 0x563783fa08f0 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.142+0000 7ff4be878740 1 -- 172.21.15.69:0/3327948263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4ac02f4d0 con 0x563783fa08f0 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff49d7fa640 1 -- 172.21.15.69:0/3327948263 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff4b001fc90 con 0x563783fa08f0 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff4be878740 10 client.15561 fetch_fsmap learned FSMap version 31 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff4be878740 1 -- 172.21.15.69:0/3327948263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563783fba210 con 0x563783fa08f0 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff49d7fa640 1 -- 172.21.15.69:0/3327948263 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4ac02f4d0 con 0x563783fa08f0 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff49d7fa640 1 -- 172.21.15.69:0/3327948263 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7ff4b000f040 con 0x563783fa08f0 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff4be878740 10 client.15561 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff4be878740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff4be878740 20 client.15561 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff4be878740 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563783fbb770 0x563783fbdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff4be878740 4 client.15561 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff4be878740 1 -- 172.21.15.69:0/3327948263 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563783f15b80 con 0x563783fbb770 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.143+0000 7ff4b90fa640 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563783fbb770 0x563783fbdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.144+0000 7ff49d7fa640 10 client.15561 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.144+0000 7ff4b90fa640 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563783fbb770 0x563783fbdba0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363606eed12b08fe server_cookie=677d9c49314a3a80 in_seq=0 out_seq=0 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.144+0000 7ff49d7fa640 1 -- 172.21.15.69:0/3327948263 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563783f15b80 con 0x563783fbb770 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.144+0000 7ff49d7fa640 10 client.15561 handle_command_reply: tid=0 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.144+0000 7ff4be878740 1 client.15561 shutdown 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.144+0000 7ff4be878740 1 -- 172.21.15.69:0/3327948263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 msgr2=0x563783fab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.144+0000 7ff4be878740 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fab440 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00126b0 tx=0x7ff4b00046c0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.144+0000 7ff4be878740 1 -- 172.21.15.69:0/3327948263 shutdown_connections 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.144+0000 7ff4be878740 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563783fbb770 0x563783fbdba0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.145+0000 7ff4be878740 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783fa8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.145+0000 7ff4be878740 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783fa8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.145+0000 7ff49d7fa640 0 client.15561 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.145+0000 7ff4be878740 1 --2- 172.21.15.69:0/3327948263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fab440 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.145+0000 7ff4be878740 1 -- 172.21.15.69:0/3327948263 >> 172.21.15.69:0/3327948263 conn(0x563783f9ce00 msgr2=0x563783f9c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.145+0000 7ff4be878740 1 -- 172.21.15.69:0/3327948263 shutdown_connections 2024-05-14T16:51:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.145+0000 7ff4be878740 1 -- 172.21.15.69:0/3327948263 wait complete. 2024-05-14T16:51:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.145+0000 7ff4be878740 20 client.15561 trim_cache size 0 max 16384 2024-05-14T16:51:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4be878740 1 Processor -- start 2024-05-14T16:51:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4be878740 1 -- start start 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4be878740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fb7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4be878740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783f97f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4be878740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783faaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4be878740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b7530 con 0x563783fa08f0 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4be878740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b4d70 con 0x563783fabad0 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4be878740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b2bc0 con 0x563783fa1210 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4b98fb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fb7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4b98fb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fb7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60406/0 (socket says 172.21.15.69:60406) 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4b98fb640 1 -- 172.21.15.69:0/3151805067 learned_addr learned my addr 172.21.15.69:0/3151805067 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4b98fb640 1 -- 172.21.15.69:0/3151805067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 msgr2=0x563783f97f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/3151805067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783f97f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4b98fb640 1 -- 172.21.15.69:0/3151805067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 msgr2=0x563783faaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/3151805067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783faaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4b98fb640 1 -- 172.21.15.69:0/3151805067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563783f9d9d0 con 0x563783fa08f0 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/3151805067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fb7610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0004690 tx=0x7ff4b0005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61864dc9eacc536 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.160+0000 7ff49f7fe640 1 -- 172.21.15.69:0/3151805067 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0034070 con 0x563783fa08f0 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff49f7fe640 1 -- 172.21.15.69:0/3151805067 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b003e070 con 0x563783fa08f0 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff49f7fe640 1 -- 172.21.15.69:0/3151805067 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00396a0 con 0x563783fa08f0 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 -- 172.21.15.69:0/3151805067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 msgr2=0x563783fb7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 --2- 172.21.15.69:0/3151805067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fb7610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0004690 tx=0x7ff4b0005870 comp rx=0 tx=0).stop 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 -- 172.21.15.69:0/3151805067 shutdown_connections 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 --2- 172.21.15.69:0/3151805067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783faaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 --2- 172.21.15.69:0/3151805067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa1210 0x563783f97f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 --2- 172.21.15.69:0/3151805067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa08f0 0x563783fb7610 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 -- 172.21.15.69:0/3151805067 >> 172.21.15.69:0/3151805067 conn(0x563783f9ce00 msgr2=0x563783fa3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 -- 172.21.15.69:0/3151805067 shutdown_connections 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 -- 172.21.15.69:0/3151805067 wait complete. 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 -- start start 2024-05-14T16:51:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.161+0000 7ff4be878740 1 Processor -- start 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4be878740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa08f0 0x563783fb3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4be878740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa1210 0x563783faefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4be878740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783faf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4be878740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b3230 con 0x563783fa1210 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4be878740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b46a0 con 0x563783fabad0 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4be878740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b3900 con 0x563783fa08f0 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4be878740 10 client.0 _ll_register_callbacks cb 0x7ff4b0009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4b90fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa1210 0x563783faefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4b90fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa1210 0x563783faefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60422/0 (socket says 172.21.15.69:60422) 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4b90fa640 1 -- 172.21.15.69:0/121951599 learned_addr learned my addr 172.21.15.69:0/121951599 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4ba0fc640 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783faf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa08f0 0x563783fb3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4b90fa640 1 -- 172.21.15.69:0/121951599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa08f0 msgr2=0x563783fb3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4b90fa640 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa08f0 0x563783fb3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4b90fa640 1 -- 172.21.15.69:0/121951599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 msgr2=0x563783faf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4b90fa640 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783faf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4b90fa640 1 -- 172.21.15.69:0/121951599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563783fafa80 con 0x563783fa1210 2024-05-14T16:51:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff4b90fa640 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa1210 0x563783faefd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40a9430 tx=0x7ff4a4005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519c84332ee5aa0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.162+0000 7ff49d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff49d7fa640 1 -- 172.21.15.69:0/121951599 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4020040 con 0x563783fa1210 2024-05-14T16:51:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff49d7fa640 1 -- 172.21.15.69:0/121951599 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff4a40a5730 con 0x563783fa1210 2024-05-14T16:51:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff4be878740 1 -- 172.21.15.69:0/121951599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4b003e070 con 0x563783fa1210 2024-05-14T16:51:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff49d7fa640 1 -- 172.21.15.69:0/121951599 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff4a40a8c10 con 0x563783fa1210 2024-05-14T16:51:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff4be878740 10 client.15567 fetch_fsmap learned FSMap version 31 2024-05-14T16:51:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff4be878740 1 -- 172.21.15.69:0/121951599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563784025e50 con 0x563783fa1210 2024-05-14T16:51:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff49d7fa640 1 -- 172.21.15.69:0/121951599 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b003e070 con 0x563783fa1210 2024-05-14T16:51:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff49d7fa640 1 -- 172.21.15.69:0/121951599 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7ff4a40a3bc0 con 0x563783fa1210 2024-05-14T16:51:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff4be878740 10 client.15567 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:51:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff4be878740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:51:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff4be878740 20 client.15567 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff4be878740 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563784026630 0x563784028a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff4be878740 4 client.15567 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:51:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff4be878740 1 -- 172.21.15.69:0/121951599 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563783f0f360 con 0x563784026630 2024-05-14T16:51:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.163+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563784026630 0x563784028a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.164+0000 7ff49d7fa640 10 client.15567 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.164+0000 7ff4b98fb640 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563784026630 0x563784028a50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb30b4dae8a41bb0 server_cookie=76a76c8af8f24322 in_seq=0 out_seq=0 2024-05-14T16:51:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff49d7fa640 1 -- 172.21.15.69:0/121951599 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563783f0f360 con 0x563784026630 2024-05-14T16:51:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff49d7fa640 10 client.15567 handle_command_reply: tid=0 2024-05-14T16:51:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 client.15567 shutdown 2024-05-14T16:51:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 -- 172.21.15.69:0/121951599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa1210 msgr2=0x563783faefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa1210 0x563783faefd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40a9430 tx=0x7ff4a4005d30 comp rx=0 tx=0).stop 2024-05-14T16:51:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 -- 172.21.15.69:0/121951599 shutdown_connections 2024-05-14T16:51:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563784026630 0x563784028a50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563783fabad0 0x563783faf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563783fa1210 0x563783faefd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 --2- 172.21.15.69:0/121951599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563783fa08f0 0x563783fb3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff49d7fa640 0 client.15567 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 -- 172.21.15.69:0/121951599 >> 172.21.15.69:0/121951599 conn(0x563783f9ce00 msgr2=0x563783fa3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 -- 172.21.15.69:0/121951599 shutdown_connections 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 1 -- 172.21.15.69:0/121951599 wait complete. 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.176+0000 7ff4be878740 20 client.15567 trim_cache size 0 max 16384 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:51:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:51:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/2671552707 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff4a00b2a10 msgr2=0x7ff4a00b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff4a00b2a10 0x7ff4a00b4ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0005fd0 tx=0x7ff4b00094a0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/2671552707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4b41023b0 msgr2=0x7ff4b41a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4b41023b0 0x7ff4b41a5940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400d950 tx=0x7ff4a400de10 comp rx=0 tx=0).stop 2024-05-14T16:51:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/2671552707 shutdown_connections 2024-05-14T16:51:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff4a00b2a10 0x7ff4a00b4ed0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4b4106e90 0x7ff4b419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4b41023b0 0x7ff4b41a5940 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 --2- 172.21.15.69:0/2671552707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b4101a90 0x7ff4b41a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/2671552707 >> 172.21.15.69:0/2671552707 conn(0x7ff4b4078ad0 msgr2=0x7ff4b4104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/2671552707 shutdown_connections 2024-05-14T16:51:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:43.177+0000 7ff4bbb5f640 1 -- 172.21.15.69:0/2671552707 wait complete. 2024-05-14T16:51:43.880 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e3c2299d', quiesce took 9.2 seconds. Will release after: 17.5 2024-05-14T16:51:44.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:43 smithi069 ceph-mon[31507]: pgmap v59: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-05-14T16:51:44.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:43 smithi203 ceph-mon[35787]: pgmap v59: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-05-14T16:51:44.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:43 smithi092 ceph-mon[35763]: pgmap v59: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-05-14T16:51:45.395 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 75.0 s, 1337.5 tps, lat 35.482 ms stddev 73.170 2024-05-14T16:51:46.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:45 smithi092 ceph-mon[35763]: pgmap v60: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-05-14T16:51:46.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:45 smithi069 ceph-mon[31507]: pgmap v60: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-05-14T16:51:46.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:45 smithi203 ceph-mon[35787]: pgmap v60: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-05-14T16:51:48.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:47 smithi069 ceph-mon[31507]: pgmap v61: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.61k op/s 2024-05-14T16:51:48.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:47 smithi203 ceph-mon[35787]: pgmap v61: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.61k op/s 2024-05-14T16:51:48.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:47 smithi092 ceph-mon[35763]: pgmap v61: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.61k op/s 2024-05-14T16:51:50.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:49 smithi069 ceph-mon[31507]: pgmap v62: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 1.28k op/s 2024-05-14T16:51:50.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:49 smithi203 ceph-mon[35787]: pgmap v62: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 1.28k op/s 2024-05-14T16:51:50.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:49 smithi092 ceph-mon[35763]: pgmap v62: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 1.28k op/s 2024-05-14T16:51:50.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 80.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:51:52.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:51 smithi069 ceph-mon[31507]: pgmap v63: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.38k op/s 2024-05-14T16:51:52.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:51 smithi203 ceph-mon[35787]: pgmap v63: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.38k op/s 2024-05-14T16:51:52.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:51 smithi092 ceph-mon[35763]: pgmap v63: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.38k op/s 2024-05-14T16:51:54.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:53 smithi069 ceph-mon[31507]: pgmap v64: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 MiB/s wr, 985 op/s 2024-05-14T16:51:54.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:53 smithi203 ceph-mon[35787]: pgmap v64: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 MiB/s wr, 985 op/s 2024-05-14T16:51:54.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:53 smithi092 ceph-mon[35763]: pgmap v64: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 MiB/s wr, 985 op/s 2024-05-14T16:51:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 85.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:51:56.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:55 smithi069 ceph-mon[31507]: pgmap v65: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 MiB/s wr, 985 op/s 2024-05-14T16:51:56.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:55 smithi203 ceph-mon[35787]: pgmap v65: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 MiB/s wr, 985 op/s 2024-05-14T16:51:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:55 smithi092 ceph-mon[35763]: pgmap v65: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 MiB/s wr, 985 op/s 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.276+0000 7f7c8f41c640 1 -- 172.21.15.69:0/1181603709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c88102a60 msgr2=0x7f7c88102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.276+0000 7f7c8f41c640 1 --2- 172.21.15.69:0/1181603709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c88102a60 0x7f7c88102ee0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009930 tx=0x7f7c7802a420 comp rx=0 tx=0).stop 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.276+0000 7f7c8f41c640 1 -- 172.21.15.69:0/1181603709 shutdown_connections 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.276+0000 7f7c8f41c640 1 --2- 172.21.15.69:0/1181603709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88103420 0x7f7c8810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.276+0000 7f7c8f41c640 1 --2- 172.21.15.69:0/1181603709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c88102a60 0x7f7c88102ee0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.276+0000 7f7c8f41c640 1 --2- 172.21.15.69:0/1181603709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c88108a70 0x7f7c88108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.276+0000 7f7c8f41c640 1 -- 172.21.15.69:0/1181603709 >> 172.21.15.69:0/1181603709 conn(0x7f7c880fe550 msgr2=0x7f7c88100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.276+0000 7f7c8f41c640 1 -- 172.21.15.69:0/1181603709 shutdown_connections 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.276+0000 7f7c8f41c640 1 -- 172.21.15.69:0/1181603709 wait complete. 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8f41c640 1 Processor -- start 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8f41c640 1 -- start start 2024-05-14T16:51:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8f41c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102a60 0x7f7c88199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8f41c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c88103420 0x7f7c881997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8f41c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c88108a70 0x7f7c8819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8f41c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88112d00 con 0x7f7c88103420 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8f41c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88112b80 con 0x7f7c88102a60 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8f41c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88112e80 con 0x7f7c88108a70 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8c9b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c88103420 0x7f7c881997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8c9b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c88103420 0x7f7c881997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45400/0 (socket says 172.21.15.69:45400) 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8c9b7640 1 -- 172.21.15.69:0/4249817141 learned_addr learned my addr 172.21.15.69:0/4249817141 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c88108a70 0x7f7c8819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8d1b8640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102a60 0x7f7c88199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8c9b7640 1 -- 172.21.15.69:0/4249817141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c88108a70 msgr2=0x7f7c8819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8c9b7640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c88108a70 0x7f7c8819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8c9b7640 1 -- 172.21.15.69:0/4249817141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102a60 msgr2=0x7f7c88199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8c9b7640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102a60 0x7f7c88199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.277+0000 7f7c8c9b7640 1 -- 172.21.15.69:0/4249817141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8819e200 con 0x7f7c88103420 2024-05-14T16:51:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.278+0000 7f7c8c9b7640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c88103420 0x7f7c881997a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c78004770 tx=0x7f7c780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.278+0000 7f7c7e7fc640 1 -- 172.21.15.69:0/4249817141 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7803f070 con 0x7f7c88103420 2024-05-14T16:51:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.278+0000 7f7c7e7fc640 1 -- 172.21.15.69:0/4249817141 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c7803b460 con 0x7f7c88103420 2024-05-14T16:51:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.278+0000 7f7c7e7fc640 1 -- 172.21.15.69:0/4249817141 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78035070 con 0x7f7c88103420 2024-05-14T16:51:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.278+0000 7f7c8f41c640 1 -- 172.21.15.69:0/4249817141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8819e490 con 0x7f7c88103420 2024-05-14T16:51:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.278+0000 7f7c8f41c640 1 -- 172.21.15.69:0/4249817141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c881a5d80 con 0x7f7c88103420 2024-05-14T16:51:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.281+0000 7f7c9221a740 1 Processor -- start 2024-05-14T16:51:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.281+0000 7f7c9221a740 1 -- start start 2024-05-14T16:51:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c7e7fc640 1 -- 172.21.15.69:0/4249817141 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7c7803b8d0 con 0x7f7c88103420 2024-05-14T16:51:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 0x563bb4513cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb451e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4520ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c9221a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563bb4486c30 con 0x563bb451ead0 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c9221a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563bb44767d0 con 0x563bb4514210 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c9221a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563bb447a810 con 0x563bb45138f0 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8c9b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4520ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8c9b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4520ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45416/0 (socket says 172.21.15.69:45416) 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8c9b7640 1 -- 172.21.15.69:0/133550524 learned_addr learned my addr 172.21.15.69:0/133550524 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/133550524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 0x563bb4513cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8d1b8640 1 --2- 172.21.15.69:0/133550524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb451e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8c9b7640 1 -- 172.21.15.69:0/133550524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 msgr2=0x563bb4513cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8c9b7640 1 --2- 172.21.15.69:0/133550524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 0x563bb4513cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8c9b7640 1 -- 172.21.15.69:0/133550524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 msgr2=0x563bb451e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8c9b7640 1 --2- 172.21.15.69:0/133550524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb451e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8c9b7640 1 -- 172.21.15.69:0/133550524 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb418b210 con 0x563bb451ead0 2024-05-14T16:51:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/133550524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 0x563bb4513cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:51:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c7e7fc640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7c740b2a10 0x7f7c740b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.282+0000 7f7c8c9b7640 1 --2- 172.21.15.69:0/133550524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4520ec0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7c78100050 tx=0x7f7c78108cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a782ca8e5ace8b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.283+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7c740b2a10 0x7f7c740b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.283+0000 7f7c737fe640 1 -- 172.21.15.69:0/133550524 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c780c5b30 con 0x563bb451ead0 2024-05-14T16:51:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.283+0000 7f7c7e7fc640 1 -- 172.21.15.69:0/4249817141 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7c78102980 con 0x7f7c88103420 2024-05-14T16:51:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.283+0000 7f7c737fe640 1 -- 172.21.15.69:0/133550524 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c780c9780 con 0x563bb451ead0 2024-05-14T16:51:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.283+0000 7f7c737fe640 1 -- 172.21.15.69:0/133550524 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c780c93e0 con 0x563bb451ead0 2024-05-14T16:51:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.283+0000 7f7c9221a740 1 -- 172.21.15.69:0/133550524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 msgr2=0x563bb4520ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.283+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7c740b2a10 0x7f7c740b4ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005fd0 tx=0x7f7c80009600 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.283+0000 7f7c9221a740 1 --2- 172.21.15.69:0/133550524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4520ec0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7c78100050 tx=0x7f7c78108cb0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 -- 172.21.15.69:0/133550524 shutdown_connections 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 --2- 172.21.15.69:0/133550524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4520ec0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 --2- 172.21.15.69:0/133550524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb451e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 --2- 172.21.15.69:0/133550524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 0x563bb4513cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 -- 172.21.15.69:0/133550524 >> 172.21.15.69:0/133550524 conn(0x563bb450fe00 msgr2=0x563bb45101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 -- 172.21.15.69:0/133550524 shutdown_connections 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 -- 172.21.15.69:0/133550524 wait complete. 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 -- start start 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 Processor -- start 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb45138f0 0x563bb451e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb451b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb451ead0 0x563bb451bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563bb415c790 con 0x563bb45138f0 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563bb4432890 con 0x563bb4514210 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563bb45214c0 con 0x563bb451ead0 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.284+0000 7f7c9221a740 10 client.0 _ll_register_callbacks cb 0x563bb4435d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8d1b8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb45138f0 0x563bb451e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8d1b8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb45138f0 0x563bb451e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45418/0 (socket says 172.21.15.69:45418) 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8d1b8640 1 -- 172.21.15.69:0/2015003034 learned_addr learned my addr 172.21.15.69:0/2015003034 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8c9b7640 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb451b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb451ead0 0x563bb451bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8d1b8640 1 -- 172.21.15.69:0/2015003034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb451ead0 msgr2=0x563bb451bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8d1b8640 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb451ead0 0x563bb451bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8d1b8640 1 -- 172.21.15.69:0/2015003034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 msgr2=0x563bb451b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8d1b8640 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb451b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8d1b8640 1 -- 172.21.15.69:0/2015003034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb4257620 con 0x563bb45138f0 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c8d1b8640 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb45138f0 0x563bb451e3a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400d9a0 tx=0x7f7c8400de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5922d320161bc947 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c70ff9640 1 -- 172.21.15.69:0/2015003034 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8401b640 con 0x563bb45138f0 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c70ff9640 1 -- 172.21.15.69:0/2015003034 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7c8401b7e0 con 0x563bb45138f0 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c9221a740 1 -- 172.21.15.69:0/2015003034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c780c9780 con 0x563bb45138f0 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c70ff9640 1 -- 172.21.15.69:0/2015003034 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7c8401b9e0 con 0x563bb45138f0 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c9221a740 10 client.15579 fetch_fsmap learned FSMap version 31 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.285+0000 7f7c9221a740 1 -- 172.21.15.69:0/2015003034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563bb452cf30 con 0x563bb45138f0 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c70ff9640 1 -- 172.21.15.69:0/2015003034 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c780c9780 con 0x563bb45138f0 2024-05-14T16:51:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c70ff9640 1 -- 172.21.15.69:0/2015003034 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f7c84019020 con 0x563bb45138f0 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c9221a740 10 client.15579 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c9221a740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c9221a740 20 client.15579 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c9221a740 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563bb452e460 0x563bb45308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c9221a740 4 client.15579 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c9221a740 1 -- 172.21.15.69:0/2015003034 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bb4488b80 con 0x563bb452e460 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c8c9b7640 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563bb452e460 0x563bb45308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c8c9b7640 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563bb452e460 0x563bb45308b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b4a2f48b9a61ee0 server_cookie=13a86af2dd69990f in_seq=0 out_seq=0 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.286+0000 7f7c70ff9640 10 client.15579 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c70ff9640 1 -- 172.21.15.69:0/2015003034 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563bb4488b80 con 0x563bb452e460 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c70ff9640 10 client.15579 handle_command_reply: tid=0 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 client.15579 shutdown 2024-05-14T16:51:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 -- 172.21.15.69:0/2015003034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb45138f0 msgr2=0x563bb451e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb45138f0 0x563bb451e3a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400d9a0 tx=0x7f7c8400de60 comp rx=0 tx=0).stop 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 -- 172.21.15.69:0/2015003034 shutdown_connections 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563bb452e460 0x563bb45308b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb451ead0 0x563bb451bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c70ff9640 0 client.15579 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb451b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 --2- 172.21.15.69:0/2015003034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb45138f0 0x563bb451e3a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 -- 172.21.15.69:0/2015003034 >> 172.21.15.69:0/2015003034 conn(0x563bb450fe00 msgr2=0x563bb450f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 -- 172.21.15.69:0/2015003034 shutdown_connections 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.287+0000 7f7c9221a740 1 -- 172.21.15.69:0/2015003034 wait complete. 2024-05-14T16:51:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.288+0000 7f7c9221a740 20 client.15579 trim_cache size 0 max 16384 2024-05-14T16:51:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c9221a740 1 Processor -- start 2024-05-14T16:51:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c9221a740 1 -- start start 2024-05-14T16:51:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb45138f0 0x563bb452a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb4514210 0x563bb4536210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4536750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c9221a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881b0f90 con 0x563bb451ead0 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c9221a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881ae7d0 con 0x563bb45138f0 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c9221a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881ac620 con 0x563bb4514210 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c8d9b9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4536750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c8d9b9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4536750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45432/0 (socket says 172.21.15.69:45432) 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c8d9b9640 1 -- 172.21.15.69:0/4238991354 learned_addr learned my addr 172.21.15.69:0/4238991354 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c8d1b8640 1 --2- 172.21.15.69:0/4238991354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb45138f0 0x563bb452a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c8d9b9640 1 -- 172.21.15.69:0/4238991354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb4514210 msgr2=0x563bb4536210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/4238991354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb4514210 0x563bb4536210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c8d9b9640 1 -- 172.21.15.69:0/4238991354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb45138f0 msgr2=0x563bb452a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/4238991354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb45138f0 0x563bb452a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c8d9b9640 1 -- 172.21.15.69:0/4238991354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb451a640 con 0x563bb451ead0 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/4238991354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4536750 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7c800106e0 tx=0x7f7c80019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768ebb7acf33e469 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.303+0000 7f7c72ffd640 1 -- 172.21.15.69:0/4238991354 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80004d20 con 0x563bb451ead0 2024-05-14T16:51:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c72ffd640 1 -- 172.21.15.69:0/4238991354 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c8001c9f0 con 0x563bb451ead0 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c72ffd640 1 -- 172.21.15.69:0/4238991354 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80005650 con 0x563bb451ead0 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 -- 172.21.15.69:0/4238991354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 msgr2=0x563bb4536750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 --2- 172.21.15.69:0/4238991354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4536750 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7c800106e0 tx=0x7f7c80019000 comp rx=0 tx=0).stop 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 -- 172.21.15.69:0/4238991354 shutdown_connections 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 --2- 172.21.15.69:0/4238991354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4536750 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 --2- 172.21.15.69:0/4238991354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb4514210 0x563bb4536210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 --2- 172.21.15.69:0/4238991354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb45138f0 0x563bb452a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 -- 172.21.15.69:0/4238991354 >> 172.21.15.69:0/4238991354 conn(0x563bb450fe00 msgr2=0x563bb450f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 -- 172.21.15.69:0/4238991354 shutdown_connections 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 -- 172.21.15.69:0/4238991354 wait complete. 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 -- start start 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.304+0000 7f7c9221a740 1 Processor -- start 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 0x563bb4524450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb4524990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c9221a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4524ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c9221a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881acc90 con 0x563bb451ead0 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c9221a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881ae100 con 0x563bb4514210 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c9221a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881ad360 con 0x563bb45138f0 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c9221a740 10 client.0 _ll_register_callbacks cb 0x7f7c84004310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8d9b9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4524ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8d9b9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4524ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45438/0 (socket says 172.21.15.69:45438) 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8d9b9640 1 -- 172.21.15.69:0/1190380658 learned_addr learned my addr 172.21.15.69:0/1190380658 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8d9b9640 1 -- 172.21.15.69:0/1190380658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 msgr2=0x563bb4524450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8d1b8640 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 0x563bb4524450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 0x563bb4524450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8c9b7640 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb4524990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8d9b9640 1 -- 172.21.15.69:0/1190380658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 msgr2=0x563bb4524990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb4524990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8d9b9640 1 -- 172.21.15.69:0/1190380658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb4597f30 con 0x563bb451ead0 2024-05-14T16:51:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c8d9b9640 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4524ed0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005190 tx=0x7f7c8001cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f4fa97ff0d97bb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.305+0000 7f7c70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:51:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c70ff9640 1 -- 172.21.15.69:0/1190380658 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8002a050 con 0x563bb451ead0 2024-05-14T16:51:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c70ff9640 1 -- 172.21.15.69:0/1190380658 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7c80004a40 con 0x563bb451ead0 2024-05-14T16:51:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c9221a740 1 -- 172.21.15.69:0/1190380658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c8001c9f0 con 0x563bb451ead0 2024-05-14T16:51:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c70ff9640 1 -- 172.21.15.69:0/1190380658 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7c8003e050 con 0x563bb451ead0 2024-05-14T16:51:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c9221a740 10 client.15585 fetch_fsmap learned FSMap version 31 2024-05-14T16:51:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c9221a740 1 -- 172.21.15.69:0/1190380658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563bb4599220 con 0x563bb451ead0 2024-05-14T16:51:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c70ff9640 1 -- 172.21.15.69:0/1190380658 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c8001c9f0 con 0x563bb451ead0 2024-05-14T16:51:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c70ff9640 1 -- 172.21.15.69:0/1190380658 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f7c8000f040 con 0x563bb451ead0 2024-05-14T16:51:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c9221a740 10 client.15585 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:51:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c9221a740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:51:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c9221a740 20 client.15585 populate_metadata read hostname 'smithi069' 2024-05-14T16:51:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c9221a740 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563bb4599940 0x563bb459bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c9221a740 4 client.15585 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:51:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c9221a740 1 -- 172.21.15.69:0/1190380658 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563bb4482360 con 0x563bb4599940 2024-05-14T16:51:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.306+0000 7f7c8d1b8640 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563bb4599940 0x563bb459bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.307+0000 7f7c70ff9640 10 client.15585 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.307+0000 7f7c8d1b8640 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563bb4599940 0x563bb459bd60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899ba488ec32034c server_cookie=248947f1cf800915 in_seq=0 out_seq=0 2024-05-14T16:51:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c70ff9640 1 -- 172.21.15.69:0/1190380658 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563bb4482360 con 0x563bb4599940 2024-05-14T16:51:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c70ff9640 10 client.15585 handle_command_reply: tid=0 2024-05-14T16:51:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 client.15585 shutdown 2024-05-14T16:51:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 -- 172.21.15.69:0/1190380658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 msgr2=0x563bb4524ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4524ed0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005190 tx=0x7f7c8001cee0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 -- 172.21.15.69:0/1190380658 shutdown_connections 2024-05-14T16:51:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563bb4599940 0x563bb459bd60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bb451ead0 0x563bb4524ed0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563bb4514210 0x563bb4524990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 --2- 172.21.15.69:0/1190380658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bb45138f0 0x563bb4524450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 -- 172.21.15.69:0/1190380658 >> 172.21.15.69:0/1190380658 conn(0x563bb450fe00 msgr2=0x563bb45293b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c70ff9640 0 client.15585 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:51:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 -- 172.21.15.69:0/1190380658 shutdown_connections 2024-05-14T16:51:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 1 -- 172.21.15.69:0/1190380658 wait complete. 2024-05-14T16:51:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.320+0000 7f7c9221a740 20 client.15585 trim_cache size 0 max 16384 2024-05-14T16:51:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:51:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:51:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:51:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 -- 172.21.15.69:0/4249817141 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7c740b2a10 msgr2=0x7f7c740b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7c740b2a10 0x7f7c740b4ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005fd0 tx=0x7f7c80009600 comp rx=0 tx=0).stop 2024-05-14T16:51:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 -- 172.21.15.69:0/4249817141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c88103420 msgr2=0x7f7c881997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c88103420 0x7f7c881997a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c78004770 tx=0x7f7c780042f0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 -- 172.21.15.69:0/4249817141 shutdown_connections 2024-05-14T16:51:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7c740b2a10 0x7f7c740b4ed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c88108a70 0x7f7c8819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c88103420 0x7f7c881997a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 --2- 172.21.15.69:0/4249817141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102a60 0x7f7c88199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 -- 172.21.15.69:0/4249817141 >> 172.21.15.69:0/4249817141 conn(0x7f7c880fe550 msgr2=0x7f7c880fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 -- 172.21.15.69:0/4249817141 shutdown_connections 2024-05-14T16:51:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:51:56.321+0000 7f7c8f41c640 1 -- 172.21.15.69:0/4249817141 wait complete. 2024-05-14T16:51:58.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:57 smithi069 ceph-mon[31507]: pgmap v66: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 988 op/s 2024-05-14T16:51:58.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:57 smithi203 ceph-mon[35787]: pgmap v66: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 988 op/s 2024-05-14T16:51:58.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:57 smithi092 ceph-mon[35763]: pgmap v66: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 988 op/s 2024-05-14T16:52:00.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:51:59 smithi069 ceph-mon[31507]: pgmap v67: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.7 MiB/s wr, 96 op/s 2024-05-14T16:52:00.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:51:59 smithi203 ceph-mon[35787]: pgmap v67: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.7 MiB/s wr, 96 op/s 2024-05-14T16:52:00.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:51:59 smithi092 ceph-mon[35763]: pgmap v67: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.7 MiB/s wr, 96 op/s 2024-05-14T16:52:00.398 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 90.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:52:01.382 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:52:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:02.019 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:01 smithi069 ceph-mon[31507]: pgmap v68: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 312 op/s 2024-05-14T16:52:02.019 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:01 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:02.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:01 smithi203 ceph-mon[35787]: pgmap v68: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 312 op/s 2024-05-14T16:52:02.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:01 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:02.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:01 smithi092 ceph-mon[35763]: pgmap v68: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 312 op/s 2024-05-14T16:52:02.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:01 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:03.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:02 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:52:03.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:02 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:52:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:02 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:52:03.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:02 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:52:03.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:02 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:52:03.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:02 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:52:03.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:02 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:52:03.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:02 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:52:03.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:02 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:52:03.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.399+0000 7fb5ff519700 1 -- 172.21.15.69:0/2155131706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8102e00 msgr2=0x7fb5f810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:03.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.399+0000 7fb5ff519700 1 --2- 172.21.15.69:0/2155131706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8102e00 0x7fb5f810f2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000b0a0 tx=0x7fb5f000b3b0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.399+0000 7fb5ff519700 1 -- 172.21.15.69:0/2155131706 shutdown_connections 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.399+0000 7fb5ff519700 1 --2- 172.21.15.69:0/2155131706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8102e00 0x7fb5f810f2f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.399+0000 7fb5ff519700 1 --2- 172.21.15.69:0/2155131706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f8108520 0x7fb5f8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.399+0000 7fb5ff519700 1 --2- 172.21.15.69:0/2155131706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8102440 0x7fb5f81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.399+0000 7fb5ff519700 1 -- 172.21.15.69:0/2155131706 >> 172.21.15.69:0/2155131706 conn(0x7fb5f80793f0 msgr2=0x7fb5f8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.399+0000 7fb5ff519700 1 -- 172.21.15.69:0/2155131706 shutdown_connections 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.399+0000 7fb5ff519700 1 -- 172.21.15.69:0/2155131706 wait complete. 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5ff519700 1 Processor -- start 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5ff519700 1 -- start start 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5ff519700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8102440 0x7fb5f8199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5ff519700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f8102e00 0x7fb5f819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5ff519700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8108520 0x7fb5f819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:03.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5ff519700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8112650 con 0x7fb5f8108520 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5ff519700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f81124d0 con 0x7fb5f8102440 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5ff519700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f81127d0 con 0x7fb5f8102e00 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fdab6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8108520 0x7fb5f819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fdab6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8108520 0x7fb5f819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45442/0 (socket says 172.21.15.69:45442) 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fdab6700 1 -- 172.21.15.69:0/1183756757 learned_addr learned my addr 172.21.15.69:0/1183756757 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fdab6700 1 -- 172.21.15.69:0/1183756757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f8102e00 msgr2=0x7fb5f819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fcab4700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f8102e00 0x7fb5f819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fd2b5700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8102440 0x7fb5f8199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fdab6700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f8102e00 0x7fb5f819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fdab6700 1 -- 172.21.15.69:0/1183756757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8102440 msgr2=0x7fb5f8199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fdab6700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8102440 0x7fb5f8199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fdab6700 1 -- 172.21.15.69:0/1183756757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f819ed20 con 0x7fb5f8108520 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fcab4700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f8102e00 0x7fb5f819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:52:03.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fd2b5700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8102440 0x7fb5f8199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:52:03.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.400+0000 7fb5fdab6700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8108520 0x7fb5f819e5e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000b830 tx=0x7fb5f0004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:03.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.401+0000 7fb5f67fc700 1 -- 172.21.15.69:0/1183756757 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0021070 con 0x7fb5f8108520 2024-05-14T16:52:03.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.401+0000 7fb5f67fc700 1 -- 172.21.15.69:0/1183756757 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5f0003c00 con 0x7fb5f8108520 2024-05-14T16:52:03.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.401+0000 7fb5f67fc700 1 -- 172.21.15.69:0/1183756757 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0004730 con 0x7fb5f8108520 2024-05-14T16:52:03.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.401+0000 7fb5ff519700 1 -- 172.21.15.69:0/1183756757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f819efb0 con 0x7fb5f8108520 2024-05-14T16:52:03.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.401+0000 7fb5ff519700 1 -- 172.21.15.69:0/1183756757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f81a6890 con 0x7fb5f8108520 2024-05-14T16:52:03.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.402+0000 7fb5dffff700 1 -- 172.21.15.69:0/1183756757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f806a230 con 0x7fb5f8108520 2024-05-14T16:52:03.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.403+0000 7fb5f67fc700 1 -- 172.21.15.69:0/1183756757 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb5f0007a00 con 0x7fb5f8108520 2024-05-14T16:52:03.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.404+0000 7fb5f67fc700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5e00b2a70 0x7fb5e00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:03.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.404+0000 7fb5fd2b5700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5e00b2a70 0x7fb5e00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:03.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.404+0000 7fb5f67fc700 1 -- 172.21.15.69:0/1183756757 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb5f000f020 con 0x7fb5f8108520 2024-05-14T16:52:03.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.404+0000 7fb5fd2b5700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5e00b2a70 0x7fb5e00b4f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8007900 tx=0x7fb5e8008040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:03.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.407+0000 7fb5f67fc700 1 -- 172.21.15.69:0/1183756757 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5f002a020 con 0x7fb5f8108520 2024-05-14T16:52:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.660+0000 7fb5dffff700 1 -- 172.21.15.69:0/1183756757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5f819b070 con 0x7fb5f8108520 2024-05-14T16:52:03.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.660+0000 7fb5f67fc700 1 -- 172.21.15.69:0/1183756757 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11863 (secure 0 0 0) 0x7fb5f00abaf0 con 0x7fb5f8108520 2024-05-14T16:52:03.660 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:52:03.660 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:51:16.740826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:52:03.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 -- 172.21.15.69:0/1183756757 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5e00b2a70 msgr2=0x7fb5e00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:03.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5e00b2a70 0x7fb5e00b4f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8007900 tx=0x7fb5e8008040 comp rx=0 tx=0).stop 2024-05-14T16:52:03.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 -- 172.21.15.69:0/1183756757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8108520 msgr2=0x7fb5f819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:03.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8108520 0x7fb5f819e5e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000b830 tx=0x7fb5f0004d90 comp rx=0 tx=0).stop 2024-05-14T16:52:03.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 -- 172.21.15.69:0/1183756757 shutdown_connections 2024-05-14T16:52:03.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5e00b2a70 0x7fb5e00b4f30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8108520 0x7fb5f819e5e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8102440 0x7fb5f8199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 --2- 172.21.15.69:0/1183756757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f8102e00 0x7fb5f819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 -- 172.21.15.69:0/1183756757 >> 172.21.15.69:0/1183756757 conn(0x7fb5f80793f0 msgr2=0x7fb5f806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:03.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 -- 172.21.15.69:0/1183756757 shutdown_connections 2024-05-14T16:52:03.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:03.663+0000 7fb5dffff700 1 -- 172.21.15.69:0/1183756757 wait complete. 2024-05-14T16:52:03.664 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 31 2024-05-14T16:52:04.375 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:04 smithi069 ceph-mon[31507]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-05-14T16:52:04.376 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:04 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1183756757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:04.404 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id e3c2299d --release --await' 2024-05-14T16:52:04.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id e3c2299d --release --await 2024-05-14T16:52:04.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:04 smithi203 ceph-mon[35787]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-05-14T16:52:04.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:04 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1183756757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:04.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:04 smithi092 ceph-mon[35763]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-05-14T16:52:04.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:04 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1183756757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:04.650 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:05.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 95.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:52:06.330 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:06 smithi069 ceph-mon[31507]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-05-14T16:52:06.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.439+0000 7f9a0d864700 1 -- 172.21.15.69:0/1440521544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a0806b220 msgr2=0x7f9a0806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.439+0000 7f9a0d864700 1 --2- 172.21.15.69:0/1440521544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a0806b220 0x7f9a0806b6a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009990 tx=0x7f99fc009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.439+0000 7f9a0d864700 1 -- 172.21.15.69:0/1440521544 shutdown_connections 2024-05-14T16:52:06.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.439+0000 7f9a0d864700 1 --2- 172.21.15.69:0/1440521544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a0806bbe0 0x7f9a081103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.439+0000 7f9a0d864700 1 --2- 172.21.15.69:0/1440521544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a081095b0 0x7f9a08109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.439+0000 7f9a0d864700 1 --2- 172.21.15.69:0/1440521544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a0806b220 0x7f9a0806b6a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.439+0000 7f9a0d864700 1 -- 172.21.15.69:0/1440521544 >> 172.21.15.69:0/1440521544 conn(0x7f9a080793f0 msgr2=0x7f9a08079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:06.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.439+0000 7f9a0d864700 1 -- 172.21.15.69:0/1440521544 shutdown_connections 2024-05-14T16:52:06.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.439+0000 7f9a0d864700 1 -- 172.21.15.69:0/1440521544 wait complete. 2024-05-14T16:52:06.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.440+0000 7f9a0d864700 1 Processor -- start 2024-05-14T16:52:06.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.440+0000 7f9a0d864700 1 -- start start 2024-05-14T16:52:06.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.440+0000 7f9a0d864700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a0806b220 0x7f9a0819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.440+0000 7f9a0d864700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a0806bbe0 0x7f9a0819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.440+0000 7f9a0d864700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a081095b0 0x7f9a08199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.440+0000 7f9a0d864700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081136c0 con 0x7f9a0806bbe0 2024-05-14T16:52:06.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.440+0000 7f9a0d864700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08113540 con 0x7f9a081095b0 2024-05-14T16:52:06.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.440+0000 7f9a0d864700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08113840 con 0x7f9a0806b220 2024-05-14T16:52:06.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a067fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a0806bbe0 0x7f9a0819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a067fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a0806bbe0 0x7f9a0819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51090/0 (socket says 172.21.15.69:51090) 2024-05-14T16:52:06.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a067fc700 1 -- 172.21.15.69:0/4019651508 learned_addr learned my addr 172.21.15.69:0/4019651508 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:06.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a077fe700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a081095b0 0x7f9a08199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a0806b220 0x7f9a0819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a067fc700 1 -- 172.21.15.69:0/4019651508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a0806b220 msgr2=0x7f9a0819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a067fc700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a0806b220 0x7f9a0819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a067fc700 1 -- 172.21.15.69:0/4019651508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a081095b0 msgr2=0x7f9a08199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a067fc700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a081095b0 0x7f9a08199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a067fc700 1 -- 172.21.15.69:0/4019651508 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0819a180 con 0x7f9a0806bbe0 2024-05-14T16:52:06.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a0806b220 0x7f9a0819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:06.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a067fc700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a0806bbe0 0x7f9a0819fda0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f99fc000c00 tx=0x7f99fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:06.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a0c862700 1 -- 172.21.15.69:0/4019651508 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc027070 con 0x7f9a0806bbe0 2024-05-14T16:52:06.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a0d864700 1 -- 172.21.15.69:0/4019651508 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0819a410 con 0x7f9a0806bbe0 2024-05-14T16:52:06.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a0d864700 1 -- 172.21.15.69:0/4019651508 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a081a6730 con 0x7f9a0806bbe0 2024-05-14T16:52:06.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a0c862700 1 -- 172.21.15.69:0/4019651508 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99fc0040e0 con 0x7f9a0806bbe0 2024-05-14T16:52:06.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.441+0000 7f9a0c862700 1 -- 172.21.15.69:0/4019651508 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00f460 con 0x7f9a0806bbe0 2024-05-14T16:52:06.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.444+0000 7f9a0c862700 1 -- 172.21.15.69:0/4019651508 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f99fc02a070 con 0x7f9a0806bbe0 2024-05-14T16:52:06.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.444+0000 7f9a0c862700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99f00b2a70 0x7f99f00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99f00b2a70 0x7f99f00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a0c862700 1 -- 172.21.15.69:0/4019651508 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f99fc0e32c0 con 0x7f9a0806bbe0 2024-05-14T16:52:06.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a180fab80 1 Processor -- start 2024-05-14T16:52:06.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99f00b2a70 0x7f99f00b4f30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f99f8005ea0 tx=0x7f99f8005e30 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:06.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a180fab80 1 -- start start 2024-05-14T16:52:06.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240420 0x55f82e240800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240d40 0x55f82e23bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e23c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a180fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f82e199e70 con 0x55f82e23c2c0 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a180fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f82e189a10 con 0x55f82e240420 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a180fab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f82e18da50 con 0x55f82e240d40 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a06ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240420 0x55f82e240800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a077fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e23c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a06ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240420 0x55f82e240800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:33832/0 (socket says 172.21.15.69:33832) 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.445+0000 7f9a06ffd700 1 -- 172.21.15.69:0/1021794510 learned_addr learned my addr 172.21.15.69:0/1021794510 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f9a067fc700 1 --2- 172.21.15.69:0/1021794510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240d40 0x55f82e23bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f9a077fe700 1 -- 172.21.15.69:0/1021794510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240d40 msgr2=0x55f82e23bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f9a077fe700 1 --2- 172.21.15.69:0/1021794510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240d40 0x55f82e23bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f9a077fe700 1 -- 172.21.15.69:0/1021794510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240420 msgr2=0x55f82e240800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:06.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f9a077fe700 1 --2- 172.21.15.69:0/1021794510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240420 0x55f82e240800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f9a077fe700 1 -- 172.21.15.69:0/1021794510 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f82e1330b0 con 0x55f82e23c2c0 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f9a067fc700 1 --2- 172.21.15.69:0/1021794510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240d40 0x55f82e23bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f9a077fe700 1 --2- 172.21.15.69:0/1021794510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e23c720 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000a5d0 tx=0x7f9a000078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f37d8899a0c778 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f99edffb700 1 -- 172.21.15.69:0/1021794510 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00003a00 con 0x55f82e23c2c0 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f99edffb700 1 -- 172.21.15.69:0/1021794510 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a00003ba0 con 0x55f82e23c2c0 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.446+0000 7f99edffb700 1 -- 172.21.15.69:0/1021794510 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00014660 con 0x55f82e23c2c0 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.447+0000 7f9a180fab80 1 -- 172.21.15.69:0/1021794510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 msgr2=0x55f82e23c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.447+0000 7f9a180fab80 1 --2- 172.21.15.69:0/1021794510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e23c720 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000a5d0 tx=0x7f9a000078d0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.447+0000 7f9a180fab80 1 -- 172.21.15.69:0/1021794510 shutdown_connections 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.447+0000 7f9a180fab80 1 --2- 172.21.15.69:0/1021794510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e23c720 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.447+0000 7f9a180fab80 1 --2- 172.21.15.69:0/1021794510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240420 0x55f82e240800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.447+0000 7f9a180fab80 1 --2- 172.21.15.69:0/1021794510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240d40 0x55f82e23bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.447+0000 7f9a180fab80 1 -- 172.21.15.69:0/1021794510 >> 172.21.15.69:0/1021794510 conn(0x55f82e168eb0 msgr2=0x55f82e2364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:06.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.447+0000 7f9a180fab80 1 -- 172.21.15.69:0/1021794510 shutdown_connections 2024-05-14T16:52:06.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.447+0000 7f9a180fab80 1 -- 172.21.15.69:0/1021794510 wait complete. 2024-05-14T16:52:06.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.447+0000 7f9a180fab80 1 -- start start 2024-05-14T16:52:06.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a180fab80 1 Processor -- start 2024-05-14T16:52:06.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e23c2c0 0x55f82e247a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e244e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e245370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a180fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f82e0900a0 con 0x55f82e240d40 2024-05-14T16:52:06.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a180fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f82e0f1ea0 con 0x55f82e23c2c0 2024-05-14T16:52:06.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a180fab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f82e248880 con 0x55f82e240420 2024-05-14T16:52:06.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a180fab80 10 client.0 _ll_register_callbacks cb 0x55f82e124a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:06.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a077fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e245370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a077fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e245370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51114/0 (socket says 172.21.15.69:51114) 2024-05-14T16:52:06.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a077fe700 1 -- 172.21.15.69:0/3460855167 learned_addr learned my addr 172.21.15.69:0/3460855167 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e23c2c0 0x55f82e247a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a067fc700 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e244e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a077fe700 1 -- 172.21.15.69:0/3460855167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 msgr2=0x55f82e244e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a077fe700 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e244e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a077fe700 1 -- 172.21.15.69:0/3460855167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e23c2c0 msgr2=0x55f82e247a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a077fe700 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e23c2c0 0x55f82e247a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.448+0000 7f9a077fe700 1 -- 172.21.15.69:0/3460855167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f82e0d99d0 con 0x55f82e240d40 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f9a077fe700 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e245370 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9a00007700 tx=0x7f9a0000af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98168732fdb4954a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f99e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f99e6ffd700 1 -- 172.21.15.69:0/3460855167 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00004210 con 0x55f82e240d40 2024-05-14T16:52:06.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f99e6ffd700 1 -- 172.21.15.69:0/3460855167 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f9a000043b0 con 0x55f82e240d40 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f9a180fab80 1 -- 172.21.15.69:0/3460855167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a00003ba0 con 0x55f82e240d40 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f99e6ffd700 1 -- 172.21.15.69:0/3460855167 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9a0001e420 con 0x55f82e240d40 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f9a180fab80 10 client.15603 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f9a180fab80 1 -- 172.21.15.69:0/3460855167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f82e143860 con 0x55f82e240d40 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f99e6ffd700 1 -- 172.21.15.69:0/3460855167 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a00003ba0 con 0x55f82e240d40 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f99e6ffd700 1 -- 172.21.15.69:0/3460855167 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f9a00026590 con 0x55f82e240d40 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f9a180fab80 10 client.15603 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f9a180fab80 10 client.15603 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f9a180fab80 20 client.15603 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f9a180fab80 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f82e255370 0x55f82e257760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f9a180fab80 4 client.15603 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.449+0000 7f9a180fab80 1 -- 172.21.15.69:0/3460855167 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f82e19bdc0 con 0x55f82e255370 2024-05-14T16:52:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.450+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f82e255370 0x55f82e257760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.450+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f82e255370 0x55f82e257760 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cea0e6de10b3794 server_cookie=6ea9ce1c618fe37f in_seq=0 out_seq=0 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.450+0000 7f99e6ffd700 10 client.15603 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f99e6ffd700 1 -- 172.21.15.69:0/3460855167 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f82e19bdc0 con 0x55f82e255370 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f99e6ffd700 10 client.15603 handle_command_reply: tid=0 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 client.15603 shutdown 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 -- 172.21.15.69:0/3460855167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 msgr2=0x55f82e245370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e245370 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9a00007700 tx=0x7f9a0000af40 comp rx=0 tx=0).stop 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 -- 172.21.15.69:0/3460855167 shutdown_connections 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f82e255370 0x55f82e257760 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e245370 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e23c2c0 0x55f82e247a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 --2- 172.21.15.69:0/3460855167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e244e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 -- 172.21.15.69:0/3460855167 >> 172.21.15.69:0/3460855167 conn(0x55f82e168eb0 msgr2=0x55f82e230fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:06.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f99e6ffd700 0 client.15603 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:06.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 -- 172.21.15.69:0/3460855167 shutdown_connections 2024-05-14T16:52:06.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 1 -- 172.21.15.69:0/3460855167 wait complete. 2024-05-14T16:52:06.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.451+0000 7f9a180fab80 20 client.15603 trim_cache size 0 max 16384 2024-05-14T16:52:06.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:06 smithi203 ceph-mon[35787]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-05-14T16:52:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.470+0000 7f9a180fab80 1 Processor -- start 2024-05-14T16:52:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a180fab80 1 -- start start 2024-05-14T16:52:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e23c2c0 0x55f82e2310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e239c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e23a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a180fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f99f400fed0 con 0x55f82e240d40 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a180fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f99f400d6d0 con 0x55f82e23c2c0 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a180fab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99f400b520 con 0x55f82e240420 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a077fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e23a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a077fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e23a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51116/0 (socket says 172.21.15.69:51116) 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a077fe700 1 -- 172.21.15.69:0/2619333025 learned_addr learned my addr 172.21.15.69:0/2619333025 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a067fc700 1 --2- 172.21.15.69:0/2619333025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e239c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/2619333025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e23c2c0 0x55f82e2310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a077fe700 1 -- 172.21.15.69:0/2619333025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 msgr2=0x55f82e239c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a077fe700 1 --2- 172.21.15.69:0/2619333025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e239c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a077fe700 1 -- 172.21.15.69:0/2619333025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e23c2c0 msgr2=0x55f82e2310d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a077fe700 1 --2- 172.21.15.69:0/2619333025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e23c2c0 0x55f82e2310d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a077fe700 1 -- 172.21.15.69:0/2619333025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f82e243df0 con 0x55f82e240d40 2024-05-14T16:52:06.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a067fc700 1 --2- 172.21.15.69:0/2619333025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e239c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:06.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f9a077fe700 1 --2- 172.21.15.69:0/2619333025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e23a1d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9a00014400 tx=0x7f9a00007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3521e29fb2c7b33 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:06.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.471+0000 7f99ed7fa700 1 -- 172.21.15.69:0/2619333025 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00033070 con 0x55f82e240d40 2024-05-14T16:52:06.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f99ed7fa700 1 -- 172.21.15.69:0/2619333025 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a00003b40 con 0x55f82e240d40 2024-05-14T16:52:06.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f99ed7fa700 1 -- 172.21.15.69:0/2619333025 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00005e10 con 0x55f82e240d40 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f9a180fab80 1 -- 172.21.15.69:0/2619333025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 msgr2=0x55f82e23a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f9a180fab80 1 --2- 172.21.15.69:0/2619333025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e23a1d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9a00014400 tx=0x7f9a00007690 comp rx=0 tx=0).stop 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f9a180fab80 1 -- 172.21.15.69:0/2619333025 shutdown_connections 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f9a180fab80 1 --2- 172.21.15.69:0/2619333025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e240d40 0x55f82e23a1d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f9a180fab80 1 --2- 172.21.15.69:0/2619333025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e23c2c0 0x55f82e2310d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f9a180fab80 1 --2- 172.21.15.69:0/2619333025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e239c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f9a180fab80 1 -- 172.21.15.69:0/2619333025 >> 172.21.15.69:0/2619333025 conn(0x55f82e168eb0 msgr2=0x55f82e23a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f9a180fab80 1 -- 172.21.15.69:0/2619333025 shutdown_connections 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f9a180fab80 1 -- 172.21.15.69:0/2619333025 wait complete. 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.472+0000 7f9a180fab80 1 -- start start 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a180fab80 1 Processor -- start 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e24e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e248b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a180fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240d40 0x55f82e249040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a180fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f99f400bb90 con 0x55f82e23c2c0 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a180fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f99f400d000 con 0x55f82e240d40 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a180fab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99f400c260 con 0x55f82e240420 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a180fab80 10 client.0 _ll_register_callbacks cb 0x55f82e124a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a06ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e24e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a06ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e24e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51126/0 (socket says 172.21.15.69:51126) 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a06ffd700 1 -- 172.21.15.69:0/4202917970 learned_addr learned my addr 172.21.15.69:0/4202917970 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a06ffd700 1 -- 172.21.15.69:0/4202917970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 msgr2=0x55f82e248b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a077fe700 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240d40 0x55f82e249040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a067fc700 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e248b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e248b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a06ffd700 1 -- 172.21.15.69:0/4202917970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240d40 msgr2=0x55f82e249040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240d40 0x55f82e249040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a06ffd700 1 -- 172.21.15.69:0/4202917970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f82e249880 con 0x55f82e23c2c0 2024-05-14T16:52:06.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a067fc700 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e248b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.473+0000 7f9a077fe700 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240d40 0x55f82e249040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f9a06ffd700 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e24e100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f99f800a560 tx=0x7f99f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1642f180b3af18 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f99e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f99e6ffd700 1 -- 172.21.15.69:0/4202917970 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f802a830 con 0x55f82e23c2c0 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f99e6ffd700 1 -- 172.21.15.69:0/4202917970 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f99f802a9d0 con 0x55f82e23c2c0 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f9a180fab80 1 -- 172.21.15.69:0/4202917970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a00003b40 con 0x55f82e23c2c0 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f99e6ffd700 1 -- 172.21.15.69:0/4202917970 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f99f803a050 con 0x55f82e23c2c0 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f9a180fab80 10 client.15609 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f9a180fab80 1 -- 172.21.15.69:0/4202917970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f82e2c09f0 con 0x55f82e23c2c0 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f99e6ffd700 1 -- 172.21.15.69:0/4202917970 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a00003b40 con 0x55f82e23c2c0 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f99e6ffd700 1 -- 172.21.15.69:0/4202917970 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f99f801d440 con 0x55f82e23c2c0 2024-05-14T16:52:06.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f9a180fab80 10 client.15609 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:06.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f9a180fab80 10 client.15609 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:52:06.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f9a180fab80 20 client.15609 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:06.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f9a180fab80 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f82e2c1170 0x55f82e2c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:06.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f9a180fab80 4 client.15609 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e3c2299d", "release": true, "await": true}] 2024-05-14T16:52:06.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.474+0000 7f9a180fab80 1 -- 172.21.15.69:0/4202917970 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e3c2299d", "release": true, "await": true}) -- 0x55f82e1955a0 con 0x55f82e2c1170 2024-05-14T16:52:06.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.475+0000 7f9a067fc700 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f82e2c1170 0x55f82e2c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:06.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.475+0000 7f9a067fc700 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f82e2c1170 0x55f82e2c35d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f78c37d2514173 server_cookie=f033b30be394adfe in_seq=0 out_seq=0 2024-05-14T16:52:06.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.475+0000 7f99e6ffd700 10 client.15609 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:06.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f99e6ffd700 1 -- 172.21.15.69:0/4202917970 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f82e1955a0 con 0x55f82e2c1170 2024-05-14T16:52:06.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f99e6ffd700 10 client.15609 handle_command_reply: tid=0 2024-05-14T16:52:06.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 client.15609 shutdown 2024-05-14T16:52:06.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 -- 172.21.15.69:0/4202917970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 msgr2=0x55f82e24e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e24e100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f99f800a560 tx=0x7f99f8004750 comp rx=0 tx=0).stop 2024-05-14T16:52:06.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 -- 172.21.15.69:0/4202917970 shutdown_connections 2024-05-14T16:52:06.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f82e2c1170 0x55f82e2c35d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f82e23c2c0 0x55f82e24e100 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f82e240d40 0x55f82e249040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 --2- 172.21.15.69:0/4202917970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f82e240420 0x55f82e248b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f99e6ffd700 0 client.15609 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 -- 172.21.15.69:0/4202917970 >> 172.21.15.69:0/4202917970 conn(0x55f82e168eb0 msgr2=0x55f82e23ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 -- 172.21.15.69:0/4202917970 shutdown_connections 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 1 -- 172.21.15.69:0/4202917970 wait complete. 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.483+0000 7f9a180fab80 20 client.15609 trim_cache size 0 max 16384 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 31, 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 8, 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:52:06.484 INFO:teuthology.orchestra.run.smithi069.stdout: "e3c2299d": { 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 8, 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 146.7, 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:52:06.485 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:06.486 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:06.486 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:06.486 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:06.486 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:52:06.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 -- 172.21.15.69:0/4019651508 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99f00b2a70 msgr2=0x7f99f00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99f00b2a70 0x7f99f00b4f30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f99f8005ea0 tx=0x7f99f8005e30 comp rx=0 tx=0).stop 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 -- 172.21.15.69:0/4019651508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a0806bbe0 msgr2=0x7f9a0819fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a0806bbe0 0x7f9a0819fda0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f99fc000c00 tx=0x7f99fc0047b0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 -- 172.21.15.69:0/4019651508 shutdown_connections 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a081095b0 0x7f9a08199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a0806b220 0x7f9a0819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99f00b2a70 0x7f99f00b4f30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 --2- 172.21.15.69:0/4019651508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a0806bbe0 0x7f9a0819fda0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 -- 172.21.15.69:0/4019651508 >> 172.21.15.69:0/4019651508 conn(0x7f9a080793f0 msgr2=0x7f9a08101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 -- 172.21.15.69:0/4019651508 shutdown_connections 2024-05-14T16:52:06.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:06.484+0000 7f9a0d864700 1 -- 172.21.15.69:0/4019651508 wait complete. 2024-05-14T16:52:06.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:06 smithi092 ceph-mon[35763]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-05-14T16:52:07.184 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e3c2299d', total seconds elapsed: 32.5 2024-05-14T16:52:07.184 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.7 2024-05-14T16:52:07.185 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:52:07.548 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:08.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:08 smithi069 ceph-mon[31507]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-05-14T16:52:08.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:08 smithi203 ceph-mon[35787]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-05-14T16:52:08.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:08 smithi092 ceph-mon[35763]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-05-14T16:52:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.455+0000 7f6333854640 1 -- 172.21.15.69:0/2925848402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f632c102370 msgr2=0x7f632c06ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.455+0000 7f6333854640 1 --2- 172.21.15.69:0/2925848402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f632c102370 0x7f632c06ac70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6324009be0 tx=0x7f632402a6c0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.455+0000 7f6333854640 1 -- 172.21.15.69:0/2925848402 shutdown_connections 2024-05-14T16:52:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.455+0000 7f6333854640 1 --2- 172.21.15.69:0/2925848402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f632c102370 0x7f632c06ac70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.455+0000 7f6333854640 1 --2- 172.21.15.69:0/2925848402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f632c1019b0 0x7f632c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.455+0000 7f6333854640 1 --2- 172.21.15.69:0/2925848402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f632c107970 0x7f632c107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.455+0000 7f6333854640 1 -- 172.21.15.69:0/2925848402 >> 172.21.15.69:0/2925848402 conn(0x7f632c078aa0 msgr2=0x7f632c078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.455+0000 7f6333854640 1 -- 172.21.15.69:0/2925848402 shutdown_connections 2024-05-14T16:52:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.455+0000 7f6333854640 1 -- 172.21.15.69:0/2925848402 wait complete. 2024-05-14T16:52:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.456+0000 7f6333854640 1 Processor -- start 2024-05-14T16:52:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.456+0000 7f6333854640 1 -- start start 2024-05-14T16:52:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.456+0000 7f6333854640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f632c1019b0 0x7f632c0745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.456+0000 7f6333854640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f632c102370 0x7f632c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.456+0000 7f6333854640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f632c107970 0x7f632c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.456+0000 7f6333854640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f632c114fd0 con 0x7f632c107970 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.456+0000 7f6333854640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f632c114e50 con 0x7f632c102370 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.456+0000 7f6333854640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f632c115150 con 0x7f632c1019b0 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6330def640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f632c102370 0x7f632c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6330def640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f632c102370 0x7f632c06c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:33886/0 (socket says 172.21.15.69:33886) 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6330def640 1 -- 172.21.15.69:0/2047413417 learned_addr learned my addr 172.21.15.69:0/2047413417 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6331df1640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f632c107970 0x7f632c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6330def640 1 -- 172.21.15.69:0/2047413417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f632c1019b0 msgr2=0x7f632c0745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f63315f0640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f632c1019b0 0x7f632c0745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6330def640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f632c1019b0 0x7f632c0745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6330def640 1 -- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f632c107970 msgr2=0x7f632c06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6330def640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f632c107970 0x7f632c06caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6330def640 1 -- 172.21.15.69:0/2047413417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c06d360 con 0x7f632c102370 2024-05-14T16:52:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6331df1640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f632c107970 0x7f632c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.457+0000 7f6330def640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f632c102370 0x7f632c06c560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f632800e9c0 tx=0x7f632800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.458+0000 7f63227fc640 1 -- 172.21.15.69:0/2047413417 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632800ccc0 con 0x7f632c102370 2024-05-14T16:52:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.458+0000 7f6333854640 1 -- 172.21.15.69:0/2047413417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632c1b0150 con 0x7f632c102370 2024-05-14T16:52:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.458+0000 7f6333854640 1 -- 172.21.15.69:0/2047413417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632c1b06c0 con 0x7f632c102370 2024-05-14T16:52:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.459+0000 7f63227fc640 1 -- 172.21.15.69:0/2047413417 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63280054d0 con 0x7f632c102370 2024-05-14T16:52:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.459+0000 7f63227fc640 1 -- 172.21.15.69:0/2047413417 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328010430 con 0x7f632c102370 2024-05-14T16:52:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.462+0000 7f633656e740 1 Processor -- start 2024-05-14T16:52:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.473+0000 7f633656e740 1 -- start start 2024-05-14T16:52:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.474+0000 7f633656e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022322830 0x565022322c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.474+0000 7f633656e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 0x565022335da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.474+0000 7f633656e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 0x5650223386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.474+0000 7f633656e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565022295c30 con 0x5650223362e0 2024-05-14T16:52:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.474+0000 7f633656e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5650222857d0 con 0x565022322830 2024-05-14T16:52:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.474+0000 7f633656e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x565022289810 con 0x565022323150 2024-05-14T16:52:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.475+0000 7f63227fc640 1 -- 172.21.15.69:0/2047413417 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f63280046c0 con 0x7f632c102370 2024-05-14T16:52:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.475+0000 7f6330def640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 0x5650223386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.475+0000 7f6330def640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 0x5650223386d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51150/0 (socket says 172.21.15.69:51150) 2024-05-14T16:52:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.475+0000 7f6330def640 1 -- 172.21.15.69:0/3347504480 learned_addr learned my addr 172.21.15.69:0/3347504480 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.476+0000 7f6330def640 1 -- 172.21.15.69:0/3347504480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 msgr2=0x565022335da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.476+0000 7f6330def640 1 --2- 172.21.15.69:0/3347504480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 0x565022335da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.476+0000 7f6330def640 1 -- 172.21.15.69:0/3347504480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022322830 msgr2=0x565022322c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.476+0000 7f6330def640 1 --2- 172.21.15.69:0/3347504480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022322830 0x565022322c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.476+0000 7f6330def640 1 -- 172.21.15.69:0/3347504480 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565021f9a210 con 0x5650223362e0 2024-05-14T16:52:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.476+0000 7f6330def640 1 --2- 172.21.15.69:0/3347504480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 0x5650223386d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f63280e0000 tx=0x7f63280e8e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e14c31491d4e60d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.480+0000 7f63177fe640 1 -- 172.21.15.69:0/3347504480 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280f2020 con 0x5650223362e0 2024-05-14T16:52:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.481+0000 7f63177fe640 1 -- 172.21.15.69:0/3347504480 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63280ee460 con 0x5650223362e0 2024-05-14T16:52:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.481+0000 7f63177fe640 1 -- 172.21.15.69:0/3347504480 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280ed600 con 0x5650223362e0 2024-05-14T16:52:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.481+0000 7f63227fc640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f63180b2b10 0x7f63180b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.481+0000 7f63227fc640 1 -- 172.21.15.69:0/2047413417 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f63280e29c0 con 0x7f632c102370 2024-05-14T16:52:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.482+0000 7f633656e740 1 -- 172.21.15.69:0/3347504480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 msgr2=0x5650223386d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.482+0000 7f633656e740 1 --2- 172.21.15.69:0/3347504480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 0x5650223386d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f63280e0000 tx=0x7f63280e8e10 comp rx=0 tx=0).stop 2024-05-14T16:52:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.483+0000 7f6331df1640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f63180b2b10 0x7f63180b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.484+0000 7f633656e740 1 -- 172.21.15.69:0/3347504480 shutdown_connections 2024-05-14T16:52:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.484+0000 7f633656e740 1 --2- 172.21.15.69:0/3347504480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 0x5650223386d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.484+0000 7f633656e740 1 --2- 172.21.15.69:0/3347504480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 0x565022335da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.484+0000 7f633656e740 1 --2- 172.21.15.69:0/3347504480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022322830 0x565022322c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.484+0000 7f633656e740 1 -- 172.21.15.69:0/3347504480 >> 172.21.15.69:0/3347504480 conn(0x56502231ee00 msgr2=0x56502231f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.485+0000 7f6331df1640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f63180b2b10 0x7f63180b4fd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6324006010 tx=0x7f6324004a40 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.486+0000 7f633656e740 1 -- 172.21.15.69:0/3347504480 shutdown_connections 2024-05-14T16:52:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.486+0000 7f633656e740 1 -- 172.21.15.69:0/3347504480 wait complete. 2024-05-14T16:52:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.486+0000 7f633656e740 1 -- start start 2024-05-14T16:52:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.486+0000 7f633656e740 1 Processor -- start 2024-05-14T16:52:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.487+0000 7f633656e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022322830 0x56502232fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.487+0000 7f633656e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022323150 0x56502232ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.487+0000 7f633656e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650223362e0 0x565022335790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.487+0000 7f633656e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565021f6b790 con 0x565022323150 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.487+0000 7f633656e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565022241890 con 0x565022322830 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.487+0000 7f633656e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x565022325870 con 0x5650223362e0 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.487+0000 7f633656e740 10 client.0 _ll_register_callbacks cb 0x565022244d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.488+0000 7f6330def640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022323150 0x56502232ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.488+0000 7f6330def640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022323150 0x56502232ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51152/0 (socket says 172.21.15.69:51152) 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.488+0000 7f6330def640 1 -- 172.21.15.69:0/4064974377 learned_addr learned my addr 172.21.15.69:0/4064974377 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.489+0000 7f6330def640 1 -- 172.21.15.69:0/4064974377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650223362e0 msgr2=0x565022335790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.489+0000 7f6330def640 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650223362e0 0x565022335790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.489+0000 7f6330def640 1 -- 172.21.15.69:0/4064974377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022322830 msgr2=0x56502232fa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.489+0000 7f6330def640 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022322830 0x56502232fa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.489+0000 7f6330def640 1 -- 172.21.15.69:0/4064974377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565022066620 con 0x565022323150 2024-05-14T16:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.489+0000 7f6330def640 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022323150 0x56502232ffb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f63280a5cd0 tx=0x7f63280eeaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67502eb1e54e7de server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.489+0000 7f6314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.490+0000 7f6314ff9640 1 -- 172.21.15.69:0/4064974377 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280f2070 con 0x565022323150 2024-05-14T16:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.490+0000 7f6314ff9640 1 -- 172.21.15.69:0/4064974377 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f63280e0030 con 0x565022323150 2024-05-14T16:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.490+0000 7f633656e740 1 -- 172.21.15.69:0/4064974377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63280ee460 con 0x565022323150 2024-05-14T16:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.490+0000 7f6314ff9640 1 -- 172.21.15.69:0/4064974377 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6328091070 con 0x565022323150 2024-05-14T16:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.490+0000 7f633656e740 10 client.15621 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.491+0000 7f633656e740 1 -- 172.21.15.69:0/4064974377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56502233a320 con 0x565022323150 2024-05-14T16:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.491+0000 7f6314ff9640 1 -- 172.21.15.69:0/4064974377 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63280ee460 con 0x565022323150 2024-05-14T16:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.491+0000 7f6314ff9640 1 -- 172.21.15.69:0/4064974377 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f63280b2620 con 0x565022323150 2024-05-14T16:52:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.491+0000 7f633656e740 10 client.15621 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.491+0000 7f633656e740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:52:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.492+0000 7f633656e740 20 client.15621 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.492+0000 7f633656e740 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56502233baa0 0x56502233dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.492+0000 7f633656e740 4 client.15621 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:52:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.492+0000 7f633656e740 1 -- 172.21.15.69:0/4064974377 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565022297b80 con 0x56502233baa0 2024-05-14T16:52:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.493+0000 7f63315f0640 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56502233baa0 0x56502233dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.495+0000 7f6314ff9640 10 client.15621 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.495+0000 7f63315f0640 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56502233baa0 0x56502233dee0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf293c0b5bad71c server_cookie=5b94b4ab11711ef7 in_seq=0 out_seq=0 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.496+0000 7f6314ff9640 1 -- 172.21.15.69:0/4064974377 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565022297b80 con 0x56502233baa0 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.496+0000 7f6314ff9640 10 client.15621 handle_command_reply: tid=0 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.496+0000 7f633656e740 1 client.15621 shutdown 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 1 -- 172.21.15.69:0/4064974377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022323150 msgr2=0x56502232ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022323150 0x56502232ffb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f63280a5cd0 tx=0x7f63280eeaf0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 1 -- 172.21.15.69:0/4064974377 shutdown_connections 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56502233baa0 0x56502233dee0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650223362e0 0x565022335790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022323150 0x56502232ffb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 1 --2- 172.21.15.69:0/4064974377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022322830 0x56502232fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 1 -- 172.21.15.69:0/4064974377 >> 172.21.15.69:0/4064974377 conn(0x56502231ee00 msgr2=0x56502231e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f6314ff9640 0 client.15621 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 1 -- 172.21.15.69:0/4064974377 shutdown_connections 2024-05-14T16:52:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 1 -- 172.21.15.69:0/4064974377 wait complete. 2024-05-14T16:52:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.497+0000 7f633656e740 20 client.15621 trim_cache size 0 max 16384 2024-05-14T16:52:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.523+0000 7f633656e740 1 Processor -- start 2024-05-14T16:52:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.523+0000 7f633656e740 1 -- start start 2024-05-14T16:52:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f633656e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022322830 0x56502234b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f633656e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022323150 0x56502231df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f633656e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 0x5650223437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f633656e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63100111b0 con 0x5650223362e0 2024-05-14T16:52:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f633656e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f631000e9f0 con 0x565022323150 2024-05-14T16:52:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f633656e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f631000c840 con 0x565022322830 2024-05-14T16:52:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f6330def640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022323150 0x56502231df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f6330def640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022323150 0x56502231df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:33912/0 (socket says 172.21.15.69:33912) 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f6330def640 1 -- 172.21.15.69:0/3250344570 learned_addr learned my addr 172.21.15.69:0/3250344570 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f6331df1640 1 --2- 172.21.15.69:0/3250344570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 0x5650223437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f63315f0640 1 --2- 172.21.15.69:0/3250344570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022322830 0x56502234b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f6330def640 1 -- 172.21.15.69:0/3250344570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022322830 msgr2=0x56502234b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f6330def640 1 --2- 172.21.15.69:0/3250344570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022322830 0x56502234b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f6330def640 1 -- 172.21.15.69:0/3250344570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 msgr2=0x5650223437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.524+0000 7f6330def640 1 --2- 172.21.15.69:0/3250344570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 0x5650223437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.525+0000 7f6330def640 1 -- 172.21.15.69:0/3250344570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565022322f80 con 0x565022323150 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.525+0000 7f63315f0640 1 --2- 172.21.15.69:0/3250344570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022322830 0x56502234b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.525+0000 7f6330def640 1 --2- 172.21.15.69:0/3250344570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022323150 0x56502231df80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f632800ceb0 tx=0x7f6328009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584e44519399242 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.525+0000 7f6316ffd640 1 -- 172.21.15.69:0/3250344570 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280e6070 con 0x565022323150 2024-05-14T16:52:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.525+0000 7f6316ffd640 1 -- 172.21.15.69:0/3250344570 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6328005680 con 0x565022323150 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.525+0000 7f6316ffd640 1 -- 172.21.15.69:0/3250344570 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280f2070 con 0x565022323150 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.526+0000 7f633656e740 1 -- 172.21.15.69:0/3250344570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022323150 msgr2=0x56502231df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.526+0000 7f633656e740 1 --2- 172.21.15.69:0/3250344570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022323150 0x56502231df80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f632800ceb0 tx=0x7f6328009f90 comp rx=0 tx=0).stop 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.526+0000 7f633656e740 1 -- 172.21.15.69:0/3250344570 shutdown_connections 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.526+0000 7f633656e740 1 --2- 172.21.15.69:0/3250344570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650223362e0 0x5650223437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.527+0000 7f633656e740 1 --2- 172.21.15.69:0/3250344570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565022323150 0x56502231df80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.527+0000 7f633656e740 1 --2- 172.21.15.69:0/3250344570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022322830 0x56502234b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.527+0000 7f633656e740 1 -- 172.21.15.69:0/3250344570 >> 172.21.15.69:0/3250344570 conn(0x56502231ee00 msgr2=0x56502231d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.527+0000 7f633656e740 1 -- 172.21.15.69:0/3250344570 shutdown_connections 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.527+0000 7f633656e740 1 -- 172.21.15.69:0/3250344570 wait complete. 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.527+0000 7f633656e740 1 -- start start 2024-05-14T16:52:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.527+0000 7f633656e740 1 Processor -- start 2024-05-14T16:52:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.528+0000 7f633656e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022322830 0x565022343be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.528+0000 7f633656e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 0x565022327890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.528+0000 7f633656e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650223362e0 0x565022327dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.528+0000 7f63315f0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022322830 0x565022343be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6330def640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 0x565022327890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f633656e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f631000ceb0 con 0x565022322830 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f633656e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f631000e320 con 0x5650223362e0 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f633656e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f631000d580 con 0x565022323150 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f63315f0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022322830 0x565022343be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51170/0 (socket says 172.21.15.69:51170) 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6330def640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 0x565022327890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:58730/0 (socket says 172.21.15.69:58730) 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f63315f0640 1 -- 172.21.15.69:0/1606464212 learned_addr learned my addr 172.21.15.69:0/1606464212 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f633656e740 10 client.0 _ll_register_callbacks cb 0x7f63280e81e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6331df1640 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650223362e0 0x565022327dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6330def640 1 -- 172.21.15.69:0/1606464212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650223362e0 msgr2=0x565022327dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6330def640 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650223362e0 0x565022327dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6330def640 1 -- 172.21.15.69:0/1606464212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022322830 msgr2=0x565022343be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6330def640 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022322830 0x565022343be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6330def640 1 -- 172.21.15.69:0/1606464212 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565022328490 con 0x565022323150 2024-05-14T16:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6330def640 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 0x565022327890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f632800ceb0 tx=0x7f63280a90c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb85d8f852dd5de server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.529+0000 7f6314ff9640 1 -- 172.21.15.69:0/1606464212 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280e6070 con 0x565022323150 2024-05-14T16:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.530+0000 7f6314ff9640 1 -- 172.21.15.69:0/1606464212 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f63280a57d0 con 0x565022323150 2024-05-14T16:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.530+0000 7f633656e740 1 -- 172.21.15.69:0/1606464212 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6328005680 con 0x565022323150 2024-05-14T16:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.530+0000 7f6314ff9640 1 -- 172.21.15.69:0/1606464212 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f63280f2040 con 0x565022323150 2024-05-14T16:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.530+0000 7f633656e740 10 client.25117 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.530+0000 7f633656e740 1 -- 172.21.15.69:0/1606464212 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565022328490 con 0x565022323150 2024-05-14T16:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.530+0000 7f6314ff9640 1 -- 172.21.15.69:0/1606464212 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63280c4310 con 0x565022323150 2024-05-14T16:52:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.530+0000 7f6314ff9640 1 -- 172.21.15.69:0/1606464212 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f63280a8920 con 0x565022323150 2024-05-14T16:52:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.530+0000 7f633656e740 10 client.25117 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.531+0000 7f633656e740 10 client.25117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:52:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.531+0000 7f633656e740 20 client.25117 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.531+0000 7f633656e740 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5650223c9c20 0x5650223cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.531+0000 7f633656e740 4 client.25117 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:52:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.531+0000 7f633656e740 1 -- 172.21.15.69:0/1606464212 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6328005680 con 0x5650223c9c20 2024-05-14T16:52:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.532+0000 7f63315f0640 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5650223c9c20 0x5650223cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.533+0000 7f63315f0640 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5650223c9c20 0x5650223cc040 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dfdc59f677d1534 server_cookie=174f546bd52204c8 in_seq=0 out_seq=0 2024-05-14T16:52:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.533+0000 7f6314ff9640 10 client.25117 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.541+0000 7f6314ff9640 1 -- 172.21.15.69:0/1606464212 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f6328005680 con 0x5650223c9c20 2024-05-14T16:52:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.541+0000 7f6314ff9640 10 client.25117 handle_command_reply: tid=0 2024-05-14T16:52:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.541+0000 7f633656e740 1 client.25117 shutdown 2024-05-14T16:52:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.542+0000 7f633656e740 1 -- 172.21.15.69:0/1606464212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 msgr2=0x565022327890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.542+0000 7f633656e740 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 0x565022327890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f632800ceb0 tx=0x7f63280a90c0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.543+0000 7f633656e740 1 -- 172.21.15.69:0/1606464212 shutdown_connections 2024-05-14T16:52:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.543+0000 7f633656e740 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5650223c9c20 0x5650223cc040 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.543+0000 7f633656e740 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650223362e0 0x565022327dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.543+0000 7f6314ff9640 0 client.25117 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.543+0000 7f633656e740 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565022323150 0x565022327890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.543+0000 7f633656e740 1 --2- 172.21.15.69:0/1606464212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565022322830 0x565022343be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.543+0000 7f633656e740 1 -- 172.21.15.69:0/1606464212 >> 172.21.15.69:0/1606464212 conn(0x56502231ee00 msgr2=0x56502231e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.544+0000 7f633656e740 1 -- 172.21.15.69:0/1606464212 shutdown_connections 2024-05-14T16:52:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.544+0000 7f633656e740 1 -- 172.21.15.69:0/1606464212 wait complete. 2024-05-14T16:52:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.543+0000 7fbfc4e48700 1 -- 172.21.15.69:0/3422423334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc010f780 msgr2=0x7fbfc0117300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.543+0000 7fbfc4e48700 1 --2- 172.21.15.69:0/3422423334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc010f780 0x7fbfc0117300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009f90 tx=0x7fbfb800b330 comp rx=0 tx=0).stop 2024-05-14T16:52:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.544+0000 7fbfc4e48700 1 -- 172.21.15.69:0/3422423334 shutdown_connections 2024-05-14T16:52:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.544+0000 7fbfc4e48700 1 --2- 172.21.15.69:0/3422423334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfc010b030 0x7fbfc010b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.544+0000 7fbfc4e48700 1 --2- 172.21.15.69:0/3422423334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc010f780 0x7fbfc0117300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.544+0000 7fbfc4e48700 1 --2- 172.21.15.69:0/3422423334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc010b9e0 0x7fbfc010f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.544+0000 7fbfc4e48700 1 -- 172.21.15.69:0/3422423334 >> 172.21.15.69:0/3422423334 conn(0x7fbfc006e6b0 msgr2=0x7fbfc006eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.544+0000 7fbfc4e48700 1 -- 172.21.15.69:0/3422423334 shutdown_connections 2024-05-14T16:52:09.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.544+0000 7fbfc4e48700 1 -- 172.21.15.69:0/3422423334 wait complete. 2024-05-14T16:52:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.544+0000 7f633656e740 20 client.25117 trim_cache size 0 max 16384 2024-05-14T16:52:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:52:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:52:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:52:09.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.545+0000 7fbfc4e48700 1 Processor -- start 2024-05-14T16:52:09.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.545+0000 7fbfc4e48700 1 -- start start 2024-05-14T16:52:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.545+0000 7f6333854640 1 -- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f63180b2b10 msgr2=0x7f63180b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.545+0000 7f6333854640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f63180b2b10 0x7f63180b4fd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6324006010 tx=0x7f6324004a40 comp rx=0 tx=0).stop 2024-05-14T16:52:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.545+0000 7f6333854640 1 -- 172.21.15.69:0/2047413417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f632c102370 msgr2=0x7f632c06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.545+0000 7f6333854640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f632c102370 0x7f632c06c560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f632800e9c0 tx=0x7f632800ee80 comp rx=0 tx=0).stop 2024-05-14T16:52:09.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.545+0000 7fbfc4e48700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc010b030 0x7fbfc01a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.545+0000 7fbfc4e48700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc010b9e0 0x7fbfc01a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.545+0000 7fbfc4e48700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfc010f780 0x7fbfc01a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.545+0000 7fbfc4e48700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0119db0 con 0x7fbfc010f780 2024-05-14T16:52:09.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.545+0000 7fbfc4e48700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0119c30 con 0x7fbfc010b030 2024-05-14T16:52:09.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.545+0000 7fbfc4e48700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0119f30 con 0x7fbfc010b9e0 2024-05-14T16:52:09.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.545+0000 7fbfbdd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc010b9e0 0x7fbfc01a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbdd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc010b9e0 0x7fbfc01a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:58760/0 (socket says 172.21.15.69:58760) 2024-05-14T16:52:09.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbdd9b700 1 -- 172.21.15.69:0/669636955 learned_addr learned my addr 172.21.15.69:0/669636955 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.546+0000 7f6333854640 1 -- 172.21.15.69:0/2047413417 shutdown_connections 2024-05-14T16:52:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.546+0000 7f6333854640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f63180b2b10 0x7f63180b4fd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.546+0000 7f6333854640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f632c107970 0x7f632c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.546+0000 7f6333854640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f632c102370 0x7f632c06c560 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.546+0000 7f6333854640 1 --2- 172.21.15.69:0/2047413417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f632c1019b0 0x7f632c0745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.546+0000 7f6333854640 1 -- 172.21.15.69:0/2047413417 >> 172.21.15.69:0/2047413417 conn(0x7f632c078aa0 msgr2=0x7f632c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:09.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbe59c700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc010b030 0x7fbfc01a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbed9d700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfc010f780 0x7fbfc01a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.546+0000 7f6333854640 1 -- 172.21.15.69:0/2047413417 shutdown_connections 2024-05-14T16:52:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:09.546+0000 7f6333854640 1 -- 172.21.15.69:0/2047413417 wait complete. 2024-05-14T16:52:09.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbe59c700 1 -- 172.21.15.69:0/669636955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc010b9e0 msgr2=0x7fbfc01a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbe59c700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc010b9e0 0x7fbfc01a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbe59c700 1 -- 172.21.15.69:0/669636955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfc010f780 msgr2=0x7fbfc01a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbe59c700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfc010f780 0x7fbfc01a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbe59c700 1 -- 172.21.15.69:0/669636955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc01a51f0 con 0x7fbfc010b030 2024-05-14T16:52:09.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbdd9b700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc010b9e0 0x7fbfc01a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:09.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.546+0000 7fbfbe59c700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc010b030 0x7fbfc01a01e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000ea60 tx=0x7fbfb000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.547+0000 7fbfaf7fe700 1 -- 172.21.15.69:0/669636955 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb000ccd0 con 0x7fbfc010b030 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.547+0000 7fbfc4e48700 1 -- 172.21.15.69:0/669636955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc01a54e0 con 0x7fbfc010b030 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.547+0000 7fbfc4e48700 1 -- 172.21.15.69:0/669636955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc01b55f0 con 0x7fbfc010b030 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.547+0000 7fbfaf7fe700 1 -- 172.21.15.69:0/669636955 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb0016890 con 0x7fbfc010b030 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.547+0000 7fbfaf7fe700 1 -- 172.21.15.69:0/669636955 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0015870 con 0x7fbfc010b030 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.549+0000 7fbfc4e48700 1 -- 172.21.15.69:0/669636955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa0005330 con 0x7fbfc010b030 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.550+0000 7fbfaf7fe700 1 -- 172.21.15.69:0/669636955 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbfb00163e0 con 0x7fbfc010b030 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.551+0000 7fbfaf7fe700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbfa80b2a50 0x7fbfa80b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.551+0000 7fbfaf7fe700 1 -- 172.21.15.69:0/669636955 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fbfb00e27b0 con 0x7fbfc010b030 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.551+0000 7fbfbdd9b700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbfa80b2a50 0x7fbfa80b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:09.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.552+0000 7fbfbdd9b700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbfa80b2a50 0x7fbfa80b4f10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0074cd0 tx=0x7fbfb4009210 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:09.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.556+0000 7fbfaf7fe700 1 -- 172.21.15.69:0/669636955 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbfb00aa1a0 con 0x7fbfc010b030 2024-05-14T16:52:09.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.972+0000 7fbfc4e48700 1 -- 172.21.15.69:0/669636955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfa00058f0 con 0x7fbfc010b030 2024-05-14T16:52:09.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.973+0000 7fbfaf7fe700 1 -- 172.21.15.69:0/669636955 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11863 (secure 0 0 0) 0x7fbfb00a9aa0 con 0x7fbfc010b030 2024-05-14T16:52:09.974 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:52:09.974 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:51:16.740826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:52:09.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.977+0000 7fbfc4e48700 1 -- 172.21.15.69:0/669636955 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbfa80b2a50 msgr2=0x7fbfa80b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.977+0000 7fbfc4e48700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbfa80b2a50 0x7fbfa80b4f10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0074cd0 tx=0x7fbfb4009210 comp rx=0 tx=0).stop 2024-05-14T16:52:09.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.978+0000 7fbfc4e48700 1 -- 172.21.15.69:0/669636955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc010b030 msgr2=0x7fbfc01a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.978+0000 7fbfc4e48700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc010b030 0x7fbfc01a01e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000ea60 tx=0x7fbfb000ed70 comp rx=0 tx=0).stop 2024-05-14T16:52:09.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.978+0000 7fbfc4e48700 1 -- 172.21.15.69:0/669636955 shutdown_connections 2024-05-14T16:52:09.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.978+0000 7fbfc4e48700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbfa80b2a50 0x7fbfa80b4f10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.978+0000 7fbfc4e48700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfc010f780 0x7fbfc01a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.978+0000 7fbfc4e48700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfc010b030 0x7fbfc01a01e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.978+0000 7fbfc4e48700 1 --2- 172.21.15.69:0/669636955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc010b9e0 0x7fbfc01a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.978+0000 7fbfc4e48700 1 -- 172.21.15.69:0/669636955 >> 172.21.15.69:0/669636955 conn(0x7fbfc006e6b0 msgr2=0x7fbfc00732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:09.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.978+0000 7fbfc4e48700 1 -- 172.21.15.69:0/669636955 shutdown_connections 2024-05-14T16:52:09.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:09.978+0000 7fbfc4e48700 1 -- 172.21.15.69:0/669636955 wait complete. 2024-05-14T16:52:09.978 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 31 2024-05-14T16:52:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 100.0 s, 2502.9 tps, lat 220.942 ms stddev 2117.906 2024-05-14T16:52:10.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:10 smithi069 ceph-mon[31507]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-05-14T16:52:10.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:10 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/669636955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:10.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:10 smithi203 ceph-mon[35787]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-05-14T16:52:10.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:10 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/669636955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:10.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:10 smithi092 ceph-mon[35763]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-05-14T16:52:10.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:10 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/669636955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:10.900 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0' 2024-05-14T16:52:10.900 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0 2024-05-14T16:52:11.304 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:12.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:12 smithi069 ceph-mon[31507]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 562 op/s 2024-05-14T16:52:12.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:12 smithi203 ceph-mon[35787]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 562 op/s 2024-05-14T16:52:12.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:12 smithi092 ceph-mon[35763]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 562 op/s 2024-05-14T16:52:14.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:14 smithi069 ceph-mon[31507]: pgmap v74: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-05-14T16:52:14.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:14 smithi203 ceph-mon[35787]: pgmap v74: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-05-14T16:52:14.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:14 smithi092 ceph-mon[35763]: pgmap v74: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-05-14T16:52:14.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 -- 172.21.15.69:0/506433811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f4c1084c0 msgr2=0x7f3f4c1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 --2- 172.21.15.69:0/506433811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f4c1084c0 0x7f3f4c1088a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009990 tx=0x7f3f3c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 -- 172.21.15.69:0/506433811 shutdown_connections 2024-05-14T16:52:14.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 --2- 172.21.15.69:0/506433811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f4c1084c0 0x7f3f4c1088a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 --2- 172.21.15.69:0/506433811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f4c102e00 0x7f3f4c10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 --2- 172.21.15.69:0/506433811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c102440 0x7f3f4c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 -- 172.21.15.69:0/506433811 >> 172.21.15.69:0/506433811 conn(0x7f3f4c0793e0 msgr2=0x7f3f4c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 -- 172.21.15.69:0/506433811 shutdown_connections 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 -- 172.21.15.69:0/506433811 wait complete. 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 Processor -- start 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.690+0000 7f3f51dc9700 1 -- start start 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f51dc9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f4c102440 0x7f3f4c104090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f51dc9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c102e00 0x7f3f4c1045d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f51dc9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f4c1084c0 0x7f3f4c104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f51dc9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c1125d0 con 0x7f3f4c102440 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f51dc9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c112450 con 0x7f3f4c1084c0 2024-05-14T16:52:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f51dc9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c112750 con 0x7f3f4c102e00 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f4b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f4c102440 0x7f3f4c104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f4b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f4c102440 0x7f3f4c104090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55622/0 (socket says 172.21.15.69:55622) 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f4b7fe700 1 -- 172.21.15.69:0/853585309 learned_addr learned my addr 172.21.15.69:0/853585309 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f4c1084c0 0x7f3f4c104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f4affd700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c102e00 0x7f3f4c1045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f4affd700 1 -- 172.21.15.69:0/853585309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f4c1084c0 msgr2=0x7f3f4c104b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f4affd700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f4c1084c0 0x7f3f4c104b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.691+0000 7f3f4affd700 1 -- 172.21.15.69:0/853585309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f4c102440 msgr2=0x7f3f4c104090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.692+0000 7f3f4affd700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f4c102440 0x7f3f4c104090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.692+0000 7f3f4affd700 1 -- 172.21.15.69:0/853585309 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c1a80d0 con 0x7f3f4c102e00 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.693+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f4c102440 0x7f3f4c104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:14.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.693+0000 7f3f4affd700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c102e00 0x7f3f4c1045d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000d750 tx=0x7f3f4000db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:14.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.693+0000 7f3f48ff9700 1 -- 172.21.15.69:0/853585309 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40016070 con 0x7f3f4c102e00 2024-05-14T16:52:14.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.693+0000 7f3f48ff9700 1 -- 172.21.15.69:0/853585309 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f40004830 con 0x7f3f4c102e00 2024-05-14T16:52:14.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.693+0000 7f3f48ff9700 1 -- 172.21.15.69:0/853585309 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f400055d0 con 0x7f3f4c102e00 2024-05-14T16:52:14.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.693+0000 7f3f51dc9700 1 -- 172.21.15.69:0/853585309 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4c1a83c0 con 0x7f3f4c102e00 2024-05-14T16:52:14.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.693+0000 7f3f51dc9700 1 -- 172.21.15.69:0/853585309 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4c1a88b0 con 0x7f3f4c102e00 2024-05-14T16:52:14.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.697+0000 7f3f48ff9700 1 -- 172.21.15.69:0/853585309 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3f40004ce0 con 0x7f3f4c102e00 2024-05-14T16:52:14.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.698+0000 7f3f5c65fb80 1 Processor -- start 2024-05-14T16:52:14.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.698+0000 7f3f5c65fb80 1 -- start start 2024-05-14T16:52:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f93e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f938540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e25f896e20 con 0x55e25f938b60 2024-05-14T16:52:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e25f8869c0 con 0x55e25f93e750 2024-05-14T16:52:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e25f88aa00 con 0x55e25f93de30 2024-05-14T16:52:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f48ff9700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3f340b2ba0 0x7f3f340b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f938540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f93e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f938540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:50026/0 (socket says 172.21.15.69:50026) 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4b7fe700 1 -- 172.21.15.69:0/1094680054 learned_addr learned my addr 172.21.15.69:0/1094680054 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4affd700 1 --2- 172.21.15.69:0/1094680054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4bfff700 1 -- 172.21.15.69:0/1094680054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 msgr2=0x55e25f938540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/1094680054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f938540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4bfff700 1 -- 172.21.15.69:0/1094680054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 msgr2=0x55e25f938fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/1094680054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4bfff700 1 -- 172.21.15.69:0/1094680054 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e25f7d6a60 con 0x55e25f93de30 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4affd700 1 --2- 172.21.15.69:0/1094680054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/1094680054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f938540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.699+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3f340b2ba0 0x7f3f340b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.700+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/1094680054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f93e210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f44012580 tx=0x7f3f44012890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99dccca5f68e835a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.700+0000 7f3f48ff9700 1 -- 172.21.15.69:0/853585309 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f3f400e23b0 con 0x7f3f4c102e00 2024-05-14T16:52:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.700+0000 7f3f31ffb700 1 -- 172.21.15.69:0/1094680054 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4401a070 con 0x55e25f93de30 2024-05-14T16:52:14.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.700+0000 7f3f31ffb700 1 -- 172.21.15.69:0/1094680054 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f44016c10 con 0x55e25f93de30 2024-05-14T16:52:14.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.700+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3f340b2ba0 0x7f3f340b5060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f44021000 tx=0x7f3f440159f0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:14.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.700+0000 7f3f31ffb700 1 -- 172.21.15.69:0/1094680054 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44015660 con 0x55e25f93de30 2024-05-14T16:52:14.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.701+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/1094680054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 msgr2=0x55e25f93e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.701+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/1094680054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f93e210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f44012580 tx=0x7f3f44012890 comp rx=0 tx=0).stop 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.701+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/1094680054 shutdown_connections 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.701+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/1094680054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.701+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/1094680054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f93e210 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.701+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/1094680054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f938540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.701+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/1094680054 >> 172.21.15.69:0/1094680054 conn(0x55e25f8633e0 msgr2=0x55e25f932d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.701+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/1094680054 shutdown_connections 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.701+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/1094680054 wait complete. 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f5c65fb80 1 -- start start 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f5c65fb80 1 Processor -- start 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f938b60 0x55e25f944440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f9417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f93e750 0x55e25f941d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e25f78d130 con 0x55e25f93e750 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e25f828950 con 0x55e25f938b60 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e25f945190 con 0x55e25f93de30 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f5c65fb80 10 client.0 _ll_register_callbacks cb 0x55e25f81d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f4affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f9417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f4affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f9417e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:56062/0 (socket says 172.21.15.69:56062) 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.702+0000 7f3f4affd700 1 -- 172.21.15.69:0/172393944 learned_addr learned my addr 172.21.15.69:0/172393944 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.703+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f938b60 0x55e25f944440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.703+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f93e750 0x55e25f941d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.703+0000 7f3f4b7fe700 1 -- 172.21.15.69:0/172393944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 msgr2=0x55e25f9417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.703+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f9417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.703+0000 7f3f4b7fe700 1 -- 172.21.15.69:0/172393944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f93e750 msgr2=0x55e25f941d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.703+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f93e750 0x55e25f941d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.703+0000 7f3f4b7fe700 1 -- 172.21.15.69:0/172393944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e25f825820 con 0x55e25f938b60 2024-05-14T16:52:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.704+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f938b60 0x55e25f944440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0098e0 tx=0x7f3f3c00bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c669013e160071ee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.704+0000 7f3f2affd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.704+0000 7f3f2affd700 1 -- 172.21.15.69:0/172393944 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c005270 con 0x55e25f938b60 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.704+0000 7f3f2affd700 1 -- 172.21.15.69:0/172393944 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f3f3c005410 con 0x55e25f938b60 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.704+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/172393944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f44016c10 con 0x55e25f938b60 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.704+0000 7f3f2affd700 1 -- 172.21.15.69:0/172393944 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3f3c0059c0 con 0x55e25f938b60 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f5c65fb80 10 client.25187 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/172393944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e25f94e4c0 con 0x55e25f938b60 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f2affd700 1 -- 172.21.15.69:0/172393944 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f44016c10 con 0x55e25f938b60 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f2affd700 1 -- 172.21.15.69:0/172393944 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f3f3c03c030 con 0x55e25f938b60 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f5c65fb80 10 client.25187 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f5c65fb80 10 client.25187 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f5c65fb80 20 client.25187 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e25f950b50 0x55e25f952f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f5c65fb80 4 client.25187 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:52:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/172393944 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e25f9044f0 con 0x55e25f950b50 2024-05-14T16:52:14.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.705+0000 7f3f4affd700 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e25f950b50 0x55e25f952f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.706+0000 7f3f4affd700 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e25f950b50 0x55e25f952f80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d806ffe4072082b server_cookie=2a23e82d8e31142d in_seq=0 out_seq=0 2024-05-14T16:52:14.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.706+0000 7f3f2affd700 10 client.25187 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.707+0000 7f3f2affd700 1 -- 172.21.15.69:0/172393944 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e25f9044f0 con 0x55e25f950b50 2024-05-14T16:52:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.707+0000 7f3f2affd700 10 client.25187 handle_command_reply: tid=0 2024-05-14T16:52:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.707+0000 7f3f5c65fb80 1 client.25187 shutdown 2024-05-14T16:52:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.708+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/172393944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f938b60 msgr2=0x55e25f944440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.708+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f938b60 0x55e25f944440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0098e0 tx=0x7f3f3c00bf00 comp rx=0 tx=0).stop 2024-05-14T16:52:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.708+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/172393944 shutdown_connections 2024-05-14T16:52:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.708+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e25f950b50 0x55e25f952f80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.708+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f93e750 0x55e25f941d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.708+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f938b60 0x55e25f944440 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.708+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/172393944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f9417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.708+0000 7f3f2affd700 0 client.25187 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.708+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/172393944 >> 172.21.15.69:0/172393944 conn(0x55e25f8633e0 msgr2=0x55e25f92d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.709+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/172393944 shutdown_connections 2024-05-14T16:52:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.709+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/172393944 wait complete. 2024-05-14T16:52:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.709+0000 7f3f5c65fb80 20 client.25187 trim_cache size 0 max 16384 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f5c65fb80 1 Processor -- start 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f5c65fb80 1 -- start start 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f9407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f956840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c1ae590 con 0x55e25f938b60 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c1abd90 con 0x55e25f93e750 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c1a9d20 con 0x55e25f93de30 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f4b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f4b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55638/0 (socket says 172.21.15.69:55638) 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f4b7fe700 1 -- 172.21.15.69:0/143796769 learned_addr learned my addr 172.21.15.69:0/143796769 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/143796769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f956840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.737+0000 7f3f4affd700 1 --2- 172.21.15.69:0/143796769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f9407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.738+0000 7f3f4b7fe700 1 -- 172.21.15.69:0/143796769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 msgr2=0x55e25f9407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.738+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/143796769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f9407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.738+0000 7f3f4b7fe700 1 -- 172.21.15.69:0/143796769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 msgr2=0x55e25f956840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.738+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/143796769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f956840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.738+0000 7f3f4b7fe700 1 -- 172.21.15.69:0/143796769 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e25f93fd50 con 0x55e25f938b60 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.738+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/143796769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f956840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.738+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/143796769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938540 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00bca0 tx=0x7f3f3c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341fa2aa4162fd8d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.738+0000 7f3f317fa700 1 -- 172.21.15.69:0/143796769 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c04b070 con 0x55e25f938b60 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.738+0000 7f3f317fa700 1 -- 172.21.15.69:0/143796769 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f3c003b30 con 0x55e25f938b60 2024-05-14T16:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.738+0000 7f3f317fa700 1 -- 172.21.15.69:0/143796769 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00f7c0 con 0x55e25f938b60 2024-05-14T16:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.739+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/143796769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 msgr2=0x55e25f938540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.739+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/143796769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938540 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00bca0 tx=0x7f3f3c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.739+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/143796769 shutdown_connections 2024-05-14T16:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.739+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/143796769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f938b60 0x55e25f938540 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00bca0 tx=0x7f3f3c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.739+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/143796769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f93e750 0x55e25f956840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.739+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/143796769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93de30 0x55e25f9407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.739+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/143796769 >> 172.21.15.69:0/143796769 conn(0x55e25f8633e0 msgr2=0x55e25f93c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.740+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/143796769 shutdown_connections 2024-05-14T16:52:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.740+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/143796769 wait complete. 2024-05-14T16:52:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.741+0000 7f3f5c65fb80 1 -- start start 2024-05-14T16:52:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.741+0000 7f3f5c65fb80 1 Processor -- start 2024-05-14T16:52:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.741+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f93de30 0x55e25f95abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.741+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93e750 0x55e25f95b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.741+0000 7f3f5c65fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f950b50 0x55e25f95b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.741+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c1aa250 con 0x55e25f93de30 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.741+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c1ab6c0 con 0x55e25f950b50 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.741+0000 7f3f5c65fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c1aa920 con 0x55e25f93e750 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.741+0000 7f3f5c65fb80 10 client.0 _ll_register_callbacks cb 0x55e25f81d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f950b50 0x55e25f95b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f950b50 0x55e25f95b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:50046/0 (socket says 172.21.15.69:50046) 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4bfff700 1 -- 172.21.15.69:0/842619450 learned_addr learned my addr 172.21.15.69:0/842619450 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f93de30 0x55e25f95abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4affd700 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93e750 0x55e25f95b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4bfff700 1 -- 172.21.15.69:0/842619450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93e750 msgr2=0x55e25f95b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93e750 0x55e25f95b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4bfff700 1 -- 172.21.15.69:0/842619450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f93de30 msgr2=0x55e25f95abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f93de30 0x55e25f95abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4bfff700 1 -- 172.21.15.69:0/842619450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e25f9bcc70 con 0x55e25f950b50 2024-05-14T16:52:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f4bfff700 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f950b50 0x55e25f95b670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f44012550 tx=0x7f3f4401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df0eeb291cc101d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f2affd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f2affd700 1 -- 172.21.15.69:0/842619450 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44039060 con 0x55e25f950b50 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/842619450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f3c003b30 con 0x55e25f950b50 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f2affd700 1 -- 172.21.15.69:0/842619450 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f3f44004db0 con 0x55e25f950b50 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f2affd700 1 -- 172.21.15.69:0/842619450 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3f4401a040 con 0x55e25f950b50 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.742+0000 7f3f5c65fb80 10 client.25193 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.743+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/842619450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e25f9bdf40 con 0x55e25f950b50 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.743+0000 7f3f2affd700 1 -- 172.21.15.69:0/842619450 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f3c003b30 con 0x55e25f950b50 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.743+0000 7f3f2affd700 1 -- 172.21.15.69:0/842619450 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f3f4401e070 con 0x55e25f950b50 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.743+0000 7f3f5c65fb80 10 client.25193 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.743+0000 7f3f5c65fb80 10 client.25193 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.743+0000 7f3f5c65fb80 20 client.25193 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.743+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e25f9be3d0 0x55e25f9c07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.743+0000 7f3f5c65fb80 4 client.25193 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}] 2024-05-14T16:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.743+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/842619450 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}) -- 0x55e25f898d70 con 0x55e25f9be3d0 2024-05-14T16:52:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.744+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e25f9be3d0 0x55e25f9c07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.746+0000 7f3f4b7fe700 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e25f9be3d0 0x55e25f9c07f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b24cf158e9e36fc server_cookie=6183f3c835e57c15 in_seq=0 out_seq=0 2024-05-14T16:52:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.746+0000 7f3f2affd700 10 client.25193 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f2affd700 1 -- 172.21.15.69:0/842619450 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55e25f898d70 con 0x55e25f9be3d0 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f2affd700 10 client.25193 handle_command_reply: tid=0 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 client.25193 shutdown 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/842619450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f950b50 msgr2=0x55e25f95b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f950b50 0x55e25f95b670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f44012550 tx=0x7f3f4401c000 comp rx=0 tx=0).stop 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/842619450 shutdown_connections 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e25f9be3d0 0x55e25f9c07f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e25f93de30 0x55e25f95abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f2affd700 0 client.25193 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e25f950b50 0x55e25f95b670 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 --2- 172.21.15.69:0/842619450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e25f93e750 0x55e25f95b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/842619450 >> 172.21.15.69:0/842619450 conn(0x55e25f8633e0 msgr2=0x55e25f93d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/842619450 shutdown_connections 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 1 -- 172.21.15.69:0/842619450 wait complete. 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.928+0000 7f3f5c65fb80 20 client.25193 trim_cache size 0 max 16384 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 31, 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 10, 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stdout: "9577e2f": { 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 10, 2024-05-14T16:52:14.929 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 146.7, 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:14.930 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:14.931 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:14.931 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:52:14.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 -- 172.21.15.69:0/853585309 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3f340b2ba0 msgr2=0x7f3f340b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3f340b2ba0 0x7f3f340b5060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f44021000 tx=0x7f3f440159f0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 -- 172.21.15.69:0/853585309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c102e00 msgr2=0x7f3f4c1045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c102e00 0x7f3f4c1045d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000d750 tx=0x7f3f4000db10 comp rx=0 tx=0).stop 2024-05-14T16:52:14.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 -- 172.21.15.69:0/853585309 shutdown_connections 2024-05-14T16:52:14.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3f340b2ba0 0x7f3f340b5060 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f4c102440 0x7f3f4c104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f4c1084c0 0x7f3f4c104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 --2- 172.21.15.69:0/853585309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c102e00 0x7f3f4c1045d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 -- 172.21.15.69:0/853585309 >> 172.21.15.69:0/853585309 conn(0x7f3f4c0793e0 msgr2=0x7f3f4c06b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:14.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 -- 172.21.15.69:0/853585309 shutdown_connections 2024-05-14T16:52:14.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:14.929+0000 7f3f51dc9700 1 -- 172.21.15.69:0/853585309 wait complete. 2024-05-14T16:52:15.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 105.0 s, 2844.9 tps, lat 31.188 ms stddev 33.564 2024-05-14T16:52:15.789 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9577e2f', quiesce took 8.6 seconds. Will release after: 18.1 2024-05-14T16:52:16.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:16 smithi069 ceph-mon[31507]: pgmap v75: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-05-14T16:52:16.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:16 smithi203 ceph-mon[35787]: pgmap v75: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-05-14T16:52:16.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:16 smithi092 ceph-mon[35763]: pgmap v75: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-05-14T16:52:18.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:18 smithi069 ceph-mon[31507]: pgmap v76: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 604 op/s 2024-05-14T16:52:18.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:18 smithi203 ceph-mon[35787]: pgmap v76: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 604 op/s 2024-05-14T16:52:18.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:18 smithi092 ceph-mon[35763]: pgmap v76: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 604 op/s 2024-05-14T16:52:20.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 110.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:52:20.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:20 smithi069 ceph-mon[31507]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 MiB/s wr, 601 op/s 2024-05-14T16:52:20.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:20 smithi203 ceph-mon[35787]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 MiB/s wr, 601 op/s 2024-05-14T16:52:20.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:20 smithi092 ceph-mon[35763]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 MiB/s wr, 601 op/s 2024-05-14T16:52:22.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:22 smithi069 ceph-mon[31507]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 604 op/s 2024-05-14T16:52:22.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:22 smithi203 ceph-mon[35787]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 604 op/s 2024-05-14T16:52:22.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:22 smithi092 ceph-mon[35763]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 604 op/s 2024-05-14T16:52:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.646+0000 7fdb769f0640 1 -- 172.21.15.69:0/20745320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb70074e00 msgr2=0x7fdb700751e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.646+0000 7fdb769f0640 1 --2- 172.21.15.69:0/20745320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb70074e00 0x7fdb700751e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009900 tx=0x7fdb6002a480 comp rx=0 tx=0).stop 2024-05-14T16:52:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.646+0000 7fdb769f0640 1 -- 172.21.15.69:0/20745320 shutdown_connections 2024-05-14T16:52:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.646+0000 7fdb769f0640 1 --2- 172.21.15.69:0/20745320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb7010f2f0 0x7fdb701116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.646+0000 7fdb769f0640 1 --2- 172.21.15.69:0/20745320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb70075720 0x7fdb7010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.646+0000 7fdb769f0640 1 --2- 172.21.15.69:0/20745320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb70074e00 0x7fdb700751e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.646+0000 7fdb769f0640 1 -- 172.21.15.69:0/20745320 >> 172.21.15.69:0/20745320 conn(0x7fdb700fe500 msgr2=0x7fdb70100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.646+0000 7fdb769f0640 1 -- 172.21.15.69:0/20745320 shutdown_connections 2024-05-14T16:52:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.646+0000 7fdb769f0640 1 -- 172.21.15.69:0/20745320 wait complete. 2024-05-14T16:52:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.646+0000 7fdb769f0640 1 Processor -- start 2024-05-14T16:52:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb769f0640 1 -- start start 2024-05-14T16:52:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb769f0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb70074e00 0x7fdb7019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb769f0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb70075720 0x7fdb7019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb769f0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb7010f2f0 0x7fdb7019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb769f0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70114ec0 con 0x7fdb70075720 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb769f0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70114d40 con 0x7fdb70074e00 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb769f0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70115040 con 0x7fdb7010f2f0 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb6f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb70075720 0x7fdb7019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb6f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb70075720 0x7fdb7019b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55670/0 (socket says 172.21.15.69:55670) 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/959997671 learned_addr learned my addr 172.21.15.69:0/959997671 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb6ffff640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb70074e00 0x7fdb7019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb7010f2f0 0x7fdb7019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/959997671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb7010f2f0 msgr2=0x7fdb7019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb7010f2f0 0x7fdb7019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/959997671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb70074e00 msgr2=0x7fdb7019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb70074e00 0x7fdb7019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.647+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/959997671 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb701a0430 con 0x7fdb70075720 2024-05-14T16:52:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.648+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb70075720 0x7fdb7019b980 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800e9f0 tx=0x7fdb5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.648+0000 7fdb6d7fa640 1 -- 172.21.15.69:0/959997671 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800cd80 con 0x7fdb70075720 2024-05-14T16:52:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.648+0000 7fdb769f0640 1 -- 172.21.15.69:0/959997671 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb701a0720 con 0x7fdb70075720 2024-05-14T16:52:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.648+0000 7fdb6d7fa640 1 -- 172.21.15.69:0/959997671 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb580108e0 con 0x7fdb70075720 2024-05-14T16:52:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.648+0000 7fdb6d7fa640 1 -- 172.21.15.69:0/959997671 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58018980 con 0x7fdb70075720 2024-05-14T16:52:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.648+0000 7fdb769f0640 1 -- 172.21.15.69:0/959997671 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb701a7fb0 con 0x7fdb70075720 2024-05-14T16:52:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.650+0000 7fdb6d7fa640 1 -- 172.21.15.69:0/959997671 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdb58010430 con 0x7fdb70075720 2024-05-14T16:52:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.650+0000 7fdb797e3740 1 Processor -- start 2024-05-14T16:52:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.650+0000 7fdb797e3740 1 -- start start 2024-05-14T16:52:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645b8f0 0x564f9645bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645c210 0x564f96466590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f96468ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb797e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564f963cec30 con 0x564f96466ad0 2024-05-14T16:52:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb797e3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564f963be7d0 con 0x564f9645b8f0 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb797e3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564f963c2810 con 0x564f9645c210 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f96468ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f96468ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55682/0 (socket says 172.21.15.69:55682) 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/3130189101 learned_addr learned my addr 172.21.15.69:0/3130189101 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6ffff640 1 --2- 172.21.15.69:0/3130189101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645c210 0x564f96466590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/3130189101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645b8f0 0x564f9645bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6d7fa640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fdb5c0b2a10 0x7fdb5c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/3130189101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645c210 msgr2=0x564f96466590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/3130189101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645c210 0x564f96466590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/3130189101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645b8f0 msgr2=0x564f9645bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/3130189101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645b8f0 0x564f9645bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/3130189101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f960d3210 con 0x564f96466ad0 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fdb5c0b2a10 0x7fdb5c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6d7fa640 1 -- 172.21.15.69:0/959997671 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fdb580e29e0 con 0x7fdb70075720 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.651+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/3130189101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f96468ec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdb580e00b0 tx=0x7fdb5806f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a279c5f949ec51 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.652+0000 7fdb667fc640 1 -- 172.21.15.69:0/3130189101 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb580a9aa0 con 0x564f96466ad0 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.652+0000 7fdb667fc640 1 -- 172.21.15.69:0/3130189101 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb58010720 con 0x564f96466ad0 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.652+0000 7fdb667fc640 1 -- 172.21.15.69:0/3130189101 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb580b0870 con 0x564f96466ad0 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.652+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fdb5c0b2a10 0x7fdb5c0b4ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005fd0 tx=0x7fdb68004a40 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 -- 172.21.15.69:0/3130189101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 msgr2=0x564f96468ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 --2- 172.21.15.69:0/3130189101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f96468ec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdb580e00b0 tx=0x7fdb5806f330 comp rx=0 tx=0).stop 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 -- 172.21.15.69:0/3130189101 shutdown_connections 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 --2- 172.21.15.69:0/3130189101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f96468ec0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 --2- 172.21.15.69:0/3130189101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645c210 0x564f96466590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 --2- 172.21.15.69:0/3130189101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645b8f0 0x564f9645bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 -- 172.21.15.69:0/3130189101 >> 172.21.15.69:0/3130189101 conn(0x564f96457e00 msgr2=0x564f964581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 -- 172.21.15.69:0/3130189101 shutdown_connections 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 -- 172.21.15.69:0/3130189101 wait complete. 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 -- start start 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.653+0000 7fdb797e3740 1 Processor -- start 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f96466400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f9645c210 0x564f964636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f96466ad0 0x564f96463c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb797e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564f960a4790 con 0x564f9645c210 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb797e3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564f9637a890 con 0x564f96466ad0 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb797e3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564f964696e0 con 0x564f9645b8f0 2024-05-14T16:52:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb797e3740 10 client.0 _ll_register_callbacks cb 0x564f9637dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb6f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f9645c210 0x564f964636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb6f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f9645c210 0x564f964636f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55692/0 (socket says 172.21.15.69:55692) 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/2512695711 learned_addr learned my addr 172.21.15.69:0/2512695711 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb6ffff640 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f96466400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f96466ad0 0x564f96463c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/2512695711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 msgr2=0x564f96466400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f96466400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/2512695711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f96466ad0 msgr2=0x564f96463c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f96466ad0 0x564f96463c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb6f7fe640 1 -- 172.21.15.69:0/2512695711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f9619f620 con 0x564f9645c210 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f96466ad0 0x564f96463c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:52:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f9645c210 0x564f964636f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdb580b07e0 tx=0x7fdb580e2ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189165db0126dd28 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2512695711 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb580a9440 con 0x564f9645c210 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2512695711 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fdb580a5f60 con 0x564f9645c210 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.654+0000 7fdb797e3740 1 -- 172.21.15.69:0/2512695711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564f963d0b80 con 0x564f9645c210 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2512695711 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdb580a9620 con 0x564f9645c210 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb797e3740 10 client.15663 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb797e3740 1 -- 172.21.15.69:0/2512695711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564f964750e0 con 0x564f9645c210 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2512695711 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564f963d0b80 con 0x564f9645c210 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2512695711 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fdb5806ef10 con 0x564f9645c210 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb797e3740 10 client.15663 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb797e3740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:52:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb797e3740 20 client.15663 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564f96476430 0x564f96478820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb797e3740 4 client.15663 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb797e3740 1 -- 172.21.15.69:0/2512695711 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f963ca360 con 0x564f96476430 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.655+0000 7fdb6ffff640 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564f96476430 0x564f96478820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.656+0000 7fdb4f7fe640 10 client.15663 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.656+0000 7fdb6ffff640 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564f96476430 0x564f96478820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae78298c6c01a844 server_cookie=d01619cd759b03d6 in_seq=0 out_seq=0 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.656+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2512695711 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f963ca360 con 0x564f96476430 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.656+0000 7fdb4f7fe640 10 client.15663 handle_command_reply: tid=0 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.656+0000 7fdb797e3740 1 client.15663 shutdown 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 1 -- 172.21.15.69:0/2512695711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f9645c210 msgr2=0x564f964636f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f9645c210 0x564f964636f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdb580b07e0 tx=0x7fdb580e2ce0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 1 -- 172.21.15.69:0/2512695711 shutdown_connections 2024-05-14T16:52:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564f96476430 0x564f96478820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f96466ad0 0x564f96463c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f9645c210 0x564f964636f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb4f7fe640 0 client.15663 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2512695711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f96466400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 1 -- 172.21.15.69:0/2512695711 >> 172.21.15.69:0/2512695711 conn(0x564f96457e00 msgr2=0x564f96457100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 1 -- 172.21.15.69:0/2512695711 shutdown_connections 2024-05-14T16:52:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 1 -- 172.21.15.69:0/2512695711 wait complete. 2024-05-14T16:52:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.657+0000 7fdb797e3740 20 client.15663 trim_cache size 0 max 16384 2024-05-14T16:52:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.672+0000 7fdb797e3740 1 Processor -- start 2024-05-14T16:52:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.672+0000 7fdb797e3740 1 -- start start 2024-05-14T16:52:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.672+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f964725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.672+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 0x564f9647df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.672+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9647e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.672+0000 7fdb797e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701b3130 con 0x564f96466ad0 2024-05-14T16:52:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.672+0000 7fdb797e3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701b0970 con 0x564f9645c210 2024-05-14T16:52:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.672+0000 7fdb797e3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701ae7c0 con 0x564f9645b8f0 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb74f8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9647e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb74f8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9647e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55704/0 (socket says 172.21.15.69:55704) 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb74f8d640 1 -- 172.21.15.69:0/3719827964 learned_addr learned my addr 172.21.15.69:0/3719827964 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/3719827964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 0x564f9647df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb6ffff640 1 --2- 172.21.15.69:0/3719827964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f964725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb74f8d640 1 -- 172.21.15.69:0/3719827964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 msgr2=0x564f964725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/3719827964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f964725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb74f8d640 1 -- 172.21.15.69:0/3719827964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 msgr2=0x564f9647df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/3719827964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 0x564f9647df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb74f8d640 1 -- 172.21.15.69:0/3719827964 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f96456f50 con 0x564f96466ad0 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/3719827964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 0x564f9647df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/3719827964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9647e4a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdb6801e770 tx=0x7fdb68014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489fd6fb4937e6a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb65ffb640 1 -- 172.21.15.69:0/3719827964 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb680127b0 con 0x564f96466ad0 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb65ffb640 1 -- 172.21.15.69:0/3719827964 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb68016070 con 0x564f96466ad0 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.673+0000 7fdb65ffb640 1 -- 172.21.15.69:0/3719827964 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68027da0 con 0x564f96466ad0 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.674+0000 7fdb797e3740 1 -- 172.21.15.69:0/3719827964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 msgr2=0x564f9647e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.674+0000 7fdb797e3740 1 --2- 172.21.15.69:0/3719827964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9647e4a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdb6801e770 tx=0x7fdb68014000 comp rx=0 tx=0).stop 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.674+0000 7fdb797e3740 1 -- 172.21.15.69:0/3719827964 shutdown_connections 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.674+0000 7fdb797e3740 1 --2- 172.21.15.69:0/3719827964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9647e4a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.674+0000 7fdb797e3740 1 --2- 172.21.15.69:0/3719827964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 0x564f9647df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.674+0000 7fdb797e3740 1 --2- 172.21.15.69:0/3719827964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f964725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.674+0000 7fdb797e3740 1 -- 172.21.15.69:0/3719827964 >> 172.21.15.69:0/3719827964 conn(0x564f96457e00 msgr2=0x564f96463810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.674+0000 7fdb797e3740 1 -- 172.21.15.69:0/3719827964 shutdown_connections 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.674+0000 7fdb797e3740 1 -- 172.21.15.69:0/3719827964 wait complete. 2024-05-14T16:52:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.674+0000 7fdb797e3740 1 -- start start 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb797e3740 1 Processor -- start 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f964d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 0x564f964d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb797e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9646bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb797e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701aee30 con 0x564f96466ad0 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb797e3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701b02a0 con 0x564f9645c210 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb797e3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701af500 con 0x564f9645b8f0 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb797e3740 10 client.0 _ll_register_callbacks cb 0x7fdb580a5c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb74f8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9646bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb74f8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9646bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55706/0 (socket says 172.21.15.69:55706) 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb74f8d640 1 -- 172.21.15.69:0/2536286847 learned_addr learned my addr 172.21.15.69:0/2536286847 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb6f7fe640 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 0x564f964d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb6ffff640 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f964d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb74f8d640 1 -- 172.21.15.69:0/2536286847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 msgr2=0x564f964d8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f964d8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb74f8d640 1 -- 172.21.15.69:0/2536286847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 msgr2=0x564f964d9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 0x564f964d9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb74f8d640 1 -- 172.21.15.69:0/2536286847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f9646c4d0 con 0x564f96466ad0 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb6ffff640 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f964d8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb74f8d640 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9646bde0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800a010 tx=0x7fdb680276b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc8424a29d1b87a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.675+0000 7fdb4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2536286847 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68035060 con 0x564f96466ad0 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2536286847 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fdb68027cf0 con 0x564f96466ad0 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb797e3740 1 -- 172.21.15.69:0/2536286847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb68016070 con 0x564f96466ad0 2024-05-14T16:52:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2536286847 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdb6801e040 con 0x564f96466ad0 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb797e3740 10 client.15669 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb797e3740 1 -- 172.21.15.69:0/2536286847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564f964e0710 con 0x564f96466ad0 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2536286847 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb68016070 con 0x564f96466ad0 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2536286847 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fdb6801d440 con 0x564f96466ad0 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb797e3740 10 client.15669 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb797e3740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb797e3740 20 client.15669 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564f964e0e70 0x564f964e3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb797e3740 4 client.15669 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb797e3740 1 -- 172.21.15.69:0/2536286847 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f963445f0 con 0x564f964e0e70 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.676+0000 7fdb6ffff640 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564f964e0e70 0x564f964e3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.677+0000 7fdb6ffff640 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564f964e0e70 0x564f964e3290 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cedc26cc7381414a server_cookie=cedf1edff94ef170 in_seq=0 out_seq=0 2024-05-14T16:52:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.677+0000 7fdb4f7fe640 10 client.15669 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb4f7fe640 1 -- 172.21.15.69:0/2536286847 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564f963445f0 con 0x564f964e0e70 2024-05-14T16:52:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb4f7fe640 10 client.15669 handle_command_reply: tid=0 2024-05-14T16:52:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 client.15669 shutdown 2024-05-14T16:52:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 -- 172.21.15.69:0/2536286847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 msgr2=0x564f9646bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9646bde0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800a010 tx=0x7fdb680276b0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 -- 172.21.15.69:0/2536286847 shutdown_connections 2024-05-14T16:52:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564f964e0e70 0x564f964e3290 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f96466ad0 0x564f9646bde0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f9645c210 0x564f964d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 --2- 172.21.15.69:0/2536286847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f9645b8f0 0x564f964d8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb4f7fe640 0 client.15669 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 -- 172.21.15.69:0/2536286847 >> 172.21.15.69:0/2536286847 conn(0x564f96457e00 msgr2=0x564f964715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 -- 172.21.15.69:0/2536286847 shutdown_connections 2024-05-14T16:52:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 1 -- 172.21.15.69:0/2536286847 wait complete. 2024-05-14T16:52:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.689+0000 7fdb797e3740 20 client.15669 trim_cache size 0 max 16384 2024-05-14T16:52:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:52:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:52:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:52:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 -- 172.21.15.69:0/959997671 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fdb5c0b2a10 msgr2=0x7fdb5c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fdb5c0b2a10 0x7fdb5c0b4ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005fd0 tx=0x7fdb68004a40 comp rx=0 tx=0).stop 2024-05-14T16:52:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 -- 172.21.15.69:0/959997671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb70075720 msgr2=0x7fdb7019b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb70075720 0x7fdb7019b980 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800e9f0 tx=0x7fdb5800eeb0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 -- 172.21.15.69:0/959997671 shutdown_connections 2024-05-14T16:52:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fdb5c0b2a10 0x7fdb5c0b4ed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb7010f2f0 0x7fdb7019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb70075720 0x7fdb7019b980 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 --2- 172.21.15.69:0/959997671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb70074e00 0x7fdb7019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 -- 172.21.15.69:0/959997671 >> 172.21.15.69:0/959997671 conn(0x7fdb700fe500 msgr2=0x7fdb700ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 -- 172.21.15.69:0/959997671 shutdown_connections 2024-05-14T16:52:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:22.690+0000 7fdb769f0640 1 -- 172.21.15.69:0/959997671 wait complete. 2024-05-14T16:52:24.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:24 smithi069 ceph-mon[31507]: pgmap v79: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-05-14T16:52:24.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:24 smithi203 ceph-mon[35787]: pgmap v79: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-05-14T16:52:24.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:24 smithi092 ceph-mon[35763]: pgmap v79: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-05-14T16:52:25.397 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 115.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:52:26.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:26 smithi069 ceph-mon[31507]: pgmap v80: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-05-14T16:52:26.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:26 smithi203 ceph-mon[35787]: pgmap v80: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-05-14T16:52:26.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:26 smithi092 ceph-mon[35763]: pgmap v80: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-05-14T16:52:28.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:28 smithi069 ceph-mon[31507]: pgmap v81: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-05-14T16:52:28.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:28 smithi203 ceph-mon[35787]: pgmap v81: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-05-14T16:52:28.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:28 smithi092 ceph-mon[35763]: pgmap v81: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-05-14T16:52:30.398 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 120.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:52:30.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:30 smithi069 ceph-mon[31507]: pgmap v82: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-05-14T16:52:30.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:30 smithi203 ceph-mon[35787]: pgmap v82: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-05-14T16:52:30.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:30 smithi092 ceph-mon[35763]: pgmap v82: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-05-14T16:52:32.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:32 smithi069 ceph-mon[31507]: pgmap v83: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:32.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:32 smithi203 ceph-mon[35787]: pgmap v83: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:32.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:32 smithi092 ceph-mon[35763]: pgmap v83: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:33.890 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:52:34.147 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:34.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:34 smithi069 ceph-mon[31507]: pgmap v84: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:34.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:34 smithi203 ceph-mon[35787]: pgmap v84: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:34.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:34 smithi092 ceph-mon[35763]: pgmap v84: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 125.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:52:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.779+0000 7fc76fdf0640 1 -- 172.21.15.69:0/3391894296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76806b260 msgr2=0x7fc76806b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.779+0000 7fc76fdf0640 1 --2- 172.21.15.69:0/3391894296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76806b260 0x7fc76806b6e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc758009930 tx=0x7fc75802a420 comp rx=0 tx=0).stop 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 -- 172.21.15.69:0/3391894296 shutdown_connections 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 --2- 172.21.15.69:0/3391894296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc76806bc20 0x7fc76810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 --2- 172.21.15.69:0/3391894296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76806b260 0x7fc76806b6e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 --2- 172.21.15.69:0/3391894296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc768108b20 0x7fc768108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 -- 172.21.15.69:0/3391894296 >> 172.21.15.69:0/3391894296 conn(0x7fc7680787c0 msgr2=0x7fc768078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 -- 172.21.15.69:0/3391894296 shutdown_connections 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 -- 172.21.15.69:0/3391894296 wait complete. 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 Processor -- start 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 -- start start 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76806b260 0x7fc768101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc76806bc20 0x7fc7681021b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc768108b20 0x7fc7681026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc768112d30 con 0x7fc76806b260 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc768112bb0 con 0x7fc768108b20 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.780+0000 7fc76fdf0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc768112eb0 con 0x7fc76806bc20 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76db8c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76806b260 0x7fc768101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76db8c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76806b260 0x7fc768101c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37076/0 (socket says 172.21.15.69:37076) 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76db8c640 1 -- 172.21.15.69:0/624682064 learned_addr learned my addr 172.21.15.69:0/624682064 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76e38d640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc768108b20 0x7fc7681026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76d38b640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc76806bc20 0x7fc7681021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76db8c640 1 -- 172.21.15.69:0/624682064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc76806bc20 msgr2=0x7fc7681021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76db8c640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc76806bc20 0x7fc7681021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76db8c640 1 -- 172.21.15.69:0/624682064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc768108b20 msgr2=0x7fc7681026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76db8c640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc768108b20 0x7fc7681026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76db8c640 1 -- 172.21.15.69:0/624682064 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7681a76d0 con 0x7fc76806b260 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76db8c640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76806b260 0x7fc768101c70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc76400e9f0 tx=0x7fc76400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc75effd640 1 -- 172.21.15.69:0/624682064 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76400cd80 con 0x7fc76806b260 2024-05-14T16:52:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc75effd640 1 -- 172.21.15.69:0/624682064 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7640108e0 con 0x7fc76806b260 2024-05-14T16:52:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76fdf0640 1 -- 172.21.15.69:0/624682064 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7681a79c0 con 0x7fc76806b260 2024-05-14T16:52:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc75effd640 1 -- 172.21.15.69:0/624682064 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764018980 con 0x7fc76806b260 2024-05-14T16:52:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.781+0000 7fc76fdf0640 1 -- 172.21.15.69:0/624682064 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7681a7ee0 con 0x7fc76806b260 2024-05-14T16:52:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.784+0000 7fc772be5740 1 Processor -- start 2024-05-14T16:52:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc772be5740 1 -- start start 2024-05-14T16:52:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc772be5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c20cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc772be5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc772be5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c2dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc772be5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560eb9b93c30 con 0x560eb9c208f0 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc772be5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560eb9b837d0 con 0x560eb9c21210 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc772be5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560eb9b87810 con 0x560eb9c2bad0 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc75effd640 1 -- 172.21.15.69:0/624682064 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc764010430 con 0x7fc76806b260 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76e38d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c20cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76e38d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c20cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37080/0 (socket says 172.21.15.69:37080) 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76e38d640 1 -- 172.21.15.69:0/2583721281 learned_addr learned my addr 172.21.15.69:0/2583721281 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76d38b640 1 --2- 172.21.15.69:0/2583721281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c2dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76db8c640 1 --2- 172.21.15.69:0/2583721281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76e38d640 1 -- 172.21.15.69:0/2583721281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 msgr2=0x560eb9c2dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76e38d640 1 --2- 172.21.15.69:0/2583721281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c2dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76e38d640 1 -- 172.21.15.69:0/2583721281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 msgr2=0x560eb9c2b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76e38d640 1 --2- 172.21.15.69:0/2583721281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76e38d640 1 -- 172.21.15.69:0/2583721281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb9898210 con 0x560eb9c208f0 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76db8c640 1 --2- 172.21.15.69:0/2583721281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.785+0000 7fc76e38d640 1 --2- 172.21.15.69:0/2583721281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c20cd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc760010370 tx=0x7fc760010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d2dae5fcf8ad47 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.786+0000 7fc753fff640 1 -- 172.21.15.69:0/2583721281 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760016070 con 0x560eb9c208f0 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.786+0000 7fc753fff640 1 -- 172.21.15.69:0/2583721281 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc760004d20 con 0x560eb9c208f0 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.786+0000 7fc753fff640 1 -- 172.21.15.69:0/2583721281 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7600053a0 con 0x560eb9c208f0 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.786+0000 7fc75effd640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc7540b2a40 0x7fc7540b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.786+0000 7fc76db8c640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc7540b2a40 0x7fc7540b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.786+0000 7fc75effd640 1 -- 172.21.15.69:0/624682064 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc7640e2870 con 0x7fc76806b260 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.786+0000 7fc772be5740 1 -- 172.21.15.69:0/2583721281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 msgr2=0x560eb9c20cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.786+0000 7fc76db8c640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc7540b2a40 0x7fc7540b4f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7640e0080 tx=0x7fc76400cf20 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.786+0000 7fc772be5740 1 --2- 172.21.15.69:0/2583721281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c20cd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc760010370 tx=0x7fc760010830 comp rx=0 tx=0).stop 2024-05-14T16:52:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 -- 172.21.15.69:0/2583721281 shutdown_connections 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 --2- 172.21.15.69:0/2583721281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c2dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 --2- 172.21.15.69:0/2583721281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 --2- 172.21.15.69:0/2583721281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c20cd0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 -- 172.21.15.69:0/2583721281 >> 172.21.15.69:0/2583721281 conn(0x560eb9c1ce00 msgr2=0x560eb9c1d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 -- 172.21.15.69:0/2583721281 shutdown_connections 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 -- 172.21.15.69:0/2583721281 wait complete. 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 -- start start 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 Processor -- start 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c2b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c28770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560eb9869790 con 0x560eb9c208f0 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560eb9b3f890 con 0x560eb9c21210 2024-05-14T16:52:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560eb9c2e7a0 con 0x560eb9c2bad0 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.787+0000 7fc772be5740 10 client.0 _ll_register_callbacks cb 0x560eb9b42d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76e38d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c2b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76e38d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c2b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37094/0 (socket says 172.21.15.69:37094) 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76e38d640 1 -- 172.21.15.69:0/3132184471 learned_addr learned my addr 172.21.15.69:0/3132184471 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76d38b640 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c28770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76e38d640 1 -- 172.21.15.69:0/3132184471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 msgr2=0x560eb9c28d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76e38d640 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76e38d640 1 -- 172.21.15.69:0/3132184471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 msgr2=0x560eb9c28770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76e38d640 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c28770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76e38d640 1 -- 172.21.15.69:0/3132184471 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb9964620 con 0x560eb9c208f0 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76d38b640 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c28770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc76e38d640 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c2b480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc76000a010 tx=0x7fc760004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53331b2733ac6bdc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc7517fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc7517fa640 1 -- 172.21.15.69:0/3132184471 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760016070 con 0x560eb9c208f0 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc7517fa640 1 -- 172.21.15.69:0/3132184471 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc760012710 con 0x560eb9c208f0 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc772be5740 1 -- 172.21.15.69:0/3132184471 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc760004d20 con 0x560eb9c208f0 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc7517fa640 1 -- 172.21.15.69:0/3132184471 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc760012910 con 0x560eb9c208f0 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc772be5740 10 client.15681 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc772be5740 1 -- 172.21.15.69:0/3132184471 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560eb9c3a170 con 0x560eb9c208f0 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.788+0000 7fc7517fa640 1 -- 172.21.15.69:0/3132184471 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc760004d20 con 0x560eb9c208f0 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.789+0000 7fc7517fa640 1 -- 172.21.15.69:0/3132184471 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fc76000f040 con 0x560eb9c208f0 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.789+0000 7fc772be5740 10 client.15681 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.789+0000 7fc772be5740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.789+0000 7fc772be5740 20 client.15681 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.789+0000 7fc772be5740 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560eb9c3b520 0x560eb9c3d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.789+0000 7fc772be5740 4 client.15681 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.789+0000 7fc772be5740 1 -- 172.21.15.69:0/3132184471 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eb9b95b80 con 0x560eb9c3b520 2024-05-14T16:52:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.789+0000 7fc76d38b640 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560eb9c3b520 0x560eb9c3d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.789+0000 7fc7517fa640 10 client.15681 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.789+0000 7fc76d38b640 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560eb9c3b520 0x560eb9c3d910 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9219b67363fad82 server_cookie=15e63da226180ef1 in_seq=0 out_seq=0 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc7517fa640 1 -- 172.21.15.69:0/3132184471 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560eb9b95b80 con 0x560eb9c3b520 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc7517fa640 10 client.15681 handle_command_reply: tid=0 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 client.15681 shutdown 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 -- 172.21.15.69:0/3132184471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 msgr2=0x560eb9c2b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c2b480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc76000a010 tx=0x7fc760004b60 comp rx=0 tx=0).stop 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 -- 172.21.15.69:0/3132184471 shutdown_connections 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560eb9c3b520 0x560eb9c3d910 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc7517fa640 0 client.15681 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c28770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 --2- 172.21.15.69:0/3132184471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c2b480 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 -- 172.21.15.69:0/3132184471 >> 172.21.15.69:0/3132184471 conn(0x560eb9c1ce00 msgr2=0x560eb9c1c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 -- 172.21.15.69:0/3132184471 shutdown_connections 2024-05-14T16:52:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 1 -- 172.21.15.69:0/3132184471 wait complete. 2024-05-14T16:52:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.790+0000 7fc772be5740 20 client.15681 trim_cache size 0 max 16384 2024-05-14T16:52:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc772be5740 1 Processor -- start 2024-05-14T16:52:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc772be5740 1 -- start start 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc772be5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c208f0 0x560eb9c37670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc772be5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc772be5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c2bad0 0x560eb9c42f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc772be5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681b3220 con 0x560eb9c2bad0 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc772be5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681b0a60 con 0x560eb9c21210 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc772be5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681ae8b0 con 0x560eb9c208f0 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc76e38d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c2bad0 0x560eb9c42f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc76e38d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c2bad0 0x560eb9c42f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37110/0 (socket says 172.21.15.69:37110) 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc76e38d640 1 -- 172.21.15.69:0/155856130 learned_addr learned my addr 172.21.15.69:0/155856130 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc76e38d640 1 -- 172.21.15.69:0/155856130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c208f0 msgr2=0x560eb9c37670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc76e38d640 1 --2- 172.21.15.69:0/155856130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c208f0 0x560eb9c37670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc76e38d640 1 -- 172.21.15.69:0/155856130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 msgr2=0x560eb9c2afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc76e38d640 1 --2- 172.21.15.69:0/155856130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc76e38d640 1 -- 172.21.15.69:0/155856130 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb9c2dec0 con 0x560eb9c2bad0 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.806+0000 7fc76e38d640 1 --2- 172.21.15.69:0/155856130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c2bad0 0x560eb9c42f10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc760002230 tx=0x7fc760004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e441ed7fb7ac951 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.807+0000 7fc7537fe640 1 -- 172.21.15.69:0/155856130 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760016040 con 0x560eb9c2bad0 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.807+0000 7fc7537fe640 1 -- 172.21.15.69:0/155856130 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc760034070 con 0x560eb9c2bad0 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.807+0000 7fc7537fe640 1 -- 172.21.15.69:0/155856130 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760004e30 con 0x560eb9c2bad0 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.807+0000 7fc772be5740 1 -- 172.21.15.69:0/155856130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c2bad0 msgr2=0x560eb9c42f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.807+0000 7fc772be5740 1 --2- 172.21.15.69:0/155856130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c2bad0 0x560eb9c42f10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc760002230 tx=0x7fc760004af0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.807+0000 7fc772be5740 1 -- 172.21.15.69:0/155856130 shutdown_connections 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.807+0000 7fc772be5740 1 --2- 172.21.15.69:0/155856130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c2bad0 0x560eb9c42f10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.807+0000 7fc772be5740 1 --2- 172.21.15.69:0/155856130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.807+0000 7fc772be5740 1 --2- 172.21.15.69:0/155856130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c208f0 0x560eb9c37670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.807+0000 7fc772be5740 1 -- 172.21.15.69:0/155856130 >> 172.21.15.69:0/155856130 conn(0x560eb9c1ce00 msgr2=0x560eb9c22b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 1 -- 172.21.15.69:0/155856130 shutdown_connections 2024-05-14T16:52:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 1 -- 172.21.15.69:0/155856130 wait complete. 2024-05-14T16:52:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 1 -- start start 2024-05-14T16:52:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 1 Processor -- start 2024-05-14T16:52:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c33ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c2f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681aef20 con 0x560eb9c208f0 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681b0390 con 0x560eb9c21210 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681af5f0 con 0x560eb9c2bad0 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc772be5740 10 client.0 _ll_register_callbacks cb 0x7fc760004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.808+0000 7fc76e38d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c33ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc76e38d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c33ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37118/0 (socket says 172.21.15.69:37118) 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc76e38d640 1 -- 172.21.15.69:0/1698850802 learned_addr learned my addr 172.21.15.69:0/1698850802 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc76d38b640 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc76e38d640 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c2f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc76e38d640 1 -- 172.21.15.69:0/1698850802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 msgr2=0x560eb9c2f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc76e38d640 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c2f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc76e38d640 1 -- 172.21.15.69:0/1698850802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 msgr2=0x560eb9c2f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc76e38d640 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc76e38d640 1 -- 172.21.15.69:0/1698850802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb9c2ffa0 con 0x560eb9c208f0 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc76e38d640 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c33ea0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc760009da0 tx=0x7fc760012c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bcd600f21bbeb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc7517fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc7517fa640 1 -- 172.21.15.69:0/1698850802 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76003f060 con 0x560eb9c208f0 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc7517fa640 1 -- 172.21.15.69:0/1698850802 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc7600058f0 con 0x560eb9c208f0 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc772be5740 1 -- 172.21.15.69:0/1698850802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc760034070 con 0x560eb9c208f0 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc7517fa640 1 -- 172.21.15.69:0/1698850802 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc760016070 con 0x560eb9c208f0 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc772be5740 10 client.15687 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.809+0000 7fc772be5740 1 -- 172.21.15.69:0/1698850802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560eb9ca5a40 con 0x560eb9c208f0 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.810+0000 7fc7517fa640 1 -- 172.21.15.69:0/1698850802 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc760034070 con 0x560eb9c208f0 2024-05-14T16:52:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.810+0000 7fc7517fa640 1 -- 172.21.15.69:0/1698850802 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fc7600042f0 con 0x560eb9c208f0 2024-05-14T16:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.810+0000 7fc772be5740 10 client.15687 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.810+0000 7fc772be5740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.810+0000 7fc772be5740 20 client.15687 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.810+0000 7fc772be5740 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560eb9ca60e0 0x560eb9ca8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.810+0000 7fc772be5740 4 client.15687 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.810+0000 7fc772be5740 1 -- 172.21.15.69:0/1698850802 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560eb9b8f360 con 0x560eb9ca60e0 2024-05-14T16:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.810+0000 7fc76d38b640 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560eb9ca60e0 0x560eb9ca8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.811+0000 7fc76d38b640 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560eb9ca60e0 0x560eb9ca8500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa6552203a91bd8 server_cookie=76bfc848f6ef0227 in_seq=0 out_seq=0 2024-05-14T16:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.811+0000 7fc7517fa640 10 client.15687 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc7517fa640 1 -- 172.21.15.69:0/1698850802 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560eb9b8f360 con 0x560eb9ca60e0 2024-05-14T16:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc7517fa640 10 client.15687 handle_command_reply: tid=0 2024-05-14T16:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 client.15687 shutdown 2024-05-14T16:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 -- 172.21.15.69:0/1698850802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 msgr2=0x560eb9c33ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c33ea0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc760009da0 tx=0x7fc760012c80 comp rx=0 tx=0).stop 2024-05-14T16:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 -- 172.21.15.69:0/1698850802 shutdown_connections 2024-05-14T16:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560eb9ca60e0 0x560eb9ca8500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560eb9c2bad0 0x560eb9c2f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560eb9c21210 0x560eb9c2f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc7517fa640 0 client.15687 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 --2- 172.21.15.69:0/1698850802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560eb9c208f0 0x560eb9c33ea0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 -- 172.21.15.69:0/1698850802 >> 172.21.15.69:0/1698850802 conn(0x560eb9c1ce00 msgr2=0x560eb9c222a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 -- 172.21.15.69:0/1698850802 shutdown_connections 2024-05-14T16:52:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 1 -- 172.21.15.69:0/1698850802 wait complete. 2024-05-14T16:52:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.823+0000 7fc772be5740 20 client.15687 trim_cache size 0 max 16384 2024-05-14T16:52:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:52:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:52:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:52:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 -- 172.21.15.69:0/624682064 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc7540b2a40 msgr2=0x7fc7540b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc7540b2a40 0x7fc7540b4f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7640e0080 tx=0x7fc76400cf20 comp rx=0 tx=0).stop 2024-05-14T16:52:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 -- 172.21.15.69:0/624682064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76806b260 msgr2=0x7fc768101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76806b260 0x7fc768101c70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc76400e9f0 tx=0x7fc76400eeb0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 -- 172.21.15.69:0/624682064 shutdown_connections 2024-05-14T16:52:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc7540b2a40 0x7fc7540b4f00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc768108b20 0x7fc7681026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc76806bc20 0x7fc7681021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 --2- 172.21.15.69:0/624682064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76806b260 0x7fc768101c70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 -- 172.21.15.69:0/624682064 >> 172.21.15.69:0/624682064 conn(0x7fc7680787c0 msgr2=0x7fc7680ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 -- 172.21.15.69:0/624682064 shutdown_connections 2024-05-14T16:52:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:35.824+0000 7fc76fdf0640 1 -- 172.21.15.69:0/624682064 wait complete. 2024-05-14T16:52:35.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.913+0000 7fb5965cf700 1 -- 172.21.15.69:0/4213139910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5901085c0 msgr2=0x7fb5901089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.913+0000 7fb5965cf700 1 --2- 172.21.15.69:0/4213139910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5901085c0 0x7fb5901089a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb580009990 tx=0x7fb580009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.914+0000 7fb5965cf700 1 -- 172.21.15.69:0/4213139910 shutdown_connections 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.914+0000 7fb5965cf700 1 --2- 172.21.15.69:0/4213139910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5901085c0 0x7fb5901089a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.914+0000 7fb5965cf700 1 --2- 172.21.15.69:0/4213139910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb590103580 0x7fb59010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.914+0000 7fb5965cf700 1 --2- 172.21.15.69:0/4213139910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb590102bc0 0x7fb590103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.914+0000 7fb5965cf700 1 -- 172.21.15.69:0/4213139910 >> 172.21.15.69:0/4213139910 conn(0x7fb590078a40 msgr2=0x7fb5900ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.914+0000 7fb5965cf700 1 -- 172.21.15.69:0/4213139910 shutdown_connections 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.914+0000 7fb5965cf700 1 -- 172.21.15.69:0/4213139910 wait complete. 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.914+0000 7fb5965cf700 1 Processor -- start 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.914+0000 7fb5965cf700 1 -- start start 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb5965cf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb590102bc0 0x7fb590104120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb5965cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb590103580 0x7fb590104660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb5965cf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5901085c0 0x7fb590104ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb5965cf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb590112630 con 0x7fb590102bc0 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb5965cf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901124b0 con 0x7fb590103580 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb5965cf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901127b0 con 0x7fb5901085c0 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb590102bc0 0x7fb590104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb590102bc0 0x7fb590104120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37140/0 (socket says 172.21.15.69:37140) 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58ffff700 1 -- 172.21.15.69:0/222180201 learned_addr learned my addr 172.21.15.69:0/222180201 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58f7fe700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb590103580 0x7fb590104660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb594b6c700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5901085c0 0x7fb590104ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58ffff700 1 -- 172.21.15.69:0/222180201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5901085c0 msgr2=0x7fb590104ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58ffff700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5901085c0 0x7fb590104ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58ffff700 1 -- 172.21.15.69:0/222180201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb590103580 msgr2=0x7fb590104660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58ffff700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb590103580 0x7fb590104660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58ffff700 1 -- 172.21.15.69:0/222180201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5901a8120 con 0x7fb590102bc0 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58ffff700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb590102bc0 0x7fb590104120 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb580000c00 tx=0x7fb5800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58d7fa700 1 -- 172.21.15.69:0/222180201 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580027070 con 0x7fb590102bc0 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb5965cf700 1 -- 172.21.15.69:0/222180201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5901a83b0 con 0x7fb590102bc0 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58d7fa700 1 -- 172.21.15.69:0/222180201 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb580003fa0 con 0x7fb590102bc0 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb58d7fa700 1 -- 172.21.15.69:0/222180201 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580022dd0 con 0x7fb590102bc0 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.915+0000 7fb5965cf700 1 -- 172.21.15.69:0/222180201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901a8860 con 0x7fb590102bc0 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.916+0000 7fb5965cf700 1 -- 172.21.15.69:0/222180201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb59006a230 con 0x7fb590102bc0 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.918+0000 7fb58d7fa700 1 -- 172.21.15.69:0/222180201 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb58002a070 con 0x7fb590102bc0 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.918+0000 7fb58d7fa700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5780b29f0 0x7fb5780b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.918+0000 7fb58f7fe700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5780b29f0 0x7fb5780b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.919+0000 7fb58d7fa700 1 -- 172.21.15.69:0/222180201 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb5800e3690 con 0x7fb590102bc0 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.919+0000 7fb58f7fe700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5780b29f0 0x7fb5780b4eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb584005950 tx=0x7fb5840058e0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:35.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:35.922+0000 7fb58d7fa700 1 -- 172.21.15.69:0/222180201 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb580013070 con 0x7fb590102bc0 2024-05-14T16:52:36.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.172+0000 7fb5965cf700 1 -- 172.21.15.69:0/222180201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5901a8b80 con 0x7fb590102bc0 2024-05-14T16:52:36.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.172+0000 7fb58d7fa700 1 -- 172.21.15.69:0/222180201 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11863 (secure 0 0 0) 0x7fb5800ab1a0 con 0x7fb590102bc0 2024-05-14T16:52:36.172 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:52:36.172 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:51:16.740826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:52:36.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.174+0000 7fb5965cf700 1 -- 172.21.15.69:0/222180201 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5780b29f0 msgr2=0x7fb5780b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:36.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.174+0000 7fb5965cf700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5780b29f0 0x7fb5780b4eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb584005950 tx=0x7fb5840058e0 comp rx=0 tx=0).stop 2024-05-14T16:52:36.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.174+0000 7fb5965cf700 1 -- 172.21.15.69:0/222180201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb590102bc0 msgr2=0x7fb590104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:36.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.174+0000 7fb5965cf700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb590102bc0 0x7fb590104120 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb580000c00 tx=0x7fb5800047b0 comp rx=0 tx=0).stop 2024-05-14T16:52:36.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.175+0000 7fb5965cf700 1 -- 172.21.15.69:0/222180201 shutdown_connections 2024-05-14T16:52:36.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.175+0000 7fb5965cf700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb5780b29f0 0x7fb5780b4eb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:36.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.175+0000 7fb5965cf700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb590102bc0 0x7fb590104120 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:36.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.175+0000 7fb5965cf700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5901085c0 0x7fb590104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:36.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.175+0000 7fb5965cf700 1 --2- 172.21.15.69:0/222180201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb590103580 0x7fb590104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:36.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.175+0000 7fb5965cf700 1 -- 172.21.15.69:0/222180201 >> 172.21.15.69:0/222180201 conn(0x7fb590078a40 msgr2=0x7fb5901007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:36.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.175+0000 7fb5965cf700 1 -- 172.21.15.69:0/222180201 shutdown_connections 2024-05-14T16:52:36.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:36.175+0000 7fb5965cf700 1 -- 172.21.15.69:0/222180201 wait complete. 2024-05-14T16:52:36.175 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 31 2024-05-14T16:52:36.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:36 smithi069 ceph-mon[31507]: pgmap v85: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:36.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:36 smithi203 ceph-mon[35787]: pgmap v85: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:36.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:36 smithi092 ceph-mon[35763]: pgmap v85: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:36.875 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id 9577e2f --release --await' 2024-05-14T16:52:36.875 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id 9577e2f --release --await 2024-05-14T16:52:37.118 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:37.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:37 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/222180201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:37.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:37 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/222180201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:37.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:37 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/222180201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:38.396 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:38 smithi069 ceph-mon[31507]: pgmap v86: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:38.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:38 smithi203 ceph-mon[35787]: pgmap v86: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:38.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:38 smithi092 ceph-mon[35763]: pgmap v86: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-14T16:52:39.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.179+0000 7f75684df700 1 -- 172.21.15.69:0/4073672482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560108520 msgr2=0x7f7560108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:39.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.179+0000 7f75684df700 1 --2- 172.21.15.69:0/4073672482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560108520 0x7f7560108900 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f755c009990 tx=0x7f755c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.179+0000 7f75684df700 1 -- 172.21.15.69:0/4073672482 shutdown_connections 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.179+0000 7f75684df700 1 --2- 172.21.15.69:0/4073672482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560108520 0x7f7560108900 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.179+0000 7f75684df700 1 --2- 172.21.15.69:0/4073672482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7560102e00 0x7f756010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.179+0000 7f75684df700 1 --2- 172.21.15.69:0/4073672482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7560102440 0x7f75601028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.179+0000 7f75684df700 1 -- 172.21.15.69:0/4073672482 >> 172.21.15.69:0/4073672482 conn(0x7f75600793f0 msgr2=0x7f7560079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.179+0000 7f75684df700 1 -- 172.21.15.69:0/4073672482 shutdown_connections 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.179+0000 7f75684df700 1 -- 172.21.15.69:0/4073672482 wait complete. 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.180+0000 7f75684df700 1 Processor -- start 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.180+0000 7f75684df700 1 -- start start 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.180+0000 7f75684df700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560102440 0x7f756019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.180+0000 7f75684df700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7560102e00 0x7f756019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.180+0000 7f75684df700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7560108520 0x7f75601a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.180+0000 7f75684df700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75601125f0 con 0x7f7560102440 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.180+0000 7f75684df700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7560112470 con 0x7f7560108520 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.180+0000 7f75684df700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7560112770 con 0x7f7560102e00 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f756627b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560102440 0x7f756019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f7565a7a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7560102e00 0x7f756019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f756627b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560102440 0x7f756019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37154/0 (socket says 172.21.15.69:37154) 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f756627b700 1 -- 172.21.15.69:0/4272232839 learned_addr learned my addr 172.21.15.69:0/4272232839 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f7566a7c700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7560108520 0x7f75601a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f756627b700 1 -- 172.21.15.69:0/4272232839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7560102e00 msgr2=0x7f756019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f756627b700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7560102e00 0x7f756019afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f756627b700 1 -- 172.21.15.69:0/4272232839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7560108520 msgr2=0x7f75601a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f756627b700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7560108520 0x7f75601a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f756627b700 1 -- 172.21.15.69:0/4272232839 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75601a6030 con 0x7f7560102440 2024-05-14T16:52:39.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f756627b700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560102440 0x7f756019aa70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f755c000c00 tx=0x7f755c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:39.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f75577fe700 1 -- 172.21.15.69:0/4272232839 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c027070 con 0x7f7560102440 2024-05-14T16:52:39.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f75577fe700 1 -- 172.21.15.69:0/4272232839 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755c003f20 con 0x7f7560102440 2024-05-14T16:52:39.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f75577fe700 1 -- 172.21.15.69:0/4272232839 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c022ca0 con 0x7f7560102440 2024-05-14T16:52:39.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f75684df700 1 -- 172.21.15.69:0/4272232839 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75601a6200 con 0x7f7560102440 2024-05-14T16:52:39.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.181+0000 7f75684df700 1 -- 172.21.15.69:0/4272232839 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75601a65a0 con 0x7f7560102440 2024-05-14T16:52:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.184+0000 7f75577fe700 1 -- 172.21.15.69:0/4272232839 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f755c02a070 con 0x7f7560102440 2024-05-14T16:52:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.185+0000 7f75577fe700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f754c0b2a70 0x7f754c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.185+0000 7f7565a7a700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f754c0b2a70 0x7f754c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.185+0000 7f75577fe700 1 -- 172.21.15.69:0/4272232839 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f755c0e33e0 con 0x7f7560102440 2024-05-14T16:52:39.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.185+0000 7f7565a7a700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f754c0b2a70 0x7f754c0b4f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7560079a60 tx=0x7f755000b3c0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:39.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7572d75b80 1 Processor -- start 2024-05-14T16:52:39.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7572d75b80 1 -- start start 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4420 0x5613869f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f4d40 0x5613869efca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f02c0 0x5613869f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7572d75b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56138694de70 con 0x5613869f02c0 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7572d75b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56138693da10 con 0x5613869f4420 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7572d75b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561386941a50 con 0x5613869f4d40 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7566a7c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f02c0 0x5613869f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7566a7c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f02c0 0x5613869f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37166/0 (socket says 172.21.15.69:37166) 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7566a7c700 1 -- 172.21.15.69:0/899240328 learned_addr learned my addr 172.21.15.69:0/899240328 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f756627b700 1 --2- 172.21.15.69:0/899240328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4420 0x5613869f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7565a7a700 1 --2- 172.21.15.69:0/899240328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f4d40 0x5613869efca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7566a7c700 1 -- 172.21.15.69:0/899240328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f4d40 msgr2=0x5613869efca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7566a7c700 1 --2- 172.21.15.69:0/899240328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f4d40 0x5613869efca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7566a7c700 1 -- 172.21.15.69:0/899240328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4420 msgr2=0x5613869f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7566a7c700 1 --2- 172.21.15.69:0/899240328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4420 0x5613869f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.186+0000 7f7566a7c700 1 -- 172.21.15.69:0/899240328 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613868e70b0 con 0x5613869f02c0 2024-05-14T16:52:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.187+0000 7f7566a7c700 1 --2- 172.21.15.69:0/899240328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f02c0 0x5613869f0720 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f755800f410 tx=0x7f755800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156591f929124b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.187+0000 7f7554d70700 1 -- 172.21.15.69:0/899240328 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558018030 con 0x5613869f02c0 2024-05-14T16:52:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.187+0000 7f7554d70700 1 -- 172.21.15.69:0/899240328 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7558004bb0 con 0x5613869f02c0 2024-05-14T16:52:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.187+0000 7f7554d70700 1 -- 172.21.15.69:0/899240328 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558005020 con 0x5613869f02c0 2024-05-14T16:52:39.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.187+0000 7f7572d75b80 1 -- 172.21.15.69:0/899240328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f02c0 msgr2=0x5613869f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:39.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.187+0000 7f7572d75b80 1 --2- 172.21.15.69:0/899240328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f02c0 0x5613869f0720 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f755800f410 tx=0x7f755800f720 comp rx=0 tx=0).stop 2024-05-14T16:52:39.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 -- 172.21.15.69:0/899240328 shutdown_connections 2024-05-14T16:52:39.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 --2- 172.21.15.69:0/899240328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f02c0 0x5613869f0720 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 --2- 172.21.15.69:0/899240328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4420 0x5613869f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 --2- 172.21.15.69:0/899240328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f4d40 0x5613869efca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 -- 172.21.15.69:0/899240328 >> 172.21.15.69:0/899240328 conn(0x56138691ceb0 msgr2=0x5613869ea4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:39.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 -- 172.21.15.69:0/899240328 shutdown_connections 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 -- 172.21.15.69:0/899240328 wait complete. 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 -- start start 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 Processor -- start 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f02c0 0x5613869fba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4420 0x5613869f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869f9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5613868440a0 con 0x5613869f4d40 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5613868a5ea0 con 0x5613869f4420 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7572d75b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5613869fc880 con 0x5613869f02c0 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7566a7c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.188+0000 7f7566a7c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869f9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37170/0 (socket says 172.21.15.69:37170) 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7566a7c700 1 -- 172.21.15.69:0/409595679 learned_addr learned my addr 172.21.15.69:0/409595679 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7572d75b80 10 client.0 _ll_register_callbacks cb 0x5613868d8a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f756627b700 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f02c0 0x5613869fba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7565a7a700 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4420 0x5613869f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7566a7c700 1 -- 172.21.15.69:0/409595679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f02c0 msgr2=0x5613869fba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7566a7c700 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f02c0 0x5613869fba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7566a7c700 1 -- 172.21.15.69:0/409595679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4420 msgr2=0x5613869f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7566a7c700 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4420 0x5613869f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7566a7c700 1 -- 172.21.15.69:0/409595679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138688d9d0 con 0x5613869f4d40 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7566a7c700 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869f9370 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7558007df0 tx=0x7f7558007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d756f5fdbf21aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7549ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7549ffb700 1 -- 172.21.15.69:0/409595679 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558014980 con 0x5613869f4d40 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7549ffb700 1 -- 172.21.15.69:0/409595679 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7558004040 con 0x5613869f4d40 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7572d75b80 1 -- 172.21.15.69:0/409595679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7558004bb0 con 0x5613869f4d40 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7549ffb700 1 -- 172.21.15.69:0/409595679 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7558004240 con 0x5613869f4d40 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7572d75b80 10 client.15705 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:39.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.189+0000 7f7572d75b80 1 -- 172.21.15.69:0/409595679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5613868f7860 con 0x5613869f4d40 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.190+0000 7f7549ffb700 1 -- 172.21.15.69:0/409595679 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7558004bb0 con 0x5613869f4d40 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.190+0000 7f7549ffb700 1 -- 172.21.15.69:0/409595679 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f755801d910 con 0x5613869f4d40 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.190+0000 7f7572d75b80 10 client.15705 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.190+0000 7f7572d75b80 10 client.15705 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.190+0000 7f7572d75b80 20 client.15705 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.190+0000 7f7572d75b80 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561386a09240 0x561386a0b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.190+0000 7f7572d75b80 4 client.15705 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.190+0000 7f7572d75b80 1 -- 172.21.15.69:0/409595679 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56138694fdc0 con 0x561386a09240 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.190+0000 7f756627b700 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561386a09240 0x561386a0b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.191+0000 7f756627b700 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561386a09240 0x561386a0b630 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad74aae3d72ed105 server_cookie=c94930e2a32f67ee in_seq=0 out_seq=0 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.191+0000 7f7549ffb700 10 client.15705 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.191+0000 7f7549ffb700 1 -- 172.21.15.69:0/409595679 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56138694fdc0 con 0x561386a09240 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.191+0000 7f7549ffb700 10 client.15705 handle_command_reply: tid=0 2024-05-14T16:52:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.191+0000 7f7572d75b80 1 client.15705 shutdown 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.191+0000 7f7572d75b80 1 -- 172.21.15.69:0/409595679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 msgr2=0x5613869f9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.191+0000 7f7572d75b80 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869f9370 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7558007df0 tx=0x7f7558007710 comp rx=0 tx=0).stop 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.192+0000 7f7572d75b80 1 -- 172.21.15.69:0/409595679 shutdown_connections 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.192+0000 7f7572d75b80 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561386a09240 0x561386a0b630 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.192+0000 7f7572d75b80 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869f9370 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.192+0000 7f7572d75b80 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f02c0 0x5613869fba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.192+0000 7f7572d75b80 1 --2- 172.21.15.69:0/409595679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4420 0x5613869f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.192+0000 7f7572d75b80 1 -- 172.21.15.69:0/409595679 >> 172.21.15.69:0/409595679 conn(0x56138691ceb0 msgr2=0x5613869e4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.192+0000 7f7549ffb700 0 client.15705 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.192+0000 7f7572d75b80 1 -- 172.21.15.69:0/409595679 shutdown_connections 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.192+0000 7f7572d75b80 1 -- 172.21.15.69:0/409595679 wait complete. 2024-05-14T16:52:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.192+0000 7f7572d75b80 20 client.15705 trim_cache size 0 max 16384 2024-05-14T16:52:39.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7572d75b80 1 Processor -- start 2024-05-14T16:52:39.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7572d75b80 1 -- start start 2024-05-14T16:52:39.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f02c0 0x5613869e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4420 0x5613869edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4d40 0x5613869ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7572d75b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75601b0620 con 0x5613869f4420 2024-05-14T16:52:39.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7572d75b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ade20 con 0x5613869f4d40 2024-05-14T16:52:39.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7572d75b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75601abc70 con 0x5613869f02c0 2024-05-14T16:52:39.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7565a7a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4420 0x5613869edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7565a7a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4420 0x5613869edc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37178/0 (socket says 172.21.15.69:37178) 2024-05-14T16:52:39.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7565a7a700 1 -- 172.21.15.69:0/3150050298 learned_addr learned my addr 172.21.15.69:0/3150050298 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:39.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7566a7c700 1 --2- 172.21.15.69:0/3150050298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4d40 0x5613869ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f756627b700 1 --2- 172.21.15.69:0/3150050298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f02c0 0x5613869e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7565a7a700 1 -- 172.21.15.69:0/3150050298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f02c0 msgr2=0x5613869e50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:39.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7565a7a700 1 --2- 172.21.15.69:0/3150050298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f02c0 0x5613869e50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7565a7a700 1 -- 172.21.15.69:0/3150050298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4d40 msgr2=0x5613869ee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:39.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.211+0000 7f7565a7a700 1 --2- 172.21.15.69:0/3150050298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4d40 0x5613869ee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f7565a7a700 1 -- 172.21.15.69:0/3150050298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613869f73a0 con 0x5613869f4420 2024-05-14T16:52:39.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f7565a7a700 1 --2- 172.21.15.69:0/3150050298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4420 0x5613869edc90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7550003d40 tx=0x7f7550013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf8a35271742458 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:39.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f754bfff700 1 -- 172.21.15.69:0/3150050298 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550003e20 con 0x5613869f4420 2024-05-14T16:52:39.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f754bfff700 1 -- 172.21.15.69:0/3150050298 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7550011d00 con 0x5613869f4420 2024-05-14T16:52:39.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f754bfff700 1 -- 172.21.15.69:0/3150050298 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550005260 con 0x5613869f4420 2024-05-14T16:52:39.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f7572d75b80 1 -- 172.21.15.69:0/3150050298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4420 msgr2=0x5613869edc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:39.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f7572d75b80 1 --2- 172.21.15.69:0/3150050298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4420 0x5613869edc90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7550003d40 tx=0x7f7550013000 comp rx=0 tx=0).stop 2024-05-14T16:52:39.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f7572d75b80 1 -- 172.21.15.69:0/3150050298 shutdown_connections 2024-05-14T16:52:39.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f7572d75b80 1 --2- 172.21.15.69:0/3150050298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f4d40 0x5613869ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f7572d75b80 1 --2- 172.21.15.69:0/3150050298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f02c0 0x5613869e50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f7572d75b80 1 --2- 172.21.15.69:0/3150050298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4420 0x5613869edc90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.212+0000 7f7572d75b80 1 -- 172.21.15.69:0/3150050298 >> 172.21.15.69:0/3150050298 conn(0x56138691ceb0 msgr2=0x5613869ee7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:39.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 1 -- 172.21.15.69:0/3150050298 shutdown_connections 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 1 -- 172.21.15.69:0/3150050298 wait complete. 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 1 -- start start 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 1 Processor -- start 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f02c0 0x561386a01e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f4420 0x5613869fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869fd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ac2e0 con 0x5613869f4d40 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ad750 con 0x5613869f02c0 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ac9b0 con 0x5613869f4420 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7572d75b80 10 client.0 _ll_register_callbacks cb 0x5613868d8a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7566a7c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7566a7c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869fd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37186/0 (socket says 172.21.15.69:37186) 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f7566a7c700 1 -- 172.21.15.69:0/1249048551 learned_addr learned my addr 172.21.15.69:0/1249048551 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:39.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.213+0000 7f756627b700 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f02c0 0x561386a01e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7565a7a700 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f4420 0x5613869fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7566a7c700 1 -- 172.21.15.69:0/1249048551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f4420 msgr2=0x5613869fcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7566a7c700 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f4420 0x5613869fcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7566a7c700 1 -- 172.21.15.69:0/1249048551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f02c0 msgr2=0x561386a01e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7566a7c700 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f02c0 0x561386a01e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7566a7c700 1 -- 172.21.15.69:0/1249048551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613869fd630 con 0x5613869f4d40 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7566a7c700 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869fd070 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f755800f330 tx=0x7f7558003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2604e3c77e2eadc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7549ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7549ffb700 1 -- 172.21.15.69:0/1249048551 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558018070 con 0x5613869f4d40 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7549ffb700 1 -- 172.21.15.69:0/1249048551 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7558005450 con 0x5613869f4d40 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7572d75b80 1 -- 172.21.15.69:0/1249048551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7550011d00 con 0x5613869f4d40 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7549ffb700 1 -- 172.21.15.69:0/1249048551 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f755803c050 con 0x5613869f4d40 2024-05-14T16:52:39.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7572d75b80 10 client.15711 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7572d75b80 1 -- 172.21.15.69:0/1249048551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561386a746e0 con 0x5613869f4d40 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7549ffb700 1 -- 172.21.15.69:0/1249048551 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7550011d00 con 0x5613869f4d40 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7549ffb700 1 -- 172.21.15.69:0/1249048551 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f7558034020 con 0x5613869f4d40 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7572d75b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7572d75b80 10 client.15711 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7572d75b80 20 client.15711 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7572d75b80 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561386a74de0 0x561386a77200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7572d75b80 4 client.15711 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9577e2f", "release": true, "await": true}] 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.214+0000 7f7572d75b80 1 -- 172.21.15.69:0/1249048551 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9577e2f", "release": true, "await": true}) -- 0x5613869495a0 con 0x561386a74de0 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.215+0000 7f756627b700 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561386a74de0 0x561386a77200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.215+0000 7f756627b700 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561386a74de0 0x561386a77200 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a125d75ef5a684 server_cookie=297a395c88af9815 in_seq=0 out_seq=0 2024-05-14T16:52:39.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.215+0000 7f7549ffb700 10 client.15711 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:39.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7549ffb700 1 -- 172.21.15.69:0/1249048551 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5613869495a0 con 0x561386a74de0 2024-05-14T16:52:39.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7549ffb700 10 client.15711 handle_command_reply: tid=0 2024-05-14T16:52:39.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 client.15711 shutdown 2024-05-14T16:52:39.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 -- 172.21.15.69:0/1249048551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 msgr2=0x5613869fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:39.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869fd070 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f755800f330 tx=0x7f7558003d10 comp rx=0 tx=0).stop 2024-05-14T16:52:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 -- 172.21.15.69:0/1249048551 shutdown_connections 2024-05-14T16:52:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561386a74de0 0x561386a77200 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5613869f4d40 0x5613869fd070 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613869f02c0 0x561386a01e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 --2- 172.21.15.69:0/1249048551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613869f4420 0x5613869fcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 -- 172.21.15.69:0/1249048551 >> 172.21.15.69:0/1249048551 conn(0x56138691ceb0 msgr2=0x5613869fabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7549ffb700 0 client.15711 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 -- 172.21.15.69:0/1249048551 shutdown_connections 2024-05-14T16:52:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 1 -- 172.21.15.69:0/1249048551 wait complete. 2024-05-14T16:52:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.230+0000 7f7572d75b80 20 client.15711 trim_cache size 0 max 16384 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 31, 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 12, 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "9577e2f": { 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 12, 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:52:39.232 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 146.7, 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:39.233 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:39.234 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:52:39.234 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:52:39.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 -- 172.21.15.69:0/4272232839 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f754c0b2a70 msgr2=0x7f754c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:39.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f754c0b2a70 0x7f754c0b4f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7560079a60 tx=0x7f755000b3c0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 -- 172.21.15.69:0/4272232839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560102440 msgr2=0x7f756019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:39.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560102440 0x7f756019aa70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f755c000c00 tx=0x7f755c0047b0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 -- 172.21.15.69:0/4272232839 shutdown_connections 2024-05-14T16:52:39.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f754c0b2a70 0x7f754c0b4f30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560102440 0x7f756019aa70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7560108520 0x7f75601a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 --2- 172.21.15.69:0/4272232839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7560102e00 0x7f756019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:39.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 -- 172.21.15.69:0/4272232839 >> 172.21.15.69:0/4272232839 conn(0x7f75600793f0 msgr2=0x7f7560101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:39.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 -- 172.21.15.69:0/4272232839 shutdown_connections 2024-05-14T16:52:39.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:52:39.231+0000 7f75684df700 1 -- 172.21.15.69:0/4272232839 wait complete. 2024-05-14T16:52:40.020 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9577e2f', total seconds elapsed: 32.8 2024-05-14T16:52:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 130.0 s, 940.7 tps, lat 543.927 ms stddev 3526.998 2024-05-14T16:52:40.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:40 smithi069 ceph-mon[31507]: pgmap v87: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-05-14T16:52:40.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:40 smithi203 ceph-mon[35787]: pgmap v87: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-05-14T16:52:40.502 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:40 smithi092 ceph-mon[35763]: pgmap v87: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-05-14T16:52:42.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:42 smithi069 ceph-mon[31507]: pgmap v88: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 13 MiB/s wr, 1.17k op/s 2024-05-14T16:52:42.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:42 smithi203 ceph-mon[35787]: pgmap v88: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 13 MiB/s wr, 1.17k op/s 2024-05-14T16:52:42.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:42 smithi092 ceph-mon[35763]: pgmap v88: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 13 MiB/s wr, 1.17k op/s 2024-05-14T16:52:44.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:44 smithi069 ceph-mon[31507]: pgmap v89: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1000 KiB/s wr, 106 op/s 2024-05-14T16:52:44.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:44 smithi203 ceph-mon[35787]: pgmap v89: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1000 KiB/s wr, 106 op/s 2024-05-14T16:52:44.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:44 smithi092 ceph-mon[35763]: pgmap v89: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1000 KiB/s wr, 106 op/s 2024-05-14T16:52:45.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 135.0 s, 3597.0 tps, lat 27.663 ms stddev 29.319 2024-05-14T16:52:46.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:46 smithi069 ceph-mon[31507]: pgmap v90: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1000 KiB/s wr, 106 op/s 2024-05-14T16:52:46.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:46 smithi203 ceph-mon[35787]: pgmap v90: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1000 KiB/s wr, 106 op/s 2024-05-14T16:52:46.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:46 smithi092 ceph-mon[35763]: pgmap v90: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1000 KiB/s wr, 106 op/s 2024-05-14T16:52:48.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:48 smithi069 ceph-mon[31507]: pgmap v91: 97 pgs: 97 active+clean; 1.7 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.6 MiB/s wr, 450 op/s 2024-05-14T16:52:48.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:48 smithi203 ceph-mon[35787]: pgmap v91: 97 pgs: 97 active+clean; 1.7 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.6 MiB/s wr, 450 op/s 2024-05-14T16:52:48.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:48 smithi092 ceph-mon[35763]: pgmap v91: 97 pgs: 97 active+clean; 1.7 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.6 MiB/s wr, 450 op/s 2024-05-14T16:52:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.941+0000 7fe3275e4640 1 -- 172.21.15.69:0/2828903008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe320111240 msgr2=0x7fe320109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.941+0000 7fe3275e4640 1 --2- 172.21.15.69:0/2828903008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe320111240 0x7fe320109a50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009930 tx=0x7fe31c02a4d0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.941+0000 7fe3275e4640 1 -- 172.21.15.69:0/2828903008 shutdown_connections 2024-05-14T16:52:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.941+0000 7fe3275e4640 1 --2- 172.21.15.69:0/2828903008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe32011c230 0x7fe32011e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.941+0000 7fe3275e4640 1 --2- 172.21.15.69:0/2828903008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe320111620 0x7fe320109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.941+0000 7fe3275e4640 1 --2- 172.21.15.69:0/2828903008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe320111240 0x7fe320109a50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.941+0000 7fe3275e4640 1 -- 172.21.15.69:0/2828903008 >> 172.21.15.69:0/2828903008 conn(0x7fe32006db20 msgr2=0x7fe32006df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.941+0000 7fe3275e4640 1 -- 172.21.15.69:0/2828903008 shutdown_connections 2024-05-14T16:52:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.941+0000 7fe3275e4640 1 -- 172.21.15.69:0/2828903008 wait complete. 2024-05-14T16:52:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe3275e4640 1 Processor -- start 2024-05-14T16:52:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe3275e4640 1 -- start start 2024-05-14T16:52:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe3275e4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe320111240 0x7fe3201a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe3275e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe32011c230 0x7fe3201a8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe3275e4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3201ac9b0 0x7fe3201ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe3275e4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe320121bd0 con 0x7fe3201ac9b0 2024-05-14T16:52:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe3275e4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe320121a50 con 0x7fe32011c230 2024-05-14T16:52:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe3275e4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe320121d50 con 0x7fe320111240 2024-05-14T16:52:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe325b81640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3201ac9b0 0x7fe3201ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe324b7f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe32011c230 0x7fe3201a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe325b81640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3201ac9b0 0x7fe3201ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46686/0 (socket says 172.21.15.69:46686) 2024-05-14T16:52:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe325b81640 1 -- 172.21.15.69:0/611403051 learned_addr learned my addr 172.21.15.69:0/611403051 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.942+0000 7fe325380640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe320111240 0x7fe3201a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe325b81640 1 -- 172.21.15.69:0/611403051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe320111240 msgr2=0x7fe3201a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe325b81640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe320111240 0x7fe3201a80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe325b81640 1 -- 172.21.15.69:0/611403051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe32011c230 msgr2=0x7fe3201a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe325b81640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe32011c230 0x7fe3201a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe325b81640 1 -- 172.21.15.69:0/611403051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3201ad4a0 con 0x7fe3201ac9b0 2024-05-14T16:52:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe324b7f640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe32011c230 0x7fe3201a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe325b81640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3201ac9b0 0x7fe3201ace60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe31800b9e0 tx=0x7fe31800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe3167fc640 1 -- 172.21.15.69:0/611403051 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318019070 con 0x7fe3201ac9b0 2024-05-14T16:52:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe3167fc640 1 -- 172.21.15.69:0/611403051 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe318014440 con 0x7fe3201ac9b0 2024-05-14T16:52:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe3167fc640 1 -- 172.21.15.69:0/611403051 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3180125a0 con 0x7fe3201ac9b0 2024-05-14T16:52:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe3275e4640 1 -- 172.21.15.69:0/611403051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3201ad790 con 0x7fe3201ac9b0 2024-05-14T16:52:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.943+0000 7fe3275e4640 1 -- 172.21.15.69:0/611403051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3201b4f30 con 0x7fe3201ac9b0 2024-05-14T16:52:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.946+0000 7fe3167fc640 1 -- 172.21.15.69:0/611403051 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe31801c070 con 0x7fe3201ac9b0 2024-05-14T16:52:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe32a3da740 1 Processor -- start 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe32a3da740 1 -- start start 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 0x562ab0084cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab0085210 0x562ab008f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 0x562ab0091ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe32a3da740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562aafff7c30 con 0x562ab0085210 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe32a3da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562aaffe77d0 con 0x562ab008fad0 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe32a3da740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562aaffeb810 con 0x562ab00848f0 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe325380640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab0085210 0x562ab008f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe325380640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab0085210 0x562ab008f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46690/0 (socket says 172.21.15.69:46690) 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe325380640 1 -- 172.21.15.69:0/3886013777 learned_addr learned my addr 172.21.15.69:0/3886013777 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe3167fc640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe30c0b2ae0 0x7fe30c0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe324b7f640 1 --2- 172.21.15.69:0/3886013777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 0x562ab0091ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.947+0000 7fe324b7f640 1 --2- 172.21.15.69:0/3886013777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 0x562ab0084cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.948+0000 7fe325b81640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe30c0b2ae0 0x7fe30c0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.948+0000 7fe3167fc640 1 -- 172.21.15.69:0/611403051 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe3180e19c0 con 0x7fe3201ac9b0 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.948+0000 7fe324b7f640 1 -- 172.21.15.69:0/3886013777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 msgr2=0x562ab0084cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.948+0000 7fe324b7f640 1 --2- 172.21.15.69:0/3886013777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 0x562ab0084cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.948+0000 7fe324b7f640 1 -- 172.21.15.69:0/3886013777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab0085210 msgr2=0x562ab008f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.948+0000 7fe324b7f640 1 --2- 172.21.15.69:0/3886013777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab0085210 0x562ab008f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.948+0000 7fe324b7f640 1 -- 172.21.15.69:0/3886013777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aafcfc210 con 0x562ab008fad0 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.948+0000 7fe324b7f640 1 --2- 172.21.15.69:0/3886013777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 0x562ab0091ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe32006c120 tx=0x7fe310004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41fd1eb919e4d6c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.948+0000 7fe325b81640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe30c0b2ae0 0x7fe30c0b4fa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe3180df720 tx=0x7fe318098040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.948+0000 7fe30b7fe640 1 -- 172.21.15.69:0/3886013777 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310011050 con 0x562ab008fad0 2024-05-14T16:52:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.950+0000 7fe30b7fe640 1 -- 172.21.15.69:0/3886013777 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3100111f0 con 0x562ab008fad0 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.951+0000 7fe30b7fe640 1 -- 172.21.15.69:0/3886013777 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31000fea0 con 0x562ab008fad0 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.951+0000 7fe32a3da740 1 -- 172.21.15.69:0/3886013777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 msgr2=0x562ab0091ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.951+0000 7fe32a3da740 1 --2- 172.21.15.69:0/3886013777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 0x562ab0091ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe32006c120 tx=0x7fe310004730 comp rx=0 tx=0).stop 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.951+0000 7fe32a3da740 1 -- 172.21.15.69:0/3886013777 shutdown_connections 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.951+0000 7fe32a3da740 1 --2- 172.21.15.69:0/3886013777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 0x562ab0091ec0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.951+0000 7fe32a3da740 1 --2- 172.21.15.69:0/3886013777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab0085210 0x562ab008f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.951+0000 7fe32a3da740 1 --2- 172.21.15.69:0/3886013777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 0x562ab0084cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.951+0000 7fe32a3da740 1 -- 172.21.15.69:0/3886013777 >> 172.21.15.69:0/3886013777 conn(0x562ab0080e00 msgr2=0x562ab00811e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.951+0000 7fe32a3da740 1 -- 172.21.15.69:0/3886013777 shutdown_connections 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.951+0000 7fe32a3da740 1 -- 172.21.15.69:0/3886013777 wait complete. 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe32a3da740 1 -- start start 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe32a3da740 1 Processor -- start 2024-05-14T16:52:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 0x562ab008f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab0085210 0x562ab008c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 0x562ab008cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe32a3da740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562aafccd790 con 0x562ab00848f0 2024-05-14T16:52:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe32a3da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562aaffa3890 con 0x562ab008fad0 2024-05-14T16:52:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe32a3da740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562ab00927a0 con 0x562ab0085210 2024-05-14T16:52:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe32a3da740 10 client.0 _ll_register_callbacks cb 0x562aaffa6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe324b7f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 0x562ab008f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.952+0000 7fe324b7f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 0x562ab008f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46704/0 (socket says 172.21.15.69:46704) 2024-05-14T16:52:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe324b7f640 1 -- 172.21.15.69:0/1482500677 learned_addr learned my addr 172.21.15.69:0/1482500677 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe325380640 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab0085210 0x562ab008c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe324b7f640 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 0x562ab008cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe324b7f640 1 -- 172.21.15.69:0/1482500677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab0085210 msgr2=0x562ab008c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe324b7f640 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab0085210 0x562ab008c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe324b7f640 1 -- 172.21.15.69:0/1482500677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 msgr2=0x562ab008cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe324b7f640 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 0x562ab008cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe324b7f640 1 -- 172.21.15.69:0/1482500677 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aafdc8620 con 0x562ab00848f0 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe324b7f640 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 0x562ab008f420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe310009d80 tx=0x7fe31000fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c02773d3be27a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe308ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe308ff9640 1 -- 172.21.15.69:0/1482500677 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310005220 con 0x562ab00848f0 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe308ff9640 1 -- 172.21.15.69:0/1482500677 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe3100053c0 con 0x562ab00848f0 2024-05-14T16:52:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe32a3da740 1 -- 172.21.15.69:0/1482500677 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3100111f0 con 0x562ab00848f0 2024-05-14T16:52:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe308ff9640 1 -- 172.21.15.69:0/1482500677 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe310032050 con 0x562ab00848f0 2024-05-14T16:52:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe32a3da740 10 client.15723 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.953+0000 7fe32a3da740 1 -- 172.21.15.69:0/1482500677 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562ab009e250 con 0x562ab00848f0 2024-05-14T16:52:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.954+0000 7fe308ff9640 1 -- 172.21.15.69:0/1482500677 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3100111f0 con 0x562ab00848f0 2024-05-14T16:52:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.954+0000 7fe308ff9640 1 -- 172.21.15.69:0/1482500677 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fe31002a020 con 0x562ab00848f0 2024-05-14T16:52:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.954+0000 7fe32a3da740 10 client.15723 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.954+0000 7fe32a3da740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:52:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.954+0000 7fe32a3da740 20 client.15723 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.954+0000 7fe32a3da740 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ab009f5a0 0x562ab00a1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.954+0000 7fe32a3da740 4 client.15723 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:52:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.954+0000 7fe32a3da740 1 -- 172.21.15.69:0/1482500677 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aafff9b80 con 0x562ab009f5a0 2024-05-14T16:52:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.955+0000 7fe325380640 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ab009f5a0 0x562ab00a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.955+0000 7fe325380640 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ab009f5a0 0x562ab00a1990 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d6234391f574c6 server_cookie=78532204d272281d in_seq=0 out_seq=0 2024-05-14T16:52:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.955+0000 7fe308ff9640 10 client.15723 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe308ff9640 1 -- 172.21.15.69:0/1482500677 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562aafff9b80 con 0x562ab009f5a0 2024-05-14T16:52:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe308ff9640 10 client.15723 handle_command_reply: tid=0 2024-05-14T16:52:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 client.15723 shutdown 2024-05-14T16:52:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 -- 172.21.15.69:0/1482500677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 msgr2=0x562ab008f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 0x562ab008f420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe310009d80 tx=0x7fe31000fc80 comp rx=0 tx=0).stop 2024-05-14T16:52:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 -- 172.21.15.69:0/1482500677 shutdown_connections 2024-05-14T16:52:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ab009f5a0 0x562ab00a1990 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab008fad0 0x562ab008cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab0085210 0x562ab008c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 --2- 172.21.15.69:0/1482500677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 0x562ab008f420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe308ff9640 0 client.15723 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 -- 172.21.15.69:0/1482500677 >> 172.21.15.69:0/1482500677 conn(0x562ab0080e00 msgr2=0x562ab0080100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 -- 172.21.15.69:0/1482500677 shutdown_connections 2024-05-14T16:52:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 1 -- 172.21.15.69:0/1482500677 wait complete. 2024-05-14T16:52:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.956+0000 7fe32a3da740 20 client.15723 trim_cache size 0 max 16384 2024-05-14T16:52:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.987+0000 7fe32a3da740 1 Processor -- start 2024-05-14T16:52:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.987+0000 7fe32a3da740 1 -- start start 2024-05-14T16:52:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.988+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.988+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 0x562ab008ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.988+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab0085210 0x562ab00a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.988+0000 7fe32a3da740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201bfea0 con 0x562ab00848f0 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.988+0000 7fe32a3da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201bd6e0 con 0x562ab0080e00 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.988+0000 7fe32a3da740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201bb530 con 0x562ab0085210 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.988+0000 7fe324b7f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.988+0000 7fe324b7f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00869b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:49624/0 (socket says 172.21.15.69:49624) 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.988+0000 7fe324b7f640 1 -- 172.21.15.69:0/831680352 learned_addr learned my addr 172.21.15.69:0/831680352 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.988+0000 7fe324b7f640 1 --2- 172.21.15.69:0/831680352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab0085210 0x562ab00a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.989+0000 7fe324b7f640 1 -- 172.21.15.69:0/831680352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab0085210 msgr2=0x562ab00a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.989+0000 7fe324b7f640 1 --2- 172.21.15.69:0/831680352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab0085210 0x562ab00a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.989+0000 7fe324b7f640 1 -- 172.21.15.69:0/831680352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 msgr2=0x562ab008ef50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:52:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.989+0000 7fe324b7f640 1 --2- 172.21.15.69:0/831680352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 0x562ab008ef50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.989+0000 7fe324b7f640 1 -- 172.21.15.69:0/831680352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ab0080c30 con 0x562ab0080e00 2024-05-14T16:52:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.989+0000 7fe324b7f640 1 --2- 172.21.15.69:0/831680352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00869b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe310011940 tx=0x7fe3100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3719983e864eb962 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.989+0000 7fe30affd640 1 -- 172.21.15.69:0/831680352 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31001e5f0 con 0x562ab0080e00 2024-05-14T16:52:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.989+0000 7fe30affd640 1 -- 172.21.15.69:0/831680352 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31001f930 con 0x562ab0080e00 2024-05-14T16:52:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.989+0000 7fe30affd640 1 -- 172.21.15.69:0/831680352 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31001e790 con 0x562ab0080e00 2024-05-14T16:52:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.990+0000 7fe32a3da740 1 -- 172.21.15.69:0/831680352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 msgr2=0x562ab00869b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.990+0000 7fe32a3da740 1 --2- 172.21.15.69:0/831680352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00869b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe310011940 tx=0x7fe3100118d0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.990+0000 7fe32a3da740 1 -- 172.21.15.69:0/831680352 shutdown_connections 2024-05-14T16:52:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.990+0000 7fe32a3da740 1 --2- 172.21.15.69:0/831680352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab0085210 0x562ab00a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.990+0000 7fe32a3da740 1 --2- 172.21.15.69:0/831680352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab00848f0 0x562ab008ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.990+0000 7fe32a3da740 1 --2- 172.21.15.69:0/831680352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00869b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.990+0000 7fe32a3da740 1 -- 172.21.15.69:0/831680352 >> 172.21.15.69:0/831680352 conn(0x562ab00a1d00 msgr2=0x562ab00862a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.990+0000 7fe32a3da740 1 -- 172.21.15.69:0/831680352 shutdown_connections 2024-05-14T16:52:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.990+0000 7fe32a3da740 1 -- 172.21.15.69:0/831680352 wait complete. 2024-05-14T16:52:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.991+0000 7fe32a3da740 1 -- start start 2024-05-14T16:52:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.991+0000 7fe32a3da740 1 Processor -- start 2024-05-14T16:52:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00979c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 0x562ab0097f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe32a3da740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab0085210 0x562ab0092e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe32a3da740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201bbba0 con 0x562ab0085210 2024-05-14T16:52:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe32a3da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201bd010 con 0x562ab0080e00 2024-05-14T16:52:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe32a3da740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201bc270 con 0x562ab00848f0 2024-05-14T16:52:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe32a3da740 10 client.0 _ll_register_callbacks cb 0x7fe310011a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:52:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe324b7f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe324b7f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:49638/0 (socket says 172.21.15.69:49638) 2024-05-14T16:52:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe324b7f640 1 -- 172.21.15.69:0/3346234877 learned_addr learned my addr 172.21.15.69:0/3346234877 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe325380640 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 0x562ab0097f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe325380640 1 -- 172.21.15.69:0/3346234877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 msgr2=0x562ab00979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe325380640 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe325380640 1 -- 172.21.15.69:0/3346234877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab0085210 msgr2=0x562ab0092e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe325380640 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab0085210 0x562ab0092e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.992+0000 7fe325380640 1 -- 172.21.15.69:0/3346234877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ab00935f0 con 0x562ab00848f0 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.993+0000 7fe325380640 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 0x562ab0097f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x562ab0085e90 tx=0x7fe31c009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82489d11d15facbd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.993+0000 7fe308ff9640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.993+0000 7fe308ff9640 1 -- 172.21.15.69:0/3346234877 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c035040 con 0x562ab00848f0 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.993+0000 7fe308ff9640 1 -- 172.21.15.69:0/3346234877 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe31c005590 con 0x562ab00848f0 2024-05-14T16:52:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.993+0000 7fe32a3da740 1 -- 172.21.15.69:0/3346234877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31001f930 con 0x562ab00848f0 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.993+0000 7fe308ff9640 1 -- 172.21.15.69:0/3346234877 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe31c052050 con 0x562ab00848f0 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.993+0000 7fe32a3da740 10 client.25204 fetch_fsmap learned FSMap version 31 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.993+0000 7fe32a3da740 1 -- 172.21.15.69:0/3346234877 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562ab01097f0 con 0x562ab00848f0 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.994+0000 7fe308ff9640 1 -- 172.21.15.69:0/3346234877 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31001f930 con 0x562ab00848f0 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.994+0000 7fe308ff9640 1 -- 172.21.15.69:0/3346234877 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fe31c04d030 con 0x562ab00848f0 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.994+0000 7fe32a3da740 10 client.25204 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.994+0000 7fe32a3da740 10 client.25204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.994+0000 7fe32a3da740 20 client.25204 populate_metadata read hostname 'smithi069' 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.994+0000 7fe32a3da740 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ab008fad0 0x562ab010c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.994+0000 7fe32a3da740 4 client.25204 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:52:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.994+0000 7fe32a3da740 1 -- 172.21.15.69:0/3346234877 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562aafff3360 con 0x562ab008fad0 2024-05-14T16:52:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.994+0000 7fe324b7f640 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ab008fad0 0x562ab010c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.996+0000 7fe324b7f640 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ab008fad0 0x562ab010c090 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7329145b4f74748 server_cookie=579c65355b01026e in_seq=0 out_seq=0 2024-05-14T16:52:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:48.996+0000 7fe308ff9640 10 client.25204 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe308ff9640 1 -- 172.21.15.69:0/3346234877 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562aafff3360 con 0x562ab008fad0 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe308ff9640 10 client.25204 handle_command_reply: tid=0 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 client.25204 shutdown 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 -- 172.21.15.69:0/3346234877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 msgr2=0x562ab0097f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 0x562ab0097f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x562ab0085e90 tx=0x7fe31c009540 comp rx=0 tx=0).stop 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 -- 172.21.15.69:0/3346234877 shutdown_connections 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ab008fad0 0x562ab010c090 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ab0085210 0x562ab0092e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ab00848f0 0x562ab0097f00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 --2- 172.21.15.69:0/3346234877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ab0080e00 0x562ab00979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 -- 172.21.15.69:0/3346234877 >> 172.21.15.69:0/3346234877 conn(0x562ab00a1d00 msgr2=0x562ab0080100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe308ff9640 0 client.25204 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:52:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 -- 172.21.15.69:0/3346234877 shutdown_connections 2024-05-14T16:52:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 1 -- 172.21.15.69:0/3346234877 wait complete. 2024-05-14T16:52:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.006+0000 7fe32a3da740 20 client.25204 trim_cache size 0 max 16384 2024-05-14T16:52:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:52:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:52:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:52:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 -- 172.21.15.69:0/611403051 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe30c0b2ae0 msgr2=0x7fe30c0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe30c0b2ae0 0x7fe30c0b4fa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe3180df720 tx=0x7fe318098040 comp rx=0 tx=0).stop 2024-05-14T16:52:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 -- 172.21.15.69:0/611403051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3201ac9b0 msgr2=0x7fe3201ace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3201ac9b0 0x7fe3201ace60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe31800b9e0 tx=0x7fe31800bea0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 -- 172.21.15.69:0/611403051 shutdown_connections 2024-05-14T16:52:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe30c0b2ae0 0x7fe30c0b4fa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3201ac9b0 0x7fe3201ace60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe32011c230 0x7fe3201a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 --2- 172.21.15.69:0/611403051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe320111240 0x7fe3201a80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 -- 172.21.15.69:0/611403051 >> 172.21.15.69:0/611403051 conn(0x7fe32006db20 msgr2=0x7fe32010ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 -- 172.21.15.69:0/611403051 shutdown_connections 2024-05-14T16:52:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:52:49.008+0000 7fe3275e4640 1 -- 172.21.15.69:0/611403051 wait complete. 2024-05-14T16:52:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 140.0 s, 3521.6 tps, lat 28.526 ms stddev 29.280 2024-05-14T16:52:50.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:50 smithi069 ceph-mon[31507]: pgmap v92: 97 pgs: 97 active+clean; 1.7 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.6 MiB/s wr, 447 op/s 2024-05-14T16:52:50.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:50 smithi203 ceph-mon[35787]: pgmap v92: 97 pgs: 97 active+clean; 1.7 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.6 MiB/s wr, 447 op/s 2024-05-14T16:52:50.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:50 smithi092 ceph-mon[35763]: pgmap v92: 97 pgs: 97 active+clean; 1.7 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.6 MiB/s wr, 447 op/s 2024-05-14T16:52:52.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:52 smithi069 ceph-mon[31507]: pgmap v93: 97 pgs: 97 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.7 MiB/s wr, 759 op/s 2024-05-14T16:52:52.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:52 smithi203 ceph-mon[35787]: pgmap v93: 97 pgs: 97 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.7 MiB/s wr, 759 op/s 2024-05-14T16:52:52.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:52 smithi092 ceph-mon[35763]: pgmap v93: 97 pgs: 97 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.7 MiB/s wr, 759 op/s 2024-05-14T16:52:54.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:54 smithi069 ceph-mon[31507]: pgmap v94: 97 pgs: 97 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 656 op/s 2024-05-14T16:52:54.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:54 smithi203 ceph-mon[35787]: pgmap v94: 97 pgs: 97 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 656 op/s 2024-05-14T16:52:54.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:54 smithi092 ceph-mon[35763]: pgmap v94: 97 pgs: 97 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 656 op/s 2024-05-14T16:52:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 145.0 s, 3536.2 tps, lat 28.295 ms stddev 29.499 2024-05-14T16:52:56.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:56 smithi069 ceph-mon[31507]: pgmap v95: 97 pgs: 97 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 656 op/s 2024-05-14T16:52:56.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:56 smithi203 ceph-mon[35787]: pgmap v95: 97 pgs: 97 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 656 op/s 2024-05-14T16:52:56.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:56 smithi092 ceph-mon[35763]: pgmap v95: 97 pgs: 97 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 656 op/s 2024-05-14T16:52:58.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:58 smithi069 ceph-mon[31507]: pgmap v96: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 977 op/s 2024-05-14T16:52:58.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:58 smithi203 ceph-mon[35787]: pgmap v96: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 977 op/s 2024-05-14T16:52:58.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:58 smithi092 ceph-mon[35763]: pgmap v96: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 977 op/s 2024-05-14T16:52:59.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:52:59 smithi069 ceph-mon[31507]: pgmap v97: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 633 op/s 2024-05-14T16:52:59.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:52:59 smithi203 ceph-mon[35787]: pgmap v97: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 633 op/s 2024-05-14T16:52:59.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:52:59 smithi092 ceph-mon[35763]: pgmap v97: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 633 op/s 2024-05-14T16:53:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 150.0 s, 3542.2 tps, lat 28.218 ms stddev 29.712 2024-05-14T16:53:02.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:01 smithi092 ceph-mon[35763]: pgmap v98: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 960 op/s 2024-05-14T16:53:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.163+0000 7f61287c9640 1 -- 172.21.15.69:0/459347752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6120101ad0 msgr2=0x7f6120101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.163+0000 7f61287c9640 1 --2- 172.21.15.69:0/459347752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6120101ad0 0x7f6120101eb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6110009890 tx=0x7f611002a420 comp rx=0 tx=0).stop 2024-05-14T16:53:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.164+0000 7f61287c9640 1 -- 172.21.15.69:0/459347752 shutdown_connections 2024-05-14T16:53:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.164+0000 7f61287c9640 1 --2- 172.21.15.69:0/459347752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6120106ed0 0x7f6120107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.164+0000 7f61287c9640 1 --2- 172.21.15.69:0/459347752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61201023f0 0x7f61201068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.164+0000 7f61287c9640 1 --2- 172.21.15.69:0/459347752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6120101ad0 0x7f6120101eb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.164+0000 7f61287c9640 1 -- 172.21.15.69:0/459347752 >> 172.21.15.69:0/459347752 conn(0x7f6120078a70 msgr2=0x7f6120078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.167+0000 7f61287c9640 1 -- 172.21.15.69:0/459347752 shutdown_connections 2024-05-14T16:53:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.167+0000 7f61287c9640 1 -- 172.21.15.69:0/459347752 wait complete. 2024-05-14T16:53:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.168+0000 7f61287c9640 1 Processor -- start 2024-05-14T16:53:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.169+0000 7f61287c9640 1 -- start start 2024-05-14T16:53:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.169+0000 7f61287c9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6120101ad0 0x7f6120115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.169+0000 7f61287c9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61201023f0 0x7f6120111db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.169+0000 7f61287c9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6120106ed0 0x7f61201122f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.169+0000 7f61287c9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f612010a750 con 0x7f6120101ad0 2024-05-14T16:53:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.169+0000 7f61287c9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f612010a5d0 con 0x7f61201023f0 2024-05-14T16:53:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.169+0000 7f61287c9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f612010a8d0 con 0x7f6120106ed0 2024-05-14T16:53:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.169+0000 7f6125d64640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61201023f0 0x7f6120111db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.169+0000 7f6125d64640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61201023f0 0x7f6120111db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:47394/0 (socket says 172.21.15.69:47394) 2024-05-14T16:53:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.169+0000 7f6125d64640 1 -- 172.21.15.69:0/3518251612 learned_addr learned my addr 172.21.15.69:0/3518251612 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.170+0000 7f6126565640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6120101ad0 0x7f6120115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.170+0000 7f6126d66640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6120106ed0 0x7f61201122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.170+0000 7f6125d64640 1 -- 172.21.15.69:0/3518251612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6120106ed0 msgr2=0x7f61201122f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.170+0000 7f6125d64640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6120106ed0 0x7f61201122f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.170+0000 7f6125d64640 1 -- 172.21.15.69:0/3518251612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6120101ad0 msgr2=0x7f6120115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.170+0000 7f6125d64640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6120101ad0 0x7f6120115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.170+0000 7f6125d64640 1 -- 172.21.15.69:0/3518251612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61201128b0 con 0x7f61201023f0 2024-05-14T16:53:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.171+0000 7f6125d64640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61201023f0 0x7f6120111db0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f611c00b450 tx=0x7f611c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.171+0000 7f61177fe640 1 -- 172.21.15.69:0/3518251612 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c015020 con 0x7f61201023f0 2024-05-14T16:53:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.172+0000 7f61177fe640 1 -- 172.21.15.69:0/3518251612 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c011d40 con 0x7f61201023f0 2024-05-14T16:53:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.172+0000 7f61177fe640 1 -- 172.21.15.69:0/3518251612 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c00f340 con 0x7f61201023f0 2024-05-14T16:53:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.172+0000 7f61287c9640 1 -- 172.21.15.69:0/3518251612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6120112ba0 con 0x7f61201023f0 2024-05-14T16:53:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.173+0000 7f61287c9640 1 -- 172.21.15.69:0/3518251612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61201a3b30 con 0x7f61201023f0 2024-05-14T16:53:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.179+0000 7f612b5bd740 1 Processor -- start 2024-05-14T16:53:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.179+0000 7f61177fe640 1 -- 172.21.15.69:0/3518251612 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f611c018020 con 0x7f61201023f0 2024-05-14T16:53:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.180+0000 7f61177fe640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f610c0b2bd0 0x7f610c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.181+0000 7f61177fe640 1 -- 172.21.15.69:0/3518251612 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f611c0e1db0 con 0x7f61201023f0 2024-05-14T16:53:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.181+0000 7f612b5bd740 1 -- start start 2024-05-14T16:53:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.182+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 0x56082f89fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.182+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8a0210 0x56082f8aa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.182+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f8aaad0 0x56082f8acec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.182+0000 7f612b5bd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56082f812c30 con 0x56082f8aaad0 2024-05-14T16:53:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.182+0000 7f612b5bd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56082f8027d0 con 0x56082f89f8f0 2024-05-14T16:53:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.182+0000 7f612b5bd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56082f806810 con 0x56082f8a0210 2024-05-14T16:53:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.183+0000 7f6126d66640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f610c0b2bd0 0x7f610c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.183+0000 7f6126d66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f8aaad0 0x56082f8acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.183+0000 7f6125d64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8a0210 0x56082f8aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.183+0000 7f6125d64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8a0210 0x56082f8aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:59082/0 (socket says 172.21.15.69:59082) 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.183+0000 7f6125d64640 1 -- 172.21.15.69:0/1264426708 learned_addr learned my addr 172.21.15.69:0/1264426708 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.183+0000 7f6126565640 1 --2- 172.21.15.69:0/1264426708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 0x56082f89fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.184+0000 7f6126d66640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f610c0b2bd0 0x7f610c0b5090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6118005fd0 tx=0x7f611801aad0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.184+0000 7f6126d66640 1 -- 172.21.15.69:0/1264426708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8a0210 msgr2=0x56082f8aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.184+0000 7f6126d66640 1 --2- 172.21.15.69:0/1264426708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8a0210 0x56082f8aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.184+0000 7f6126d66640 1 -- 172.21.15.69:0/1264426708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 msgr2=0x56082f89fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.184+0000 7f6126d66640 1 --2- 172.21.15.69:0/1264426708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 0x56082f89fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.184+0000 7f6126d66640 1 -- 172.21.15.69:0/1264426708 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082f517210 con 0x56082f8aaad0 2024-05-14T16:53:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.184+0000 7f6126d66640 1 --2- 172.21.15.69:0/1264426708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f8aaad0 0x56082f8acec0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6118003bb0 tx=0x7f611802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e037e09d682e383 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.185+0000 7f6114ff9640 1 -- 172.21.15.69:0/1264426708 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611801f4c0 con 0x56082f8aaad0 2024-05-14T16:53:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.185+0000 7f6114ff9640 1 -- 172.21.15.69:0/1264426708 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6118036070 con 0x56082f8aaad0 2024-05-14T16:53:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.185+0000 7f6114ff9640 1 -- 172.21.15.69:0/1264426708 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118030820 con 0x56082f8aaad0 2024-05-14T16:53:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.186+0000 7f612b5bd740 1 -- 172.21.15.69:0/1264426708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f8aaad0 msgr2=0x56082f8acec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.187+0000 7f612b5bd740 1 --2- 172.21.15.69:0/1264426708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f8aaad0 0x56082f8acec0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6118003bb0 tx=0x7f611802e000 comp rx=0 tx=0).stop 2024-05-14T16:53:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.187+0000 7f612b5bd740 1 -- 172.21.15.69:0/1264426708 shutdown_connections 2024-05-14T16:53:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.187+0000 7f612b5bd740 1 --2- 172.21.15.69:0/1264426708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f8aaad0 0x56082f8acec0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.187+0000 7f612b5bd740 1 --2- 172.21.15.69:0/1264426708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8a0210 0x56082f8aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.187+0000 7f612b5bd740 1 --2- 172.21.15.69:0/1264426708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 0x56082f89fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.187+0000 7f612b5bd740 1 -- 172.21.15.69:0/1264426708 >> 172.21.15.69:0/1264426708 conn(0x56082f89be00 msgr2=0x56082f89c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.188+0000 7f612b5bd740 1 -- 172.21.15.69:0/1264426708 shutdown_connections 2024-05-14T16:53:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.188+0000 7f612b5bd740 1 -- 172.21.15.69:0/1264426708 wait complete. 2024-05-14T16:53:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.189+0000 7f612b5bd740 1 -- start start 2024-05-14T16:53:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.189+0000 7f612b5bd740 1 Processor -- start 2024-05-14T16:53:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.190+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 0x56082f8aa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.190+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f8a0210 0x56082f8a7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.190+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8aaad0 0x56082f8a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.190+0000 7f6126565640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 0x56082f8aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.190+0000 7f6126565640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 0x56082f8aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:47406/0 (socket says 172.21.15.69:47406) 2024-05-14T16:53:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.190+0000 7f6126565640 1 -- 172.21.15.69:0/3591843023 learned_addr learned my addr 172.21.15.69:0/3591843023 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.190+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56082f4e8790 con 0x56082f8a0210 2024-05-14T16:53:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.190+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56082f7be890 con 0x56082f89f8f0 2024-05-14T16:53:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.190+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56082f8ad740 con 0x56082f8aaad0 2024-05-14T16:53:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.191+0000 7f6126d66640 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8aaad0 0x56082f8a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.191+0000 7f6126565640 1 -- 172.21.15.69:0/3591843023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8aaad0 msgr2=0x56082f8a7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.191+0000 7f6126565640 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8aaad0 0x56082f8a7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.191+0000 7f6126565640 1 -- 172.21.15.69:0/3591843023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f8a0210 msgr2=0x56082f8a7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.191+0000 7f6126565640 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f8a0210 0x56082f8a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.191+0000 7f6126565640 1 -- 172.21.15.69:0/3591843023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082f5e3620 con 0x56082f89f8f0 2024-05-14T16:53:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.191+0000 7f612b5bd740 10 client.0 _ll_register_callbacks cb 0x56082f7c1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.191+0000 7f6126565640 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 0x56082f8aa380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6110009890 tx=0x7f6110030b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ad7f169832fbae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.191+0000 7f6109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:53:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.191+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6118036070 con 0x56082f89f8f0 2024-05-14T16:53:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.193+0000 7f6109ffb640 1 -- 172.21.15.69:0/3591843023 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110005030 con 0x56082f89f8f0 2024-05-14T16:53:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.193+0000 7f6109ffb640 1 -- 172.21.15.69:0/3591843023 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f61100051d0 con 0x56082f89f8f0 2024-05-14T16:53:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.193+0000 7f6109ffb640 1 -- 172.21.15.69:0/3591843023 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f61100053d0 con 0x56082f89f8f0 2024-05-14T16:53:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.196+0000 7f612b5bd740 10 client.25268 fetch_fsmap learned FSMap version 31 2024-05-14T16:53:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.196+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({fsmap=31}) -- 0x56082f8b8fa0 con 0x56082f89f8f0 2024-05-14T16:53:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.198+0000 7f6109ffb640 1 -- 172.21.15.69:0/3591843023 <== mon.2 v2:172.21.15.92:3300/0 4 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f61100046c0 con 0x56082f89f8f0 2024-05-14T16:53:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.198+0000 7f612b5bd740 10 client.25268 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:53:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.198+0000 7f612b5bd740 10 client.25268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:53:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.198+0000 7f612b5bd740 20 client.25268 populate_metadata read hostname 'smithi069' 2024-05-14T16:53:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.198+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56082f8ba9d0 0x56082f8bce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.198+0000 7f612b5bd740 4 client.25268 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:53:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.198+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56082f814b80 con 0x56082f8ba9d0 2024-05-14T16:53:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.200+0000 7f6125d64640 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56082f8ba9d0 0x56082f8bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.202+0000 7f6125d64640 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56082f8ba9d0 0x56082f8bce90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c939f098d18d8a server_cookie=47299dceddc9573 in_seq=0 out_seq=0 2024-05-14T16:53:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.203+0000 7f6109ffb640 10 client.25268 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:02.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:01 smithi069 ceph-mon[31507]: pgmap v98: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 960 op/s 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.203+0000 7f6109ffb640 1 -- 172.21.15.69:0/3591843023 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56082f814b80 con 0x56082f8ba9d0 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.203+0000 7f6109ffb640 10 client.25268 handle_command_reply: tid=0 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 client.25268 shutdown 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 msgr2=0x56082f8aa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 0x56082f8aa380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6110009890 tx=0x7f6110030b90 comp rx=0 tx=0).stop 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 shutdown_connections 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56082f8ba9d0 0x56082f8bce90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8aaad0 0x56082f8a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f8a0210 0x56082f8a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3591843023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89f8f0 0x56082f8aa380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 >> 172.21.15.69:0/3591843023 conn(0x56082f89be00 msgr2=0x56082f89b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f6109ffb640 0 client.25268 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 shutdown_connections 2024-05-14T16:53:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 1 -- 172.21.15.69:0/3591843023 wait complete. 2024-05-14T16:53:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.204+0000 7f612b5bd740 20 client.25268 trim_cache size 0 max 16384 2024-05-14T16:53:02.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:01 smithi203 ceph-mon[35787]: pgmap v98: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 960 op/s 2024-05-14T16:53:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.235+0000 7f612b5bd740 1 Processor -- start 2024-05-14T16:53:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.235+0000 7f612b5bd740 1 -- start start 2024-05-14T16:53:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89be00 0x56082f8b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f89f8f0 0x56082f8a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f8a0210 0x56082f8c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f612b5bd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6104011150 con 0x56082f89be00 2024-05-14T16:53:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f612b5bd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f610400e990 con 0x56082f8a0210 2024-05-14T16:53:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f612b5bd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f610400c7e0 con 0x56082f89f8f0 2024-05-14T16:53:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f6126565640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89be00 0x56082f8b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f6126565640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89be00 0x56082f8b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51688/0 (socket says 172.21.15.69:51688) 2024-05-14T16:53:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f6126565640 1 -- 172.21.15.69:0/1337999504 learned_addr learned my addr 172.21.15.69:0/1337999504 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f6125d64640 1 --2- 172.21.15.69:0/1337999504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f89f8f0 0x56082f8a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.236+0000 7f6126d66640 1 --2- 172.21.15.69:0/1337999504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f8a0210 0x56082f8c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.237+0000 7f6125d64640 1 -- 172.21.15.69:0/1337999504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f8a0210 msgr2=0x56082f8c27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.237+0000 7f6125d64640 1 --2- 172.21.15.69:0/1337999504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f8a0210 0x56082f8c27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.237+0000 7f6125d64640 1 -- 172.21.15.69:0/1337999504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89be00 msgr2=0x56082f8b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.237+0000 7f6125d64640 1 --2- 172.21.15.69:0/1337999504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89be00 0x56082f8b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.237+0000 7f6125d64640 1 -- 172.21.15.69:0/1337999504 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082f8ad230 con 0x56082f89f8f0 2024-05-14T16:53:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.237+0000 7f6125d64640 1 --2- 172.21.15.69:0/1337999504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f89f8f0 0x56082f8a9eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f611c0a8e50 tx=0x7f611c012f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ced2170d4b2ad server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.237+0000 7f610bfff640 1 -- 172.21.15.69:0/1337999504 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0b1fb0 con 0x56082f89f8f0 2024-05-14T16:53:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.238+0000 7f610bfff640 1 -- 172.21.15.69:0/1337999504 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c0a53e0 con 0x56082f89f8f0 2024-05-14T16:53:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.238+0000 7f610bfff640 1 -- 172.21.15.69:0/1337999504 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0a8610 con 0x56082f89f8f0 2024-05-14T16:53:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 -- 172.21.15.69:0/1337999504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f89f8f0 msgr2=0x56082f8a9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 --2- 172.21.15.69:0/1337999504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f89f8f0 0x56082f8a9eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f611c0a8e50 tx=0x7f611c012f80 comp rx=0 tx=0).stop 2024-05-14T16:53:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 -- 172.21.15.69:0/1337999504 shutdown_connections 2024-05-14T16:53:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 --2- 172.21.15.69:0/1337999504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f8a0210 0x56082f8c27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 --2- 172.21.15.69:0/1337999504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f89f8f0 0x56082f8a9eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 --2- 172.21.15.69:0/1337999504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89be00 0x56082f8b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 -- 172.21.15.69:0/1337999504 >> 172.21.15.69:0/1337999504 conn(0x56082f8a7d10 msgr2=0x56082f8a2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 -- 172.21.15.69:0/1337999504 shutdown_connections 2024-05-14T16:53:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 -- 172.21.15.69:0/1337999504 wait complete. 2024-05-14T16:53:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 -- start start 2024-05-14T16:53:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.239+0000 7f612b5bd740 1 Processor -- start 2024-05-14T16:53:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.240+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89be00 0x56082f8b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.240+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89f8f0 0x56082f8b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.240+0000 7f612b5bd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8a0210 0x56082f8ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.240+0000 7f612b5bd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f610400ce50 con 0x56082f89f8f0 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.240+0000 7f612b5bd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f610400e2c0 con 0x56082f89be00 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.240+0000 7f612b5bd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f610400d520 con 0x56082f8a0210 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.240+0000 7f612b5bd740 10 client.0 _ll_register_callbacks cb 0x7f6110030740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.240+0000 7f6125d64640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89f8f0 0x56082f8b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.240+0000 7f6125d64640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89f8f0 0x56082f8b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51702/0 (socket says 172.21.15.69:51702) 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.240+0000 7f6125d64640 1 -- 172.21.15.69:0/3035305148 learned_addr learned my addr 172.21.15.69:0/3035305148 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.241+0000 7f6125d64640 1 -- 172.21.15.69:0/3035305148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8a0210 msgr2=0x56082f8ae1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.241+0000 7f6125d64640 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8a0210 0x56082f8ae1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.241+0000 7f6125d64640 1 -- 172.21.15.69:0/3035305148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89be00 msgr2=0x56082f8b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.241+0000 7f6125d64640 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89be00 0x56082f8b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.241+0000 7f6125d64640 1 -- 172.21.15.69:0/3035305148 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082f8ae9a0 con 0x56082f89f8f0 2024-05-14T16:53:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.244+0000 7f6125d64640 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89f8f0 0x56082f8b30b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f611c0a50c0 tx=0x7f611c0ad400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a058be46c307c58b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.244+0000 7f6109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:53:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.247+0000 7f6109ffb640 1 -- 172.21.15.69:0/3035305148 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0bc070 con 0x56082f89f8f0 2024-05-14T16:53:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.247+0000 7f612b5bd740 1 -- 172.21.15.69:0/3035305148 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611c0a53e0 con 0x56082f89f8f0 2024-05-14T16:53:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.247+0000 7f6109ffb640 1 -- 172.21.15.69:0/3035305148 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f611c011aa0 con 0x56082f89f8f0 2024-05-14T16:53:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.247+0000 7f6109ffb640 1 -- 172.21.15.69:0/3035305148 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f611c0b1fb0 con 0x56082f89f8f0 2024-05-14T16:53:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.248+0000 7f612b5bd740 10 client.15738 fetch_fsmap learned FSMap version 31 2024-05-14T16:53:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.248+0000 7f612b5bd740 1 -- 172.21.15.69:0/3035305148 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56082f936000 con 0x56082f89f8f0 2024-05-14T16:53:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.248+0000 7f6109ffb640 1 -- 172.21.15.69:0/3035305148 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c0a53e0 con 0x56082f89f8f0 2024-05-14T16:53:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.248+0000 7f6109ffb640 1 -- 172.21.15.69:0/3035305148 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f611c0a4cb0 con 0x56082f89f8f0 2024-05-14T16:53:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.248+0000 7f612b5bd740 10 client.15738 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:53:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.248+0000 7f612b5bd740 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:53:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.248+0000 7f612b5bd740 20 client.15738 populate_metadata read hostname 'smithi069' 2024-05-14T16:53:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.248+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56082f8aaad0 0x56082f9387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.248+0000 7f612b5bd740 4 client.15738 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:53:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.248+0000 7f612b5bd740 1 -- 172.21.15.69:0/3035305148 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56082f80e360 con 0x56082f8aaad0 2024-05-14T16:53:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.249+0000 7f6126565640 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56082f8aaad0 0x56082f9387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.250+0000 7f6126565640 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56082f8aaad0 0x56082f9387a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4b4af2a1206d0e server_cookie=64dc3d915b2cae9d in_seq=0 out_seq=0 2024-05-14T16:53:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.251+0000 7f6109ffb640 10 client.15738 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.366+0000 7f6109ffb640 1 -- 172.21.15.69:0/3035305148 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56082f80e360 con 0x56082f8aaad0 2024-05-14T16:53:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.366+0000 7f6109ffb640 10 client.15738 handle_command_reply: tid=0 2024-05-14T16:53:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.366+0000 7f612b5bd740 1 client.15738 shutdown 2024-05-14T16:53:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 1 -- 172.21.15.69:0/3035305148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89f8f0 msgr2=0x56082f8b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89f8f0 0x56082f8b30b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f611c0a50c0 tx=0x7f611c0ad400 comp rx=0 tx=0).stop 2024-05-14T16:53:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 1 -- 172.21.15.69:0/3035305148 shutdown_connections 2024-05-14T16:53:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56082f8aaad0 0x56082f9387a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56082f8a0210 0x56082f8ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56082f89f8f0 0x56082f8b30b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 1 --2- 172.21.15.69:0/3035305148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082f89be00 0x56082f8b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 1 -- 172.21.15.69:0/3035305148 >> 172.21.15.69:0/3035305148 conn(0x56082f8a7d10 msgr2=0x56082f8b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f6109ffb640 0 client.15738 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 1 -- 172.21.15.69:0/3035305148 shutdown_connections 2024-05-14T16:53:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 1 -- 172.21.15.69:0/3035305148 wait complete. 2024-05-14T16:53:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.367+0000 7f612b5bd740 20 client.15738 trim_cache size 0 max 16384 2024-05-14T16:53:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:53:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:53:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:53:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.370+0000 7f61287c9640 1 -- 172.21.15.69:0/3518251612 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f610c0b2bd0 msgr2=0x7f610c0b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.370+0000 7f61287c9640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f610c0b2bd0 0x7f610c0b5090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6118005fd0 tx=0x7f611801aad0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.370+0000 7f61287c9640 1 -- 172.21.15.69:0/3518251612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61201023f0 msgr2=0x7f6120111db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.370+0000 7f61287c9640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61201023f0 0x7f6120111db0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f611c00b450 tx=0x7f611c00b910 comp rx=0 tx=0).stop 2024-05-14T16:53:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.371+0000 7f61287c9640 1 -- 172.21.15.69:0/3518251612 shutdown_connections 2024-05-14T16:53:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.371+0000 7f61287c9640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f610c0b2bd0 0x7f610c0b5090 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.371+0000 7f61287c9640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6120106ed0 0x7f61201122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.372+0000 7f61287c9640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61201023f0 0x7f6120111db0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.372+0000 7f61287c9640 1 --2- 172.21.15.69:0/3518251612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6120101ad0 0x7f6120115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.372+0000 7f61287c9640 1 -- 172.21.15.69:0/3518251612 >> 172.21.15.69:0/3518251612 conn(0x7f6120078a70 msgr2=0x7f6120104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.372+0000 7f61287c9640 1 -- 172.21.15.69:0/3518251612 shutdown_connections 2024-05-14T16:53:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:02.373+0000 7f61287c9640 1 -- 172.21.15.69:0/3518251612 wait complete. 2024-05-14T16:53:03.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:02 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:03.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:02 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:03.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:02 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:03 smithi092 ceph-mon[35763]: pgmap v99: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 648 op/s 2024-05-14T16:53:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:03 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:03 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:04.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:03 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:53:04.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:03 smithi069 ceph-mon[31507]: pgmap v99: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 648 op/s 2024-05-14T16:53:04.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:03 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:04.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:03 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:04.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:03 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:53:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:03 smithi203 ceph-mon[35787]: pgmap v99: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 648 op/s 2024-05-14T16:53:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:03 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:03 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:03 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:53:05.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 155.0 s, 1760.4 tps, lat 56.702 ms stddev 105.140 2024-05-14T16:53:05.999 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:05 smithi092 ceph-mon[35763]: pgmap v100: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 648 op/s 2024-05-14T16:53:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:05 smithi069 ceph-mon[31507]: pgmap v100: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 648 op/s 2024-05-14T16:53:06.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:05 smithi203 ceph-mon[35787]: pgmap v100: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 648 op/s 2024-05-14T16:53:08.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:07 smithi092 ceph-mon[35763]: pgmap v101: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 2.08k op/s 2024-05-14T16:53:08.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:07 smithi069 ceph-mon[31507]: pgmap v101: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 2.08k op/s 2024-05-14T16:53:08.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:07 smithi203 ceph-mon[35787]: pgmap v101: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 2.08k op/s 2024-05-14T16:53:10.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:09 smithi092 ceph-mon[35763]: pgmap v102: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 MiB/s wr, 1.76k op/s 2024-05-14T16:53:10.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:09 smithi069 ceph-mon[31507]: pgmap v102: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 MiB/s wr, 1.76k op/s 2024-05-14T16:53:10.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:09 smithi203 ceph-mon[35787]: pgmap v102: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 MiB/s wr, 1.76k op/s 2024-05-14T16:53:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 160.0 s, 3280.0 tps, lat 30.540 ms stddev 32.710 2024-05-14T16:53:12.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:11 smithi092 ceph-mon[35763]: pgmap v103: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 2.07k op/s 2024-05-14T16:53:12.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:11 smithi069 ceph-mon[31507]: pgmap v103: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 2.07k op/s 2024-05-14T16:53:12.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:11 smithi203 ceph-mon[35787]: pgmap v103: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 2.07k op/s 2024-05-14T16:53:14.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:13 smithi092 ceph-mon[35763]: pgmap v104: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2024-05-14T16:53:14.010 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:13 smithi203 ceph-mon[35787]: pgmap v104: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2024-05-14T16:53:14.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:13 smithi069 ceph-mon[31507]: pgmap v104: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2024-05-14T16:53:15.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:14 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:53:15.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:14 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:53:15.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:14 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:53:15.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 165.0 s, 3186.0 tps, lat 31.277 ms stddev 33.157 2024-05-14T16:53:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.499+0000 7fe5b871c640 1 -- 172.21.15.69:0/3201960602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b0107940 msgr2=0x7fe5b0107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.499+0000 7fe5b871c640 1 --2- 172.21.15.69:0/3201960602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b0107940 0x7fe5b0107d20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009900 tx=0x7fe5ac02a410 comp rx=0 tx=0).stop 2024-05-14T16:53:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.499+0000 7fe5b871c640 1 -- 172.21.15.69:0/3201960602 shutdown_connections 2024-05-14T16:53:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.499+0000 7fe5b871c640 1 --2- 172.21.15.69:0/3201960602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5b0102340 0x7fe5b010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.499+0000 7fe5b871c640 1 --2- 172.21.15.69:0/3201960602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0101980 0x7fe5b0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.499+0000 7fe5b871c640 1 --2- 172.21.15.69:0/3201960602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b0107940 0x7fe5b0107d20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.499+0000 7fe5b871c640 1 -- 172.21.15.69:0/3201960602 >> 172.21.15.69:0/3201960602 conn(0x7fe5b0078ad0 msgr2=0x7fe5b0078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.499+0000 7fe5b871c640 1 -- 172.21.15.69:0/3201960602 shutdown_connections 2024-05-14T16:53:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.499+0000 7fe5b871c640 1 -- 172.21.15.69:0/3201960602 wait complete. 2024-05-14T16:53:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.500+0000 7fe5b871c640 1 Processor -- start 2024-05-14T16:53:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.500+0000 7fe5b871c640 1 -- start start 2024-05-14T16:53:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.500+0000 7fe5b871c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b0101980 0x7fe5b019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.500+0000 7fe5b871c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5b0102340 0x7fe5b019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.500+0000 7fe5b871c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0107940 0x7fe5b019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b871c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0110ae0 con 0x7fe5b0107940 2024-05-14T16:53:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b6cb9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0107940 0x7fe5b019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b6cb9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0107940 0x7fe5b019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49682/0 (socket says 172.21.15.69:49682) 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/135713643 learned_addr learned my addr 172.21.15.69:0/135713643 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b64b8640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b0101980 0x7fe5b019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b871c640 1 -- 172.21.15.69:0/135713643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0110960 con 0x7fe5b0101980 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b871c640 1 -- 172.21.15.69:0/135713643 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0110c60 con 0x7fe5b0102340 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b5cb7640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5b0102340 0x7fe5b019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b64b8640 1 -- 172.21.15.69:0/135713643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5b0102340 msgr2=0x7fe5b019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b64b8640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5b0102340 0x7fe5b019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b64b8640 1 -- 172.21.15.69:0/135713643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0107940 msgr2=0x7fe5b019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b64b8640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0107940 0x7fe5b019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.501+0000 7fe5b64b8640 1 -- 172.21.15.69:0/135713643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b01a0370 con 0x7fe5b0101980 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.502+0000 7fe5b64b8640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b0101980 0x7fe5b019b380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac02a930 tx=0x7fe5ac003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.502+0000 7fe5a77fe640 1 -- 172.21.15.69:0/135713643 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac03f070 con 0x7fe5b0101980 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.502+0000 7fe5a77fe640 1 -- 172.21.15.69:0/135713643 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ac03bb10 con 0x7fe5b0101980 2024-05-14T16:53:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.502+0000 7fe5b871c640 1 -- 172.21.15.69:0/135713643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b01a0600 con 0x7fe5b0101980 2024-05-14T16:53:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.502+0000 7fe5a77fe640 1 -- 172.21.15.69:0/135713643 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac035030 con 0x7fe5b0101980 2024-05-14T16:53:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.502+0000 7fe5b871c640 1 -- 172.21.15.69:0/135713643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b01a7ef0 con 0x7fe5b0101980 2024-05-14T16:53:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.505+0000 7fe5bb431740 1 Processor -- start 2024-05-14T16:53:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.505+0000 7fe5bb431740 1 -- start start 2024-05-14T16:53:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.506+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 0x563247e31cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.506+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e32210 0x563247e3c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.506+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e3cad0 0x563247e3eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.506+0000 7fe5bb431740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563247da4c30 con 0x563247e32210 2024-05-14T16:53:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.506+0000 7fe5bb431740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563247d947d0 con 0x563247e3cad0 2024-05-14T16:53:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.506+0000 7fe5bb431740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563247d98810 con 0x563247e318f0 2024-05-14T16:53:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.506+0000 7fe5b6cb9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 0x563247e31cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.506+0000 7fe5b6cb9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 0x563247e31cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:39806/0 (socket says 172.21.15.69:39806) 2024-05-14T16:53:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.506+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/2110064671 learned_addr learned my addr 172.21.15.69:0/2110064671 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.507+0000 7fe5a77fe640 1 -- 172.21.15.69:0/135713643 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe5ac042070 con 0x7fe5b0101980 2024-05-14T16:53:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.507+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/2110064671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e3cad0 msgr2=0x563247e3eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:53:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.507+0000 7fe5b5cb7640 1 --2- 172.21.15.69:0/2110064671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e3cad0 0x563247e3eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.507+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/2110064671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e3cad0 0x563247e3eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.507+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/2110064671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e32210 msgr2=0x563247e3c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.507+0000 7fe5b64b8640 1 --2- 172.21.15.69:0/2110064671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e32210 0x563247e3c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.507+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/2110064671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e32210 0x563247e3c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.507+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/2110064671 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563247aa9210 con 0x563247e318f0 2024-05-14T16:53:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.507+0000 7fe5b64b8640 1 --2- 172.21.15.69:0/2110064671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e32210 0x563247e3c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.508+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/2110064671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 0x563247e31cd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009fb0 tx=0x7fe5a8011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5cf518bc4ab5037 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.508+0000 7fe5a77fe640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe59c0b2a90 0x7fe59c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.508+0000 7fe5b64b8640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe59c0b2a90 0x7fe59c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.508+0000 7fe5a4ff9640 1 -- 172.21.15.69:0/2110064671 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80127d0 con 0x563247e318f0 2024-05-14T16:53:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.508+0000 7fe5a4ff9640 1 -- 172.21.15.69:0/2110064671 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a8012e20 con 0x563247e318f0 2024-05-14T16:53:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.508+0000 7fe5a4ff9640 1 -- 172.21.15.69:0/2110064671 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a801ae40 con 0x563247e318f0 2024-05-14T16:53:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.508+0000 7fe5a77fe640 1 -- 172.21.15.69:0/135713643 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe5ac102b40 con 0x7fe5b0101980 2024-05-14T16:53:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.509+0000 7fe5bb431740 1 -- 172.21.15.69:0/2110064671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 msgr2=0x563247e31cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.509+0000 7fe5b64b8640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe59c0b2a90 0x7fe59c0b4f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac100350 tx=0x7fe5ac0c5eb0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.509+0000 7fe5bb431740 1 --2- 172.21.15.69:0/2110064671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 0x563247e31cd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009fb0 tx=0x7fe5a8011870 comp rx=0 tx=0).stop 2024-05-14T16:53:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.509+0000 7fe5bb431740 1 -- 172.21.15.69:0/2110064671 shutdown_connections 2024-05-14T16:53:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.509+0000 7fe5bb431740 1 --2- 172.21.15.69:0/2110064671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e3cad0 0x563247e3eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.509+0000 7fe5bb431740 1 --2- 172.21.15.69:0/2110064671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e32210 0x563247e3c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.509+0000 7fe5bb431740 1 --2- 172.21.15.69:0/2110064671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 0x563247e31cd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.509+0000 7fe5bb431740 1 -- 172.21.15.69:0/2110064671 >> 172.21.15.69:0/2110064671 conn(0x563247e2de00 msgr2=0x563247e2e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.509+0000 7fe5bb431740 1 -- 172.21.15.69:0/2110064671 shutdown_connections 2024-05-14T16:53:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.509+0000 7fe5bb431740 1 -- 172.21.15.69:0/2110064671 wait complete. 2024-05-14T16:53:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.510+0000 7fe5bb431740 1 -- start start 2024-05-14T16:53:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.510+0000 7fe5bb431740 1 Processor -- start 2024-05-14T16:53:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 0x563247e3c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e32210 0x563247e39790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247e39da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5bb431740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563247a7a790 con 0x563247e3cad0 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5bb431740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563247d50890 con 0x563247e32210 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5bb431740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563247e47fb0 con 0x563247e318f0 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5bb431740 10 client.0 _ll_register_callbacks cb 0x563247d53d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5b5cb7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e32210 0x563247e39790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5b6cb9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247e39da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5b5cb7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e32210 0x563247e39790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:51930/0 (socket says 172.21.15.69:51930) 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5b5cb7640 1 -- 172.21.15.69:0/679669607 learned_addr learned my addr 172.21.15.69:0/679669607 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 0x563247e3c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5b5cb7640 1 -- 172.21.15.69:0/679669607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 msgr2=0x563247e3c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5b5cb7640 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 0x563247e3c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5b5cb7640 1 -- 172.21.15.69:0/679669607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 msgr2=0x563247e39da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5b5cb7640 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247e39da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.511+0000 7fe5b5cb7640 1 -- 172.21.15.69:0/679669607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563247b75620 con 0x563247e32210 2024-05-14T16:53:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe5b5cb7640 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e32210 0x563247e39790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0013710 tx=0x7fe5a0013bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=157b4d6d4e43c1d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe599ffb640 1 -- 172.21.15.69:0/679669607 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0004460 con 0x563247e32210 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe5bb431740 1 -- 172.21.15.69:0/679669607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a8012e20 con 0x563247e32210 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe599ffb640 1 -- 172.21.15.69:0/679669607 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe5a00119c0 con 0x563247e32210 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe599ffb640 1 -- 172.21.15.69:0/679669607 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe5a0011bc0 con 0x563247e32210 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe5bb431740 10 client.25280 fetch_fsmap learned FSMap version 32 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe5bb431740 1 -- 172.21.15.69:0/679669607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563247e537f0 con 0x563247e32210 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe599ffb640 1 -- 172.21.15.69:0/679669607 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a8012e20 con 0x563247e32210 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe599ffb640 1 -- 172.21.15.69:0/679669607 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe5a001a070 con 0x563247e32210 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe5bb431740 10 client.25280 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.512+0000 7fe5bb431740 10 client.25280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.513+0000 7fe5bb431740 20 client.25280 populate_metadata read hostname 'smithi069' 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.513+0000 7fe5bb431740 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563247e54d70 0x563247e57160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.513+0000 7fe5bb431740 4 client.25280 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.513+0000 7fe5bb431740 1 -- 172.21.15.69:0/679669607 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563247da6b80 con 0x563247e54d70 2024-05-14T16:53:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.513+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563247e54d70 0x563247e57160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.513+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563247e54d70 0x563247e57160 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781fb78a7418b0ff server_cookie=58b5b546a04b1d1e in_seq=0 out_seq=0 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.513+0000 7fe599ffb640 10 client.25280 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe599ffb640 1 -- 172.21.15.69:0/679669607 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563247da6b80 con 0x563247e54d70 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe599ffb640 10 client.25280 handle_command_reply: tid=0 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe5bb431740 1 client.25280 shutdown 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe5bb431740 1 -- 172.21.15.69:0/679669607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e32210 msgr2=0x563247e39790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe5bb431740 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e32210 0x563247e39790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0013710 tx=0x7fe5a0013bd0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe5bb431740 1 -- 172.21.15.69:0/679669607 shutdown_connections 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe5bb431740 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563247e54d70 0x563247e57160 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe5bb431740 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247e39da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe5bb431740 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e32210 0x563247e39790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe5bb431740 1 --2- 172.21.15.69:0/679669607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e318f0 0x563247e3c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.514+0000 7fe5bb431740 1 -- 172.21.15.69:0/679669607 >> 172.21.15.69:0/679669607 conn(0x563247e2de00 msgr2=0x563247e2d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.515+0000 7fe599ffb640 0 client.25280 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.515+0000 7fe5bb431740 1 -- 172.21.15.69:0/679669607 shutdown_connections 2024-05-14T16:53:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.515+0000 7fe5bb431740 1 -- 172.21.15.69:0/679669607 wait complete. 2024-05-14T16:53:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.515+0000 7fe5bb431740 20 client.25280 trim_cache size 0 max 16384 2024-05-14T16:53:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.536+0000 7fe5bb431740 1 Processor -- start 2024-05-14T16:53:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.536+0000 7fe5bb431740 1 -- start start 2024-05-14T16:53:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.536+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e50e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e32210 0x563247e34c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247e3bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5bb431740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01aed40 con 0x563247e3cad0 2024-05-14T16:53:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5bb431740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01ac5b0 con 0x563247e318f0 2024-05-14T16:53:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5bb431740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01aa440 con 0x563247e32210 2024-05-14T16:53:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b6cb9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e50e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b6cb9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e50e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:51940/0 (socket says 172.21.15.69:51940) 2024-05-14T16:53:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/1728578525 learned_addr learned my addr 172.21.15.69:0/1728578525 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/1728578525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247e3bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b5cb7640 1 --2- 172.21.15.69:0/1728578525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e32210 0x563247e34c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/1728578525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e32210 msgr2=0x563247e34c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/1728578525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e32210 0x563247e34c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/1728578525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 msgr2=0x563247e3bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/1728578525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247e3bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/1728578525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563247e32750 con 0x563247e318f0 2024-05-14T16:53:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.537+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/1728578525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e50e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x563247e32630 tx=0x7fe5a8009270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed7074c42e45d258 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.538+0000 7fe59bfff640 1 -- 172.21.15.69:0/1728578525 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a802d070 con 0x563247e318f0 2024-05-14T16:53:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.538+0000 7fe59bfff640 1 -- 172.21.15.69:0/1728578525 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a801a3f0 con 0x563247e318f0 2024-05-14T16:53:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.538+0000 7fe59bfff640 1 -- 172.21.15.69:0/1728578525 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8005420 con 0x563247e318f0 2024-05-14T16:53:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.539+0000 7fe5bb431740 1 -- 172.21.15.69:0/1728578525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 msgr2=0x563247e50e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.539+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1728578525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e50e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x563247e32630 tx=0x7fe5a8009270 comp rx=0 tx=0).stop 2024-05-14T16:53:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.539+0000 7fe5bb431740 1 -- 172.21.15.69:0/1728578525 shutdown_connections 2024-05-14T16:53:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.539+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1728578525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247e3bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.539+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1728578525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e32210 0x563247e34c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.539+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1728578525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e50e80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.539+0000 7fe5bb431740 1 -- 172.21.15.69:0/1728578525 >> 172.21.15.69:0/1728578525 conn(0x563247e2de00 msgr2=0x563247e34530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.539+0000 7fe5bb431740 1 -- 172.21.15.69:0/1728578525 shutdown_connections 2024-05-14T16:53:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.539+0000 7fe5bb431740 1 -- 172.21.15.69:0/1728578525 wait complete. 2024-05-14T16:53:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.540+0000 7fe5bb431740 1 -- start start 2024-05-14T16:53:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.540+0000 7fe5bb431740 1 Processor -- start 2024-05-14T16:53:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.540+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.540+0000 7fe5bb431740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e32210 0x563247e48d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.541+0000 7fe5b6cb9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.541+0000 7fe5b6cb9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:51944/0 (socket says 172.21.15.69:51944) 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.541+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/1752240533 learned_addr learned my addr 172.21.15.69:0/1752240533 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.541+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247eb7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.541+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01aaa90 con 0x563247e3cad0 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.541+0000 7fe5b5cb7640 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e32210 0x563247e48d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.541+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01abee0 con 0x563247e318f0 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.541+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01ab140 con 0x563247e32210 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.541+0000 7fe5bb431740 10 client.0 _ll_register_callbacks cb 0x7fe5a0009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.541+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247eb7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.542+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/1752240533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e32210 msgr2=0x563247e48d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.542+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e32210 0x563247e48d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.542+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/1752240533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 msgr2=0x563247eb7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.542+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247eb7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.542+0000 7fe5b6cb9640 1 -- 172.21.15.69:0/1752240533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563247eb7bd0 con 0x563247e318f0 2024-05-14T16:53:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.542+0000 7fe5b6cb9640 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e487c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8002230 tx=0x7fe5a8005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=776eb8daae9b430f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.542+0000 7fe599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:53:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.543+0000 7fe599ffb640 1 -- 172.21.15.69:0/1752240533 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a802d070 con 0x563247e318f0 2024-05-14T16:53:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.543+0000 7fe599ffb640 1 -- 172.21.15.69:0/1752240533 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe5a80052c0 con 0x563247e318f0 2024-05-14T16:53:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.543+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a801a3f0 con 0x563247e318f0 2024-05-14T16:53:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.543+0000 7fe599ffb640 1 -- 172.21.15.69:0/1752240533 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe5a80054c0 con 0x563247e318f0 2024-05-14T16:53:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.543+0000 7fe5bb431740 10 client.25286 fetch_fsmap learned FSMap version 32 2024-05-14T16:53:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.543+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563247ebef70 con 0x563247e318f0 2024-05-14T16:53:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.544+0000 7fe599ffb640 1 -- 172.21.15.69:0/1752240533 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a8005660 con 0x563247e318f0 2024-05-14T16:53:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.544+0000 7fe599ffb640 1 -- 172.21.15.69:0/1752240533 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe5a8045070 con 0x563247e318f0 2024-05-14T16:53:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.544+0000 7fe5bb431740 10 client.25286 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T16:53:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.544+0000 7fe5bb431740 10 client.25286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:53:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.544+0000 7fe5bb431740 20 client.25286 populate_metadata read hostname 'smithi069' 2024-05-14T16:53:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.544+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563247ebf350 0x563247ec1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.544+0000 7fe5bb431740 4 client.25286 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:53:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.544+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe5a801a3f0 con 0x563247ebf350 2024-05-14T16:53:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.544+0000 7fe5b5cb7640 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563247ebf350 0x563247ec1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.545+0000 7fe5b5cb7640 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563247ebf350 0x563247ec1770 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58c495a2418f904 server_cookie=420a4a69db0ec955 in_seq=0 out_seq=0 2024-05-14T16:53:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.545+0000 7fe599ffb640 10 client.25286 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.585+0000 7fe599ffb640 1 -- 172.21.15.69:0/1752240533 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fe5a801a3f0 con 0x563247ebf350 2024-05-14T16:53:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.585+0000 7fe599ffb640 10 client.25286 handle_command_reply: tid=0 2024-05-14T16:53:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.585+0000 7fe5bb431740 1 client.25286 shutdown 2024-05-14T16:53:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 msgr2=0x563247e487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e487c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8002230 tx=0x7fe5a8005020 comp rx=0 tx=0).stop 2024-05-14T16:53:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 shutdown_connections 2024-05-14T16:53:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563247ebf350 0x563247ec1770 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563247e3cad0 0x563247eb7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563247e32210 0x563247e48d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe599ffb640 0 client.25286 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 1 --2- 172.21.15.69:0/1752240533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563247e318f0 0x563247e487c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 >> 172.21.15.69:0/1752240533 conn(0x563247e2de00 msgr2=0x563247e33b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 shutdown_connections 2024-05-14T16:53:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 1 -- 172.21.15.69:0/1752240533 wait complete. 2024-05-14T16:53:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.587+0000 7fe5bb431740 20 client.25286 trim_cache size 0 max 16384 2024-05-14T16:53:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:53:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:53:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:53:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 -- 172.21.15.69:0/135713643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe59c0b2a90 msgr2=0x7fe59c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe59c0b2a90 0x7fe59c0b4f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac100350 tx=0x7fe5ac0c5eb0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 -- 172.21.15.69:0/135713643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b0101980 msgr2=0x7fe5b019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b0101980 0x7fe5b019b380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac02a930 tx=0x7fe5ac003b90 comp rx=0 tx=0).stop 2024-05-14T16:53:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 -- 172.21.15.69:0/135713643 shutdown_connections 2024-05-14T16:53:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe59c0b2a90 0x7fe59c0b4f50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0107940 0x7fe5b019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5b0102340 0x7fe5b019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 --2- 172.21.15.69:0/135713643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b0101980 0x7fe5b019b380 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 -- 172.21.15.69:0/135713643 >> 172.21.15.69:0/135713643 conn(0x7fe5b0078ad0 msgr2=0x7fe5b0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 -- 172.21.15.69:0/135713643 shutdown_connections 2024-05-14T16:53:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:15.588+0000 7fe5b871c640 1 -- 172.21.15.69:0/135713643 wait complete. 2024-05-14T16:53:16.150 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:15 smithi092 ceph-mon[35763]: pgmap v105: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2024-05-14T16:53:16.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:15 smithi069 ceph-mon[31507]: pgmap v105: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2024-05-14T16:53:16.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:15 smithi203 ceph-mon[35787]: pgmap v105: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2024-05-14T16:53:18.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:17 smithi069 ceph-mon[31507]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 2.19k op/s 2024-05-14T16:53:18.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:17 smithi203 ceph-mon[35787]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 2.19k op/s 2024-05-14T16:53:18.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:17 smithi092 ceph-mon[35763]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 2.19k op/s 2024-05-14T16:53:20.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:19 smithi069 ceph-mon[31507]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 760 op/s 2024-05-14T16:53:20.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:19 smithi203 ceph-mon[35787]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 760 op/s 2024-05-14T16:53:20.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:19 smithi092 ceph-mon[35763]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 760 op/s 2024-05-14T16:53:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 170.0 s, 3662.2 tps, lat 27.410 ms stddev 27.119 2024-05-14T16:53:22.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:21 smithi069 ceph-mon[31507]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-05-14T16:53:22.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:21 smithi203 ceph-mon[35787]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-05-14T16:53:22.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:21 smithi092 ceph-mon[35763]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-05-14T16:53:24.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:23 smithi069 ceph-mon[31507]: pgmap v109: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-05-14T16:53:24.208 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:23 smithi203 ceph-mon[35787]: pgmap v109: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-05-14T16:53:24.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:23 smithi092 ceph-mon[35763]: pgmap v109: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-05-14T16:53:25.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 175.0 s, 3247.0 tps, lat 30.759 ms stddev 31.207 2024-05-14T16:53:26.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:25 smithi069 ceph-mon[31507]: pgmap v110: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-05-14T16:53:26.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:25 smithi203 ceph-mon[35787]: pgmap v110: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-05-14T16:53:26.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:25 smithi092 ceph-mon[35763]: pgmap v110: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-05-14T16:53:28.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:27 smithi069 ceph-mon[31507]: pgmap v111: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-05-14T16:53:28.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:27 smithi203 ceph-mon[35787]: pgmap v111: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-05-14T16:53:28.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:27 smithi092 ceph-mon[35763]: pgmap v111: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-05-14T16:53:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.727+0000 7f3579358640 1 -- 172.21.15.69:0/1339241094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3574103d20 msgr2=0x7f3574104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.727+0000 7f3579358640 1 --2- 172.21.15.69:0/1339241094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3574103d20 0x7f3574104100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3564009900 tx=0x7f356402a480 comp rx=0 tx=0).stop 2024-05-14T16:53:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.727+0000 7f3579358640 1 -- 172.21.15.69:0/1339241094 shutdown_connections 2024-05-14T16:53:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.727+0000 7f3579358640 1 --2- 172.21.15.69:0/1339241094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f357410d060 0x7f357410f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.727+0000 7f3579358640 1 --2- 172.21.15.69:0/1339241094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3574104640 0x7f357410cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.727+0000 7f3579358640 1 --2- 172.21.15.69:0/1339241094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3574103d20 0x7f3574104100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.727+0000 7f3579358640 1 -- 172.21.15.69:0/1339241094 >> 172.21.15.69:0/1339241094 conn(0x7f3574078ad0 msgr2=0x7f3574078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.727+0000 7f3579358640 1 -- 172.21.15.69:0/1339241094 shutdown_connections 2024-05-14T16:53:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.727+0000 7f3579358640 1 -- 172.21.15.69:0/1339241094 wait complete. 2024-05-14T16:53:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.728+0000 7f3579358640 1 Processor -- start 2024-05-14T16:53:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.728+0000 7f3579358640 1 -- start start 2024-05-14T16:53:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.728+0000 7f3579358640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3574103d20 0x7f357419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.728+0000 7f3579358640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3574104640 0x7f357419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f3579358640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f357410d060 0x7f357419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f3572ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3574103d20 0x7f357419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f3572ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3574103d20 0x7f357419b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:57994/0 (socket says 172.21.15.69:57994) 2024-05-14T16:53:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f3572ffd640 1 -- 172.21.15.69:0/411907353 learned_addr learned my addr 172.21.15.69:0/411907353 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3574112cc0 con 0x7f357410d060 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3574112b40 con 0x7f3574104640 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3574112e40 con 0x7f3574103d20 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f35737fe640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f357410d060 0x7f357419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f35727fc640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3574104640 0x7f357419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f35727fc640 1 -- 172.21.15.69:0/411907353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3574103d20 msgr2=0x7f357419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f35727fc640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3574103d20 0x7f357419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f35727fc640 1 -- 172.21.15.69:0/411907353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f357410d060 msgr2=0x7f357419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f35727fc640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f357410d060 0x7f357419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f35727fc640 1 -- 172.21.15.69:0/411907353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35741a0450 con 0x7f3574104640 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.729+0000 7f3572ffd640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3574103d20 0x7f357419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.730+0000 7f35727fc640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3574104640 0x7f357419b9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f356800e9f0 tx=0x7f356800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.730+0000 7f355bfff640 1 -- 172.21.15.69:0/411907353 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356800ccc0 con 0x7f3574104640 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.730+0000 7f355bfff640 1 -- 172.21.15.69:0/411907353 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35680108e0 con 0x7f3574104640 2024-05-14T16:53:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.730+0000 7f355bfff640 1 -- 172.21.15.69:0/411907353 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568018840 con 0x7f3574104640 2024-05-14T16:53:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.730+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35741a0740 con 0x7f3574104640 2024-05-14T16:53:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.730+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35741a7fd0 con 0x7f3574104640 2024-05-14T16:53:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.734+0000 7f355bfff640 1 -- 172.21.15.69:0/411907353 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3568010430 con 0x7f3574104640 2024-05-14T16:53:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.734+0000 7f357c075740 1 Processor -- start 2024-05-14T16:53:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.734+0000 7f357c075740 1 -- start start 2024-05-14T16:53:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.734+0000 7f357c075740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a7c8f0 0x564477a7ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.734+0000 7f357c075740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 0x564477a87590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.734+0000 7f357c075740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a87ad0 0x564477a89ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.734+0000 7f357c075740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5644779efc30 con 0x564477a7d210 2024-05-14T16:53:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f357c075740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5644779df7d0 con 0x564477a7c8f0 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f357c075740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5644779e3810 con 0x564477a87ad0 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f3572ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 0x564477a87590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f3572ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 0x564477a87590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42130/0 (socket says 172.21.15.69:42130) 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f3572ffd640 1 -- 172.21.15.69:0/481231257 learned_addr learned my addr 172.21.15.69:0/481231257 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35727fc640 1 --2- 172.21.15.69:0/481231257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a87ad0 0x564477a89ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35737fe640 1 --2- 172.21.15.69:0/481231257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a7c8f0 0x564477a7ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35737fe640 1 -- 172.21.15.69:0/481231257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a87ad0 msgr2=0x564477a89ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35737fe640 1 --2- 172.21.15.69:0/481231257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a87ad0 0x564477a89ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35737fe640 1 -- 172.21.15.69:0/481231257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 msgr2=0x564477a87590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f355bfff640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f355c0b2a90 0x7f355c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35737fe640 1 --2- 172.21.15.69:0/481231257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 0x564477a87590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35737fe640 1 -- 172.21.15.69:0/481231257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644776f4210 con 0x564477a7c8f0 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f3572ffd640 1 --2- 172.21.15.69:0/481231257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 0x564477a87590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f355bfff640 1 -- 172.21.15.69:0/411907353 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f35680e2800 con 0x7f3574104640 2024-05-14T16:53:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35737fe640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f355c0b2a90 0x7f355c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35737fe640 1 --2- 172.21.15.69:0/481231257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a7c8f0 0x564477a7ccd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f356c009f10 tx=0x7f356c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a327efb6993862cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35597fa640 1 -- 172.21.15.69:0/481231257 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c004050 con 0x564477a7c8f0 2024-05-14T16:53:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35597fa640 1 -- 172.21.15.69:0/481231257 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356c003e30 con 0x564477a7c8f0 2024-05-14T16:53:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.735+0000 7f35597fa640 1 -- 172.21.15.69:0/481231257 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c019e10 con 0x564477a7c8f0 2024-05-14T16:53:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.736+0000 7f357c075740 1 -- 172.21.15.69:0/481231257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a7c8f0 msgr2=0x564477a7ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.736+0000 7f35737fe640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f355c0b2a90 0x7f355c0b4f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f356c01d000 tx=0x7f356c00e430 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.736+0000 7f357c075740 1 --2- 172.21.15.69:0/481231257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a7c8f0 0x564477a7ccd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f356c009f10 tx=0x7f356c011810 comp rx=0 tx=0).stop 2024-05-14T16:53:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.736+0000 7f357c075740 1 -- 172.21.15.69:0/481231257 shutdown_connections 2024-05-14T16:53:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.736+0000 7f357c075740 1 --2- 172.21.15.69:0/481231257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a87ad0 0x564477a89ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.736+0000 7f357c075740 1 --2- 172.21.15.69:0/481231257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 0x564477a87590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.736+0000 7f357c075740 1 --2- 172.21.15.69:0/481231257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a7c8f0 0x564477a7ccd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.736+0000 7f357c075740 1 -- 172.21.15.69:0/481231257 >> 172.21.15.69:0/481231257 conn(0x564477a78e00 msgr2=0x564477a791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.736+0000 7f357c075740 1 -- 172.21.15.69:0/481231257 shutdown_connections 2024-05-14T16:53:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.736+0000 7f357c075740 1 -- 172.21.15.69:0/481231257 wait complete. 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.737+0000 7f357c075740 1 -- start start 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.737+0000 7f357c075740 1 Processor -- start 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.737+0000 7f357c075740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7c8f0 0x564477a87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.737+0000 7f357c075740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 0x564477a84640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f357c075740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a87ad0 0x564477a84b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f357c075740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5644776c5790 con 0x564477a7d210 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f3572ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7c8f0 0x564477a87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f35727fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 0x564477a84640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f3572ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7c8f0 0x564477a87350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:58020/0 (socket says 172.21.15.69:58020) 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f3572ffd640 1 -- 172.21.15.69:0/896212212 learned_addr learned my addr 172.21.15.69:0/896212212 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f357c075740 1 -- 172.21.15.69:0/896212212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56447799b890 con 0x564477a87ad0 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f357c075740 1 -- 172.21.15.69:0/896212212 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564477a8a7a0 con 0x564477a7c8f0 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f357c075740 10 client.0 _ll_register_callbacks cb 0x56447799ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f35737fe640 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a87ad0 0x564477a84b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f3572ffd640 1 -- 172.21.15.69:0/896212212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a87ad0 msgr2=0x564477a84b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f3572ffd640 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a87ad0 0x564477a84b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f3572ffd640 1 -- 172.21.15.69:0/896212212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 msgr2=0x564477a84640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f3572ffd640 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 0x564477a84640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f3572ffd640 1 -- 172.21.15.69:0/896212212 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644777c0620 con 0x564477a7c8f0 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f357c075740 1 -- 172.21.15.69:0/896212212 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f356c003e30 con 0x564477a7c8f0 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f3572ffd640 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7c8f0 0x564477a87350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3564005c40 tx=0x7f35640316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3152912b0227ad52 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.738+0000 7f3556ff5640 10 client.25246 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.739+0000 7f3556ff5640 1 -- 172.21.15.69:0/896212212 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564035040 con 0x564477a7c8f0 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.739+0000 7f3556ff5640 1 -- 172.21.15.69:0/896212212 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f3564004250 con 0x564477a7c8f0 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.739+0000 7f3556ff5640 1 -- 172.21.15.69:0/896212212 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3564004450 con 0x564477a7c8f0 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.739+0000 7f357c075740 10 client.25246 fetch_fsmap learned FSMap version 32 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.739+0000 7f357c075740 1 -- 172.21.15.69:0/896212212 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({fsmap=32}) -- 0x564477a9e840 con 0x564477a7c8f0 2024-05-14T16:53:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.740+0000 7f3556ff5640 1 -- 172.21.15.69:0/896212212 <== mon.1 v2:172.21.15.203:3300/0 4 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f3564005040 con 0x564477a7c8f0 2024-05-14T16:53:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.740+0000 7f357c075740 10 client.25246 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T16:53:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.740+0000 7f357c075740 10 client.25246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:53:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.740+0000 7f357c075740 20 client.25246 populate_metadata read hostname 'smithi069' 2024-05-14T16:53:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.740+0000 7f357c075740 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564477a9fcb0 0x564477aa20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.740+0000 7f357c075740 4 client.25246 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:53:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.740+0000 7f357c075740 1 -- 172.21.15.69:0/896212212 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644779f1b80 con 0x564477a9fcb0 2024-05-14T16:53:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.740+0000 7f35727fc640 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564477a9fcb0 0x564477aa20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.740+0000 7f3556ff5640 10 client.25246 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.740+0000 7f35727fc640 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564477a9fcb0 0x564477aa20c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d083bcd5b18ec7 server_cookie=d42f4138040bec53 in_seq=0 out_seq=0 2024-05-14T16:53:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.741+0000 7f3556ff5640 1 -- 172.21.15.69:0/896212212 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644779f1b80 con 0x564477a9fcb0 2024-05-14T16:53:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.741+0000 7f3556ff5640 10 client.25246 handle_command_reply: tid=0 2024-05-14T16:53:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.741+0000 7f357c075740 1 client.25246 shutdown 2024-05-14T16:53:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.741+0000 7f357c075740 1 -- 172.21.15.69:0/896212212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7c8f0 msgr2=0x564477a87350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.741+0000 7f357c075740 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7c8f0 0x564477a87350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3564005c40 tx=0x7f35640316d0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.741+0000 7f357c075740 1 -- 172.21.15.69:0/896212212 shutdown_connections 2024-05-14T16:53:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.741+0000 7f357c075740 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564477a9fcb0 0x564477aa20c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.741+0000 7f357c075740 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a87ad0 0x564477a84b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.741+0000 7f357c075740 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a7d210 0x564477a84640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.741+0000 7f357c075740 1 --2- 172.21.15.69:0/896212212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7c8f0 0x564477a87350 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3564005c40 tx=0x7f35640316d0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.742+0000 7f357c075740 1 -- 172.21.15.69:0/896212212 >> 172.21.15.69:0/896212212 conn(0x564477a78e00 msgr2=0x564477a78100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.742+0000 7f3556ff5640 0 client.25246 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.742+0000 7f357c075740 1 -- 172.21.15.69:0/896212212 shutdown_connections 2024-05-14T16:53:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.743+0000 7f357c075740 1 -- 172.21.15.69:0/896212212 wait complete. 2024-05-14T16:53:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.743+0000 7f357c075740 20 client.25246 trim_cache size 0 max 16384 2024-05-14T16:53:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f357c075740 1 Processor -- start 2024-05-14T16:53:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f357c075740 1 -- start start 2024-05-14T16:53:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f357c075740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a78e00 0x564477a9bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f357c075740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a7d210 0x564477a82f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f357c075740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 0x564477a86e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f357c075740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35741aed80 con 0x564477a87ad0 2024-05-14T16:53:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f357c075740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f35741ac5f0 con 0x564477a7d210 2024-05-14T16:53:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f357c075740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f35741aa440 con 0x564477a78e00 2024-05-14T16:53:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f3572ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a78e00 0x564477a9bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f3572ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a78e00 0x564477a9bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:58032/0 (socket says 172.21.15.69:58032) 2024-05-14T16:53:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.767+0000 7f3572ffd640 1 -- 172.21.15.69:0/865297462 learned_addr learned my addr 172.21.15.69:0/865297462 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.768+0000 7f3572ffd640 1 -- 172.21.15.69:0/865297462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a7d210 msgr2=0x564477a82f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:53:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.768+0000 7f3572ffd640 1 --2- 172.21.15.69:0/865297462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a7d210 0x564477a82f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.768+0000 7f3572ffd640 1 -- 172.21.15.69:0/865297462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 msgr2=0x564477a86e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.768+0000 7f3572ffd640 1 --2- 172.21.15.69:0/865297462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 0x564477a86e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.768+0000 7f3572ffd640 1 -- 172.21.15.69:0/865297462 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564477a835f0 con 0x564477a78e00 2024-05-14T16:53:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.768+0000 7f3572ffd640 1 --2- 172.21.15.69:0/865297462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a78e00 0x564477a9bf00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3564009230 tx=0x7f356403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99f596492ce77737 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.768+0000 7f3558ff9640 1 -- 172.21.15.69:0/865297462 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564035030 con 0x564477a78e00 2024-05-14T16:53:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.768+0000 7f3558ff9640 1 -- 172.21.15.69:0/865297462 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3564066070 con 0x564477a78e00 2024-05-14T16:53:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.768+0000 7f3558ff9640 1 -- 172.21.15.69:0/865297462 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356403d6b0 con 0x564477a78e00 2024-05-14T16:53:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.769+0000 7f357c075740 1 -- 172.21.15.69:0/865297462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a78e00 msgr2=0x564477a9bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.770+0000 7f357c075740 1 --2- 172.21.15.69:0/865297462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a78e00 0x564477a9bf00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3564009230 tx=0x7f356403f000 comp rx=0 tx=0).stop 2024-05-14T16:53:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.770+0000 7f357c075740 1 -- 172.21.15.69:0/865297462 shutdown_connections 2024-05-14T16:53:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.770+0000 7f357c075740 1 --2- 172.21.15.69:0/865297462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 0x564477a86e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.770+0000 7f357c075740 1 --2- 172.21.15.69:0/865297462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a7d210 0x564477a82f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.770+0000 7f357c075740 1 --2- 172.21.15.69:0/865297462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a78e00 0x564477a9bf00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.770+0000 7f357c075740 1 -- 172.21.15.69:0/865297462 >> 172.21.15.69:0/865297462 conn(0x564477a84ce0 msgr2=0x564477a77870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.770+0000 7f357c075740 1 -- 172.21.15.69:0/865297462 shutdown_connections 2024-05-14T16:53:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.770+0000 7f357c075740 1 -- 172.21.15.69:0/865297462 wait complete. 2024-05-14T16:53:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f357c075740 1 -- start start 2024-05-14T16:53:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f357c075740 1 Processor -- start 2024-05-14T16:53:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f357c075740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a78e00 0x564477a905d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f357c075740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7d210 0x564477a90b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f357c075740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 0x564477a98300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f357c075740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35741aaab0 con 0x564477a87ad0 2024-05-14T16:53:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f357c075740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f35741abf20 con 0x564477a78e00 2024-05-14T16:53:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f357c075740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f35741ab180 con 0x564477a7d210 2024-05-14T16:53:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f357c075740 10 client.0 _ll_register_callbacks cb 0x7f3564009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f35737fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 0x564477a98300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f3572ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a78e00 0x564477a905d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f35737fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 0x564477a98300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42158/0 (socket says 172.21.15.69:42158) 2024-05-14T16:53:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f35737fe640 1 -- 172.21.15.69:0/1602420808 learned_addr learned my addr 172.21.15.69:0/1602420808 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.771+0000 7f35727fc640 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7d210 0x564477a90b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.772+0000 7f35737fe640 1 -- 172.21.15.69:0/1602420808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7d210 msgr2=0x564477a90b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.772+0000 7f35737fe640 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7d210 0x564477a90b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.772+0000 7f35737fe640 1 -- 172.21.15.69:0/1602420808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a78e00 msgr2=0x564477a905d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.772+0000 7f35737fe640 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a78e00 0x564477a905d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.772+0000 7f35737fe640 1 -- 172.21.15.69:0/1602420808 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564477a98b30 con 0x564477a87ad0 2024-05-14T16:53:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.772+0000 7f3572ffd640 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a78e00 0x564477a905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.772+0000 7f35737fe640 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 0x564477a98300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f356c009ee0 tx=0x7f356c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c534a6d6f2b4383 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.772+0000 7f3556ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:53:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.773+0000 7f3556ff5640 1 -- 172.21.15.69:0/1602420808 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c019ac0 con 0x564477a87ad0 2024-05-14T16:53:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.773+0000 7f3556ff5640 1 -- 172.21.15.69:0/1602420808 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f356c019c60 con 0x564477a87ad0 2024-05-14T16:53:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.773+0000 7f357c075740 1 -- 172.21.15.69:0/1602420808 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3564066070 con 0x564477a87ad0 2024-05-14T16:53:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.773+0000 7f3556ff5640 1 -- 172.21.15.69:0/1602420808 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f356c004050 con 0x564477a87ad0 2024-05-14T16:53:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.773+0000 7f357c075740 10 client.15768 fetch_fsmap learned FSMap version 32 2024-05-14T16:53:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.773+0000 7f357c075740 1 -- 172.21.15.69:0/1602420808 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564477b02540 con 0x564477a87ad0 2024-05-14T16:53:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.774+0000 7f3556ff5640 1 -- 172.21.15.69:0/1602420808 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3564066070 con 0x564477a87ad0 2024-05-14T16:53:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.774+0000 7f3556ff5640 1 -- 172.21.15.69:0/1602420808 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f356c004800 con 0x564477a87ad0 2024-05-14T16:53:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.774+0000 7f357c075740 10 client.15768 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T16:53:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.774+0000 7f357c075740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:53:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.774+0000 7f357c075740 20 client.15768 populate_metadata read hostname 'smithi069' 2024-05-14T16:53:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.774+0000 7f357c075740 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564477b029d0 0x564477b04df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.775+0000 7f357c075740 4 client.15768 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:53:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.775+0000 7f357c075740 1 -- 172.21.15.69:0/1602420808 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644779eb360 con 0x564477b029d0 2024-05-14T16:53:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.775+0000 7f3572ffd640 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564477b029d0 0x564477b04df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.776+0000 7f3572ffd640 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564477b029d0 0x564477b04df0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d5ef5b1552d9c0f server_cookie=fed233b2b93b6243 in_seq=0 out_seq=0 2024-05-14T16:53:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.776+0000 7f3556ff5640 10 client.15768 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.786+0000 7f3556ff5640 1 -- 172.21.15.69:0/1602420808 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5644779eb360 con 0x564477b029d0 2024-05-14T16:53:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.786+0000 7f3556ff5640 10 client.15768 handle_command_reply: tid=0 2024-05-14T16:53:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.786+0000 7f357c075740 1 client.15768 shutdown 2024-05-14T16:53:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.786+0000 7f357c075740 1 -- 172.21.15.69:0/1602420808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 msgr2=0x564477a98300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.786+0000 7f357c075740 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 0x564477a98300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f356c009ee0 tx=0x7f356c015000 comp rx=0 tx=0).stop 2024-05-14T16:53:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.786+0000 7f357c075740 1 -- 172.21.15.69:0/1602420808 shutdown_connections 2024-05-14T16:53:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.786+0000 7f357c075740 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564477b029d0 0x564477b04df0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f357c075740 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564477a87ad0 0x564477a98300 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f357c075740 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564477a7d210 0x564477a90b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f357c075740 1 --2- 172.21.15.69:0/1602420808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564477a78e00 0x564477a905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f357c075740 1 -- 172.21.15.69:0/1602420808 >> 172.21.15.69:0/1602420808 conn(0x564477a84ce0 msgr2=0x564477a9aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f3556ff5640 0 client.15768 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f357c075740 1 -- 172.21.15.69:0/1602420808 shutdown_connections 2024-05-14T16:53:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f357c075740 1 -- 172.21.15.69:0/1602420808 wait complete. 2024-05-14T16:53:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f357c075740 20 client.15768 trim_cache size 0 max 16384 2024-05-14T16:53:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:53:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:53:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:53:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f355c0b2a90 msgr2=0x7f355c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f3579358640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f355c0b2a90 0x7f355c0b4f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f356c01d000 tx=0x7f356c00e430 comp rx=0 tx=0).stop 2024-05-14T16:53:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3574104640 msgr2=0x7f357419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.787+0000 7f3579358640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3574104640 0x7f357419b9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f356800e9f0 tx=0x7f356800edf0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.788+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 shutdown_connections 2024-05-14T16:53:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.788+0000 7f3579358640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f355c0b2a90 0x7f355c0b4f50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.788+0000 7f3579358640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f357410d060 0x7f357419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.788+0000 7f3579358640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3574104640 0x7f357419b9a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.788+0000 7f3579358640 1 --2- 172.21.15.69:0/411907353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3574103d20 0x7f357419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.788+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 >> 172.21.15.69:0/411907353 conn(0x7f3574078ad0 msgr2=0x7f3574109070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.788+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 shutdown_connections 2024-05-14T16:53:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:28.789+0000 7f3579358640 1 -- 172.21.15.69:0/411907353 wait complete. 2024-05-14T16:53:30.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:29 smithi069 ceph-mon[31507]: pgmap v112: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 617 op/s 2024-05-14T16:53:30.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:29 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:53:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:29 smithi203 ceph-mon[35787]: pgmap v112: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 617 op/s 2024-05-14T16:53:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:29 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:53:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:29 smithi092 ceph-mon[35763]: pgmap v112: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 617 op/s 2024-05-14T16:53:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:29 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:53:30.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 180.0 s, 3352.6 tps, lat 29.780 ms stddev 33.241 2024-05-14T16:53:32.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:31 smithi069 ceph-mon[31507]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 940 op/s 2024-05-14T16:53:32.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:31 smithi203 ceph-mon[35787]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 940 op/s 2024-05-14T16:53:32.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:31 smithi092 ceph-mon[35763]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 940 op/s 2024-05-14T16:53:34.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:33 smithi069 ceph-mon[31507]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-05-14T16:53:34.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:33 smithi203 ceph-mon[35787]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-05-14T16:53:34.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:33 smithi092 ceph-mon[35763]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-05-14T16:53:35.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 185.0 s, 3190.4 tps, lat 30.964 ms stddev 33.309 2024-05-14T16:53:36.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:35 smithi069 ceph-mon[31507]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-05-14T16:53:36.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:35 smithi203 ceph-mon[35787]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-05-14T16:53:36.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:35 smithi092 ceph-mon[35763]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-05-14T16:53:38.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:37 smithi069 ceph-mon[31507]: pgmap v116: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 920 op/s 2024-05-14T16:53:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:37 smithi203 ceph-mon[35787]: pgmap v116: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 920 op/s 2024-05-14T16:53:38.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:37 smithi092 ceph-mon[35763]: pgmap v116: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 920 op/s 2024-05-14T16:53:40.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:39 smithi069 ceph-mon[31507]: pgmap v117: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.5 MiB/s wr, 618 op/s 2024-05-14T16:53:40.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:39 smithi203 ceph-mon[35787]: pgmap v117: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.5 MiB/s wr, 618 op/s 2024-05-14T16:53:40.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:39 smithi092 ceph-mon[35763]: pgmap v117: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.5 MiB/s wr, 618 op/s 2024-05-14T16:53:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 190.0 s, 3226.7 tps, lat 31.369 ms stddev 34.208 2024-05-14T16:53:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.922+0000 7fef2be2a640 1 -- 172.21.15.69:0/1474054031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24102660 msgr2=0x7fef2410e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.922+0000 7fef2be2a640 1 --2- 172.21.15.69:0/1474054031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24102660 0x7fef2410e770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009ca0 tx=0x7fef1c02a5c0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.922+0000 7fef2be2a640 1 -- 172.21.15.69:0/1474054031 shutdown_connections 2024-05-14T16:53:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.922+0000 7fef2be2a640 1 --2- 172.21.15.69:0/1474054031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24102660 0x7fef2410e770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.922+0000 7fef2be2a640 1 --2- 172.21.15.69:0/1474054031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef24101ca0 0x7fef24102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.922+0000 7fef2be2a640 1 --2- 172.21.15.69:0/1474054031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef241068f0 0x7fef24106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.922+0000 7fef2be2a640 1 -- 172.21.15.69:0/1474054031 >> 172.21.15.69:0/1474054031 conn(0x7fef24077750 msgr2=0x7fef24077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.922+0000 7fef2be2a640 1 -- 172.21.15.69:0/1474054031 shutdown_connections 2024-05-14T16:53:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.922+0000 7fef2be2a640 1 -- 172.21.15.69:0/1474054031 wait complete. 2024-05-14T16:53:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef2be2a640 1 Processor -- start 2024-05-14T16:53:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef2be2a640 1 -- start start 2024-05-14T16:53:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef2be2a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24101ca0 0x7fef24196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef2be2a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24102660 0x7fef24197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef2be2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef241068f0 0x7fef2419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef2be2a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef24110b10 con 0x7fef241068f0 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef2be2a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef24110990 con 0x7fef24101ca0 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef2be2a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef24110c90 con 0x7fef24102660 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef29bc6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24101ca0 0x7fef24196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef29bc6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24101ca0 0x7fef24196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:47996/0 (socket says 172.21.15.69:47996) 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.923+0000 7fef29bc6640 1 -- 172.21.15.69:0/349548895 learned_addr learned my addr 172.21.15.69:0/349548895 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef2a3c7640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef241068f0 0x7fef2419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef293c5640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24102660 0x7fef24197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef29bc6640 1 -- 172.21.15.69:0/349548895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24102660 msgr2=0x7fef24197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef29bc6640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24102660 0x7fef24197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef29bc6640 1 -- 172.21.15.69:0/349548895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef241068f0 msgr2=0x7fef2419b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef29bc6640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef241068f0 0x7fef2419b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef29bc6640 1 -- 172.21.15.69:0/349548895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2419bde0 con 0x7fef24101ca0 2024-05-14T16:53:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef293c5640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24102660 0x7fef24197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef29bc6640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24101ca0 0x7fef24196df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fef2000c920 tx=0x7fef2000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef1affd640 1 -- 172.21.15.69:0/349548895 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20022070 con 0x7fef24101ca0 2024-05-14T16:53:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef1affd640 1 -- 172.21.15.69:0/349548895 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef20007e50 con 0x7fef24101ca0 2024-05-14T16:53:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.924+0000 7fef1affd640 1 -- 172.21.15.69:0/349548895 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2001ddb0 con 0x7fef24101ca0 2024-05-14T16:53:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.925+0000 7fef2be2a640 1 -- 172.21.15.69:0/349548895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2419c0d0 con 0x7fef24101ca0 2024-05-14T16:53:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.925+0000 7fef2be2a640 1 -- 172.21.15.69:0/349548895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef241a3960 con 0x7fef24101ca0 2024-05-14T16:53:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.928+0000 7fef1affd640 1 -- 172.21.15.69:0/349548895 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fef200079a0 con 0x7fef24101ca0 2024-05-14T16:53:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.928+0000 7fef1affd640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fef100b2b50 0x7fef100b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.929+0000 7fef1affd640 1 -- 172.21.15.69:0/349548895 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fef200ea370 con 0x7fef24101ca0 2024-05-14T16:53:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.929+0000 7fef293c5640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fef100b2b50 0x7fef100b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.929+0000 7fef2ec21740 1 Processor -- start 2024-05-14T16:53:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.929+0000 7fef293c5640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fef100b2b50 0x7fef100b5010 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef241016a0 tx=0x7fef14011040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.929+0000 7fef2ec21740 1 -- start start 2024-05-14T16:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.930+0000 7fef2ec21740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce56f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.930+0000 7fef2ec21740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56f4210 0x559ce56fe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.930+0000 7fef2ec21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56fead0 0x559ce5700ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.930+0000 7fef2ec21740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559ce5666c30 con 0x559ce56f38f0 2024-05-14T16:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.930+0000 7fef2ec21740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ce56567d0 con 0x559ce56fead0 2024-05-14T16:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.930+0000 7fef2ec21740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559ce565a810 con 0x559ce56f4210 2024-05-14T16:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.930+0000 7fef29bc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce56f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.930+0000 7fef293c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56f4210 0x559ce56fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.930+0000 7fef29bc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce56f3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46790/0 (socket says 172.21.15.69:46790) 2024-05-14T16:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.930+0000 7fef29bc6640 1 -- 172.21.15.69:0/1388895722 learned_addr learned my addr 172.21.15.69:0/1388895722 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.931+0000 7fef2a3c7640 1 --2- 172.21.15.69:0/1388895722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56fead0 0x559ce5700ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.931+0000 7fef29bc6640 1 -- 172.21.15.69:0/1388895722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56f4210 msgr2=0x559ce56fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.931+0000 7fef29bc6640 1 --2- 172.21.15.69:0/1388895722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56f4210 0x559ce56fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.931+0000 7fef29bc6640 1 -- 172.21.15.69:0/1388895722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56fead0 msgr2=0x559ce5700ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.931+0000 7fef29bc6640 1 --2- 172.21.15.69:0/1388895722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56fead0 0x559ce5700ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.931+0000 7fef29bc6640 1 -- 172.21.15.69:0/1388895722 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce536b210 con 0x559ce56f38f0 2024-05-14T16:53:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.931+0000 7fef293c5640 1 --2- 172.21.15.69:0/1388895722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56f4210 0x559ce56fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.931+0000 7fef29bc6640 1 --2- 172.21.15.69:0/1388895722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce56f3cd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fef200b1700 tx=0x7fef2000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf24cbf9b513151 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.932+0000 7fef0ffff640 1 -- 172.21.15.69:0/1388895722 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef200b1e70 con 0x559ce56f38f0 2024-05-14T16:53:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.932+0000 7fef0ffff640 1 -- 172.21.15.69:0/1388895722 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef200b1770 con 0x559ce56f38f0 2024-05-14T16:53:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.932+0000 7fef0ffff640 1 -- 172.21.15.69:0/1388895722 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef200ba140 con 0x559ce56f38f0 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.932+0000 7fef2ec21740 1 -- 172.21.15.69:0/1388895722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 msgr2=0x559ce56f3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.932+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1388895722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce56f3cd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fef200b1700 tx=0x7fef2000cf90 comp rx=0 tx=0).stop 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.932+0000 7fef2ec21740 1 -- 172.21.15.69:0/1388895722 shutdown_connections 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.932+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1388895722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56fead0 0x559ce5700ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.932+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1388895722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56f4210 0x559ce56fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.932+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1388895722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce56f3cd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.932+0000 7fef2ec21740 1 -- 172.21.15.69:0/1388895722 >> 172.21.15.69:0/1388895722 conn(0x559ce56efe00 msgr2=0x559ce56f01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.933+0000 7fef2ec21740 1 -- 172.21.15.69:0/1388895722 shutdown_connections 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.933+0000 7fef2ec21740 1 -- 172.21.15.69:0/1388895722 wait complete. 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.933+0000 7fef2ec21740 1 -- start start 2024-05-14T16:53:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.933+0000 7fef2ec21740 1 Processor -- start 2024-05-14T16:53:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.934+0000 7fef2ec21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f38f0 0x559ce56fe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.934+0000 7fef2ec21740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f4210 0x559ce56fb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.934+0000 7fef2ec21740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 0x559ce56fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.934+0000 7fef293c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f4210 0x559ce56fb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.934+0000 7fef293c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f4210 0x559ce56fb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46804/0 (socket says 172.21.15.69:46804) 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.934+0000 7fef293c5640 1 -- 172.21.15.69:0/441210727 learned_addr learned my addr 172.21.15.69:0/441210727 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.934+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559ce533c790 con 0x559ce56f4210 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.934+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ce5612890 con 0x559ce56f38f0 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.934+0000 7fef29bc6640 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f38f0 0x559ce56fe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.934+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559ce5701740 con 0x559ce56fead0 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.935+0000 7fef2ec21740 10 client.0 _ll_register_callbacks cb 0x559ce5615d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.935+0000 7fef293c5640 1 -- 172.21.15.69:0/441210727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 msgr2=0x559ce56fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.935+0000 7fef293c5640 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 0x559ce56fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.935+0000 7fef293c5640 1 -- 172.21.15.69:0/441210727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f38f0 msgr2=0x559ce56fe440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.935+0000 7fef293c5640 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f38f0 0x559ce56fe440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.935+0000 7fef293c5640 1 -- 172.21.15.69:0/441210727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce5437620 con 0x559ce56f4210 2024-05-14T16:53:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.935+0000 7fef293c5640 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f4210 0x559ce56fb730 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fef1401b690 tx=0x7fef140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c84aad901c8c968 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.935+0000 7fef0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:53:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef0d7fa640 1 -- 172.21.15.69:0/441210727 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1401be10 con 0x559ce56f4210 2024-05-14T16:53:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef200b1770 con 0x559ce56f4210 2024-05-14T16:53:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef0d7fa640 1 -- 172.21.15.69:0/441210727 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fef14016070 con 0x559ce56f4210 2024-05-14T16:53:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef0d7fa640 1 -- 172.21.15.69:0/441210727 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fef14004030 con 0x559ce56f4210 2024-05-14T16:53:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef2ec21740 10 client.15780 fetch_fsmap learned FSMap version 33 2024-05-14T16:53:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559ce570d210 con 0x559ce56f4210 2024-05-14T16:53:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef0d7fa640 1 -- 172.21.15.69:0/441210727 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef200b1770 con 0x559ce56f4210 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef0d7fa640 1 -- 172.21.15.69:0/441210727 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fef14025400 con 0x559ce56f4210 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef2ec21740 10 client.15780 fetch_fsmap finished waiting for FSMap version 33 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef2ec21740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef2ec21740 20 client.15780 populate_metadata read hostname 'smithi069' 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef2ec21740 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559ce570e7e0 0x559ce5710c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef2ec21740 4 client.15780 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.936+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ce5668b80 con 0x559ce570e7e0 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.937+0000 7fef29bc6640 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559ce570e7e0 0x559ce5710c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.937+0000 7fef0d7fa640 10 client.15780 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.937+0000 7fef29bc6640 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559ce570e7e0 0x559ce5710c80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51fca49ffb70df5 server_cookie=e45c9cc9e7042b1f in_seq=0 out_seq=0 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.938+0000 7fef0d7fa640 1 -- 172.21.15.69:0/441210727 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ce5668b80 con 0x559ce570e7e0 2024-05-14T16:53:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.938+0000 7fef0d7fa640 10 client.15780 handle_command_reply: tid=0 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 client.15780 shutdown 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f4210 msgr2=0x559ce56fb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f4210 0x559ce56fb730 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fef1401b690 tx=0x7fef140046c0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 shutdown_connections 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559ce570e7e0 0x559ce5710c80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 0x559ce56fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f4210 0x559ce56fb730 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 --2- 172.21.15.69:0/441210727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f38f0 0x559ce56fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef0d7fa640 0 client.15780 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 >> 172.21.15.69:0/441210727 conn(0x559ce56efe00 msgr2=0x559ce56ef100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 shutdown_connections 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 1 -- 172.21.15.69:0/441210727 wait complete. 2024-05-14T16:53:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.939+0000 7fef2ec21740 20 client.15780 trim_cache size 0 max 16384 2024-05-14T16:53:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.962+0000 7fef2ec21740 1 Processor -- start 2024-05-14T16:53:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.962+0000 7fef2ec21740 1 -- start start 2024-05-14T16:53:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.962+0000 7fef2ec21740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce570a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.962+0000 7fef2ec21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 0x559ce5716550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef29bc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce570a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef29bc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce570a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46820/0 (socket says 172.21.15.69:46820) 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef29bc6640 1 -- 172.21.15.69:0/3812072364 learned_addr learned my addr 172.21.15.69:0/3812072364 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef2ec21740 1 --2- 172.21.15.69:0/3812072364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 0x559ce5716a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef293c5640 1 --2- 172.21.15.69:0/3812072364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 0x559ce5716550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef2ec21740 1 -- 172.21.15.69:0/3812072364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef241aec50 con 0x559ce56f38f0 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef2ec21740 1 -- 172.21.15.69:0/3812072364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef241ac490 con 0x559ce56f4210 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef2ec21740 1 -- 172.21.15.69:0/3812072364 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef241aa2e0 con 0x559ce56fead0 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef29bc6640 1 -- 172.21.15.69:0/3812072364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 msgr2=0x559ce5716a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef29bc6640 1 --2- 172.21.15.69:0/3812072364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 0x559ce5716a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef29bc6640 1 -- 172.21.15.69:0/3812072364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 msgr2=0x559ce5716550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef29bc6640 1 --2- 172.21.15.69:0/3812072364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 0x559ce5716550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef29bc6640 1 -- 172.21.15.69:0/3812072364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce56fc000 con 0x559ce56f38f0 2024-05-14T16:53:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef293c5640 1 --2- 172.21.15.69:0/3812072364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 0x559ce5716550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef29bc6640 1 --2- 172.21.15.69:0/3812072364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce570a610 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fef200e80d0 tx=0x7fef20000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f864b14b53d8300 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.963+0000 7fef0f7fe640 1 -- 172.21.15.69:0/3812072364 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef200b1770 con 0x559ce56f38f0 2024-05-14T16:53:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.964+0000 7fef0f7fe640 1 -- 172.21.15.69:0/3812072364 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef200b5b20 con 0x559ce56f38f0 2024-05-14T16:53:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.964+0000 7fef0f7fe640 1 -- 172.21.15.69:0/3812072364 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef200ba140 con 0x559ce56f38f0 2024-05-14T16:53:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.964+0000 7fef2ec21740 1 -- 172.21.15.69:0/3812072364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 msgr2=0x559ce570a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.964+0000 7fef2ec21740 1 --2- 172.21.15.69:0/3812072364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce570a610 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fef200e80d0 tx=0x7fef20000bc0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.965+0000 7fef2ec21740 1 -- 172.21.15.69:0/3812072364 shutdown_connections 2024-05-14T16:53:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.965+0000 7fef2ec21740 1 --2- 172.21.15.69:0/3812072364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 0x559ce5716a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.965+0000 7fef2ec21740 1 --2- 172.21.15.69:0/3812072364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 0x559ce5716550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.965+0000 7fef2ec21740 1 --2- 172.21.15.69:0/3812072364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce570a610 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.965+0000 7fef2ec21740 1 -- 172.21.15.69:0/3812072364 >> 172.21.15.69:0/3812072364 conn(0x559ce56efe00 msgr2=0x559ce56ef100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.965+0000 7fef2ec21740 1 -- 172.21.15.69:0/3812072364 shutdown_connections 2024-05-14T16:53:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.966+0000 7fef2ec21740 1 -- 172.21.15.69:0/3812072364 wait complete. 2024-05-14T16:53:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.966+0000 7fef2ec21740 1 -- start start 2024-05-14T16:53:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.966+0000 7fef2ec21740 1 Processor -- start 2024-05-14T16:53:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.967+0000 7fef2ec21740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce57045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef2ec21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 0x559ce5704b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef29bc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce57045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef29bc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce57045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46830/0 (socket says 172.21.15.69:46830) 2024-05-14T16:53:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef29bc6640 1 -- 172.21.15.69:0/1211580258 learned_addr learned my addr 172.21.15.69:0/1211580258 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 0x559ce5705040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef241aa950 con 0x559ce56f38f0 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef241abdc0 con 0x559ce56f4210 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef241ab020 con 0x559ce56fead0 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef293c5640 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 0x559ce5704b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef2ec21740 10 client.0 _ll_register_callbacks cb 0x7fef14009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef2a3c7640 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 0x559ce5705040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef293c5640 1 -- 172.21.15.69:0/1211580258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 msgr2=0x559ce5705040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef293c5640 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 0x559ce5705040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef293c5640 1 -- 172.21.15.69:0/1211580258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 msgr2=0x559ce57045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef293c5640 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce57045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.968+0000 7fef293c5640 1 -- 172.21.15.69:0/1211580258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce5778160 con 0x559ce56f4210 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef293c5640 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 0x559ce5704b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef14025870 tx=0x7fef14027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d8cdbc8df540c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef0d7fa640 1 -- 172.21.15.69:0/1211580258 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef140043e0 con 0x559ce56f4210 2024-05-14T16:53:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef0d7fa640 1 -- 172.21.15.69:0/1211580258 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fef14022040 con 0x559ce56f4210 2024-05-14T16:53:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559ce5662360 con 0x559ce56f4210 2024-05-14T16:53:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef0d7fa640 1 -- 172.21.15.69:0/1211580258 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fef14005040 con 0x559ce56f4210 2024-05-14T16:53:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef2ec21740 10 client.25319 fetch_fsmap learned FSMap version 33 2024-05-14T16:53:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559ce5779500 con 0x559ce56f4210 2024-05-14T16:53:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef0d7fa640 1 -- 172.21.15.69:0/1211580258 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559ce5662360 con 0x559ce56f4210 2024-05-14T16:53:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef0d7fa640 1 -- 172.21.15.69:0/1211580258 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fef14038070 con 0x559ce56f4210 2024-05-14T16:53:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef2ec21740 10 client.25319 fetch_fsmap finished waiting for FSMap version 33 2024-05-14T16:53:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef2ec21740 10 client.25319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:53:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef2ec21740 20 client.25319 populate_metadata read hostname 'smithi069' 2024-05-14T16:53:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.969+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559ce5779d30 0x559ce577c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.970+0000 7fef2ec21740 4 client.25319 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:53:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.970+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ce55dc5f0 con 0x559ce5779d30 2024-05-14T16:53:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.970+0000 7fef29bc6640 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559ce5779d30 0x559ce577c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.971+0000 7fef29bc6640 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559ce5779d30 0x559ce577c150 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ebebfa45f36f104 server_cookie=b06225d635be33f9 in_seq=0 out_seq=0 2024-05-14T16:53:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.971+0000 7fef0d7fa640 10 client.25319 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef0d7fa640 1 -- 172.21.15.69:0/1211580258 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559ce55dc5f0 con 0x559ce5779d30 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef0d7fa640 10 client.25319 handle_command_reply: tid=0 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 client.25319 shutdown 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 msgr2=0x559ce5704b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 0x559ce5704b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef14025870 tx=0x7fef14027000 comp rx=0 tx=0).stop 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 shutdown_connections 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559ce5779d30 0x559ce577c150 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ce56fead0 0x559ce5705040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce56f4210 0x559ce5704b00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef0d7fa640 0 client.25319 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 --2- 172.21.15.69:0/1211580258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ce56f38f0 0x559ce57045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 >> 172.21.15.69:0/1211580258 conn(0x559ce56efe00 msgr2=0x559ce5709600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 shutdown_connections 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 1 -- 172.21.15.69:0/1211580258 wait complete. 2024-05-14T16:53:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.992+0000 7fef2ec21740 20 client.25319 trim_cache size 0 max 16384 2024-05-14T16:53:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:53:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:53:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:53:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 -- 172.21.15.69:0/349548895 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fef100b2b50 msgr2=0x7fef100b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fef100b2b50 0x7fef100b5010 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef241016a0 tx=0x7fef14011040 comp rx=0 tx=0).stop 2024-05-14T16:53:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 -- 172.21.15.69:0/349548895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24101ca0 msgr2=0x7fef24196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24101ca0 0x7fef24196df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fef2000c920 tx=0x7fef2000cde0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 -- 172.21.15.69:0/349548895 shutdown_connections 2024-05-14T16:53:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fef100b2b50 0x7fef100b5010 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef241068f0 0x7fef2419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24102660 0x7fef24197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 --2- 172.21.15.69:0/349548895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24101ca0 0x7fef24196df0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 -- 172.21.15.69:0/349548895 >> 172.21.15.69:0/349548895 conn(0x7fef24077750 msgr2=0x7fef240ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.994+0000 7fef2be2a640 1 -- 172.21.15.69:0/349548895 shutdown_connections 2024-05-14T16:53:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:41.995+0000 7fef2be2a640 1 -- 172.21.15.69:0/349548895 wait complete. 2024-05-14T16:53:42.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:41 smithi069 ceph-mon[31507]: pgmap v118: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 10 MiB/s wr, 925 op/s 2024-05-14T16:53:42.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:41 smithi203 ceph-mon[35787]: pgmap v118: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 10 MiB/s wr, 925 op/s 2024-05-14T16:53:42.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:41 smithi092 ceph-mon[35763]: pgmap v118: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 10 MiB/s wr, 925 op/s 2024-05-14T16:53:44.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:43 smithi069 ceph-mon[31507]: pgmap v119: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2024-05-14T16:53:44.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:43 smithi203 ceph-mon[35787]: pgmap v119: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2024-05-14T16:53:44.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:43 smithi092 ceph-mon[35763]: pgmap v119: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2024-05-14T16:53:45.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 195.0 s, 2930.2 tps, lat 33.943 ms stddev 41.325 2024-05-14T16:53:46.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:45 smithi069 ceph-mon[31507]: pgmap v120: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2024-05-14T16:53:46.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:45 smithi203 ceph-mon[35787]: pgmap v120: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2024-05-14T16:53:46.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:45 smithi092 ceph-mon[35763]: pgmap v120: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2024-05-14T16:53:48.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:47 smithi069 ceph-mon[31507]: pgmap v121: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-05-14T16:53:48.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:47 smithi203 ceph-mon[35787]: pgmap v121: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-05-14T16:53:48.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:47 smithi092 ceph-mon[35763]: pgmap v121: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-05-14T16:53:50.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:49 smithi069 ceph-mon[31507]: pgmap v122: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 875 op/s 2024-05-14T16:53:50.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:49 smithi203 ceph-mon[35787]: pgmap v122: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 875 op/s 2024-05-14T16:53:50.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:49 smithi092 ceph-mon[35763]: pgmap v122: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 875 op/s 2024-05-14T16:53:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 200.0 s, 3085.0 tps, lat 32.597 ms stddev 32.943 2024-05-14T16:53:52.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:51 smithi069 ceph-mon[31507]: pgmap v123: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-05-14T16:53:52.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:51 smithi203 ceph-mon[35787]: pgmap v123: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-05-14T16:53:52.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:51 smithi092 ceph-mon[35763]: pgmap v123: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-05-14T16:53:54.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:53 smithi069 ceph-mon[31507]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 864 op/s 2024-05-14T16:53:54.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:53 smithi203 ceph-mon[35787]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 864 op/s 2024-05-14T16:53:54.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:53 smithi092 ceph-mon[35763]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 864 op/s 2024-05-14T16:53:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.115+0000 7f83cce75640 1 -- 172.21.15.69:0/632068509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c8100f50 msgr2=0x7f83c81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.115+0000 7f83cce75640 1 --2- 172.21.15.69:0/632068509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c8100f50 0x7f83c81013d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009930 tx=0x7f83b002a420 comp rx=0 tx=0).stop 2024-05-14T16:53:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.115+0000 7f83cce75640 1 -- 172.21.15.69:0/632068509 shutdown_connections 2024-05-14T16:53:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.115+0000 7f83cce75640 1 --2- 172.21.15.69:0/632068509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83c8101910 0x7f83c810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.115+0000 7f83cce75640 1 --2- 172.21.15.69:0/632068509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c8100f50 0x7f83c81013d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.115+0000 7f83cce75640 1 --2- 172.21.15.69:0/632068509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c8107a30 0x7f83c8107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.115+0000 7f83cce75640 1 -- 172.21.15.69:0/632068509 >> 172.21.15.69:0/632068509 conn(0x7f83c8077ee0 msgr2=0x7f83c80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.115+0000 7f83cce75640 1 -- 172.21.15.69:0/632068509 shutdown_connections 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.115+0000 7f83cce75640 1 -- 172.21.15.69:0/632068509 wait complete. 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83cce75640 1 Processor -- start 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83cce75640 1 -- start start 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83cce75640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c8100f50 0x7f83c819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83cce75640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c8101910 0x7f83c819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83cce75640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83c8107a30 0x7f83c819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83cce75640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8103bd0 con 0x7f83c8100f50 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83cce75640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8103a50 con 0x7f83c8107a30 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83cce75640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8103d50 con 0x7f83c8101910 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83c659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c8100f50 0x7f83c819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83c659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c8100f50 0x7f83c819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34756/0 (socket says 172.21.15.69:34756) 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83c659c640 1 -- 172.21.15.69:0/3418431824 learned_addr learned my addr 172.21.15.69:0/3418431824 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83bffff640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c8101910 0x7f83c819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83c6d9d640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83c8107a30 0x7f83c819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83c659c640 1 -- 172.21.15.69:0/3418431824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c8101910 msgr2=0x7f83c819b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83c659c640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c8101910 0x7f83c819b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83c659c640 1 -- 172.21.15.69:0/3418431824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83c8107a30 msgr2=0x7f83c819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83c659c640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83c8107a30 0x7f83c819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.116+0000 7f83c659c640 1 -- 172.21.15.69:0/3418431824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c81a03a0 con 0x7f83c8100f50 2024-05-14T16:53:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.117+0000 7f83c659c640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c8100f50 0x7f83c819b400 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f83b800d770 tx=0x7f83b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.117+0000 7f83bf7fe640 1 -- 172.21.15.69:0/3418431824 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8017070 con 0x7f83c8100f50 2024-05-14T16:53:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.117+0000 7f83bf7fe640 1 -- 172.21.15.69:0/3418431824 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b80054d0 con 0x7f83c8100f50 2024-05-14T16:53:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.117+0000 7f83bf7fe640 1 -- 172.21.15.69:0/3418431824 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8004c30 con 0x7f83c8100f50 2024-05-14T16:53:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.117+0000 7f83cce75640 1 -- 172.21.15.69:0/3418431824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c81a0690 con 0x7f83c8100f50 2024-05-14T16:53:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.117+0000 7f83cce75640 1 -- 172.21.15.69:0/3418431824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c81a7f20 con 0x7f83c8100f50 2024-05-14T16:53:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.120+0000 7f83cfb9a740 1 Processor -- start 2024-05-14T16:53:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.120+0000 7f83cfb9a740 1 -- start start 2024-05-14T16:53:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db2594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db263da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 0x5617db2666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5617db1c3c30 con 0x5617db259100 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5617db1b37d0 con 0x5617db2642e0 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5617db1b7810 con 0x5617db259a20 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83c6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db2594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83c6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db2594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34758/0 (socket says 172.21.15.69:34758) 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83c6d9d640 1 -- 172.21.15.69:0/1563992066 learned_addr learned my addr 172.21.15.69:0/1563992066 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83bffff640 1 --2- 172.21.15.69:0/1563992066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 0x5617db2666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83c6d9d640 1 -- 172.21.15.69:0/1563992066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 msgr2=0x5617db263da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83c6d9d640 1 --2- 172.21.15.69:0/1563992066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db263da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83c6d9d640 1 -- 172.21.15.69:0/1563992066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 msgr2=0x5617db2666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83c6d9d640 1 --2- 172.21.15.69:0/1563992066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 0x5617db2666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83c6d9d640 1 -- 172.21.15.69:0/1563992066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617daec8210 con 0x5617db259100 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.121+0000 7f83c6d9d640 1 --2- 172.21.15.69:0/1563992066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db2594e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009fc0 tx=0x7f83c0010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfe795d63ac5b22 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83bcff9640 1 -- 172.21.15.69:0/1563992066 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0003b70 con 0x5617db259100 2024-05-14T16:53:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83bcff9640 1 -- 172.21.15.69:0/1563992066 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c0003d10 con 0x5617db259100 2024-05-14T16:53:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83bf7fe640 1 -- 172.21.15.69:0/3418431824 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f83b8004e10 con 0x7f83c8100f50 2024-05-14T16:53:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83bcff9640 1 -- 172.21.15.69:0/1563992066 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0011950 con 0x5617db259100 2024-05-14T16:53:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1563992066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 msgr2=0x5617db2594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1563992066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db2594e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009fc0 tx=0x7f83c0010860 comp rx=0 tx=0).stop 2024-05-14T16:53:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1563992066 shutdown_connections 2024-05-14T16:53:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1563992066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 0x5617db2666d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1563992066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db263da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1563992066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db2594e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.122+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1563992066 >> 172.21.15.69:0/1563992066 conn(0x5617db255610 msgr2=0x5617db2559f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.123+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1563992066 shutdown_connections 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.123+0000 7f83bf7fe640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f83b40b2920 0x7f83b40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.123+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1563992066 wait complete. 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.123+0000 7f83bffff640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f83b40b2920 0x7f83b40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.124+0000 7f83bf7fe640 1 -- 172.21.15.69:0/3418431824 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f83b80e21d0 con 0x7f83c8100f50 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.124+0000 7f83cfb9a740 1 -- start start 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.124+0000 7f83cfb9a740 1 Processor -- start 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.124+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 0x5617db263b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.124+0000 7f83bffff640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f83b40b2920 0x7f83b40b4de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83c81062c0 tx=0x7f83b0005a20 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.124+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db260e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.124+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db2642e0 0x5617db261370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.124+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5617dae99790 con 0x5617db2642e0 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5617db16f890 con 0x5617db259100 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5617db266fb0 con 0x5617db259a20 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83cfb9a740 10 client.0 _ll_register_callbacks cb 0x5617db172d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83c659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 0x5617db263b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83c659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 0x5617db263b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:51648/0 (socket says 172.21.15.69:51648) 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83c659c640 1 -- 172.21.15.69:0/921689304 learned_addr learned my addr 172.21.15.69:0/921689304 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83bffff640 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db260e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83c6d9d640 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db2642e0 0x5617db261370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83c659c640 1 -- 172.21.15.69:0/921689304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 msgr2=0x5617db260e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83c659c640 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db260e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83c659c640 1 -- 172.21.15.69:0/921689304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db2642e0 msgr2=0x5617db261370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83c659c640 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db2642e0 0x5617db261370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83c659c640 1 -- 172.21.15.69:0/921689304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617daf94620 con 0x5617db259100 2024-05-14T16:53:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83c659c640 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 0x5617db263b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f83b80df9e0 tx=0x7f83b80a9090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b39df67f40cf44ca server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.125+0000 7f83adffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:53:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.126+0000 7f83adffb640 1 -- 172.21.15.69:0/921689304 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8020070 con 0x5617db259100 2024-05-14T16:53:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.126+0000 7f83adffb640 1 -- 172.21.15.69:0/921689304 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f83b80a9720 con 0x5617db259100 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.126+0000 7f83cfb9a740 1 -- 172.21.15.69:0/921689304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c0003d10 con 0x5617db259100 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.127+0000 7f83adffb640 1 -- 172.21.15.69:0/921689304 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f83b80a94c0 con 0x5617db259100 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.127+0000 7f83cfb9a740 10 client.25331 fetch_fsmap learned FSMap version 34 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.127+0000 7f83cfb9a740 1 -- 172.21.15.69:0/921689304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5617db2729e0 con 0x5617db259100 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.127+0000 7f83adffb640 1 -- 172.21.15.69:0/921689304 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b80ad870 con 0x5617db259100 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.128+0000 7f83adffb640 1 -- 172.21.15.69:0/921689304 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 34) ==== 8342+0+0 (secure 0 0 0) 0x7f83b80a4e30 con 0x5617db259100 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.128+0000 7f83cfb9a740 10 client.25331 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.128+0000 7f83cfb9a740 10 client.25331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.128+0000 7f83cfb9a740 20 client.25331 populate_metadata read hostname 'smithi069' 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.128+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5617db273e50 0x5617db276240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.128+0000 7f83cfb9a740 4 client.25331 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.128+0000 7f83cfb9a740 1 -- 172.21.15.69:0/921689304 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f83c0003d10 con 0x5617db273e50 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.128+0000 7f83bffff640 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5617db273e50 0x5617db276240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.128+0000 7f83adffb640 10 client.25331 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.128+0000 7f83bffff640 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5617db273e50 0x5617db276240 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d4a39a092f27ba6 server_cookie=88bd44aae0dbb23c in_seq=0 out_seq=0 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83adffb640 1 -- 172.21.15.69:0/921689304 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f83c0003d10 con 0x5617db273e50 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83adffb640 10 client.25331 handle_command_reply: tid=0 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 client.25331 shutdown 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 -- 172.21.15.69:0/921689304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 msgr2=0x5617db263b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 0x5617db263b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f83b80df9e0 tx=0x7f83b80a9090 comp rx=0 tx=0).stop 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 -- 172.21.15.69:0/921689304 shutdown_connections 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5617db273e50 0x5617db276240 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db2642e0 0x5617db261370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db260e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/921689304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 0x5617db263b40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83adffb640 0 client.25331 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 -- 172.21.15.69:0/921689304 >> 172.21.15.69:0/921689304 conn(0x5617db255610 msgr2=0x5617db25aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 -- 172.21.15.69:0/921689304 shutdown_connections 2024-05-14T16:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 1 -- 172.21.15.69:0/921689304 wait complete. 2024-05-14T16:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.129+0000 7f83cfb9a740 20 client.25331 trim_cache size 0 max 16384 2024-05-14T16:53:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.158+0000 7f83cfb9a740 1 Processor -- start 2024-05-14T16:53:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.158+0000 7f83cfb9a740 1 -- start start 2024-05-14T16:53:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.158+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 0x5617db25b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.158+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259a20 0x5617db263670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.158+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db2642e0 0x5617db27ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.158+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81aed90 con 0x5617db259a20 2024-05-14T16:53:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.158+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81ac630 con 0x5617db259100 2024-05-14T16:53:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.158+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81aa480 con 0x5617db2642e0 2024-05-14T16:53:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.159+0000 7f83c6d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db2642e0 0x5617db27ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.159+0000 7f83c6d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db2642e0 0x5617db27ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:33694/0 (socket says 172.21.15.69:33694) 2024-05-14T16:53:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.159+0000 7f83c6d9d640 1 -- 172.21.15.69:0/1625710859 learned_addr learned my addr 172.21.15.69:0/1625710859 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.160+0000 7f83c6d9d640 1 -- 172.21.15.69:0/1625710859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 msgr2=0x5617db25b1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:53:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.160+0000 7f83c6d9d640 1 --2- 172.21.15.69:0/1625710859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 0x5617db25b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.160+0000 7f83c6d9d640 1 -- 172.21.15.69:0/1625710859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259a20 msgr2=0x5617db263670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:53:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.160+0000 7f83c6d9d640 1 --2- 172.21.15.69:0/1625710859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259a20 0x5617db263670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.160+0000 7f83c6d9d640 1 -- 172.21.15.69:0/1625710859 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617db276820 con 0x5617db2642e0 2024-05-14T16:53:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.162+0000 7f83c6d9d640 1 --2- 172.21.15.69:0/1625710859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db2642e0 0x5617db27ba50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009270 tx=0x7f83c00092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e316f81f659f5108 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.162+0000 7f83affff640 1 -- 172.21.15.69:0/1625710859 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0003cf0 con 0x5617db2642e0 2024-05-14T16:53:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.162+0000 7f83affff640 1 -- 172.21.15.69:0/1625710859 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c00119b0 con 0x5617db2642e0 2024-05-14T16:53:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.162+0000 7f83affff640 1 -- 172.21.15.69:0/1625710859 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0005460 con 0x5617db2642e0 2024-05-14T16:53:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.163+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1625710859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db2642e0 msgr2=0x5617db27ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.163+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1625710859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db2642e0 0x5617db27ba50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009270 tx=0x7f83c00092a0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.163+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1625710859 shutdown_connections 2024-05-14T16:53:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.163+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1625710859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db2642e0 0x5617db27ba50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.163+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1625710859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259a20 0x5617db263670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.163+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1625710859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db259100 0x5617db25b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.163+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1625710859 >> 172.21.15.69:0/1625710859 conn(0x5617db255610 msgr2=0x5617db25aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.165+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1625710859 shutdown_connections 2024-05-14T16:53:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.166+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1625710859 wait complete. 2024-05-14T16:53:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.166+0000 7f83cfb9a740 1 -- start start 2024-05-14T16:53:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.166+0000 7f83cfb9a740 1 Processor -- start 2024-05-14T16:53:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db26c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db26c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83cfb9a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 0x5617db268960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81aaaf0 con 0x5617db259100 2024-05-14T16:53:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81abf60 con 0x5617db2642e0 2024-05-14T16:53:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83cfb9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81ab1c0 con 0x5617db259a20 2024-05-14T16:53:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83cfb9a740 10 client.0 _ll_register_callbacks cb 0x7f83b80a8c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83c6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 0x5617db268960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83c659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db26c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83c6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 0x5617db268960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:51660/0 (socket says 172.21.15.69:51660) 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83c6d9d640 1 -- 172.21.15.69:0/1787103086 learned_addr learned my addr 172.21.15.69:0/1787103086 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83bffff640 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db26c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83c659c640 1 -- 172.21.15.69:0/1787103086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 msgr2=0x5617db26c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83c659c640 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db26c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83c659c640 1 -- 172.21.15.69:0/1787103086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 msgr2=0x5617db268960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83c659c640 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 0x5617db268960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83c659c640 1 -- 172.21.15.69:0/1787103086 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617db269090 con 0x5617db259100 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83c659c640 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db26c3e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f83b80a9450 tx=0x7f83b80a8600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bdbe0d151c5002 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.167+0000 7f83adffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83adffb640 1 -- 172.21.15.69:0/1787103086 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8020070 con 0x5617db259100 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83adffb640 1 -- 172.21.15.69:0/1787103086 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f83b80a9720 con 0x5617db259100 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1787103086 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c00119b0 con 0x5617db259100 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83adffb640 1 -- 172.21.15.69:0/1787103086 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f83b80a5380 con 0x5617db259100 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83cfb9a740 10 client.15801 fetch_fsmap learned FSMap version 34 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1787103086 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5617db24d400 con 0x5617db259100 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83adffb640 1 -- 172.21.15.69:0/1787103086 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c00119b0 con 0x5617db259100 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83adffb640 1 -- 172.21.15.69:0/1787103086 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 34) ==== 8342+0+0 (secure 0 0 0) 0x7f83b80ca370 con 0x5617db259100 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83cfb9a740 10 client.15801 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83cfb9a740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83cfb9a740 20 client.15801 populate_metadata read hostname 'smithi069' 2024-05-14T16:53:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5617db24d990 0x5617db24fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83cfb9a740 4 client.15801 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:53:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.168+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1787103086 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617db1c5b80 con 0x5617db24d990 2024-05-14T16:53:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.169+0000 7f83bffff640 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5617db24d990 0x5617db24fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.169+0000 7f83adffb640 10 client.15801 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.169+0000 7f83bffff640 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5617db24d990 0x5617db24fdb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e244c63a9b391f server_cookie=8475c7138eecd98d in_seq=0 out_seq=0 2024-05-14T16:53:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83adffb640 1 -- 172.21.15.69:0/1787103086 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5617db1c5b80 con 0x5617db24d990 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83adffb640 10 client.15801 handle_command_reply: tid=0 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83cfb9a740 1 client.15801 shutdown 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1787103086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 msgr2=0x5617db26c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db26c3e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f83b80a9450 tx=0x7f83b80a8600 comp rx=0 tx=0).stop 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1787103086 shutdown_connections 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5617db24d990 0x5617db24fdb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5617db2642e0 0x5617db268960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83adffb640 0 client.15801 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5617db259a20 0x5617db26c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83cfb9a740 1 --2- 172.21.15.69:0/1787103086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5617db259100 0x5617db26c3e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.182+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1787103086 >> 172.21.15.69:0/1787103086 conn(0x5617db255610 msgr2=0x5617db26f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.183+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1787103086 shutdown_connections 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.183+0000 7f83cfb9a740 1 -- 172.21.15.69:0/1787103086 wait complete. 2024-05-14T16:53:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.183+0000 7f83cfb9a740 20 client.15801 trim_cache size 0 max 16384 2024-05-14T16:53:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:53:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:53:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.184+0000 7f83cce75640 1 -- 172.21.15.69:0/3418431824 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f83b40b2920 msgr2=0x7f83b40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.184+0000 7f83cce75640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f83b40b2920 0x7f83b40b4de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83c81062c0 tx=0x7f83b0005a20 comp rx=0 tx=0).stop 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.185+0000 7f83cce75640 1 -- 172.21.15.69:0/3418431824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c8100f50 msgr2=0x7f83c819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.185+0000 7f83cce75640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c8100f50 0x7f83c819b400 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f83b800d770 tx=0x7f83b800dc30 comp rx=0 tx=0).stop 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.185+0000 7f83cce75640 1 -- 172.21.15.69:0/3418431824 shutdown_connections 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.185+0000 7f83cce75640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f83b40b2920 0x7f83b40b4de0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.185+0000 7f83cce75640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83c8107a30 0x7f83c819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.185+0000 7f83cce75640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c8101910 0x7f83c819b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.185+0000 7f83cce75640 1 --2- 172.21.15.69:0/3418431824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c8100f50 0x7f83c819b400 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.185+0000 7f83cce75640 1 -- 172.21.15.69:0/3418431824 >> 172.21.15.69:0/3418431824 conn(0x7f83c8077ee0 msgr2=0x7f83c80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.185+0000 7f83cce75640 1 -- 172.21.15.69:0/3418431824 shutdown_connections 2024-05-14T16:53:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:53:55.185+0000 7f83cce75640 1 -- 172.21.15.69:0/3418431824 wait complete. 2024-05-14T16:53:55.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:54 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:53:55.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:54 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:53:55.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:54 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:53:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 205.0 s, 3088.4 tps, lat 32.280 ms stddev 32.461 2024-05-14T16:53:56.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:55 smithi069 ceph-mon[31507]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 864 op/s 2024-05-14T16:53:56.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:55 smithi203 ceph-mon[35787]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 864 op/s 2024-05-14T16:53:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:55 smithi092 ceph-mon[35763]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 864 op/s 2024-05-14T16:53:58.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:57 smithi069 ceph-mon[31507]: pgmap v126: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-05-14T16:53:58.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:57 smithi203 ceph-mon[35787]: pgmap v126: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-05-14T16:53:58.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:57 smithi092 ceph-mon[35763]: pgmap v126: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-05-14T16:54:00.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:53:59 smithi069 ceph-mon[31507]: pgmap v127: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 600 op/s 2024-05-14T16:54:00.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:53:59 smithi203 ceph-mon[35787]: pgmap v127: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 600 op/s 2024-05-14T16:54:00.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:53:59 smithi092 ceph-mon[35763]: pgmap v127: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 600 op/s 2024-05-14T16:54:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 210.0 s, 3745.8 tps, lat 26.841 ms stddev 29.258 2024-05-14T16:54:02.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:01 smithi069 ceph-mon[31507]: pgmap v128: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 933 op/s 2024-05-14T16:54:02.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:01 smithi203 ceph-mon[35787]: pgmap v128: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 933 op/s 2024-05-14T16:54:02.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:01 smithi092 ceph-mon[35763]: pgmap v128: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 933 op/s 2024-05-14T16:54:04.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:03 smithi069 ceph-mon[31507]: pgmap v129: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 638 op/s 2024-05-14T16:54:04.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:03 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:03 smithi203 ceph-mon[35787]: pgmap v129: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 638 op/s 2024-05-14T16:54:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:03 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:04.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:03 smithi092 ceph-mon[35763]: pgmap v129: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 638 op/s 2024-05-14T16:54:04.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:03 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:05.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:04 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:05.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:04 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:05.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:04 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:54:05.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:04 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:05.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:04 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:05.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:04 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:54:05.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:04 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:05.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:04 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:05.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:04 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:54:05.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 215.0 s, 403.8 tps, lat 239.495 ms stddev 167.249 2024-05-14T16:54:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:05 smithi069 ceph-mon[31507]: pgmap v130: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 638 op/s 2024-05-14T16:54:06.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:05 smithi203 ceph-mon[35787]: pgmap v130: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 638 op/s 2024-05-14T16:54:06.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:05 smithi092 ceph-mon[35763]: pgmap v130: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 638 op/s 2024-05-14T16:54:08.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:07 smithi069 ceph-mon[31507]: pgmap v131: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.5 MiB/s wr, 889 op/s 2024-05-14T16:54:08.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:07 smithi203 ceph-mon[35787]: pgmap v131: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.5 MiB/s wr, 889 op/s 2024-05-14T16:54:08.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:07 smithi092 ceph-mon[35763]: pgmap v131: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.5 MiB/s wr, 889 op/s 2024-05-14T16:54:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.348+0000 7ff03d674640 1 -- 172.21.15.69:0/2175119459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff038111240 msgr2=0x7ff038109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.348+0000 7ff03d674640 1 --2- 172.21.15.69:0/2175119459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff038111240 0x7ff038109a50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff028009890 tx=0x7ff02802a440 comp rx=0 tx=0).stop 2024-05-14T16:54:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.349+0000 7ff03d674640 1 -- 172.21.15.69:0/2175119459 shutdown_connections 2024-05-14T16:54:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.349+0000 7ff03d674640 1 --2- 172.21.15.69:0/2175119459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff03811c230 0x7ff03811e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.349+0000 7ff03d674640 1 --2- 172.21.15.69:0/2175119459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff038111620 0x7ff038109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.349+0000 7ff03d674640 1 --2- 172.21.15.69:0/2175119459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff038111240 0x7ff038109a50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.349+0000 7ff03d674640 1 -- 172.21.15.69:0/2175119459 >> 172.21.15.69:0/2175119459 conn(0x7ff03806db20 msgr2=0x7ff03806df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.350+0000 7ff03d674640 1 -- 172.21.15.69:0/2175119459 shutdown_connections 2024-05-14T16:54:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.352+0000 7ff03d674640 1 -- 172.21.15.69:0/2175119459 wait complete. 2024-05-14T16:54:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.352+0000 7ff03d674640 1 Processor -- start 2024-05-14T16:54:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.357+0000 7ff03d674640 1 -- start start 2024-05-14T16:54:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.357+0000 7ff03d674640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff038111240 0x7ff0381a8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.357+0000 7ff03d674640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff03811c230 0x7ff0381a8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.357+0000 7ff03d674640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0381ac9d0 0x7ff0381ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.357+0000 7ff03d674640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff038121bd0 con 0x7ff038111240 2024-05-14T16:54:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.357+0000 7ff03d674640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff038121a50 con 0x7ff0381ac9d0 2024-05-14T16:54:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.357+0000 7ff03d674640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff038121d50 con 0x7ff03811c230 2024-05-14T16:54:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.358+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff038111240 0x7ff0381a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.358+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff038111240 0x7ff0381a8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40940/0 (socket says 172.21.15.69:40940) 2024-05-14T16:54:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.358+0000 7ff036ffd640 1 -- 172.21.15.69:0/207767829 learned_addr learned my addr 172.21.15.69:0/207767829 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.358+0000 7ff0367fc640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff03811c230 0x7ff0381a8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.358+0000 7ff036ffd640 1 -- 172.21.15.69:0/207767829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff03811c230 msgr2=0x7ff0381a8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.358+0000 7ff036ffd640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff03811c230 0x7ff0381a8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.359+0000 7ff036ffd640 1 -- 172.21.15.69:0/207767829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0381ac9d0 msgr2=0x7ff0381ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.359+0000 7ff036ffd640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0381ac9d0 0x7ff0381ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.359+0000 7ff036ffd640 1 -- 172.21.15.69:0/207767829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0381ad4c0 con 0x7ff038111240 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.359+0000 7ff036ffd640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff038111240 0x7ff0381a8110 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff028009460 tx=0x7ff028030f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.359+0000 7ff01ffff640 1 -- 172.21.15.69:0/207767829 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028005410 con 0x7ff038111240 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.359+0000 7ff01ffff640 1 -- 172.21.15.69:0/207767829 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0280055b0 con 0x7ff038111240 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.359+0000 7ff03d674640 1 -- 172.21.15.69:0/207767829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0381ad750 con 0x7ff038111240 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.360+0000 7ff03d674640 1 -- 172.21.15.69:0/207767829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0381b4f10 con 0x7ff038111240 2024-05-14T16:54:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.360+0000 7ff01ffff640 1 -- 172.21.15.69:0/207767829 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028004e10 con 0x7ff038111240 2024-05-14T16:54:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.364+0000 7ff04038b740 1 Processor -- start 2024-05-14T16:54:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.368+0000 7ff04038b740 1 -- start start 2024-05-14T16:54:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.369+0000 7ff04038b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c958cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.369+0000 7ff04038b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c959210 0x55ed0c963590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.369+0000 7ff04038b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c963ad0 0x55ed0c965ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.369+0000 7ff04038b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ed0c8cbc30 con 0x55ed0c9588f0 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.369+0000 7ff04038b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ed0c8bb7d0 con 0x55ed0c963ad0 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.369+0000 7ff04038b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ed0c8bf810 con 0x55ed0c959210 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.370+0000 7ff01ffff640 1 -- 172.21.15.69:0/207767829 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff028039040 con 0x7ff038111240 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.371+0000 7ff01ffff640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff0200b2c20 0x7ff0200b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.371+0000 7ff0367fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c963ad0 0x55ed0c965ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.371+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c958cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.371+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c958cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40950/0 (socket says 172.21.15.69:40950) 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.371+0000 7ff0377fe640 1 -- 172.21.15.69:0/1968671369 learned_addr learned my addr 172.21.15.69:0/1968671369 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.371+0000 7ff036ffd640 1 --2- 172.21.15.69:0/1968671369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c959210 0x55ed0c963590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.372+0000 7ff01ffff640 1 -- 172.21.15.69:0/207767829 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff0281038f0 con 0x7ff038111240 2024-05-14T16:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.374+0000 7ff0377fe640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff0200b2c20 0x7ff0200b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.377+0000 7ff0377fe640 1 -- 172.21.15.69:0/1968671369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c959210 msgr2=0x55ed0c963590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.377+0000 7ff0377fe640 1 --2- 172.21.15.69:0/1968671369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c959210 0x55ed0c963590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.377+0000 7ff0377fe640 1 -- 172.21.15.69:0/1968671369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c963ad0 msgr2=0x55ed0c965ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.377+0000 7ff0377fe640 1 --2- 172.21.15.69:0/1968671369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c963ad0 0x55ed0c965ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.377+0000 7ff0377fe640 1 -- 172.21.15.69:0/1968671369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed0c5d0210 con 0x55ed0c9588f0 2024-05-14T16:54:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.382+0000 7ff0377fe640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff0200b2c20 0x7ff0200b50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff03000cc40 tx=0x7ff03001b000 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.382+0000 7ff0377fe640 1 --2- 172.21.15.69:0/1968671369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c958cd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff030010dc0 tx=0x7ff03000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3583159f8d3875 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.382+0000 7ff01d7fa640 1 -- 172.21.15.69:0/1968671369 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03000bbf0 con 0x55ed0c9588f0 2024-05-14T16:54:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.382+0000 7ff01d7fa640 1 -- 172.21.15.69:0/1968671369 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03000bd90 con 0x55ed0c9588f0 2024-05-14T16:54:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.383+0000 7ff01d7fa640 1 -- 172.21.15.69:0/1968671369 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030003bf0 con 0x55ed0c9588f0 2024-05-14T16:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.383+0000 7ff04038b740 1 -- 172.21.15.69:0/1968671369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 msgr2=0x55ed0c958cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.383+0000 7ff04038b740 1 --2- 172.21.15.69:0/1968671369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c958cd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff030010dc0 tx=0x7ff03000c740 comp rx=0 tx=0).stop 2024-05-14T16:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.384+0000 7ff04038b740 1 -- 172.21.15.69:0/1968671369 shutdown_connections 2024-05-14T16:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.384+0000 7ff04038b740 1 --2- 172.21.15.69:0/1968671369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c963ad0 0x55ed0c965ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.384+0000 7ff04038b740 1 --2- 172.21.15.69:0/1968671369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c959210 0x55ed0c963590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.384+0000 7ff04038b740 1 --2- 172.21.15.69:0/1968671369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c958cd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.384+0000 7ff04038b740 1 -- 172.21.15.69:0/1968671369 >> 172.21.15.69:0/1968671369 conn(0x55ed0c954e00 msgr2=0x55ed0c9551e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.387+0000 7ff04038b740 1 -- 172.21.15.69:0/1968671369 shutdown_connections 2024-05-14T16:54:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.390+0000 7ff04038b740 1 -- 172.21.15.69:0/1968671369 wait complete. 2024-05-14T16:54:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.390+0000 7ff04038b740 1 -- start start 2024-05-14T16:54:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.390+0000 7ff04038b740 1 Processor -- start 2024-05-14T16:54:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.391+0000 7ff04038b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c9588f0 0x55ed0c963470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.391+0000 7ff04038b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c960760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.391+0000 7ff04038b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c963ad0 0x55ed0c960d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.391+0000 7ff04038b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ed0c5a1790 con 0x55ed0c963ad0 2024-05-14T16:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.391+0000 7ff04038b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ed0c877890 con 0x55ed0c959210 2024-05-14T16:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.391+0000 7ff04038b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ed0c977750 con 0x55ed0c9588f0 2024-05-14T16:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.391+0000 7ff04038b740 10 client.0 _ll_register_callbacks cb 0x55ed0c87ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.391+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c963ad0 0x55ed0c960d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.391+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c963ad0 0x55ed0c960d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40960/0 (socket says 172.21.15.69:40960) 2024-05-14T16:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.391+0000 7ff0377fe640 1 -- 172.21.15.69:0/3087995107 learned_addr learned my addr 172.21.15.69:0/3087995107 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.392+0000 7ff0377fe640 1 -- 172.21.15.69:0/3087995107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c9588f0 msgr2=0x55ed0c963470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.392+0000 7ff0377fe640 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c9588f0 0x55ed0c963470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.392+0000 7ff0377fe640 1 -- 172.21.15.69:0/3087995107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 msgr2=0x55ed0c960760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.392+0000 7ff0377fe640 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c960760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.392+0000 7ff0377fe640 1 -- 172.21.15.69:0/3087995107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed0c69c620 con 0x55ed0c963ad0 2024-05-14T16:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.393+0000 7ff0377fe640 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c963ad0 0x55ed0c960d00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff030005bf0 tx=0x7ff030004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c875291165e531ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.394+0000 7ff01aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:54:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.394+0000 7ff01aff5640 1 -- 172.21.15.69:0/3087995107 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030028020 con 0x55ed0c963ad0 2024-05-14T16:54:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.394+0000 7ff04038b740 1 -- 172.21.15.69:0/3087995107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03000bd90 con 0x55ed0c963ad0 2024-05-14T16:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.394+0000 7ff01aff5640 1 -- 172.21.15.69:0/3087995107 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff030003d80 con 0x55ed0c963ad0 2024-05-14T16:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.395+0000 7ff01aff5640 1 -- 172.21.15.69:0/3087995107 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff030033050 con 0x55ed0c963ad0 2024-05-14T16:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.395+0000 7ff04038b740 10 client.15813 fetch_fsmap learned FSMap version 34 2024-05-14T16:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.395+0000 7ff04038b740 1 -- 172.21.15.69:0/3087995107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ed0c983250 con 0x55ed0c963ad0 2024-05-14T16:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.395+0000 7ff01aff5640 1 -- 172.21.15.69:0/3087995107 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03000bd90 con 0x55ed0c963ad0 2024-05-14T16:54:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.396+0000 7ff01aff5640 1 -- 172.21.15.69:0/3087995107 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 34) ==== 8342+0+0 (secure 0 0 0) 0x7ff03000b040 con 0x55ed0c963ad0 2024-05-14T16:54:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.397+0000 7ff04038b740 10 client.15813 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T16:54:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.397+0000 7ff04038b740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:54:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.397+0000 7ff04038b740 20 client.15813 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.397+0000 7ff04038b740 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ed0c984bf0 0x55ed0c9870b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.397+0000 7ff04038b740 4 client.15813 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:54:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.397+0000 7ff04038b740 1 -- 172.21.15.69:0/3087995107 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed0c8cdb80 con 0x55ed0c984bf0 2024-05-14T16:54:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.399+0000 7ff036ffd640 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ed0c984bf0 0x55ed0c9870b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.404+0000 7ff036ffd640 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ed0c984bf0 0x55ed0c9870b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf4fcbd9227be42 server_cookie=7f209f0ef0d471fb in_seq=0 out_seq=0 2024-05-14T16:54:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.405+0000 7ff01aff5640 10 client.15813 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.409+0000 7ff01aff5640 1 -- 172.21.15.69:0/3087995107 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed0c8cdb80 con 0x55ed0c984bf0 2024-05-14T16:54:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.409+0000 7ff01aff5640 10 client.15813 handle_command_reply: tid=0 2024-05-14T16:54:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.409+0000 7ff04038b740 1 client.15813 shutdown 2024-05-14T16:54:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.418+0000 7ff04038b740 1 -- 172.21.15.69:0/3087995107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c963ad0 msgr2=0x55ed0c960d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.418+0000 7ff04038b740 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c963ad0 0x55ed0c960d00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff030005bf0 tx=0x7ff030004dd0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.421+0000 7ff04038b740 1 -- 172.21.15.69:0/3087995107 shutdown_connections 2024-05-14T16:54:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.421+0000 7ff04038b740 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ed0c984bf0 0x55ed0c9870b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.421+0000 7ff04038b740 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c963ad0 0x55ed0c960d00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.421+0000 7ff04038b740 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c960760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.421+0000 7ff04038b740 1 --2- 172.21.15.69:0/3087995107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c9588f0 0x55ed0c963470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.421+0000 7ff04038b740 1 -- 172.21.15.69:0/3087995107 >> 172.21.15.69:0/3087995107 conn(0x55ed0c954e00 msgr2=0x55ed0c954100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.422+0000 7ff01aff5640 0 client.15813 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.423+0000 7ff04038b740 1 -- 172.21.15.69:0/3087995107 shutdown_connections 2024-05-14T16:54:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.425+0000 7ff04038b740 1 -- 172.21.15.69:0/3087995107 wait complete. 2024-05-14T16:54:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.427+0000 7ff04038b740 20 client.15813 trim_cache size 0 max 16384 2024-05-14T16:54:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.463+0000 7ff04038b740 1 Processor -- start 2024-05-14T16:54:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.464+0000 7ff04038b740 1 -- start start 2024-05-14T16:54:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.465+0000 7ff04038b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c980650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.465+0000 7ff04038b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c962fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.465+0000 7ff04038b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c963ad0 0x55ed0c98c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.465+0000 7ff04038b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff024010fc0 con 0x55ed0c9588f0 2024-05-14T16:54:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.465+0000 7ff04038b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff02400e860 con 0x55ed0c959210 2024-05-14T16:54:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.465+0000 7ff04038b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff02400c760 con 0x55ed0c963ad0 2024-05-14T16:54:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.465+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c980650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.465+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c980650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40970/0 (socket says 172.21.15.69:40970) 2024-05-14T16:54:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.465+0000 7ff036ffd640 1 -- 172.21.15.69:0/3113341396 learned_addr learned my addr 172.21.15.69:0/3113341396 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.467+0000 7ff036ffd640 1 -- 172.21.15.69:0/3113341396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c963ad0 msgr2=0x55ed0c98c980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:54:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.467+0000 7ff036ffd640 1 --2- 172.21.15.69:0/3113341396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c963ad0 0x55ed0c98c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.467+0000 7ff036ffd640 1 -- 172.21.15.69:0/3113341396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 msgr2=0x55ed0c962fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:54:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.467+0000 7ff036ffd640 1 --2- 172.21.15.69:0/3113341396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c962fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.467+0000 7ff036ffd640 1 -- 172.21.15.69:0/3113341396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed0c95ecb0 con 0x55ed0c9588f0 2024-05-14T16:54:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.468+0000 7ff036ffd640 1 --2- 172.21.15.69:0/3113341396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c980650 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff0280ca880 tx=0x7ff0280ed040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daff4d04723901ed server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.468+0000 7ff01cff9640 1 -- 172.21.15.69:0/3113341396 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0280e3030 con 0x55ed0c9588f0 2024-05-14T16:54:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.468+0000 7ff01cff9640 1 -- 172.21.15.69:0/3113341396 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0280cb280 con 0x55ed0c9588f0 2024-05-14T16:54:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.468+0000 7ff01cff9640 1 -- 172.21.15.69:0/3113341396 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0280046c0 con 0x55ed0c9588f0 2024-05-14T16:54:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.469+0000 7ff04038b740 1 -- 172.21.15.69:0/3113341396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 msgr2=0x55ed0c980650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.469+0000 7ff04038b740 1 --2- 172.21.15.69:0/3113341396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c980650 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff0280ca880 tx=0x7ff0280ed040 comp rx=0 tx=0).stop 2024-05-14T16:54:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.471+0000 7ff04038b740 1 -- 172.21.15.69:0/3113341396 shutdown_connections 2024-05-14T16:54:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.471+0000 7ff04038b740 1 --2- 172.21.15.69:0/3113341396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c963ad0 0x55ed0c98c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.471+0000 7ff04038b740 1 --2- 172.21.15.69:0/3113341396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c962fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.471+0000 7ff04038b740 1 --2- 172.21.15.69:0/3113341396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c980650 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.471+0000 7ff04038b740 1 -- 172.21.15.69:0/3113341396 >> 172.21.15.69:0/3113341396 conn(0x55ed0c954e00 msgr2=0x55ed0c95bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.471+0000 7ff04038b740 1 -- 172.21.15.69:0/3113341396 shutdown_connections 2024-05-14T16:54:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.471+0000 7ff04038b740 1 -- 172.21.15.69:0/3113341396 wait complete. 2024-05-14T16:54:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.471+0000 7ff04038b740 1 -- start start 2024-05-14T16:54:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.471+0000 7ff04038b740 1 Processor -- start 2024-05-14T16:54:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.472+0000 7ff04038b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c9e7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.472+0000 7ff04038b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c9e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.472+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c9e7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.472+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c9e7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40986/0 (socket says 172.21.15.69:40986) 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.473+0000 7ff036ffd640 1 -- 172.21.15.69:0/1299857356 learned_addr learned my addr 172.21.15.69:0/1299857356 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.473+0000 7ff04038b740 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c963ad0 0x55ed0c977f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.473+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff02400cd20 con 0x55ed0c9588f0 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.473+0000 7ff0367fc640 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c9e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.473+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff02400e190 con 0x55ed0c959210 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.473+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff02400d3f0 con 0x55ed0c963ad0 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.473+0000 7ff04038b740 10 client.0 _ll_register_callbacks cb 0x7ff030010bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.473+0000 7ff0377fe640 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c963ad0 0x55ed0c977f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.474+0000 7ff0367fc640 1 -- 172.21.15.69:0/1299857356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c963ad0 msgr2=0x55ed0c977f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.474+0000 7ff0367fc640 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c963ad0 0x55ed0c977f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.474+0000 7ff0367fc640 1 -- 172.21.15.69:0/1299857356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 msgr2=0x55ed0c9e7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.474+0000 7ff0367fc640 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c9e7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.474+0000 7ff0367fc640 1 -- 172.21.15.69:0/1299857356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed0c978620 con 0x55ed0c959210 2024-05-14T16:54:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.475+0000 7ff0367fc640 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c9e7dd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55ed0c95ffc0 tx=0x7ff02c0119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=698a4e66c81ad1a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.475+0000 7ff01aff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:54:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.475+0000 7ff01aff5640 1 -- 172.21.15.69:0/1299857356 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c013670 con 0x55ed0c959210 2024-05-14T16:54:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.476+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0280cb280 con 0x55ed0c959210 2024-05-14T16:54:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.476+0000 7ff01aff5640 1 -- 172.21.15.69:0/1299857356 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff02c013810 con 0x55ed0c959210 2024-05-14T16:54:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.476+0000 7ff01aff5640 1 -- 172.21.15.69:0/1299857356 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff02c013a10 con 0x55ed0c959210 2024-05-14T16:54:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.476+0000 7ff04038b740 10 client.25340 fetch_fsmap learned FSMap version 34 2024-05-14T16:54:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.476+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ed0c9ef080 con 0x55ed0c959210 2024-05-14T16:54:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.477+0000 7ff01aff5640 1 -- 172.21.15.69:0/1299857356 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff02c013bb0 con 0x55ed0c959210 2024-05-14T16:54:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.477+0000 7ff01aff5640 1 -- 172.21.15.69:0/1299857356 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 34) ==== 8342+0+0 (secure 0 0 0) 0x7ff02c01d400 con 0x55ed0c959210 2024-05-14T16:54:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.477+0000 7ff04038b740 10 client.25340 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T16:54:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.477+0000 7ff04038b740 10 client.25340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:54:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.477+0000 7ff04038b740 20 client.25340 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.477+0000 7ff04038b740 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ed0c9ef7b0 0x55ed0c9f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.477+0000 7ff04038b740 4 client.25340 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:54:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.477+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff0280cb280 con 0x55ed0c9ef7b0 2024-05-14T16:54:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.481+0000 7ff036ffd640 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ed0c9ef7b0 0x55ed0c9f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.487+0000 7ff036ffd640 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ed0c9ef7b0 0x55ed0c9f1bd0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73ad988b45717ce server_cookie=d317fb6b4a9a1898 in_seq=0 out_seq=0 2024-05-14T16:54:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.487+0000 7ff01aff5640 10 client.25340 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.524+0000 7ff01aff5640 1 -- 172.21.15.69:0/1299857356 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7ff0280cb280 con 0x55ed0c9ef7b0 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff01aff5640 10 client.25340 handle_command_reply: tid=0 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 client.25340 shutdown 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 msgr2=0x55ed0c9e7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c9e7dd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55ed0c95ffc0 tx=0x7ff02c0119d0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 shutdown_connections 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ed0c9ef7b0 0x55ed0c9f1bd0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed0c963ad0 0x55ed0c977f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff01aff5640 0 client.25340 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed0c959210 0x55ed0c9e7dd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 --2- 172.21.15.69:0/1299857356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ed0c9588f0 0x55ed0c9e7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 >> 172.21.15.69:0/1299857356 conn(0x55ed0c954e00 msgr2=0x55ed0c95a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 shutdown_connections 2024-05-14T16:54:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 1 -- 172.21.15.69:0/1299857356 wait complete. 2024-05-14T16:54:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.525+0000 7ff04038b740 20 client.25340 trim_cache size 0 max 16384 2024-05-14T16:54:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:54:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:54:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 -- 172.21.15.69:0/207767829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff0200b2c20 msgr2=0x7ff0200b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff0200b2c20 0x7ff0200b50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff03000cc40 tx=0x7ff03001b000 comp rx=0 tx=0).stop 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 -- 172.21.15.69:0/207767829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff038111240 msgr2=0x7ff0381a8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff038111240 0x7ff0381a8110 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff028009460 tx=0x7ff028030f60 comp rx=0 tx=0).stop 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 -- 172.21.15.69:0/207767829 shutdown_connections 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff0200b2c20 0x7ff0200b50e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0381ac9d0 0x7ff0381ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff03811c230 0x7ff0381a8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 --2- 172.21.15.69:0/207767829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff038111240 0x7ff0381a8110 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 -- 172.21.15.69:0/207767829 >> 172.21.15.69:0/207767829 conn(0x7ff03806db20 msgr2=0x7ff03810ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 -- 172.21.15.69:0/207767829 shutdown_connections 2024-05-14T16:54:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:08.526+0000 7ff03d674640 1 -- 172.21.15.69:0/207767829 wait complete. 2024-05-14T16:54:10.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:09 smithi069 ceph-mon[31507]: pgmap v132: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 584 op/s 2024-05-14T16:54:10.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:09 smithi203 ceph-mon[35787]: pgmap v132: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 584 op/s 2024-05-14T16:54:10.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:09 smithi092 ceph-mon[35763]: pgmap v132: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 584 op/s 2024-05-14T16:54:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 220.0 s, 920.6 tps, lat 111.680 ms stddev 191.440 2024-05-14T16:54:12.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:11 smithi069 ceph-mon[31507]: pgmap v133: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 1.69k op/s 2024-05-14T16:54:12.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:11 smithi203 ceph-mon[35787]: pgmap v133: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 1.69k op/s 2024-05-14T16:54:12.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:11 smithi092 ceph-mon[35763]: pgmap v133: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 1.69k op/s 2024-05-14T16:54:14.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:13 smithi069 ceph-mon[31507]: pgmap v134: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 1.36k op/s 2024-05-14T16:54:14.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:13 smithi203 ceph-mon[35787]: pgmap v134: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 1.36k op/s 2024-05-14T16:54:14.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:13 smithi092 ceph-mon[35763]: pgmap v134: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 1.36k op/s 2024-05-14T16:54:15.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:14 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:54:15.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:14 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:54:15.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:14 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:54:15.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 225.0 s, 3262.0 tps, lat 30.729 ms stddev 31.910 2024-05-14T16:54:16.150 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:15 smithi092 ceph-mon[35763]: pgmap v135: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 1.36k op/s 2024-05-14T16:54:16.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:15 smithi069 ceph-mon[31507]: pgmap v135: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 1.36k op/s 2024-05-14T16:54:16.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:15 smithi203 ceph-mon[35787]: pgmap v135: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 1.36k op/s 2024-05-14T16:54:18.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:17 smithi069 ceph-mon[31507]: pgmap v136: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2024-05-14T16:54:18.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:17 smithi203 ceph-mon[35787]: pgmap v136: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2024-05-14T16:54:18.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:17 smithi092 ceph-mon[35763]: pgmap v136: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2024-05-14T16:54:19.192 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.8 2024-05-14T16:54:19.193 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:54:19.546 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:19.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:19 smithi069 ceph-mon[31507]: pgmap v137: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 1.55k op/s 2024-05-14T16:54:20.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:19 smithi203 ceph-mon[35787]: pgmap v137: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 1.55k op/s 2024-05-14T16:54:20.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:19 smithi092 ceph-mon[35763]: pgmap v137: 97 pgs: 97 active+clean; 2.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 1.55k op/s 2024-05-14T16:54:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 230.0 s, 3226.0 tps, lat 31.047 ms stddev 32.474 2024-05-14T16:54:21.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.379+0000 7f8a72df8700 1 -- 172.21.15.69:0/3798668489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a6c06f9a0 msgr2=0x7f8a6c11c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.379+0000 7f8a72df8700 1 --2- 172.21.15.69:0/3798668489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a6c06f9a0 0x7f8a6c11c690 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009990 tx=0x7f8a58009ca0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.379+0000 7f8a72df8700 1 -- 172.21.15.69:0/3798668489 shutdown_connections 2024-05-14T16:54:21.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.379+0000 7f8a72df8700 1 --2- 172.21.15.69:0/3798668489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6c11cbd0 0x7f8a6c11efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.379+0000 7f8a72df8700 1 --2- 172.21.15.69:0/3798668489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a6c06f080 0x7f8a6c06f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.379+0000 7f8a72df8700 1 --2- 172.21.15.69:0/3798668489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a6c06f9a0 0x7f8a6c11c690 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.379+0000 7f8a72df8700 1 -- 172.21.15.69:0/3798668489 >> 172.21.15.69:0/3798668489 conn(0x7f8a6c06e700 msgr2=0x7f8a6c10c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:21.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.379+0000 7f8a72df8700 1 -- 172.21.15.69:0/3798668489 shutdown_connections 2024-05-14T16:54:21.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.379+0000 7f8a72df8700 1 -- 172.21.15.69:0/3798668489 wait complete. 2024-05-14T16:54:21.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.380+0000 7f8a72df8700 1 Processor -- start 2024-05-14T16:54:21.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.380+0000 7f8a72df8700 1 -- start start 2024-05-14T16:54:21.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.380+0000 7f8a72df8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a6c06f080 0x7f8a6c1ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a72df8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a6c06f9a0 0x7f8a6c1aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a72df8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6c11cbd0 0x7f8a6c1a8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a72df8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c122550 con 0x7f8a6c06f9a0 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a72df8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1223d0 con 0x7f8a6c11cbd0 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a72df8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1226d0 con 0x7f8a6c06f080 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a70b94700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a6c06f080 0x7f8a6c1ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a70b94700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a6c06f080 0x7f8a6c1ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:43654/0 (socket says 172.21.15.69:43654) 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a70b94700 1 -- 172.21.15.69:0/973411130 learned_addr learned my addr 172.21.15.69:0/973411130 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a6bfff700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a6c06f9a0 0x7f8a6c1aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a71395700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6c11cbd0 0x7f8a6c1a8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a70b94700 1 -- 172.21.15.69:0/973411130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6c11cbd0 msgr2=0x7f8a6c1a8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a70b94700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6c11cbd0 0x7f8a6c1a8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a70b94700 1 -- 172.21.15.69:0/973411130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a6c06f9a0 msgr2=0x7f8a6c1aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a70b94700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a6c06f9a0 0x7f8a6c1aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.381+0000 7f8a70b94700 1 -- 172.21.15.69:0/973411130 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c1a8ff0 con 0x7f8a6c06f080 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.382+0000 7f8a70b94700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a6c06f080 0x7f8a6c1ae6d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000d950 tx=0x7f8a6000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.382+0000 7f8a69ffb700 1 -- 172.21.15.69:0/973411130 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60016070 con 0x7f8a6c06f080 2024-05-14T16:54:21.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.382+0000 7f8a69ffb700 1 -- 172.21.15.69:0/973411130 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a6000fd40 con 0x7f8a6c06f080 2024-05-14T16:54:21.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.382+0000 7f8a72df8700 1 -- 172.21.15.69:0/973411130 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6c1a92e0 con 0x7f8a6c06f080 2024-05-14T16:54:21.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.382+0000 7f8a69ffb700 1 -- 172.21.15.69:0/973411130 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60016070 con 0x7f8a6c06f080 2024-05-14T16:54:21.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.382+0000 7f8a72df8700 1 -- 172.21.15.69:0/973411130 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6c1b55a0 con 0x7f8a6c06f080 2024-05-14T16:54:21.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.383+0000 7f8a72df8700 1 -- 172.21.15.69:0/973411130 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a6c06a230 con 0x7f8a6c06f080 2024-05-14T16:54:21.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.385+0000 7f8a69ffb700 1 -- 172.21.15.69:0/973411130 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8a60019070 con 0x7f8a6c06f080 2024-05-14T16:54:21.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.386+0000 7f8a69ffb700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8a5c0b2bf0 0x7f8a5c0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.386+0000 7f8a69ffb700 1 -- 172.21.15.69:0/973411130 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f8a600e1d40 con 0x7f8a6c06f080 2024-05-14T16:54:21.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.386+0000 7f8a6bfff700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8a5c0b2bf0 0x7f8a5c0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.386+0000 7f8a6bfff700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8a5c0b2bf0 0x7f8a5c0b50b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800b5c0 tx=0x7f8a58005960 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:21.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.390+0000 7f8a69ffb700 1 -- 172.21.15.69:0/973411130 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a60010460 con 0x7f8a6c06f080 2024-05-14T16:54:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.647+0000 7ff970ae1640 1 -- 172.21.15.69:0/1170141109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c10d020 msgr2=0x7ff96c10f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.647+0000 7ff970ae1640 1 --2- 172.21.15.69:0/1170141109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c10d020 0x7ff96c10f4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff964009ca0 tx=0x7ff96402a9e0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.647+0000 7ff970ae1640 1 -- 172.21.15.69:0/1170141109 shutdown_connections 2024-05-14T16:54:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.647+0000 7ff970ae1640 1 --2- 172.21.15.69:0/1170141109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c10d020 0x7ff96c10f4c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.647+0000 7ff970ae1640 1 --2- 172.21.15.69:0/1170141109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c104600 0x7ff96c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.647+0000 7ff970ae1640 1 --2- 172.21.15.69:0/1170141109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff96c103ce0 0x7ff96c1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.647+0000 7ff970ae1640 1 -- 172.21.15.69:0/1170141109 >> 172.21.15.69:0/1170141109 conn(0x7ff96c078ad0 msgr2=0x7ff96c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.647+0000 7ff970ae1640 1 -- 172.21.15.69:0/1170141109 shutdown_connections 2024-05-14T16:54:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.647+0000 7ff970ae1640 1 -- 172.21.15.69:0/1170141109 wait complete. 2024-05-14T16:54:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.648+0000 7ff970ae1640 1 Processor -- start 2024-05-14T16:54:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.648+0000 7ff970ae1640 1 -- start start 2024-05-14T16:54:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.648+0000 7ff970ae1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff96c103ce0 0x7ff96c1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.648+0000 7ff970ae1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c104600 0x7ff96c1a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.648+0000 7ff970ae1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c10d020 0x7ff96c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.648+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff96c103ce0 0x7ff96c1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff970ae1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c110ae0 con 0x7ff96c103ce0 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff970ae1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c110960 con 0x7ff96c10d020 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff96c103ce0 0x7ff96c1a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37650/0 (socket says 172.21.15.69:37650) 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff96a59c640 1 -- 172.21.15.69:0/1984260238 learned_addr learned my addr 172.21.15.69:0/1984260238 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff970ae1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c110c60 con 0x7ff96c104600 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff969d9b640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c104600 0x7ff96c1a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff96ad9d640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c10d020 0x7ff96c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff969d9b640 1 -- 172.21.15.69:0/1984260238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c10d020 msgr2=0x7ff96c19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff969d9b640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c10d020 0x7ff96c19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff969d9b640 1 -- 172.21.15.69:0/1984260238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff96c103ce0 msgr2=0x7ff96c1a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff969d9b640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff96c103ce0 0x7ff96c1a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff969d9b640 1 -- 172.21.15.69:0/1984260238 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c19fd90 con 0x7ff96c104600 2024-05-14T16:54:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff96ad9d640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c10d020 0x7ff96c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff969d9b640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c104600 0x7ff96c1a59a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff96000e9f0 tx=0x7ff96000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.649+0000 7ff95b7fe640 1 -- 172.21.15.69:0/1984260238 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96000ccc0 con 0x7ff96c104600 2024-05-14T16:54:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.650+0000 7ff970ae1640 1 -- 172.21.15.69:0/1984260238 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c1a0080 con 0x7ff96c104600 2024-05-14T16:54:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.650+0000 7ff970ae1640 1 -- 172.21.15.69:0/1984260238 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c1ac320 con 0x7ff96c104600 2024-05-14T16:54:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.650+0000 7ff95b7fe640 1 -- 172.21.15.69:0/1984260238 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9600108e0 con 0x7ff96c104600 2024-05-14T16:54:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.650+0000 7ff95b7fe640 1 -- 172.21.15.69:0/1984260238 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960018880 con 0x7ff96c104600 2024-05-14T16:54:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.653+0000 7ff95b7fe640 1 -- 172.21.15.69:0/1984260238 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff960010430 con 0x7ff96c104600 2024-05-14T16:54:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.654+0000 7ff95b7fe640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff9540c3b20 0x7ff9540c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.654+0000 7ff95b7fe640 1 -- 172.21.15.69:0/1984260238 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff9600e2920 con 0x7ff96c104600 2024-05-14T16:54:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.654+0000 7ff96a59c640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff9540c3b20 0x7ff9540c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.655+0000 7ff96a59c640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff9540c3b20 0x7ff9540c5fe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00fc20 tx=0x7ff95c005f90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.655+0000 7ff973804740 1 Processor -- start 2024-05-14T16:54:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.655+0000 7ff973804740 1 -- start start 2024-05-14T16:54:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.655+0000 7ff973804740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c62cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.655+0000 7ff973804740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c63210 0x564576c6d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.655+0000 7ff973804740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 0x564576c6fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.655+0000 7ff973804740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564576bd5c30 con 0x564576c63210 2024-05-14T16:54:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.655+0000 7ff973804740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564576bc57d0 con 0x564576c628f0 2024-05-14T16:54:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.655+0000 7ff973804740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564576bc9810 con 0x564576c6dad0 2024-05-14T16:54:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c62cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff969d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c63210 0x564576c6d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c62cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:52418/0 (socket says 172.21.15.69:52418) 2024-05-14T16:54:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff96a59c640 1 -- 172.21.15.69:0/2349944161 learned_addr learned my addr 172.21.15.69:0/2349944161 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff96ad9d640 1 --2- 172.21.15.69:0/2349944161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 0x564576c6fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff96a59c640 1 -- 172.21.15.69:0/2349944161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 msgr2=0x564576c6fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff96a59c640 1 --2- 172.21.15.69:0/2349944161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 0x564576c6fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff96a59c640 1 -- 172.21.15.69:0/2349944161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c63210 msgr2=0x564576c6d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff96a59c640 1 --2- 172.21.15.69:0/2349944161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c63210 0x564576c6d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff96a59c640 1 -- 172.21.15.69:0/2349944161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645768da210 con 0x564576c628f0 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff96a59c640 1 --2- 172.21.15.69:0/2349944161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c62cd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00ff50 tx=0x7ff95c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4af4849aca8ab7a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff958ff9640 1 -- 172.21.15.69:0/2349944161 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c025400 con 0x564576c628f0 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff958ff9640 1 -- 172.21.15.69:0/2349944161 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c025a50 con 0x564576c628f0 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.656+0000 7ff958ff9640 1 -- 172.21.15.69:0/2349944161 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c01a6f0 con 0x564576c628f0 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.657+0000 7ff973804740 1 -- 172.21.15.69:0/2349944161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 msgr2=0x564576c62cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.657+0000 7ff973804740 1 --2- 172.21.15.69:0/2349944161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c62cd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00ff50 tx=0x7ff95c014000 comp rx=0 tx=0).stop 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.658+0000 7ff973804740 1 -- 172.21.15.69:0/2349944161 shutdown_connections 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.658+0000 7ff973804740 1 --2- 172.21.15.69:0/2349944161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 0x564576c6fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.658+0000 7ff973804740 1 --2- 172.21.15.69:0/2349944161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c63210 0x564576c6d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.658+0000 7ff973804740 1 --2- 172.21.15.69:0/2349944161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c62cd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.658+0000 7ff973804740 1 -- 172.21.15.69:0/2349944161 >> 172.21.15.69:0/2349944161 conn(0x564576c5ee00 msgr2=0x564576c5f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.658+0000 7ff973804740 1 -- 172.21.15.69:0/2349944161 shutdown_connections 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.658+0000 7ff973804740 1 -- 172.21.15.69:0/2349944161 wait complete. 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.658+0000 7ff973804740 1 -- start start 2024-05-14T16:54:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff973804740 1 Processor -- start 2024-05-14T16:54:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff973804740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c6d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff973804740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c63210 0x564576c6a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff973804740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 0x564576c6ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff973804740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5645768ab790 con 0x564576c63210 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff973804740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564576b81890 con 0x564576c628f0 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff973804740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564576c707a0 con 0x564576c6dad0 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff96ad9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 0x564576c6ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff96ad9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 0x564576c6ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:43690/0 (socket says 172.21.15.69:43690) 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff96ad9d640 1 -- 172.21.15.69:0/1387115671 learned_addr learned my addr 172.21.15.69:0/1387115671 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.659+0000 7ff96a59c640 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c6d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff973804740 10 client.0 _ll_register_callbacks cb 0x564576b84d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff96ad9d640 1 -- 172.21.15.69:0/1387115671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 msgr2=0x564576c6d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff96ad9d640 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c6d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff96ad9d640 1 -- 172.21.15.69:0/1387115671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c63210 msgr2=0x564576c6a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff96ad9d640 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c63210 0x564576c6a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff96ad9d640 1 -- 172.21.15.69:0/1387115671 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645769a6620 con 0x564576c6dad0 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff973804740 1 -- 172.21.15.69:0/1387115671 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c025a50 con 0x564576c6dad0 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff96ad9d640 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 0x564576c6ac70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff964035000 tx=0x7ff96402f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb70a21ed1888e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff951ffb640 10 client.25303 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff951ffb640 1 -- 172.21.15.69:0/1387115671 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964033040 con 0x564576c6dad0 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.660+0000 7ff951ffb640 1 -- 172.21.15.69:0/1387115671 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff96402fc20 con 0x564576c6dad0 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff951ffb640 1 -- 172.21.15.69:0/1387115671 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff96402fe20 con 0x564576c6dad0 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff973804740 10 client.25303 fetch_fsmap learned FSMap version 35 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff973804740 1 -- 172.21.15.69:0/1387115671 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({fsmap=35}) -- 0x564576c7c0a0 con 0x564576c6dad0 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff951ffb640 1 -- 172.21.15.69:0/1387115671 <== mon.1 v2:172.21.15.203:3300/0 4 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7ff96404d030 con 0x564576c6dad0 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff973804740 10 client.25303 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff973804740 10 client.25303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff973804740 20 client.25303 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff973804740 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564576c7d560 0x564576c7f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff973804740 4 client.25303 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:54:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff973804740 1 -- 172.21.15.69:0/1387115671 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564576bd7b80 con 0x564576c7d560 2024-05-14T16:54:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.661+0000 7ff96a59c640 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564576c7d560 0x564576c7f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.662+0000 7ff951ffb640 10 client.25303 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.662+0000 7ff96a59c640 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564576c7d560 0x564576c7f9d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d25fc2add07522b server_cookie=89ea560247c23b96 in_seq=0 out_seq=0 2024-05-14T16:54:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.662+0000 7ff951ffb640 1 -- 172.21.15.69:0/1387115671 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564576bd7b80 con 0x564576c7d560 2024-05-14T16:54:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.662+0000 7ff951ffb640 10 client.25303 handle_command_reply: tid=0 2024-05-14T16:54:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.662+0000 7ff973804740 1 client.25303 shutdown 2024-05-14T16:54:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 1 -- 172.21.15.69:0/1387115671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 msgr2=0x564576c6ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 0x564576c6ac70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff964035000 tx=0x7ff96402f870 comp rx=0 tx=0).stop 2024-05-14T16:54:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 1 -- 172.21.15.69:0/1387115671 shutdown_connections 2024-05-14T16:54:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564576c7d560 0x564576c7f9d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c6dad0 0x564576c6ac70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff951ffb640 0 client.25303 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c63210 0x564576c6a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 1 --2- 172.21.15.69:0/1387115671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c6d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 1 -- 172.21.15.69:0/1387115671 >> 172.21.15.69:0/1387115671 conn(0x564576c5ee00 msgr2=0x564576c5e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 1 -- 172.21.15.69:0/1387115671 shutdown_connections 2024-05-14T16:54:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 1 -- 172.21.15.69:0/1387115671 wait complete. 2024-05-14T16:54:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.663+0000 7ff973804740 20 client.25303 trim_cache size 0 max 16384 2024-05-14T16:54:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff973804740 1 Processor -- start 2024-05-14T16:54:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff973804740 1 -- start start 2024-05-14T16:54:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff973804740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c79670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff973804740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c63210 0x564576c852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff973804740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c6dad0 0x564576c85820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff973804740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1aec20 con 0x564576c6dad0 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff973804740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1ac5a0 con 0x564576c628f0 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff973804740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c072980 con 0x564576c63210 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c6dad0 0x564576c85820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c79670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff969d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c63210 0x564576c852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c79670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:52424/0 (socket says 172.21.15.69:52424) 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c6dad0 0x564576c85820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37676/0 (socket says 172.21.15.69:37676) 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96a59c640 1 -- 172.21.15.69:0/1615185566 learned_addr learned my addr 172.21.15.69:0/1615185566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96a59c640 1 -- 172.21.15.69:0/1615185566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c63210 msgr2=0x564576c852e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96a59c640 1 --2- 172.21.15.69:0/1615185566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c63210 0x564576c852e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96a59c640 1 -- 172.21.15.69:0/1615185566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c6dad0 msgr2=0x564576c85820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96a59c640 1 --2- 172.21.15.69:0/1615185566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c6dad0 0x564576c85820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96a59c640 1 -- 172.21.15.69:0/1615185566 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564576c696c0 con 0x564576c628f0 2024-05-14T16:54:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.683+0000 7ff96ad9d640 1 --2- 172.21.15.69:0/1615185566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c6dad0 0x564576c85820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff96a59c640 1 --2- 172.21.15.69:0/1615185566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c79670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff95c01a520 tx=0x7ff95c01a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffec56d8eac5fe60 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff953fff640 1 -- 172.21.15.69:0/1615185566 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c025400 con 0x564576c628f0 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff953fff640 1 -- 172.21.15.69:0/1615185566 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c03a070 con 0x564576c628f0 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff953fff640 1 -- 172.21.15.69:0/1615185566 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0048c0 con 0x564576c628f0 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff973804740 1 -- 172.21.15.69:0/1615185566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 msgr2=0x564576c79670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff973804740 1 --2- 172.21.15.69:0/1615185566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c79670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff95c01a520 tx=0x7ff95c01a490 comp rx=0 tx=0).stop 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff973804740 1 -- 172.21.15.69:0/1615185566 shutdown_connections 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff973804740 1 --2- 172.21.15.69:0/1615185566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c6dad0 0x564576c85820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff973804740 1 --2- 172.21.15.69:0/1615185566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c63210 0x564576c852e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff973804740 1 --2- 172.21.15.69:0/1615185566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c628f0 0x564576c79670 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.684+0000 7ff973804740 1 -- 172.21.15.69:0/1615185566 >> 172.21.15.69:0/1615185566 conn(0x564576c5ee00 msgr2=0x564576c5e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.685+0000 7ff973804740 1 -- 172.21.15.69:0/1615185566 shutdown_connections 2024-05-14T16:54:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.685+0000 7ff973804740 1 -- 172.21.15.69:0/1615185566 wait complete. 2024-05-14T16:54:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.685+0000 7ff973804740 1 -- start start 2024-05-14T16:54:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.686+0000 7ff973804740 1 Processor -- start 2024-05-14T16:54:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.686+0000 7ff973804740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c628f0 0x564576c75e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff973804740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c63210 0x564576c731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c628f0 0x564576c75e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c628f0 0x564576c75e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37686/0 (socket says 172.21.15.69:37686) 2024-05-14T16:54:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff96a59c640 1 -- 172.21.15.69:0/958086707 learned_addr learned my addr 172.21.15.69:0/958086707 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff973804740 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c6dad0 0x564576c73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c072ff0 con 0x564576c628f0 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff969d9b640 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c63210 0x564576c731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c074460 con 0x564576c6dad0 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c0736c0 con 0x564576c63210 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff973804740 10 client.0 _ll_register_callbacks cb 0x7ff96402e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff96ad9d640 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c6dad0 0x564576c73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff96a59c640 1 -- 172.21.15.69:0/958086707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c63210 msgr2=0x564576c731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff96a59c640 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c63210 0x564576c731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff96a59c640 1 -- 172.21.15.69:0/958086707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c6dad0 msgr2=0x564576c73720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff96a59c640 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c6dad0 0x564576c73720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff96a59c640 1 -- 172.21.15.69:0/958086707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564576c73dc0 con 0x564576c628f0 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.687+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c03a070 con 0x564576c628f0 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.688+0000 7ff96a59c640 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c628f0 0x564576c75e00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff95c035fd0 tx=0x7ff95c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ddbe51527fd5f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.688+0000 7ff96ad9d640 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c6dad0 0x564576c73720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.688+0000 7ff951ffb640 10 client.15840 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.688+0000 7ff951ffb640 1 -- 172.21.15.69:0/958086707 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c044020 con 0x564576c628f0 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.688+0000 7ff951ffb640 1 -- 172.21.15.69:0/958086707 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff95c025400 con 0x564576c628f0 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.688+0000 7ff951ffb640 1 -- 172.21.15.69:0/958086707 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff95c005630 con 0x564576c628f0 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.688+0000 7ff973804740 10 client.15840 fetch_fsmap learned FSMap version 35 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.688+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=35}) -- 0x564576ce7c20 con 0x564576c628f0 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.689+0000 7ff951ffb640 1 -- 172.21.15.69:0/958086707 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7ff95c051060 con 0x564576c628f0 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.689+0000 7ff973804740 10 client.15840 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.689+0000 7ff973804740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.689+0000 7ff973804740 20 client.15840 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.689+0000 7ff973804740 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564576c7d560 0x564576cea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.689+0000 7ff973804740 4 client.15840 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.689+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564576bd1360 con 0x564576c7d560 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.689+0000 7ff969d9b640 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564576c7d560 0x564576cea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.689+0000 7ff951ffb640 10 client.15840 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.689+0000 7ff969d9b640 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564576c7d560 0x564576cea0a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd98f3d412ff0899 server_cookie=c3052c66053a6004 in_seq=0 out_seq=0 2024-05-14T16:54:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff951ffb640 1 -- 172.21.15.69:0/958086707 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564576bd1360 con 0x564576c7d560 2024-05-14T16:54:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff951ffb640 10 client.15840 handle_command_reply: tid=0 2024-05-14T16:54:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 client.15840 shutdown 2024-05-14T16:54:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c628f0 msgr2=0x564576c75e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c628f0 0x564576c75e00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff95c035fd0 tx=0x7ff95c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 shutdown_connections 2024-05-14T16:54:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564576c7d560 0x564576cea0a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564576c6dad0 0x564576c73720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564576c63210 0x564576c731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 --2- 172.21.15.69:0/958086707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564576c628f0 0x564576c75e00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff951ffb640 0 client.15840 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 >> 172.21.15.69:0/958086707 conn(0x564576c5ee00 msgr2=0x564576c78660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 shutdown_connections 2024-05-14T16:54:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 1 -- 172.21.15.69:0/958086707 wait complete. 2024-05-14T16:54:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.701+0000 7ff973804740 20 client.15840 trim_cache size 0 max 16384 2024-05-14T16:54:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:54:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:54:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:54:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 -- 172.21.15.69:0/1984260238 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff9540c3b20 msgr2=0x7ff9540c5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff9540c3b20 0x7ff9540c5fe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00fc20 tx=0x7ff95c005f90 comp rx=0 tx=0).stop 2024-05-14T16:54:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 -- 172.21.15.69:0/1984260238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c104600 msgr2=0x7ff96c1a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c104600 0x7ff96c1a59a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff96000e9f0 tx=0x7ff96000eeb0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 -- 172.21.15.69:0/1984260238 shutdown_connections 2024-05-14T16:54:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff9540c3b20 0x7ff9540c5fe0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c10d020 0x7ff96c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c104600 0x7ff96c1a59a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 --2- 172.21.15.69:0/1984260238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff96c103ce0 0x7ff96c1a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 -- 172.21.15.69:0/1984260238 >> 172.21.15.69:0/1984260238 conn(0x7ff96c078ad0 msgr2=0x7ff96c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 -- 172.21.15.69:0/1984260238 shutdown_connections 2024-05-14T16:54:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:21.702+0000 7ff970ae1640 1 -- 172.21.15.69:0/1984260238 wait complete. 2024-05-14T16:54:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.767+0000 7f8a72df8700 1 -- 172.21.15.69:0/973411130 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a6c02e560 con 0x7f8a6c06f080 2024-05-14T16:54:21.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.771+0000 7f8a69ffb700 1 -- 172.21.15.69:0/973411130 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11863 (secure 0 0 0) 0x7f8a600a96d0 con 0x7f8a6c06f080 2024-05-14T16:54:21.771 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:54:21.772 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:54:13.610116+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24446,24422,24433]},"id":1}]} 2024-05-14T16:54:21.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.776+0000 7f8a72df8700 1 -- 172.21.15.69:0/973411130 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8a5c0b2bf0 msgr2=0x7f8a5c0b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.776+0000 7f8a72df8700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8a5c0b2bf0 0x7f8a5c0b50b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800b5c0 tx=0x7f8a58005960 comp rx=0 tx=0).stop 2024-05-14T16:54:21.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.776+0000 7f8a72df8700 1 -- 172.21.15.69:0/973411130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a6c06f080 msgr2=0x7f8a6c1ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:21.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.776+0000 7f8a72df8700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a6c06f080 0x7f8a6c1ae6d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000d950 tx=0x7f8a6000dc60 comp rx=0 tx=0).stop 2024-05-14T16:54:21.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.777+0000 7f8a72df8700 1 -- 172.21.15.69:0/973411130 shutdown_connections 2024-05-14T16:54:21.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.777+0000 7f8a72df8700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6c11cbd0 0x7f8a6c1a8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.777+0000 7f8a72df8700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a6c06f080 0x7f8a6c1ae6d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.777+0000 7f8a72df8700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8a5c0b2bf0 0x7f8a5c0b50b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.777+0000 7f8a72df8700 1 --2- 172.21.15.69:0/973411130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a6c06f9a0 0x7f8a6c1aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.777+0000 7f8a72df8700 1 -- 172.21.15.69:0/973411130 >> 172.21.15.69:0/973411130 conn(0x7f8a6c06e700 msgr2=0x7f8a6c072530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.778+0000 7f8a72df8700 1 -- 172.21.15.69:0/973411130 shutdown_connections 2024-05-14T16:54:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:21.778+0000 7f8a72df8700 1 -- 172.21.15.69:0/973411130 wait complete. 2024-05-14T16:54:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 35 2024-05-14T16:54:22.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:21 smithi069 ceph-mon[31507]: pgmap v138: 97 pgs: 97 active+clean; 2.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 1.86k op/s 2024-05-14T16:54:22.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:21 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/973411130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:22.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:21 smithi203 ceph-mon[35787]: pgmap v138: 97 pgs: 97 active+clean; 2.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 1.86k op/s 2024-05-14T16:54:22.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:21 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/973411130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:22.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:21 smithi092 ceph-mon[35763]: pgmap v138: 97 pgs: 97 active+clean; 2.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 1.86k op/s 2024-05-14T16:54:22.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:21 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/973411130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:22.513 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0' 2024-05-14T16:54:22.514 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0 2024-05-14T16:54:22.936 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:24.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:23 smithi069 ceph-mon[31507]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.9 MiB/s wr, 748 op/s 2024-05-14T16:54:24.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:23 smithi203 ceph-mon[35787]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.9 MiB/s wr, 748 op/s 2024-05-14T16:54:24.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:23 smithi092 ceph-mon[35763]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.9 MiB/s wr, 748 op/s 2024-05-14T16:54:25.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.124+0000 7f4105570700 1 -- 172.21.15.69:0/1667452645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f410010eed0 msgr2=0x7f4100116aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:25.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.124+0000 7f4105570700 1 --2- 172.21.15.69:0/1667452645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f410010eed0 0x7f4100116aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009920 tx=0x7f40f4009c30 comp rx=0 tx=0).stop 2024-05-14T16:54:25.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.129+0000 7f4105570700 1 -- 172.21.15.69:0/1667452645 shutdown_connections 2024-05-14T16:54:25.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.129+0000 7f4105570700 1 --2- 172.21.15.69:0/1667452645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4100109df0 0x7f410010a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.129+0000 7f4105570700 1 --2- 172.21.15.69:0/1667452645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f410010eed0 0x7f4100116aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.129+0000 7f4105570700 1 --2- 172.21.15.69:0/1667452645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f410010a7a0 0x7f410010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.129+0000 7f4105570700 1 -- 172.21.15.69:0/1667452645 >> 172.21.15.69:0/1667452645 conn(0x7f4100076d50 msgr2=0x7f4100108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:25.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.131+0000 7f4105570700 1 -- 172.21.15.69:0/1667452645 shutdown_connections 2024-05-14T16:54:25.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.133+0000 7f4105570700 1 -- 172.21.15.69:0/1667452645 wait complete. 2024-05-14T16:54:25.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.134+0000 7f4105570700 1 Processor -- start 2024-05-14T16:54:25.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.135+0000 7f4105570700 1 -- start start 2024-05-14T16:54:25.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.135+0000 7f4105570700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4100109df0 0x7f41001a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.136+0000 7f4105570700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f410010a7a0 0x7f41001a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.136+0000 7f4105570700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f410010eed0 0x7f41001a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.136+0000 7f4105570700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4100119db0 con 0x7f410010a7a0 2024-05-14T16:54:25.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.136+0000 7f4105570700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4100119c30 con 0x7f4100109df0 2024-05-14T16:54:25.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.136+0000 7f4105570700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4100119f30 con 0x7f410010eed0 2024-05-14T16:54:25.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.137+0000 7f40feffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4100109df0 0x7f41001a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.137+0000 7f40ff7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f410010eed0 0x7f41001a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.137+0000 7f40feffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4100109df0 0x7f41001a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:35354/0 (socket says 172.21.15.69:35354) 2024-05-14T16:54:25.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.137+0000 7f40feffd700 1 -- 172.21.15.69:0/1677073291 learned_addr learned my addr 172.21.15.69:0/1677073291 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:25.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.137+0000 7f40ff7fe700 1 -- 172.21.15.69:0/1677073291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4100109df0 msgr2=0x7f41001a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:25.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.137+0000 7f40ff7fe700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4100109df0 0x7f41001a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.137+0000 7f40ff7fe700 1 -- 172.21.15.69:0/1677073291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f410010a7a0 msgr2=0x7f41001a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:25.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.137+0000 7f40ff7fe700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f410010a7a0 0x7f41001a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.137+0000 7f40ff7fe700 1 -- 172.21.15.69:0/1677073291 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41001a5280 con 0x7f410010eed0 2024-05-14T16:54:25.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.139+0000 7f40ff7fe700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f410010eed0 0x7f41001a4b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009230 tx=0x7f40f40052e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:25.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.139+0000 7f40e7fff700 1 -- 172.21.15.69:0/1677073291 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f400b760 con 0x7f410010eed0 2024-05-14T16:54:25.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.139+0000 7f4105570700 1 -- 172.21.15.69:0/1677073291 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41001a5510 con 0x7f410010eed0 2024-05-14T16:54:25.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.139+0000 7f4105570700 1 -- 172.21.15.69:0/1677073291 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41001acdf0 con 0x7f410010eed0 2024-05-14T16:54:25.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.140+0000 7f40e7fff700 1 -- 172.21.15.69:0/1677073291 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f4003680 con 0x7f410010eed0 2024-05-14T16:54:25.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.140+0000 7f40e7fff700 1 -- 172.21.15.69:0/1677073291 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40216a0 con 0x7f410010eed0 2024-05-14T16:54:25.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.145+0000 7f40e7fff700 1 -- 172.21.15.69:0/1677073291 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f40f400b900 con 0x7f410010eed0 2024-05-14T16:54:25.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.145+0000 7f410fe06b80 1 Processor -- start 2024-05-14T16:54:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.145+0000 7f410fe06b80 1 -- start start 2024-05-14T16:54:25.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.145+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652e4b0 0x55ccc652e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652edd0 0x55ccc6529d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 0x55ccc652a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f410fe06b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ccc6487f00 con 0x55ccc652edd0 2024-05-14T16:54:25.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f410fe06b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ccc6477aa0 con 0x55ccc652a350 2024-05-14T16:54:25.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f410fe06b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ccc647bae0 con 0x55ccc652e4b0 2024-05-14T16:54:25.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f40fe7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652e4b0 0x55ccc652e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f40fe7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652e4b0 0x55ccc652e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:34622/0 (socket says 172.21.15.69:34622) 2024-05-14T16:54:25.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f40fe7fc700 1 -- 172.21.15.69:0/2332164906 learned_addr learned my addr 172.21.15.69:0/2332164906 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f40feffd700 1 --2- 172.21.15.69:0/2332164906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652edd0 0x55ccc6529d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/2332164906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 0x55ccc652a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f40fe7fc700 1 -- 172.21.15.69:0/2332164906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 msgr2=0x55ccc652a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/2332164906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 0x55ccc652a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f40fe7fc700 1 -- 172.21.15.69:0/2332164906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652edd0 msgr2=0x55ccc6529d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/2332164906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652edd0 0x55ccc6529d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.146+0000 7f40fe7fc700 1 -- 172.21.15.69:0/2332164906 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc6421140 con 0x55ccc652e4b0 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.147+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/2332164906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652e4b0 0x55ccc652e890 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f40f0012790 tx=0x7f40f0009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4448a6aaea4ab96c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.147+0000 7f40e57fa700 1 -- 172.21.15.69:0/2332164906 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0004a30 con 0x55ccc652e4b0 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.147+0000 7f40e57fa700 1 -- 172.21.15.69:0/2332164906 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f00054d0 con 0x55ccc652e4b0 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.147+0000 7f40e57fa700 1 -- 172.21.15.69:0/2332164906 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f000f6c0 con 0x55ccc652e4b0 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.147+0000 7f40e7fff700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40e80b2bf0 0x7f40e80b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f40feffd700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40e80b2bf0 0x7f40e80b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f40e7fff700 1 -- 172.21.15.69:0/1677073291 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f40f40e3470 con 0x7f410010eed0 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f410fe06b80 1 -- 172.21.15.69:0/2332164906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652e4b0 msgr2=0x55ccc652e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f410fe06b80 1 --2- 172.21.15.69:0/2332164906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652e4b0 0x55ccc652e890 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f40f0012790 tx=0x7f40f0009740 comp rx=0 tx=0).stop 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f410fe06b80 1 -- 172.21.15.69:0/2332164906 shutdown_connections 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f410fe06b80 1 --2- 172.21.15.69:0/2332164906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 0x55ccc652a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f410fe06b80 1 --2- 172.21.15.69:0/2332164906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652e4b0 0x55ccc652e890 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f410fe06b80 1 --2- 172.21.15.69:0/2332164906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652edd0 0x55ccc6529d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f410fe06b80 1 -- 172.21.15.69:0/2332164906 >> 172.21.15.69:0/2332164906 conn(0x55ccc6456f40 msgr2=0x55ccc6524530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f40feffd700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40e80b2bf0 0x7f40e80b50b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55ccc65250f0 tx=0x7f40f800b040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f410fe06b80 1 -- 172.21.15.69:0/2332164906 shutdown_connections 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f410fe06b80 1 -- 172.21.15.69:0/2332164906 wait complete. 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.148+0000 7f410fe06b80 1 -- start start 2024-05-14T16:54:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f410fe06b80 1 Processor -- start 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 0x55ccc6535ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652e4b0 0x55ccc6532e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc65333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f410fe06b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ccc637e130 con 0x55ccc652e4b0 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f410fe06b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ccc63dff30 con 0x55ccc652a350 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f410fe06b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ccc6536950 con 0x55ccc652edd0 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f410fe06b80 10 client.0 _ll_register_callbacks cb 0x55ccc6412ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40feffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 0x55ccc6535ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40feffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 0x55ccc6535ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:35386/0 (socket says 172.21.15.69:35386) 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40feffd700 1 -- 172.21.15.69:0/4173126404 learned_addr learned my addr 172.21.15.69:0/4173126404 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652e4b0 0x55ccc6532e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40ff7fe700 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc65333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40feffd700 1 -- 172.21.15.69:0/4173126404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 msgr2=0x55ccc65333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40feffd700 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc65333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40feffd700 1 -- 172.21.15.69:0/4173126404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652e4b0 msgr2=0x55ccc6532e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40feffd700 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652e4b0 0x55ccc6532e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40feffd700 1 -- 172.21.15.69:0/4173126404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc63c7a60 con 0x55ccc652a350 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.149+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652e4b0 0x55ccc6532e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f40feffd700 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 0x55ccc6535ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f40f80125f0 tx=0x7f40f8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c71b862ca3b08fb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f40e2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f40e2ff5700 1 -- 172.21.15.69:0/4173126404 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8027070 con 0x55ccc652a350 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f40e2ff5700 1 -- 172.21.15.69:0/4173126404 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f40f8004db0 con 0x55ccc652a350 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f410fe06b80 1 -- 172.21.15.69:0/4173126404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f00054d0 con 0x55ccc652a350 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f40e2ff5700 1 -- 172.21.15.69:0/4173126404 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f40f8032050 con 0x55ccc652a350 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f410fe06b80 10 client.25370 fetch_fsmap learned FSMap version 35 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f410fe06b80 1 -- 172.21.15.69:0/4173126404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ccc64318f0 con 0x55ccc652a350 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f40e2ff5700 1 -- 172.21.15.69:0/4173126404 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f00054d0 con 0x55ccc652a350 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f40e2ff5700 1 -- 172.21.15.69:0/4173126404 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f40f80147d0 con 0x55ccc652a350 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f410fe06b80 10 client.25370 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f410fe06b80 10 client.25370 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:54:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f410fe06b80 20 client.25370 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f410fe06b80 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ccc65436b0 0x55ccc6545b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f410fe06b80 4 client.25370 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.150+0000 7f410fe06b80 1 -- 172.21.15.69:0/4173126404 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccc6489e50 con 0x55ccc65436b0 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.151+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ccc65436b0 0x55ccc6545b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.151+0000 7f40e2ff5700 10 client.25370 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.151+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ccc65436b0 0x55ccc6545b30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2249de373611075e server_cookie=fd1344bc03c1e812 in_seq=0 out_seq=0 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f40e2ff5700 1 -- 172.21.15.69:0/4173126404 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ccc6489e50 con 0x55ccc65436b0 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f40e2ff5700 10 client.25370 handle_command_reply: tid=0 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 client.25370 shutdown 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 -- 172.21.15.69:0/4173126404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 msgr2=0x55ccc6535ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 0x55ccc6535ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f40f80125f0 tx=0x7f40f8004b40 comp rx=0 tx=0).stop 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 -- 172.21.15.69:0/4173126404 shutdown_connections 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ccc65436b0 0x55ccc6545b30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc65333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f40e2ff5700 0 client.25370 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:25.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652a350 0x55ccc6535ac0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 --2- 172.21.15.69:0/4173126404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652e4b0 0x55ccc6532e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 -- 172.21.15.69:0/4173126404 >> 172.21.15.69:0/4173126404 conn(0x55ccc6456f40 msgr2=0x55ccc651f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:25.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 -- 172.21.15.69:0/4173126404 shutdown_connections 2024-05-14T16:54:25.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 1 -- 172.21.15.69:0/4173126404 wait complete. 2024-05-14T16:54:25.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.152+0000 7f410fe06b80 20 client.25370 trim_cache size 0 max 16384 2024-05-14T16:54:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f410fe06b80 1 Processor -- start 2024-05-14T16:54:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f410fe06b80 1 -- start start 2024-05-14T16:54:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 0x55ccc65295b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 0x55ccc651f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc652d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f410fe06b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec00fed0 con 0x55ccc652a350 2024-05-14T16:54:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f40fe7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 0x55ccc651f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f40fe7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 0x55ccc651f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:35402/0 (socket says 172.21.15.69:35402) 2024-05-14T16:54:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f40fe7fc700 1 -- 172.21.15.69:0/3315306207 learned_addr learned my addr 172.21.15.69:0/3315306207 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f40feffd700 1 --2- 172.21.15.69:0/3315306207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 0x55ccc65295b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.178+0000 7f410fe06b80 1 -- 172.21.15.69:0/3315306207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec00d6d0 con 0x55ccc652e4b0 2024-05-14T16:54:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.179+0000 7f410fe06b80 1 -- 172.21.15.69:0/3315306207 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec00b520 con 0x55ccc652edd0 2024-05-14T16:54:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.179+0000 7f40ff7fe700 1 --2- 172.21.15.69:0/3315306207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc652d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.179+0000 7f40fe7fc700 1 -- 172.21.15.69:0/3315306207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 msgr2=0x55ccc652d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.179+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/3315306207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc652d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.179+0000 7f40fe7fc700 1 -- 172.21.15.69:0/3315306207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 msgr2=0x55ccc65295b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.179+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/3315306207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 0x55ccc65295b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.179+0000 7f40fe7fc700 1 -- 172.21.15.69:0/3315306207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc6545e90 con 0x55ccc652e4b0 2024-05-14T16:54:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.180+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/3315306207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 0x55ccc651f160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f40f0004d70 tx=0x7f40f001f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc34a676989dbf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.180+0000 7f40e4ff9700 1 -- 172.21.15.69:0/3315306207 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0004b40 con 0x55ccc652e4b0 2024-05-14T16:54:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.180+0000 7f40e4ff9700 1 -- 172.21.15.69:0/3315306207 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f0030020 con 0x55ccc652e4b0 2024-05-14T16:54:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.180+0000 7f40e4ff9700 1 -- 172.21.15.69:0/3315306207 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f002b8e0 con 0x55ccc652e4b0 2024-05-14T16:54:25.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.181+0000 7f410fe06b80 1 -- 172.21.15.69:0/3315306207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 msgr2=0x55ccc651f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:25.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.181+0000 7f410fe06b80 1 --2- 172.21.15.69:0/3315306207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 0x55ccc651f160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f40f0004d70 tx=0x7f40f001f040 comp rx=0 tx=0).stop 2024-05-14T16:54:25.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.181+0000 7f410fe06b80 1 -- 172.21.15.69:0/3315306207 shutdown_connections 2024-05-14T16:54:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.181+0000 7f410fe06b80 1 --2- 172.21.15.69:0/3315306207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 0x55ccc65295b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.181+0000 7f410fe06b80 1 --2- 172.21.15.69:0/3315306207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc652d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.181+0000 7f410fe06b80 1 --2- 172.21.15.69:0/3315306207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 0x55ccc651f160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.181+0000 7f410fe06b80 1 -- 172.21.15.69:0/3315306207 >> 172.21.15.69:0/3315306207 conn(0x55ccc6456f40 msgr2=0x55ccc652d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.182+0000 7f410fe06b80 1 -- 172.21.15.69:0/3315306207 shutdown_connections 2024-05-14T16:54:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.182+0000 7f410fe06b80 1 -- 172.21.15.69:0/3315306207 wait complete. 2024-05-14T16:54:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.182+0000 7f410fe06b80 1 -- start start 2024-05-14T16:54:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f410fe06b80 1 Processor -- start 2024-05-14T16:54:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 0x55ccc653bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 0x55ccc653c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f410fe06b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc6538c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f410fe06b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec00bb90 con 0x55ccc652a350 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f40feffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 0x55ccc653bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f410fe06b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec00d000 con 0x55ccc652e4b0 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f40feffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 0x55ccc653bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57032/0 (socket says 172.21.15.69:57032) 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f40feffd700 1 -- 172.21.15.69:0/3892327833 learned_addr learned my addr 172.21.15.69:0/3892327833 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f40ff7fe700 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc6538c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f40fe7fc700 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 0x55ccc653c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f410fe06b80 1 -- 172.21.15.69:0/3892327833 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec00c260 con 0x55ccc652edd0 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.183+0000 7f410fe06b80 10 client.0 _ll_register_callbacks cb 0x55ccc6412ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f40ff7fe700 1 -- 172.21.15.69:0/3892327833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 msgr2=0x55ccc653c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f40ff7fe700 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 0x55ccc653c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f40ff7fe700 1 -- 172.21.15.69:0/3892327833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 msgr2=0x55ccc653bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f40ff7fe700 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 0x55ccc653bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f40ff7fe700 1 -- 172.21.15.69:0/3892327833 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc6539430 con 0x55ccc652edd0 2024-05-14T16:54:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f40ff7fe700 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc6538c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f40f40e0b70 tx=0x7f40f40e1070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32285ca5a7142f16 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f40e2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f40e2ff5700 1 -- 172.21.15.69:0/3892327833 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40c4060 con 0x55ccc652edd0 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f40e2ff5700 1 -- 172.21.15.69:0/3892327833 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f40f40922f0 con 0x55ccc652edd0 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f410fe06b80 1 -- 172.21.15.69:0/3892327833 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f0030020 con 0x55ccc652edd0 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.184+0000 7f40e2ff5700 1 -- 172.21.15.69:0/3892327833 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f40f40aa560 con 0x55ccc652edd0 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f410fe06b80 10 client.25327 fetch_fsmap learned FSMap version 35 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f410fe06b80 1 -- 172.21.15.69:0/3892327833 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ccc65bfaf0 con 0x55ccc652edd0 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f40e2ff5700 1 -- 172.21.15.69:0/3892327833 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f4084e50 con 0x55ccc652edd0 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f40e2ff5700 1 -- 172.21.15.69:0/3892327833 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f40f4029030 con 0x55ccc652edd0 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f410fe06b80 10 client.25327 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f410fe06b80 10 client.25327 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f410fe06b80 20 client.25327 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f410fe06b80 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ccc65436b0 0x55ccc65c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f410fe06b80 4 client.25327 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}] 2024-05-14T16:54:25.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f410fe06b80 1 -- 172.21.15.69:0/3892327833 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}) -- 0x55ccc6483630 con 0x55ccc65436b0 2024-05-14T16:54:25.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.185+0000 7f40feffd700 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ccc65436b0 0x55ccc65c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:25.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.186+0000 7f40feffd700 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ccc65436b0 0x55ccc65c2120 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f0d9ce451404a2 server_cookie=6d1364dff43b80e1 in_seq=0 out_seq=0 2024-05-14T16:54:25.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.186+0000 7f40e2ff5700 10 client.25327 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:25.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 235.0 s, 2879.3 tps, lat 33.200 ms stddev 34.767 2024-05-14T16:54:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.429+0000 7f40e2ff5700 1 -- 172.21.15.69:0/3892327833 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ccc6483630 con 0x55ccc65436b0 2024-05-14T16:54:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.429+0000 7f40e2ff5700 10 client.25327 handle_command_reply: tid=0 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.429+0000 7f410fe06b80 1 client.25327 shutdown 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.429+0000 7f410fe06b80 1 -- 172.21.15.69:0/3892327833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 msgr2=0x55ccc6538c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f410fe06b80 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc6538c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f40f40e0b70 tx=0x7f40f40e1070 comp rx=0 tx=0).stop 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f410fe06b80 1 -- 172.21.15.69:0/3892327833 shutdown_connections 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f410fe06b80 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ccc65436b0 0x55ccc65c2120 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f410fe06b80 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ccc652a350 0x55ccc653bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f40e2ff5700 0 client.25327 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f410fe06b80 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ccc652edd0 0x55ccc6538c40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f410fe06b80 1 --2- 172.21.15.69:0/3892327833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccc652e4b0 0x55ccc653c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f410fe06b80 1 -- 172.21.15.69:0/3892327833 >> 172.21.15.69:0/3892327833 conn(0x55ccc6456f40 msgr2=0x55ccc652da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f410fe06b80 1 -- 172.21.15.69:0/3892327833 shutdown_connections 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f410fe06b80 1 -- 172.21.15.69:0/3892327833 wait complete. 2024-05-14T16:54:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f410fe06b80 20 client.25327 trim_cache size 0 max 16384 2024-05-14T16:54:25.431 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:54:25.431 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 35, 2024-05-14T16:54:25.431 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:54:25.431 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 14, 2024-05-14T16:54:25.431 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:54:25.431 INFO:teuthology.orchestra.run.smithi069.stdout: "b13943b": { 2024-05-14T16:54:25.431 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 14, 2024-05-14T16:54:25.431 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:54:25.431 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:54:25.431 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 146.8, 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:54:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:54:25.433 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:54:25.433 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:54:25.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f4105570700 1 -- 172.21.15.69:0/1677073291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40e80b2bf0 msgr2=0x7f40e80b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:25.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f4105570700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40e80b2bf0 0x7f40e80b50b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55ccc65250f0 tx=0x7f40f800b040 comp rx=0 tx=0).stop 2024-05-14T16:54:25.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f4105570700 1 -- 172.21.15.69:0/1677073291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f410010eed0 msgr2=0x7f41001a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:25.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f4105570700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f410010eed0 0x7f41001a4b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009230 tx=0x7f40f40052e0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f4105570700 1 -- 172.21.15.69:0/1677073291 shutdown_connections 2024-05-14T16:54:25.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f4105570700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f410010eed0 0x7f41001a4b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f4105570700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4100109df0 0x7f41001a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f4105570700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40e80b2bf0 0x7f40e80b50b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f4105570700 1 --2- 172.21.15.69:0/1677073291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f410010a7a0 0x7f41001a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.430+0000 7f4105570700 1 -- 172.21.15.69:0/1677073291 >> 172.21.15.69:0/1677073291 conn(0x7f4100076d50 msgr2=0x7f4100077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.431+0000 7f4105570700 1 -- 172.21.15.69:0/1677073291 shutdown_connections 2024-05-14T16:54:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:25.431+0000 7f4105570700 1 -- 172.21.15.69:0/1677073291 wait complete. 2024-05-14T16:54:26.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:25 smithi069 ceph-mon[31507]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.9 MiB/s wr, 748 op/s 2024-05-14T16:54:26.109 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b13943b', quiesce took 6.9 seconds. Will release after: 19.9 2024-05-14T16:54:26.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:25 smithi203 ceph-mon[35787]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.9 MiB/s wr, 748 op/s 2024-05-14T16:54:26.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:25 smithi092 ceph-mon[35763]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.9 MiB/s wr, 748 op/s 2024-05-14T16:54:28.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:27 smithi069 ceph-mon[31507]: pgmap v141: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-05-14T16:54:28.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:27 smithi203 ceph-mon[35787]: pgmap v141: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-05-14T16:54:28.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:27 smithi092 ceph-mon[35763]: pgmap v141: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-05-14T16:54:30.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:29 smithi069 ceph-mon[31507]: pgmap v142: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.3 MiB/s wr, 588 op/s 2024-05-14T16:54:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:29 smithi203 ceph-mon[35787]: pgmap v142: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.3 MiB/s wr, 588 op/s 2024-05-14T16:54:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:29 smithi092 ceph-mon[35763]: pgmap v142: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.3 MiB/s wr, 588 op/s 2024-05-14T16:54:30.395 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 240.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:54:32.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:31 smithi069 ceph-mon[31507]: pgmap v143: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.3 MiB/s wr, 592 op/s 2024-05-14T16:54:32.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:31 smithi203 ceph-mon[35787]: pgmap v143: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.3 MiB/s wr, 592 op/s 2024-05-14T16:54:32.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:31 smithi092 ceph-mon[35763]: pgmap v143: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.3 MiB/s wr, 592 op/s 2024-05-14T16:54:34.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:33 smithi069 ceph-mon[31507]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2024-05-14T16:54:34.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:33 smithi203 ceph-mon[35787]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2024-05-14T16:54:34.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:33 smithi092 ceph-mon[35763]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2024-05-14T16:54:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.803+0000 7fa4b5714640 1 -- 172.21.15.69:0/4058534570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4b0102a30 msgr2=0x7fa4b0102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.803+0000 7fa4b5714640 1 --2- 172.21.15.69:0/4058534570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4b0102a30 0x7fa4b0102eb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009930 tx=0x7fa4a402a420 comp rx=0 tx=0).stop 2024-05-14T16:54:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.803+0000 7fa4b5714640 1 -- 172.21.15.69:0/4058534570 shutdown_connections 2024-05-14T16:54:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.803+0000 7fa4b5714640 1 --2- 172.21.15.69:0/4058534570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b01033f0 0x7fa4b010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.803+0000 7fa4b5714640 1 --2- 172.21.15.69:0/4058534570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4b0102a30 0x7fa4b0102eb0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.803+0000 7fa4b5714640 1 --2- 172.21.15.69:0/4058534570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b0108a40 0x7fa4b0108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.803+0000 7fa4b5714640 1 -- 172.21.15.69:0/4058534570 >> 172.21.15.69:0/4058534570 conn(0x7fa4b00fe500 msgr2=0x7fa4b0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.803+0000 7fa4b5714640 1 -- 172.21.15.69:0/4058534570 shutdown_connections 2024-05-14T16:54:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.803+0000 7fa4b5714640 1 -- 172.21.15.69:0/4058534570 wait complete. 2024-05-14T16:54:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.804+0000 7fa4b5714640 1 Processor -- start 2024-05-14T16:54:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.804+0000 7fa4b5714640 1 -- start start 2024-05-14T16:54:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.804+0000 7fa4b5714640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b0102a30 0x7fa4b0199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.804+0000 7fa4b5714640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b01033f0 0x7fa4b0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.804+0000 7fa4b5714640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4b0108a40 0x7fa4b019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.804+0000 7fa4b5714640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0112d00 con 0x7fa4b0108a40 2024-05-14T16:54:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.804+0000 7fa4b5714640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0112b80 con 0x7fa4b01033f0 2024-05-14T16:54:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.804+0000 7fa4b5714640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0112e80 con 0x7fa4b0102a30 2024-05-14T16:54:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4af7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4b0108a40 0x7fa4b019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4af7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4b0108a40 0x7fa4b019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45430/0 (socket says 172.21.15.69:45430) 2024-05-14T16:54:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4af7fe640 1 -- 172.21.15.69:0/676125525 learned_addr learned my addr 172.21.15.69:0/676125525 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b0102a30 0x7fa4b0199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b01033f0 0x7fa4b0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4af7fe640 1 -- 172.21.15.69:0/676125525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b0102a30 msgr2=0x7fa4b0199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4af7fe640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b0102a30 0x7fa4b0199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4af7fe640 1 -- 172.21.15.69:0/676125525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b01033f0 msgr2=0x7fa4b0199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4af7fe640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b01033f0 0x7fa4b0199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4af7fe640 1 -- 172.21.15.69:0/676125525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b019e210 con 0x7fa4b0108a40 2024-05-14T16:54:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b01033f0 0x7fa4b0199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4af7fe640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4b0108a40 0x7fa4b019dae0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800eb10 tx=0x7fa4a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa497fff640 1 -- 172.21.15.69:0/676125525 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8004b20 con 0x7fa4b0108a40 2024-05-14T16:54:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa497fff640 1 -- 172.21.15.69:0/676125525 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a8004cc0 con 0x7fa4b0108a40 2024-05-14T16:54:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4b5714640 1 -- 172.21.15.69:0/676125525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b019e500 con 0x7fa4b0108a40 2024-05-14T16:54:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa497fff640 1 -- 172.21.15.69:0/676125525 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8010670 con 0x7fa4b0108a40 2024-05-14T16:54:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.805+0000 7fa4b5714640 1 -- 172.21.15.69:0/676125525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b01a5d90 con 0x7fa4b0108a40 2024-05-14T16:54:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa497fff640 1 -- 172.21.15.69:0/676125525 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa4a8005020 con 0x7fa4b0108a40 2024-05-14T16:54:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4b842d740 1 Processor -- start 2024-05-14T16:54:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4b842d740 1 -- start start 2024-05-14T16:54:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b4018f0 0x563e5b401cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b402210 0x563e5b40c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b40eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4b842d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563e5b374c30 con 0x563e5b40cad0 2024-05-14T16:54:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4b842d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563e5b3647d0 con 0x563e5b402210 2024-05-14T16:54:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4b842d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563e5b368810 con 0x563e5b4018f0 2024-05-14T16:54:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4ae7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b40eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4ae7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b40eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45432/0 (socket says 172.21.15.69:45432) 2024-05-14T16:54:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4ae7fc640 1 -- 172.21.15.69:0/418603188 learned_addr learned my addr 172.21.15.69:0/418603188 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/418603188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b402210 0x563e5b40c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.808+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/418603188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b4018f0 0x563e5b401cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa4ae7fc640 1 -- 172.21.15.69:0/418603188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b4018f0 msgr2=0x563e5b401cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/418603188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b4018f0 0x563e5b401cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa4ae7fc640 1 -- 172.21.15.69:0/418603188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b402210 msgr2=0x563e5b40c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/418603188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b402210 0x563e5b40c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa4ae7fc640 1 -- 172.21.15.69:0/418603188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5b079210 con 0x563e5b40cad0 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa497fff640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa4980b2a10 0x7fa4980b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa4980b2a10 0x7fa4980b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/418603188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b40eec0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4033000 tx=0x7fa4a4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1f8d8a468f3539 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa497fff640 1 -- 172.21.15.69:0/676125525 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa4a80e2c20 con 0x7fa4b0108a40 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa4957fa640 1 -- 172.21.15.69:0/418603188 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4035040 con 0x563e5b40cad0 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa4957fa640 1 -- 172.21.15.69:0/418603188 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a4045070 con 0x563e5b40cad0 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.809+0000 7fa4957fa640 1 -- 172.21.15.69:0/418603188 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40039e0 con 0x563e5b40cad0 2024-05-14T16:54:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 -- 172.21.15.69:0/418603188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 msgr2=0x563e5b40eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa4980b2a10 0x7fa4980b4ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x563e5b402e90 tx=0x7fa4a0005f90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 --2- 172.21.15.69:0/418603188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b40eec0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4033000 tx=0x7fa4a4031400 comp rx=0 tx=0).stop 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 -- 172.21.15.69:0/418603188 shutdown_connections 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 --2- 172.21.15.69:0/418603188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b40eec0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 --2- 172.21.15.69:0/418603188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b402210 0x563e5b40c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 --2- 172.21.15.69:0/418603188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b4018f0 0x563e5b401cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 -- 172.21.15.69:0/418603188 >> 172.21.15.69:0/418603188 conn(0x563e5b3fde00 msgr2=0x563e5b3fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 -- 172.21.15.69:0/418603188 shutdown_connections 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 -- 172.21.15.69:0/418603188 wait complete. 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 -- start start 2024-05-14T16:54:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 Processor -- start 2024-05-14T16:54:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.810+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 0x563e5b40c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b402210 0x563e5b409730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b409c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4b842d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563e5b04a790 con 0x563e5b40cad0 2024-05-14T16:54:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4b842d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563e5b320890 con 0x563e5b4018f0 2024-05-14T16:54:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4b842d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563e5b40f740 con 0x563e5b402210 2024-05-14T16:54:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4b842d740 10 client.0 _ll_register_callbacks cb 0x563e5b323d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4af7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b409c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4af7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b409c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45444/0 (socket says 172.21.15.69:45444) 2024-05-14T16:54:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4af7fe640 1 -- 172.21.15.69:0/2543812160 learned_addr learned my addr 172.21.15.69:0/2543812160 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 0x563e5b40c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b402210 0x563e5b409730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4af7fe640 1 -- 172.21.15.69:0/2543812160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b402210 msgr2=0x563e5b409730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4af7fe640 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b402210 0x563e5b409730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4af7fe640 1 -- 172.21.15.69:0/2543812160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 msgr2=0x563e5b40c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4af7fe640 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 0x563e5b40c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4af7fe640 1 -- 172.21.15.69:0/2543812160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5b145620 con 0x563e5b40cad0 2024-05-14T16:54:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4af7fe640 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b409c90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80a5f90 tx=0x7fa4a80a9990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e7eb2b5864b85d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa492ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:54:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa492ff5640 1 -- 172.21.15.69:0/2543812160 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80aa610 con 0x563e5b40cad0 2024-05-14T16:54:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa492ff5640 1 -- 172.21.15.69:0/2543812160 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa4a80aa380 con 0x563e5b40cad0 2024-05-14T16:54:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4b842d740 1 -- 172.21.15.69:0/2543812160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a4045070 con 0x563e5b40cad0 2024-05-14T16:54:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa492ff5640 1 -- 172.21.15.69:0/2543812160 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa4a80a9f10 con 0x563e5b40cad0 2024-05-14T16:54:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4b842d740 10 client.15867 fetch_fsmap learned FSMap version 35 2024-05-14T16:54:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.811+0000 7fa4b842d740 1 -- 172.21.15.69:0/2543812160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563e5b41b1d0 con 0x563e5b40cad0 2024-05-14T16:54:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa492ff5640 1 -- 172.21.15.69:0/2543812160 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a4045070 con 0x563e5b40cad0 2024-05-14T16:54:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa492ff5640 1 -- 172.21.15.69:0/2543812160 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fa4a80a5880 con 0x563e5b40cad0 2024-05-14T16:54:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa4b842d740 10 client.15867 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:54:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa4b842d740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:54:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa4b842d740 20 client.15867 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa4b842d740 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563e5b41c690 0x563e5b41eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa4b842d740 4 client.15867 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:54:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa4b842d740 1 -- 172.21.15.69:0/2543812160 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e5b376b80 con 0x563e5b41c690 2024-05-14T16:54:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563e5b41c690 0x563e5b41eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa492ff5640 10 client.15867 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.812+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563e5b41c690 0x563e5b41eaa0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55760d541cf43b3 server_cookie=9d8d42a953f46ea6 in_seq=0 out_seq=0 2024-05-14T16:54:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa492ff5640 1 -- 172.21.15.69:0/2543812160 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e5b376b80 con 0x563e5b41c690 2024-05-14T16:54:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa492ff5640 10 client.15867 handle_command_reply: tid=0 2024-05-14T16:54:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 client.15867 shutdown 2024-05-14T16:54:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 -- 172.21.15.69:0/2543812160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 msgr2=0x563e5b409c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b409c90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80a5f90 tx=0x7fa4a80a9990 comp rx=0 tx=0).stop 2024-05-14T16:54:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 -- 172.21.15.69:0/2543812160 shutdown_connections 2024-05-14T16:54:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563e5b41c690 0x563e5b41eaa0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b40cad0 0x563e5b409c90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b402210 0x563e5b409730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa492ff5640 0 client.15867 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 --2- 172.21.15.69:0/2543812160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 0x563e5b40c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 -- 172.21.15.69:0/2543812160 >> 172.21.15.69:0/2543812160 conn(0x563e5b3fde00 msgr2=0x563e5b3fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 -- 172.21.15.69:0/2543812160 shutdown_connections 2024-05-14T16:54:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.813+0000 7fa4b842d740 1 -- 172.21.15.69:0/2543812160 wait complete. 2024-05-14T16:54:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.814+0000 7fa4b842d740 20 client.15867 trim_cache size 0 max 16384 2024-05-14T16:54:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.828+0000 7fa4b842d740 1 Processor -- start 2024-05-14T16:54:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.828+0000 7fa4b842d740 1 -- start start 2024-05-14T16:54:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 0x563e5b418640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b402210 0x563e5b40bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b40cad0 0x563e5b424370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4b842d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01b0fe0 con 0x563e5b402210 2024-05-14T16:54:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4b842d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01ae820 con 0x563e5b4018f0 2024-05-14T16:54:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4b842d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01ac670 con 0x563e5b40cad0 2024-05-14T16:54:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4ae7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b402210 0x563e5b40bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4ae7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b402210 0x563e5b40bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45448/0 (socket says 172.21.15.69:45448) 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4ae7fc640 1 -- 172.21.15.69:0/2682706960 learned_addr learned my addr 172.21.15.69:0/2682706960 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4af7fe640 1 --2- 172.21.15.69:0/2682706960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b40cad0 0x563e5b424370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/2682706960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 0x563e5b418640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4ae7fc640 1 -- 172.21.15.69:0/2682706960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b40cad0 msgr2=0x563e5b424370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/2682706960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b40cad0 0x563e5b424370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4ae7fc640 1 -- 172.21.15.69:0/2682706960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 msgr2=0x563e5b418640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/2682706960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 0x563e5b418640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4ae7fc640 1 -- 172.21.15.69:0/2682706960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5b3fdc30 con 0x563e5b402210 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/2682706960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 0x563e5b418640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/2682706960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b402210 0x563e5b40bf70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa4a402a460 tx=0x7fa4a4032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814b009f4101db20 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa494ff9640 1 -- 172.21.15.69:0/2682706960 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a404d020 con 0x563e5b402210 2024-05-14T16:54:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa494ff9640 1 -- 172.21.15.69:0/2682706960 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a4045050 con 0x563e5b402210 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.829+0000 7fa494ff9640 1 -- 172.21.15.69:0/2682706960 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4035040 con 0x563e5b402210 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 -- 172.21.15.69:0/2682706960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b402210 msgr2=0x563e5b40bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 --2- 172.21.15.69:0/2682706960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b402210 0x563e5b40bf70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa4a402a460 tx=0x7fa4a4032f30 comp rx=0 tx=0).stop 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 -- 172.21.15.69:0/2682706960 shutdown_connections 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 --2- 172.21.15.69:0/2682706960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b40cad0 0x563e5b424370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 --2- 172.21.15.69:0/2682706960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b402210 0x563e5b40bf70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 --2- 172.21.15.69:0/2682706960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b4018f0 0x563e5b418640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 -- 172.21.15.69:0/2682706960 >> 172.21.15.69:0/2682706960 conn(0x563e5b3fde00 msgr2=0x563e5b403b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 -- 172.21.15.69:0/2682706960 shutdown_connections 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 -- 172.21.15.69:0/2682706960 wait complete. 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 -- start start 2024-05-14T16:54:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.830+0000 7fa4b842d740 1 Processor -- start 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b4018f0 0x563e5b410800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b402210 0x563e5b410d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4b842d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b40cad0 0x563e5b411280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4b842d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01acce0 con 0x563e5b4018f0 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4b842d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01ae150 con 0x563e5b40cad0 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4b842d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01ad3b0 con 0x563e5b402210 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4b842d740 10 client.0 _ll_register_callbacks cb 0x7fa4a80a5e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4aeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b4018f0 0x563e5b410800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4aeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b4018f0 0x563e5b410800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45456/0 (socket says 172.21.15.69:45456) 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4aeffd640 1 -- 172.21.15.69:0/616722109 learned_addr learned my addr 172.21.15.69:0/616722109 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4af7fe640 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b40cad0 0x563e5b411280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b402210 0x563e5b410d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4aeffd640 1 -- 172.21.15.69:0/616722109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b402210 msgr2=0x563e5b410d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b402210 0x563e5b410d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4aeffd640 1 -- 172.21.15.69:0/616722109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b40cad0 msgr2=0x563e5b411280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b40cad0 0x563e5b411280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4aeffd640 1 -- 172.21.15.69:0/616722109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5b402cc0 con 0x563e5b4018f0 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4af7fe640 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b40cad0 0x563e5b411280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4aeffd640 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b4018f0 0x563e5b410800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009fa0 tx=0x7fa4a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e5dc2cea207959 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa492ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa492ff5640 1 -- 172.21.15.69:0/616722109 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a002f070 con 0x563e5b4018f0 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa492ff5640 1 -- 172.21.15.69:0/616722109 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa4a002b630 con 0x563e5b4018f0 2024-05-14T16:54:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.831+0000 7fa4b842d740 1 -- 172.21.15.69:0/616722109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a4045050 con 0x563e5b4018f0 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa492ff5640 1 -- 172.21.15.69:0/616722109 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa4a0038050 con 0x563e5b4018f0 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa4b842d740 10 client.15873 fetch_fsmap learned FSMap version 35 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa4b842d740 1 -- 172.21.15.69:0/616722109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563e5b4872d0 con 0x563e5b4018f0 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa492ff5640 1 -- 172.21.15.69:0/616722109 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a4045050 con 0x563e5b4018f0 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa492ff5640 1 -- 172.21.15.69:0/616722109 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fa4a0032020 con 0x563e5b4018f0 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa4b842d740 10 client.15873 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa4b842d740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa4b842d740 20 client.15873 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa4b842d740 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563e5b487a20 0x563e5b489e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa4b842d740 4 client.15873 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:54:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa4b842d740 1 -- 172.21.15.69:0/616722109 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e5b370360 con 0x563e5b487a20 2024-05-14T16:54:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.832+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563e5b487a20 0x563e5b489e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.833+0000 7fa492ff5640 10 client.15873 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.833+0000 7fa4ae7fc640 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563e5b487a20 0x563e5b489e80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e44d5aaf93f775 server_cookie=4772c437e1966e6d in_seq=0 out_seq=0 2024-05-14T16:54:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa492ff5640 1 -- 172.21.15.69:0/616722109 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563e5b370360 con 0x563e5b487a20 2024-05-14T16:54:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa492ff5640 10 client.15873 handle_command_reply: tid=0 2024-05-14T16:54:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 client.15873 shutdown 2024-05-14T16:54:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 -- 172.21.15.69:0/616722109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b4018f0 msgr2=0x563e5b410800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b4018f0 0x563e5b410800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009fa0 tx=0x7fa4a00046c0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 -- 172.21.15.69:0/616722109 shutdown_connections 2024-05-14T16:54:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563e5b487a20 0x563e5b489e80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e5b40cad0 0x563e5b411280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563e5b402210 0x563e5b410d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa492ff5640 0 client.15873 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 --2- 172.21.15.69:0/616722109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563e5b4018f0 0x563e5b410800 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 -- 172.21.15.69:0/616722109 >> 172.21.15.69:0/616722109 conn(0x563e5b3fde00 msgr2=0x563e5b417630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 -- 172.21.15.69:0/616722109 shutdown_connections 2024-05-14T16:54:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 1 -- 172.21.15.69:0/616722109 wait complete. 2024-05-14T16:54:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.846+0000 7fa4b842d740 20 client.15873 trim_cache size 0 max 16384 2024-05-14T16:54:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:54:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:54:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:54:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 -- 172.21.15.69:0/676125525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa4980b2a10 msgr2=0x7fa4980b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa4980b2a10 0x7fa4980b4ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x563e5b402e90 tx=0x7fa4a0005f90 comp rx=0 tx=0).stop 2024-05-14T16:54:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 -- 172.21.15.69:0/676125525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4b0108a40 msgr2=0x7fa4b019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4b0108a40 0x7fa4b019dae0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800eb10 tx=0x7fa4a800c2d0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 -- 172.21.15.69:0/676125525 shutdown_connections 2024-05-14T16:54:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa4980b2a10 0x7fa4980b4ed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4b0108a40 0x7fa4b019dae0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b01033f0 0x7fa4b0199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 --2- 172.21.15.69:0/676125525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b0102a30 0x7fa4b0199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 -- 172.21.15.69:0/676125525 >> 172.21.15.69:0/676125525 conn(0x7fa4b00fe500 msgr2=0x7fa4b00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 -- 172.21.15.69:0/676125525 shutdown_connections 2024-05-14T16:54:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:34.847+0000 7fa4b5714640 1 -- 172.21.15.69:0/676125525 wait complete. 2024-05-14T16:54:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 245.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:54:36.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:35 smithi069 ceph-mon[31507]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2024-05-14T16:54:36.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:35 smithi203 ceph-mon[35787]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2024-05-14T16:54:36.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:35 smithi092 ceph-mon[35763]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2024-05-14T16:54:38.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:37 smithi069 ceph-mon[31507]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-05-14T16:54:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:37 smithi203 ceph-mon[35787]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-05-14T16:54:38.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:37 smithi092 ceph-mon[35763]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-05-14T16:54:39.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:39 smithi069 ceph-mon[31507]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-05-14T16:54:39.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:39 smithi203 ceph-mon[35787]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-05-14T16:54:39.999 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:39 smithi092 ceph-mon[35763]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-05-14T16:54:40.396 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 250.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:54:42.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:41 smithi069 ceph-mon[31507]: pgmap v148: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 MiB/s wr, 579 op/s 2024-05-14T16:54:42.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:41 smithi203 ceph-mon[35787]: pgmap v148: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 MiB/s wr, 579 op/s 2024-05-14T16:54:42.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:41 smithi092 ceph-mon[35763]: pgmap v148: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 MiB/s wr, 579 op/s 2024-05-14T16:54:44.011 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:43 smithi203 ceph-mon[35787]: pgmap v149: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 575 op/s 2024-05-14T16:54:44.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:43 smithi069 ceph-mon[31507]: pgmap v149: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 575 op/s 2024-05-14T16:54:44.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:43 smithi092 ceph-mon[35763]: pgmap v149: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 575 op/s 2024-05-14T16:54:45.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 255.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:54:46.010 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:54:46.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:45 smithi069 ceph-mon[31507]: pgmap v150: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 575 op/s 2024-05-14T16:54:46.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:45 smithi092 ceph-mon[35763]: pgmap v150: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 575 op/s 2024-05-14T16:54:46.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:45 smithi203 ceph-mon[35787]: pgmap v150: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 575 op/s 2024-05-14T16:54:46.266 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.944+0000 7f813b119640 1 -- 172.21.15.69:0/386265287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134101980 msgr2=0x7f8134101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.944+0000 7f813b119640 1 --2- 172.21.15.69:0/386265287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134101980 0x7f8134101e00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8124009cd0 tx=0x7f812402a8e0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.944+0000 7f813b119640 1 -- 172.21.15.69:0/386265287 shutdown_connections 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.944+0000 7f813b119640 1 --2- 172.21.15.69:0/386265287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8134102340 0x7f813410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.944+0000 7f813b119640 1 --2- 172.21.15.69:0/386265287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134101980 0x7f8134101e00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.944+0000 7f813b119640 1 --2- 172.21.15.69:0/386265287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134107940 0x7f8134107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.944+0000 7f813b119640 1 -- 172.21.15.69:0/386265287 >> 172.21.15.69:0/386265287 conn(0x7f8134078ad0 msgr2=0x7f8134078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.944+0000 7f813b119640 1 -- 172.21.15.69:0/386265287 shutdown_connections 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.944+0000 7f813b119640 1 -- 172.21.15.69:0/386265287 wait complete. 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.944+0000 7f813b119640 1 Processor -- start 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f813b119640 1 -- start start 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f813b119640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8134101980 0x7f8134199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f813b119640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134102340 0x7f81341997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f813b119640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134107940 0x7f813419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f813b119640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8134111c00 con 0x7f8134102340 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f813b119640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8134111a80 con 0x7f8134107940 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f813b119640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8134111d80 con 0x7f8134101980 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f8133fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134102340 0x7f81341997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f8133fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134102340 0x7f81341997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40516/0 (socket says 172.21.15.69:40516) 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f8133fff640 1 -- 172.21.15.69:0/3990809242 learned_addr learned my addr 172.21.15.69:0/3990809242 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f81396b6640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134107940 0x7f813419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f8138eb5640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8134101980 0x7f8134199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f8133fff640 1 -- 172.21.15.69:0/3990809242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8134101980 msgr2=0x7f8134199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f8133fff640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8134101980 0x7f8134199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f8133fff640 1 -- 172.21.15.69:0/3990809242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134107940 msgr2=0x7f813419db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f8133fff640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134107940 0x7f813419db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f8133fff640 1 -- 172.21.15.69:0/3990809242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813419e250 con 0x7f8134102340 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f81396b6640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134107940 0x7f813419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.945+0000 7f8133fff640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134102340 0x7f81341997a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f812402ae00 tx=0x7f8124003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.946+0000 7f8131ffb640 1 -- 172.21.15.69:0/3990809242 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124004240 con 0x7f8134102340 2024-05-14T16:54:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.946+0000 7f8131ffb640 1 -- 172.21.15.69:0/3990809242 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81240305a0 con 0x7f8134102340 2024-05-14T16:54:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.946+0000 7f813b119640 1 -- 172.21.15.69:0/3990809242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813419e4e0 con 0x7f8134102340 2024-05-14T16:54:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.946+0000 7f8131ffb640 1 -- 172.21.15.69:0/3990809242 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240466e0 con 0x7f8134102340 2024-05-14T16:54:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.946+0000 7f813b119640 1 -- 172.21.15.69:0/3990809242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81341a5dd0 con 0x7f8134102340 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.948+0000 7f8131ffb640 1 -- 172.21.15.69:0/3990809242 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8124030a10 con 0x7f8134102340 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f813de37740 1 Processor -- start 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f813de37740 1 -- start start 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f813de37740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f813de37740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653d7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f813de37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f813de37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56486533fc30 con 0x5648653d7ad0 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f813de37740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56486532f7d0 con 0x5648653cc8f0 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f813de37740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564865333810 con 0x5648653cd210 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f8133fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f8133fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40518/0 (socket says 172.21.15.69:40518) 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f8133fff640 1 -- 172.21.15.69:0/1823204456 learned_addr learned my addr 172.21.15.69:0/1823204456 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f8138eb5640 1 --2- 172.21.15.69:0/1823204456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f81396b6640 1 --2- 172.21.15.69:0/1823204456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f8131ffb640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81200b2a10 0x7f81200b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f81396b6640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81200b2a10 0x7f81200b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f8133fff640 1 -- 172.21.15.69:0/1823204456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 msgr2=0x5648653d7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f8133fff640 1 --2- 172.21.15.69:0/1823204456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653d7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f8133fff640 1 -- 172.21.15.69:0/1823204456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 msgr2=0x5648653cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.949+0000 7f8133fff640 1 --2- 172.21.15.69:0/1823204456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.950+0000 7f8133fff640 1 -- 172.21.15.69:0/1823204456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564865044210 con 0x5648653d7ad0 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.950+0000 7f8131ffb640 1 -- 172.21.15.69:0/3990809242 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f812402e030 con 0x7f8134102340 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.950+0000 7f8133fff640 1 --2- 172.21.15.69:0/1823204456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d9ec0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8124101510 tx=0x7f8124084ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c1f46f2028cf66 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.950+0000 7f81396b6640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81200b2a10 0x7f81200b4ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f812c00bfd0 tx=0x7f812c004730 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.950+0000 7f812affd640 1 -- 172.21.15.69:0/1823204456 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240caf30 con 0x5648653d7ad0 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.950+0000 7f812affd640 1 -- 172.21.15.69:0/1823204456 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8124088be0 con 0x5648653d7ad0 2024-05-14T16:54:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.950+0000 7f812affd640 1 -- 172.21.15.69:0/1823204456 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812402f550 con 0x5648653d7ad0 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 -- 172.21.15.69:0/1823204456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 msgr2=0x5648653d9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 --2- 172.21.15.69:0/1823204456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d9ec0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8124101510 tx=0x7f8124084ee0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 -- 172.21.15.69:0/1823204456 shutdown_connections 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 --2- 172.21.15.69:0/1823204456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d9ec0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 --2- 172.21.15.69:0/1823204456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653d7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 --2- 172.21.15.69:0/1823204456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 -- 172.21.15.69:0/1823204456 >> 172.21.15.69:0/1823204456 conn(0x5648653c8e00 msgr2=0x5648653c91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 -- 172.21.15.69:0/1823204456 shutdown_connections 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 -- 172.21.15.69:0/1823204456 wait complete. 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 -- start start 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.951+0000 7f813de37740 1 Processor -- start 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f813de37740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653d7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f813de37740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f813de37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f813de37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564865015790 con 0x5648653d7ad0 2024-05-14T16:54:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f813de37740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5648652eb890 con 0x5648653cc8f0 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f813de37740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5648653da4c0 con 0x5648653cd210 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f813de37740 10 client.0 _ll_register_callbacks cb 0x5648652eed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f81396b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f81396b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40526/0 (socket says 172.21.15.69:40526) 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f81396b6640 1 -- 172.21.15.69:0/148649785 learned_addr learned my addr 172.21.15.69:0/148649785 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f8133fff640 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f8138eb5640 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f81396b6640 1 -- 172.21.15.69:0/148649785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 msgr2=0x5648653d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f81396b6640 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f81396b6640 1 -- 172.21.15.69:0/148649785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 msgr2=0x5648653d7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f81396b6640 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653d7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f81396b6640 1 -- 172.21.15.69:0/148649785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564865110620 con 0x5648653d7ad0 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f81396b6640 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d4ba0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f812c011840 tx=0x7f812c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0b9e6af76e4407 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f8113fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:54:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f8113fff640 1 -- 172.21.15.69:0/148649785 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c019070 con 0x5648653d7ad0 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f8113fff640 1 -- 172.21.15.69:0/148649785 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f812c010de0 con 0x5648653d7ad0 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.952+0000 7f813de37740 1 -- 172.21.15.69:0/148649785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564865341b80 con 0x5648653d7ad0 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f8113fff640 1 -- 172.21.15.69:0/148649785 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f812c028400 con 0x5648653d7ad0 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f813de37740 10 client.15885 fetch_fsmap learned FSMap version 35 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f813de37740 1 -- 172.21.15.69:0/148649785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5648653e5f40 con 0x5648653d7ad0 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f8113fff640 1 -- 172.21.15.69:0/148649785 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564865341b80 con 0x5648653d7ad0 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f8113fff640 1 -- 172.21.15.69:0/148649785 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f812c0315e0 con 0x5648653d7ad0 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f813de37740 10 client.15885 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f813de37740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f813de37740 20 client.15885 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f813de37740 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5648653e7420 0x5648653e9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f813de37740 4 client.15885 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f813de37740 1 -- 172.21.15.69:0/148649785 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56486533b360 con 0x5648653e7420 2024-05-14T16:54:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.953+0000 7f8138eb5640 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5648653e7420 0x5648653e9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.954+0000 7f8138eb5640 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5648653e7420 0x5648653e9830 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e25775352e45baa0 server_cookie=f7361071a00ea505 in_seq=0 out_seq=0 2024-05-14T16:54:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.954+0000 7f8113fff640 10 client.15885 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.954+0000 7f8113fff640 1 -- 172.21.15.69:0/148649785 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56486533b360 con 0x5648653e7420 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.954+0000 7f8113fff640 10 client.15885 handle_command_reply: tid=0 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 client.15885 shutdown 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 -- 172.21.15.69:0/148649785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 msgr2=0x5648653d4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d4ba0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f812c011840 tx=0x7f812c0046c0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 -- 172.21.15.69:0/148649785 shutdown_connections 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5648653e7420 0x5648653e9830 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653d4ba0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f8113fff640 0 client.15885 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 --2- 172.21.15.69:0/148649785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653d7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 -- 172.21.15.69:0/148649785 >> 172.21.15.69:0/148649785 conn(0x5648653c8e00 msgr2=0x5648653c8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 -- 172.21.15.69:0/148649785 shutdown_connections 2024-05-14T16:54:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 1 -- 172.21.15.69:0/148649785 wait complete. 2024-05-14T16:54:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.955+0000 7f813de37740 20 client.15885 trim_cache size 0 max 16384 2024-05-14T16:54:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f813de37740 1 Processor -- start 2024-05-14T16:54:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f813de37740 1 -- start start 2024-05-14T16:54:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f813de37740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cc8f0 0x5648653ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f813de37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653cd210 0x5648653d6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f813de37740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653d7ad0 0x5648653ef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f813de37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81341b0ed0 con 0x5648653cd210 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f813de37740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81341ae710 con 0x5648653d7ad0 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f813de37740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81341ac560 con 0x5648653cc8f0 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f8133fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653cd210 0x5648653d6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f8133fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653cd210 0x5648653d6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40534/0 (socket says 172.21.15.69:40534) 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.970+0000 7f8133fff640 1 -- 172.21.15.69:0/1956617042 learned_addr learned my addr 172.21.15.69:0/1956617042 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.971+0000 7f81396b6640 1 --2- 172.21.15.69:0/1956617042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653d7ad0 0x5648653ef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.971+0000 7f8133fff640 1 -- 172.21.15.69:0/1956617042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cc8f0 msgr2=0x5648653ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.971+0000 7f8133fff640 1 --2- 172.21.15.69:0/1956617042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cc8f0 0x5648653ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.971+0000 7f8133fff640 1 -- 172.21.15.69:0/1956617042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653d7ad0 msgr2=0x5648653ef190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.971+0000 7f8133fff640 1 --2- 172.21.15.69:0/1956617042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653d7ad0 0x5648653ef190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.971+0000 7f8133fff640 1 -- 172.21.15.69:0/1956617042 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648653cd040 con 0x5648653cd210 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.971+0000 7f8133fff640 1 --2- 172.21.15.69:0/1956617042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653cd210 0x5648653d6ea0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8124107040 tx=0x7f8124103e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e5874dd296c228 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.971+0000 7f812a7fc640 1 -- 172.21.15.69:0/1956617042 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240ca8d0 con 0x5648653cd210 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.971+0000 7f812a7fc640 1 -- 172.21.15.69:0/1956617042 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8124036070 con 0x5648653cd210 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.971+0000 7f812a7fc640 1 -- 172.21.15.69:0/1956617042 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240caab0 con 0x5648653cd210 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 -- 172.21.15.69:0/1956617042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653cd210 msgr2=0x5648653d6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 --2- 172.21.15.69:0/1956617042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653cd210 0x5648653d6ea0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8124107040 tx=0x7f8124103e40 comp rx=0 tx=0).stop 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 -- 172.21.15.69:0/1956617042 shutdown_connections 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 --2- 172.21.15.69:0/1956617042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653d7ad0 0x5648653ef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 --2- 172.21.15.69:0/1956617042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653cd210 0x5648653d6ea0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 --2- 172.21.15.69:0/1956617042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cc8f0 0x5648653ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 -- 172.21.15.69:0/1956617042 >> 172.21.15.69:0/1956617042 conn(0x5648653c8e00 msgr2=0x5648653ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 -- 172.21.15.69:0/1956617042 shutdown_connections 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 -- 172.21.15.69:0/1956617042 wait complete. 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 -- start start 2024-05-14T16:54:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 Processor -- start 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653dfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653db4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81341acbd0 con 0x5648653d7ad0 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81341ae040 con 0x5648653cc8f0 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81341ad2a0 con 0x5648653cd210 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f813de37740 10 client.0 _ll_register_callbacks cb 0x7f812c00be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.972+0000 7f81396b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653db4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f81396b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653db4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40548/0 (socket says 172.21.15.69:40548) 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f81396b6640 1 -- 172.21.15.69:0/1882590777 learned_addr learned my addr 172.21.15.69:0/1882590777 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f8138eb5640 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653dfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f81396b6640 1 -- 172.21.15.69:0/1882590777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 msgr2=0x5648653daf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f8133fff640 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f81396b6640 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653daf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f81396b6640 1 -- 172.21.15.69:0/1882590777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 msgr2=0x5648653dfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f81396b6640 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653dfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f81396b6640 1 -- 172.21.15.69:0/1882590777 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648653dbc00 con 0x5648653d7ad0 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f8138eb5640 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653dfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f81396b6640 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653db4c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f812c0112f0 tx=0x7f812c011320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6e661eca2cc96c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f8113fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f8113fff640 1 -- 172.21.15.69:0/1882590777 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c019070 con 0x5648653d7ad0 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f8113fff640 1 -- 172.21.15.69:0/1882590777 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f812c010de0 con 0x5648653d7ad0 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f813de37740 1 -- 172.21.15.69:0/1882590777 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8124036070 con 0x5648653d7ad0 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f8113fff640 1 -- 172.21.15.69:0/1882590777 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f812c037070 con 0x5648653d7ad0 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f813de37740 10 client.15891 fetch_fsmap learned FSMap version 35 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f813de37740 1 -- 172.21.15.69:0/1882590777 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564865451c80 con 0x5648653d7ad0 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.973+0000 7f8113fff640 1 -- 172.21.15.69:0/1882590777 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8124036070 con 0x5648653d7ad0 2024-05-14T16:54:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.974+0000 7f8113fff640 1 -- 172.21.15.69:0/1882590777 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f812c028b20 con 0x5648653d7ad0 2024-05-14T16:54:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.974+0000 7f813de37740 10 client.15891 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:54:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.974+0000 7f813de37740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:54:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.974+0000 7f813de37740 20 client.15891 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.974+0000 7f813de37740 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564865452420 0x564865454840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.974+0000 7f813de37740 4 client.15891 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:54:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.974+0000 7f813de37740 1 -- 172.21.15.69:0/1882590777 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648652b55f0 con 0x564865452420 2024-05-14T16:54:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.974+0000 7f8138eb5640 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564865452420 0x564865454840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.974+0000 7f8138eb5640 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564865452420 0x564865454840 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6bd8f2bf3124311 server_cookie=7fc0325547521880 in_seq=0 out_seq=0 2024-05-14T16:54:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.974+0000 7f8113fff640 10 client.15891 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.985+0000 7f8113fff640 1 -- 172.21.15.69:0/1882590777 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5648652b55f0 con 0x564865452420 2024-05-14T16:54:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.985+0000 7f8113fff640 10 client.15891 handle_command_reply: tid=0 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.985+0000 7f813de37740 1 client.15891 shutdown 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.985+0000 7f813de37740 1 -- 172.21.15.69:0/1882590777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 msgr2=0x5648653db4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.985+0000 7f813de37740 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653db4c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f812c0112f0 tx=0x7f812c011320 comp rx=0 tx=0).stop 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813de37740 1 -- 172.21.15.69:0/1882590777 shutdown_connections 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813de37740 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564865452420 0x564865454840 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813de37740 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648653d7ad0 0x5648653db4c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813de37740 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5648653cd210 0x5648653daf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813de37740 1 --2- 172.21.15.69:0/1882590777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648653cc8f0 0x5648653dfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f8113fff640 0 client.15891 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813de37740 1 -- 172.21.15.69:0/1882590777 >> 172.21.15.69:0/1882590777 conn(0x5648653c8e00 msgr2=0x5648653ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813de37740 1 -- 172.21.15.69:0/1882590777 shutdown_connections 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813de37740 1 -- 172.21.15.69:0/1882590777 wait complete. 2024-05-14T16:54:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813de37740 20 client.15891 trim_cache size 0 max 16384 2024-05-14T16:54:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:54:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:54:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 -- 172.21.15.69:0/3990809242 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81200b2a10 msgr2=0x7f81200b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81200b2a10 0x7f81200b4ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f812c00bfd0 tx=0x7f812c004730 comp rx=0 tx=0).stop 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 -- 172.21.15.69:0/3990809242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134102340 msgr2=0x7f81341997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134102340 0x7f81341997a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f812402ae00 tx=0x7f8124003ba0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 -- 172.21.15.69:0/3990809242 shutdown_connections 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81200b2a10 0x7f81200b4ed0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134107940 0x7f813419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134102340 0x7f81341997a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 --2- 172.21.15.69:0/3990809242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8134101980 0x7f8134199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 -- 172.21.15.69:0/3990809242 >> 172.21.15.69:0/3990809242 conn(0x7f8134078ad0 msgr2=0x7f813406a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 -- 172.21.15.69:0/3990809242 shutdown_connections 2024-05-14T16:54:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:54:47.986+0000 7f813b119640 1 -- 172.21.15.69:0/3990809242 wait complete. 2024-05-14T16:54:48.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:47 smithi069 ceph-mon[31507]: pgmap v151: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-05-14T16:54:48.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.111+0000 7fa5dc552700 1 -- 172.21.15.69:0/3732450620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4102b10 msgr2=0x7fa5d41068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:48.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.111+0000 7fa5dc552700 1 --2- 172.21.15.69:0/3732450620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4102b10 0x7fa5d41068d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009990 tx=0x7fa5c4009ca0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.112+0000 7fa5dc552700 1 -- 172.21.15.69:0/3732450620 shutdown_connections 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.112+0000 7fa5dc552700 1 --2- 172.21.15.69:0/3732450620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5d4107000 0x7fa5d410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.112+0000 7fa5dc552700 1 --2- 172.21.15.69:0/3732450620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d4102160 0x7fa5d4102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.112+0000 7fa5dc552700 1 --2- 172.21.15.69:0/3732450620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4102b10 0x7fa5d41068d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.112+0000 7fa5dc552700 1 -- 172.21.15.69:0/3732450620 >> 172.21.15.69:0/3732450620 conn(0x7fa5d4077780 msgr2=0x7fa5d4077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.112+0000 7fa5dc552700 1 -- 172.21.15.69:0/3732450620 shutdown_connections 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.112+0000 7fa5dc552700 1 -- 172.21.15.69:0/3732450620 wait complete. 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.112+0000 7fa5dc552700 1 Processor -- start 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.112+0000 7fa5dc552700 1 -- start start 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5dc552700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d4102160 0x7fa5d419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5dc552700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4102b10 0x7fa5d419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5dc552700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5d4107000 0x7fa5d4199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5dc552700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d41137f0 con 0x7fa5d4102b10 2024-05-14T16:54:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5dc552700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d4113670 con 0x7fa5d4107000 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5dc552700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d4113970 con 0x7fa5d4102160 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5d9aed700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4102b10 0x7fa5d419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5d9aed700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4102b10 0x7fa5d419fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40570/0 (socket says 172.21.15.69:40570) 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5d9aed700 1 -- 172.21.15.69:0/4287314731 learned_addr learned my addr 172.21.15.69:0/4287314731 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5da2ee700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d4102160 0x7fa5d419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5daaef700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5d4107000 0x7fa5d4199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5d9aed700 1 -- 172.21.15.69:0/4287314731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d4102160 msgr2=0x7fa5d419f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5d9aed700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d4102160 0x7fa5d419f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5d9aed700 1 -- 172.21.15.69:0/4287314731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5d4107000 msgr2=0x7fa5d4199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5d9aed700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5d4107000 0x7fa5d4199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5d9aed700 1 -- 172.21.15.69:0/4287314731 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d419a230 con 0x7fa5d4102b10 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5da2ee700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d4102160 0x7fa5d419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5daaef700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5d4107000 0x7fa5d4199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5d9aed700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4102b10 0x7fa5d419fe50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4006010 tx=0x7fa5c4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5cb7fe700 1 -- 172.21.15.69:0/4287314731 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4027070 con 0x7fa5d4102b10 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.113+0000 7fa5cb7fe700 1 -- 172.21.15.69:0/4287314731 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c4003fe0 con 0x7fa5d4102b10 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.114+0000 7fa5dc552700 1 -- 172.21.15.69:0/4287314731 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d419a4c0 con 0x7fa5d4102b10 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.114+0000 7fa5cb7fe700 1 -- 172.21.15.69:0/4287314731 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4022c40 con 0x7fa5d4102b10 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.114+0000 7fa5dc552700 1 -- 172.21.15.69:0/4287314731 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d41a67e0 con 0x7fa5d4102b10 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.114+0000 7fa5dc552700 1 -- 172.21.15.69:0/4287314731 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d406a230 con 0x7fa5d4102b10 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.116+0000 7fa5cb7fe700 1 -- 172.21.15.69:0/4287314731 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa5c402a070 con 0x7fa5d4102b10 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.116+0000 7fa5cb7fe700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa5c00b2a70 0x7fa5c00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.117+0000 7fa5da2ee700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa5c00b2a70 0x7fa5c00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.117+0000 7fa5cb7fe700 1 -- 172.21.15.69:0/4287314731 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa5c4022e00 con 0x7fa5d4102b10 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.117+0000 7fa5da2ee700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa5c00b2a70 0x7fa5c00b4f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0007900 tx=0x7fa5d0008040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.120+0000 7fa5cb7fe700 1 -- 172.21.15.69:0/4287314731 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa5c4013070 con 0x7fa5d4102b10 2024-05-14T16:54:48.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:47 smithi203 ceph-mon[35787]: pgmap v151: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-05-14T16:54:48.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:47 smithi092 ceph-mon[35763]: pgmap v151: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-05-14T16:54:48.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.367+0000 7fa5dc552700 1 -- 172.21.15.69:0/4287314731 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5d410a190 con 0x7fa5d4102b10 2024-05-14T16:54:48.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.368+0000 7fa5cb7fe700 1 -- 172.21.15.69:0/4287314731 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11863 (secure 0 0 0) 0x7fa5c40ab9d0 con 0x7fa5d4102b10 2024-05-14T16:54:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:54:48.368 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:54:13.610116+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24446,24422,24433]},"id":1}]} 2024-05-14T16:54:48.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 -- 172.21.15.69:0/4287314731 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa5c00b2a70 msgr2=0x7fa5c00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa5c00b2a70 0x7fa5c00b4f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0007900 tx=0x7fa5d0008040 comp rx=0 tx=0).stop 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 -- 172.21.15.69:0/4287314731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4102b10 msgr2=0x7fa5d419fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4102b10 0x7fa5d419fe50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4006010 tx=0x7fa5c4005ef0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 -- 172.21.15.69:0/4287314731 shutdown_connections 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5d4107000 0x7fa5d4199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d4102160 0x7fa5d419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa5c00b2a70 0x7fa5c00b4f30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 --2- 172.21.15.69:0/4287314731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4102b10 0x7fa5d419fe50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 -- 172.21.15.69:0/4287314731 >> 172.21.15.69:0/4287314731 conn(0x7fa5d4077780 msgr2=0x7fa5d4100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 -- 172.21.15.69:0/4287314731 shutdown_connections 2024-05-14T16:54:48.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:48.370+0000 7fa5dc552700 1 -- 172.21.15.69:0/4287314731 wait complete. 2024-05-14T16:54:48.371 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 35 2024-05-14T16:54:49.005 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:48 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/4287314731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:49.034 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id b13943b --release --await' 2024-05-14T16:54:49.034 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id b13943b --release --await 2024-05-14T16:54:49.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:48 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/4287314731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:49.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:48 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/4287314731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:49.316 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:50.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:49 smithi069 ceph-mon[31507]: pgmap v152: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 MiB/s wr, 1.19k op/s 2024-05-14T16:54:50.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:49 smithi203 ceph-mon[35787]: pgmap v152: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 MiB/s wr, 1.19k op/s 2024-05-14T16:54:50.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:49 smithi092 ceph-mon[35763]: pgmap v152: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 MiB/s wr, 1.19k op/s 2024-05-14T16:54:50.396 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 260.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.286+0000 7fa7f9af5700 1 -- 172.21.15.69:0/1493300357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7f406c470 msgr2=0x7fa7f41103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.286+0000 7fa7f9af5700 1 --2- 172.21.15.69:0/1493300357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7f406c470 0x7fa7f41103c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00b0a0 tx=0x7fa7ec00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.286+0000 7fa7f9af5700 1 -- 172.21.15.69:0/1493300357 shutdown_connections 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.286+0000 7fa7f9af5700 1 --2- 172.21.15.69:0/1493300357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7f406c470 0x7fa7f41103c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.286+0000 7fa7f9af5700 1 --2- 172.21.15.69:0/1493300357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f41095a0 0x7fa7f4109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.286+0000 7fa7f9af5700 1 --2- 172.21.15.69:0/1493300357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f406bab0 0x7fa7f406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.286+0000 7fa7f9af5700 1 -- 172.21.15.69:0/1493300357 >> 172.21.15.69:0/1493300357 conn(0x7fa7f40793b0 msgr2=0x7fa7f40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.286+0000 7fa7f9af5700 1 -- 172.21.15.69:0/1493300357 shutdown_connections 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.286+0000 7fa7f9af5700 1 -- 172.21.15.69:0/1493300357 wait complete. 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7f9af5700 1 Processor -- start 2024-05-14T16:54:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7f9af5700 1 -- start start 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7f9af5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f406bab0 0x7fa7f419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7f9af5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7f406c470 0x7fa7f419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7f9af5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f41095a0 0x7fa7f4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7f9af5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4113790 con 0x7fa7f406c470 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7f9af5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4113610 con 0x7fa7f41095a0 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7f9af5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4113910 con 0x7fa7f406bab0 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7eaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7f406c470 0x7fa7f419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7eaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7f406c470 0x7fa7f419fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40592/0 (socket says 172.21.15.69:40592) 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7eaffd700 1 -- 172.21.15.69:0/2353010483 learned_addr learned my addr 172.21.15.69:0/2353010483 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f406bab0 0x7fa7f419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.287+0000 7fa7f3fff700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f41095a0 0x7fa7f4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7eaffd700 1 -- 172.21.15.69:0/2353010483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f406bab0 msgr2=0x7fa7f419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f406bab0 0x7fa7f419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7eaffd700 1 -- 172.21.15.69:0/2353010483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f41095a0 msgr2=0x7fa7f4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f41095a0 0x7fa7f4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7eaffd700 1 -- 172.21.15.69:0/2353010483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f419a210 con 0x7fa7f406c470 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7f406c470 0x7fa7f419fde0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00d950 tx=0x7fa7dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7f17fa700 1 -- 172.21.15.69:0/2353010483 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc016070 con 0x7fa7f406c470 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7f17fa700 1 -- 172.21.15.69:0/2353010483 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7dc00fd40 con 0x7fa7f406c470 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7f9af5700 1 -- 172.21.15.69:0/2353010483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f419a500 con 0x7fa7f406c470 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7f17fa700 1 -- 172.21.15.69:0/2353010483 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc00e3b0 con 0x7fa7f406c470 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.288+0000 7fa7f9af5700 1 -- 172.21.15.69:0/2353010483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f41a6770 con 0x7fa7f406c470 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.290+0000 7fa7f17fa700 1 -- 172.21.15.69:0/2353010483 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa7dc019070 con 0x7fa7f406c470 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.291+0000 7fa7f17fa700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7e00b29f0 0x7fa7e00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.291+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7e00b29f0 0x7fa7e00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.291+0000 7fa7f17fa700 1 -- 172.21.15.69:0/2353010483 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa7dc0e2100 con 0x7fa7f406c470 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa80438bb80 1 Processor -- start 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7e00b29f0 0x7fa7e00b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4100a60 tx=0x7fa7e4009190 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa80438bb80 1 -- start start 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207b420 0x5565f207b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207bd40 0x5565f2076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f2077720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa80438bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5565f1fd4e70 con 0x5565f20772c0 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa80438bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5565f1fc4a10 con 0x5565f207bd40 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa80438bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5565f1fc8a50 con 0x5565f207b420 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f2077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f2077720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40600/0 (socket says 172.21.15.69:40600) 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7f3fff700 1 -- 172.21.15.69:0/1729550990 learned_addr learned my addr 172.21.15.69:0/1729550990 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7f3fff700 1 -- 172.21.15.69:0/1729550990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207b420 msgr2=0x5565f207b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7f3fff700 1 --2- 172.21.15.69:0/1729550990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207b420 0x5565f207b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7f3fff700 1 -- 172.21.15.69:0/1729550990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207bd40 msgr2=0x5565f2076ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/1729550990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207bd40 0x5565f2076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7f3fff700 1 --2- 172.21.15.69:0/1729550990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207bd40 0x5565f2076ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7f3fff700 1 -- 172.21.15.69:0/1729550990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f1f6e0b0 con 0x5565f20772c0 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.292+0000 7fa7f3fff700 1 --2- 172.21.15.69:0/1729550990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f2077720 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec015000 tx=0x7fa7ec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32354881c7b7060 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa7e9ffb700 1 -- 172.21.15.69:0/1729550990 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec020070 con 0x5565f20772c0 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa7e9ffb700 1 -- 172.21.15.69:0/1729550990 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7ec009ca0 con 0x5565f20772c0 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa7e9ffb700 1 -- 172.21.15.69:0/1729550990 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0052a0 con 0x5565f20772c0 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa80438bb80 1 -- 172.21.15.69:0/1729550990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 msgr2=0x5565f2077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa80438bb80 1 --2- 172.21.15.69:0/1729550990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f2077720 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec015000 tx=0x7fa7ec009310 comp rx=0 tx=0).stop 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa80438bb80 1 -- 172.21.15.69:0/1729550990 shutdown_connections 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa80438bb80 1 --2- 172.21.15.69:0/1729550990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f2077720 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa80438bb80 1 --2- 172.21.15.69:0/1729550990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207b420 0x5565f207b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa80438bb80 1 --2- 172.21.15.69:0/1729550990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207bd40 0x5565f2076ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa80438bb80 1 -- 172.21.15.69:0/1729550990 >> 172.21.15.69:0/1729550990 conn(0x5565f1fa3eb0 msgr2=0x5565f20714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa80438bb80 1 -- 172.21.15.69:0/1729550990 shutdown_connections 2024-05-14T16:54:51.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.293+0000 7fa80438bb80 1 -- 172.21.15.69:0/1729550990 wait complete. 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa80438bb80 1 -- start start 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa80438bb80 1 Processor -- start 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 0x5565f2082b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207b420 0x5565f207ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207bd40 0x5565f2080490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa80438bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5565f1ecb0a0 con 0x5565f207bd40 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa80438bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5565f1f2cea0 con 0x5565f20772c0 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa80438bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5565f2083820 con 0x5565f207b420 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa80438bb80 10 client.0 _ll_register_callbacks cb 0x5565f1f5fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa7f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207bd40 0x5565f2080490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa7f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207bd40 0x5565f2080490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40612/0 (socket says 172.21.15.69:40612) 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa7f3fff700 1 -- 172.21.15.69:0/1160667774 learned_addr learned my addr 172.21.15.69:0/1160667774 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.294+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 0x5565f2082b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207b420 0x5565f207ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7f3fff700 1 -- 172.21.15.69:0/1160667774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207b420 msgr2=0x5565f207ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7f3fff700 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207b420 0x5565f207ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7f3fff700 1 -- 172.21.15.69:0/1160667774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 msgr2=0x5565f2082b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7f3fff700 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 0x5565f2082b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7f3fff700 1 -- 172.21.15.69:0/1160667774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f1f149d0 con 0x5565f207bd40 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 0x5565f2082b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7f3fff700 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207bd40 0x5565f2080490 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00b830 tx=0x7fa7ec007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f23292e7e3e1b20 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/1160667774 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec020040 con 0x5565f207bd40 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/1160667774 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa7ec0050c0 con 0x5565f207bd40 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa80438bb80 1 -- 172.21.15.69:0/1160667774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ec009ca0 con 0x5565f207bd40 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/1160667774 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa7ec0052c0 con 0x5565f207bd40 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa80438bb80 10 client.15909 fetch_fsmap learned FSMap version 35 2024-05-14T16:54:51.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa80438bb80 1 -- 172.21.15.69:0/1160667774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5565f1f7e860 con 0x5565f207bd40 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/1160667774 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7ec009ca0 con 0x5565f207bd40 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.295+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/1160667774 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fa7ec011070 con 0x5565f207bd40 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.296+0000 7fa80438bb80 10 client.15909 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.296+0000 7fa80438bb80 10 client.15909 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.296+0000 7fa80438bb80 20 client.15909 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.296+0000 7fa80438bb80 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f2090270 0x5565f2092660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.296+0000 7fa80438bb80 4 client.15909 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.296+0000 7fa80438bb80 1 -- 172.21.15.69:0/1160667774 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565f1fd6dc0 con 0x5565f2090270 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.296+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f2090270 0x5565f2092660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.296+0000 7fa7d2ffd700 10 client.15909 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.296+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f2090270 0x5565f2092660 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e399bd977681c7f server_cookie=b7d1d8a9848a273d in_seq=0 out_seq=0 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/1160667774 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5565f1fd6dc0 con 0x5565f2090270 2024-05-14T16:54:51.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa7d2ffd700 10 client.15909 handle_command_reply: tid=0 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa80438bb80 1 client.15909 shutdown 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa80438bb80 1 -- 172.21.15.69:0/1160667774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207bd40 msgr2=0x5565f2080490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa80438bb80 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207bd40 0x5565f2080490 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00b830 tx=0x7fa7ec007f90 comp rx=0 tx=0).stop 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa80438bb80 1 -- 172.21.15.69:0/1160667774 shutdown_connections 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa80438bb80 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f2090270 0x5565f2092660 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa80438bb80 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207bd40 0x5565f2080490 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa7d2ffd700 0 client.15909 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa80438bb80 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 0x5565f2082b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa80438bb80 1 --2- 172.21.15.69:0/1160667774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207b420 0x5565f207ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa80438bb80 1 -- 172.21.15.69:0/1160667774 >> 172.21.15.69:0/1160667774 conn(0x5565f1fa3eb0 msgr2=0x5565f206bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.297+0000 7fa80438bb80 1 -- 172.21.15.69:0/1160667774 shutdown_connections 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.298+0000 7fa80438bb80 1 -- 172.21.15.69:0/1160667774 wait complete. 2024-05-14T16:54:51.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.298+0000 7fa80438bb80 20 client.15909 trim_cache size 0 max 16384 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa80438bb80 1 Processor -- start 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa80438bb80 1 -- start start 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f206c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207b420 0x5565f208c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207bd40 0x5565f208c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa80438bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41b0550 con 0x5565f20772c0 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa80438bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41add50 con 0x5565f207b420 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa80438bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41abba0 con 0x5565f207bd40 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa7f37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f206c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa7f37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f206c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40622/0 (socket says 172.21.15.69:40622) 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa7f37fe700 1 -- 172.21.15.69:0/97548923 learned_addr learned my addr 172.21.15.69:0/97548923 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.318+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/97548923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207b420 0x5565f208c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7f3fff700 1 --2- 172.21.15.69:0/97548923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207bd40 0x5565f208c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7eaffd700 1 -- 172.21.15.69:0/97548923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207bd40 msgr2=0x5565f208c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/97548923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207bd40 0x5565f208c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7eaffd700 1 -- 172.21.15.69:0/97548923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 msgr2=0x5565f206c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/97548923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f206c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7eaffd700 1 -- 172.21.15.69:0/97548923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f207f150 con 0x5565f207b420 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/97548923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f206c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/97548923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207b420 0x5565f208c1c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0df860 tx=0x7fa7dc010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e7933053425a589 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7e97fa700 1 -- 172.21.15.69:0/97548923 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc020040 con 0x5565f207b420 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7e97fa700 1 -- 172.21.15.69:0/97548923 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7dc0a93f0 con 0x5565f207b420 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.319+0000 7fa7e97fa700 1 -- 172.21.15.69:0/97548923 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc0b1dc0 con 0x5565f207b420 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 -- 172.21.15.69:0/97548923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207b420 msgr2=0x5565f208c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 --2- 172.21.15.69:0/97548923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207b420 0x5565f208c1c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0df860 tx=0x7fa7dc010000 comp rx=0 tx=0).stop 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 -- 172.21.15.69:0/97548923 shutdown_connections 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 --2- 172.21.15.69:0/97548923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f20772c0 0x5565f206c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 --2- 172.21.15.69:0/97548923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207bd40 0x5565f208c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 --2- 172.21.15.69:0/97548923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f207b420 0x5565f208c1c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 -- 172.21.15.69:0/97548923 >> 172.21.15.69:0/97548923 conn(0x5565f1fa3eb0 msgr2=0x5565f208bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 -- 172.21.15.69:0/97548923 shutdown_connections 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 -- 172.21.15.69:0/97548923 wait complete. 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 -- start start 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.320+0000 7fa80438bb80 1 Processor -- start 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 0x5565f20958f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207b420 0x5565f2091c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa80438bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207bd40 0x5565f20921c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa80438bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41ac210 con 0x5565f207b420 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa80438bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41ad680 con 0x5565f20772c0 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa80438bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41ac8e0 con 0x5565f207bd40 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa80438bb80 10 client.0 _ll_register_callbacks cb 0x5565f1f5fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7eaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207b420 0x5565f2091c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7eaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207b420 0x5565f2091c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40626/0 (socket says 172.21.15.69:40626) 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7eaffd700 1 -- 172.21.15.69:0/3993378043 learned_addr learned my addr 172.21.15.69:0/3993378043 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7f3fff700 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207bd40 0x5565f20921c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 0x5565f20958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7eaffd700 1 -- 172.21.15.69:0/3993378043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207bd40 msgr2=0x5565f20921c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207bd40 0x5565f20921c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7eaffd700 1 -- 172.21.15.69:0/3993378043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 msgr2=0x5565f20958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 0x5565f20958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7eaffd700 1 -- 172.21.15.69:0/3993378043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f2092700 con 0x5565f207b420 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7eaffd700 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207b420 0x5565f2091c80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0a90a0 tx=0x7fa7dc0a9900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9394b92bc3c16da5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.321+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/3993378043 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc020070 con 0x5565f207b420 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/3993378043 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa7dc0b1fd0 con 0x5565f207b420 2024-05-14T16:54:51.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa80438bb80 1 -- 172.21.15.69:0/3993378043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7dc0a93f0 con 0x5565f207b420 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/3993378043 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa7dc0b2250 con 0x5565f207b420 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa80438bb80 10 client.15915 fetch_fsmap learned FSMap version 35 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa80438bb80 1 -- 172.21.15.69:0/3993378043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5565f20fb520 con 0x5565f207b420 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/3993378043 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7dc0a93f0 con 0x5565f207b420 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/3993378043 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fa7dc0c3ce0 con 0x5565f207b420 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa80438bb80 10 client.15915 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa80438bb80 10 client.15915 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa80438bb80 20 client.15915 populate_metadata read hostname 'smithi069' 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa80438bb80 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f20fbd00 0x5565f20fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa80438bb80 4 client.15915 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b13943b", "release": true, "await": true}] 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa80438bb80 1 -- 172.21.15.69:0/3993378043 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b13943b", "release": true, "await": true}) -- 0x5565f1fd05a0 con 0x5565f20fbd00 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.322+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f20fbd00 0x5565f20fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.323+0000 7fa7d2ffd700 10 client.15915 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:51.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.323+0000 7fa7f37fe700 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f20fbd00 0x5565f20fe0f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e24f796216e031 server_cookie=cd60e5cb739cf27b in_seq=0 out_seq=0 2024-05-14T16:54:51.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa7d2ffd700 1 -- 172.21.15.69:0/3993378043 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5565f1fd05a0 con 0x5565f20fbd00 2024-05-14T16:54:51.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa7d2ffd700 10 client.15915 handle_command_reply: tid=0 2024-05-14T16:54:51.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa80438bb80 1 client.15915 shutdown 2024-05-14T16:54:51.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa80438bb80 1 -- 172.21.15.69:0/3993378043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207b420 msgr2=0x5565f2091c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa80438bb80 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207b420 0x5565f2091c80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0a90a0 tx=0x7fa7dc0a9900 comp rx=0 tx=0).stop 2024-05-14T16:54:51.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa80438bb80 1 -- 172.21.15.69:0/3993378043 shutdown_connections 2024-05-14T16:54:51.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa80438bb80 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f20fbd00 0x5565f20fe0f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa80438bb80 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f207bd40 0x5565f20921c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa80438bb80 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f20772c0 0x5565f20958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa80438bb80 1 --2- 172.21.15.69:0/3993378043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f207b420 0x5565f2091c80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa80438bb80 1 -- 172.21.15.69:0/3993378043 >> 172.21.15.69:0/3993378043 conn(0x5565f1fa3eb0 msgr2=0x5565f208c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:51.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.335+0000 7fa7d2ffd700 0 client.15915 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:54:51.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.336+0000 7fa80438bb80 1 -- 172.21.15.69:0/3993378043 shutdown_connections 2024-05-14T16:54:51.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.336+0000 7fa80438bb80 1 -- 172.21.15.69:0/3993378043 wait complete. 2024-05-14T16:54:51.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.336+0000 7fa80438bb80 20 client.15915 trim_cache size 0 max 16384 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 35, 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 16, 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout: "b13943b": { 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 16, 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:54:51.337 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 146.8, 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:54:51.338 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:54:51.339 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:54:51.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 -- 172.21.15.69:0/2353010483 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7e00b29f0 msgr2=0x7fa7e00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7e00b29f0 0x7fa7e00b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4100a60 tx=0x7fa7e4009190 comp rx=0 tx=0).stop 2024-05-14T16:54:51.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 -- 172.21.15.69:0/2353010483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7f406c470 msgr2=0x7fa7f419fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7f406c470 0x7fa7f419fde0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00d950 tx=0x7fa7dc00dc60 comp rx=0 tx=0).stop 2024-05-14T16:54:51.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 -- 172.21.15.69:0/2353010483 shutdown_connections 2024-05-14T16:54:51.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7f41095a0 0x7fa7f4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7f406bab0 0x7fa7f419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa7e00b29f0 0x7fa7e00b4eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 --2- 172.21.15.69:0/2353010483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7f406c470 0x7fa7f419fde0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 -- 172.21.15.69:0/2353010483 >> 172.21.15.69:0/2353010483 conn(0x7fa7f40793b0 msgr2=0x7fa7f4101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:51.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 -- 172.21.15.69:0/2353010483 shutdown_connections 2024-05-14T16:54:51.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:54:51.337+0000 7fa7f9af5700 1 -- 172.21.15.69:0/2353010483 wait complete. 2024-05-14T16:54:52.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:51 smithi069 ceph-mon[31507]: pgmap v153: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-05-14T16:54:52.061 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b13943b', total seconds elapsed: 32.9 2024-05-14T16:54:52.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:51 smithi203 ceph-mon[35787]: pgmap v153: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-05-14T16:54:52.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:51 smithi092 ceph-mon[35763]: pgmap v153: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-05-14T16:54:54.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:53 smithi069 ceph-mon[31507]: pgmap v154: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 622 op/s 2024-05-14T16:54:54.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:53 smithi203 ceph-mon[35787]: pgmap v154: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 622 op/s 2024-05-14T16:54:54.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:53 smithi092 ceph-mon[35763]: pgmap v154: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 622 op/s 2024-05-14T16:54:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 265.0 s, 2313.8 tps, lat 261.238 ms stddev 2424.985 2024-05-14T16:54:56.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:55 smithi069 ceph-mon[31507]: pgmap v155: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 622 op/s 2024-05-14T16:54:56.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:55 smithi203 ceph-mon[35787]: pgmap v155: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 622 op/s 2024-05-14T16:54:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:55 smithi092 ceph-mon[35763]: pgmap v155: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 622 op/s 2024-05-14T16:54:58.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:57 smithi069 ceph-mon[31507]: pgmap v156: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 14 MiB/s wr, 882 op/s 2024-05-14T16:54:58.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:57 smithi203 ceph-mon[35787]: pgmap v156: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 14 MiB/s wr, 882 op/s 2024-05-14T16:54:58.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:57 smithi092 ceph-mon[35763]: pgmap v156: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 14 MiB/s wr, 882 op/s 2024-05-14T16:55:00.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:54:59 smithi069 ceph-mon[31507]: pgmap v157: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.8 MiB/s wr, 265 op/s 2024-05-14T16:55:00.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:54:59 smithi203 ceph-mon[35787]: pgmap v157: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.8 MiB/s wr, 265 op/s 2024-05-14T16:55:00.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:54:59 smithi092 ceph-mon[35763]: pgmap v157: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.8 MiB/s wr, 265 op/s 2024-05-14T16:55:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 270.0 s, 3735.9 tps, lat 26.782 ms stddev 28.132 2024-05-14T16:55:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.092+0000 7f9ae8445640 1 -- 172.21.15.69:0/1184157635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae0107960 msgr2=0x7f9ae0107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.092+0000 7f9ae8445640 1 --2- 172.21.15.69:0/1184157635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae0107960 0x7f9ae0107d40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009930 tx=0x7f9adc02a420 comp rx=0 tx=0).stop 2024-05-14T16:55:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.092+0000 7f9ae8445640 1 -- 172.21.15.69:0/1184157635 shutdown_connections 2024-05-14T16:55:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.092+0000 7f9ae8445640 1 --2- 172.21.15.69:0/1184157635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0102310 0x7f9ae010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.092+0000 7f9ae8445640 1 --2- 172.21.15.69:0/1184157635 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ae0101950 0x7f9ae0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.092+0000 7f9ae8445640 1 --2- 172.21.15.69:0/1184157635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae0107960 0x7f9ae0107d40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.092+0000 7f9ae8445640 1 -- 172.21.15.69:0/1184157635 >> 172.21.15.69:0/1184157635 conn(0x7f9ae0078a70 msgr2=0x7f9ae0078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.092+0000 7f9ae8445640 1 -- 172.21.15.69:0/1184157635 shutdown_connections 2024-05-14T16:55:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.092+0000 7f9ae8445640 1 -- 172.21.15.69:0/1184157635 wait complete. 2024-05-14T16:55:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae8445640 1 Processor -- start 2024-05-14T16:55:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae8445640 1 -- start start 2024-05-14T16:55:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae8445640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0101950 0x7f9ae0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae8445640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae0102310 0x7f9ae01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae8445640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ae0107960 0x7f9ae019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae8445640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae0111c20 con 0x7f9ae0102310 2024-05-14T16:55:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae8445640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae0111aa0 con 0x7f9ae0101950 2024-05-14T16:55:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae8445640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae0111da0 con 0x7f9ae0107960 2024-05-14T16:55:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae59e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae0102310 0x7f9ae01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae59e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae0102310 0x7f9ae01997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50232/0 (socket says 172.21.15.69:50232) 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae61e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0101950 0x7f9ae0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.093+0000 7f9ae59e0640 1 -- 172.21.15.69:0/860575083 learned_addr learned my addr 172.21.15.69:0/860575083 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ae0107960 0x7f9ae019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ae61e1640 1 -- 172.21.15.69:0/860575083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ae0107960 msgr2=0x7f9ae019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ae61e1640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ae0107960 0x7f9ae019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ae61e1640 1 -- 172.21.15.69:0/860575083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae0102310 msgr2=0x7f9ae01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ae61e1640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae0102310 0x7f9ae01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ae61e1640 1 -- 172.21.15.69:0/860575083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae019e230 con 0x7f9ae0101950 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ae61e1640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0101950 0x7f9ae0199290 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9adc031440 tx=0x7f9adc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ad77fe640 1 -- 172.21.15.69:0/860575083 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc03f070 con 0x7f9ae0101950 2024-05-14T16:55:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ad77fe640 1 -- 172.21.15.69:0/860575083 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9adc003aa0 con 0x7f9ae0101950 2024-05-14T16:55:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ae8445640 1 -- 172.21.15.69:0/860575083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae019e4c0 con 0x7f9ae0101950 2024-05-14T16:55:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ad77fe640 1 -- 172.21.15.69:0/860575083 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc035070 con 0x7f9ae0101950 2024-05-14T16:55:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.094+0000 7f9ae8445640 1 -- 172.21.15.69:0/860575083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae01a5db0 con 0x7f9ae0101950 2024-05-14T16:55:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.097+0000 7f9ad77fe640 1 -- 172.21.15.69:0/860575083 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9adc03b460 con 0x7f9ae0101950 2024-05-14T16:55:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.097+0000 7f9aeadfd740 1 Processor -- start 2024-05-14T16:55:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.097+0000 7f9aeadfd740 1 -- start start 2024-05-14T16:55:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.097+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d314cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d315210 0x55c78d31f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d31fad0 0x55c78d321ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c78d287c30 con 0x55c78d3148f0 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c78d2777d0 con 0x55c78d31fad0 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c78d27b810 con 0x55c78d315210 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae69e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d314cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae69e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d314cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50246/0 (socket says 172.21.15.69:50246) 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae61e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d315210 0x55c78d31f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae69e2640 1 -- 172.21.15.69:0/2329402721 learned_addr learned my addr 172.21.15.69:0/2329402721 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae59e0640 1 --2- 172.21.15.69:0/2329402721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d31fad0 0x55c78d321ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae69e2640 1 -- 172.21.15.69:0/2329402721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d315210 msgr2=0x55c78d31f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/2329402721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d315210 0x55c78d31f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae69e2640 1 -- 172.21.15.69:0/2329402721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d31fad0 msgr2=0x55c78d321ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/2329402721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d31fad0 0x55c78d321ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae69e2640 1 -- 172.21.15.69:0/2329402721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78cf8c210 con 0x55c78d3148f0 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ad77fe640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9acc0b2a90 0x7f9acc0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/2329402721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d314cd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009f50 tx=0x7f9ad80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2249a6f319b941b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ae61e1640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9acc0b2a90 0x7f9acc0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ad4bfe640 1 -- 172.21.15.69:0/2329402721 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8004d20 con 0x55c78d3148f0 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ad4bfe640 1 -- 172.21.15.69:0/2329402721 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ad8004050 con 0x55c78d3148f0 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.098+0000 7f9ad4bfe640 1 -- 172.21.15.69:0/2329402721 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8005280 con 0x55c78d3148f0 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.099+0000 7f9ad77fe640 1 -- 172.21.15.69:0/860575083 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f9adc1029c0 con 0x7f9ae0101950 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.099+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2329402721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 msgr2=0x55c78d314cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.099+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/2329402721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d314cd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009f50 tx=0x7f9ad80118d0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.099+0000 7f9ae61e1640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9acc0b2a90 0x7f9acc0b4f50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9adc1001d0 tx=0x7f9adc104040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.099+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2329402721 shutdown_connections 2024-05-14T16:55:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.099+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/2329402721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d31fad0 0x55c78d321ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.099+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/2329402721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d315210 0x55c78d31f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.099+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/2329402721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d314cd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.099+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2329402721 >> 172.21.15.69:0/2329402721 conn(0x55c78d310e00 msgr2=0x55c78d3111e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2329402721 shutdown_connections 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2329402721 wait complete. 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 1 -- start start 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 1 Processor -- start 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d3148f0 0x55c78d31f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 0x55c78d31c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d31fad0 0x55c78d31cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c78cf5d790 con 0x55c78d315210 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c78d233890 con 0x55c78d31fad0 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c78d3227a0 con 0x55c78d3148f0 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9ae59e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 0x55c78d31c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.100+0000 7f9aeadfd740 10 client.0 _ll_register_callbacks cb 0x55c78d236d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ae59e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 0x55c78d31c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50250/0 (socket says 172.21.15.69:50250) 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ae59e0640 1 -- 172.21.15.69:0/550563538 learned_addr learned my addr 172.21.15.69:0/550563538 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ae69e2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d3148f0 0x55c78d31f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d31fad0 0x55c78d31cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ae59e0640 1 -- 172.21.15.69:0/550563538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d3148f0 msgr2=0x55c78d31f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ae59e0640 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d3148f0 0x55c78d31f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ae59e0640 1 -- 172.21.15.69:0/550563538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d31fad0 msgr2=0x55c78d31cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ae59e0640 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d31fad0 0x55c78d31cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ae59e0640 1 -- 172.21.15.69:0/550563538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78d058620 con 0x55c78d315210 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ae59e0640 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 0x55c78d31c710 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0012750 tx=0x7f9ad0012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84aa430879870d95 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/550563538 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0012e60 con 0x55c78d315210 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/550563538 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f9ad000f920 con 0x55c78d315210 2024-05-14T16:55:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9aeadfd740 1 -- 172.21.15.69:0/550563538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad8004050 con 0x55c78d315210 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/550563538 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9ad000fb20 con 0x55c78d315210 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9aeadfd740 10 client.15927 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.101+0000 7f9aeadfd740 1 -- 172.21.15.69:0/550563538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c78d32e260 con 0x55c78d315210 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.102+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/550563538 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ad8004050 con 0x55c78d315210 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.102+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/550563538 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f9ad001c470 con 0x55c78d315210 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.102+0000 7f9aeadfd740 10 client.15927 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.102+0000 7f9aeadfd740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.102+0000 7f9aeadfd740 20 client.15927 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.102+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c78d32f770 0x55c78d331b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.102+0000 7f9aeadfd740 4 client.15927 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.102+0000 7f9aeadfd740 1 -- 172.21.15.69:0/550563538 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c78d289b80 con 0x55c78d32f770 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.103+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c78d32f770 0x55c78d331b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.103+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c78d32f770 0x55c78d331b60 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d41e83ceee6631 server_cookie=4c5c5f89579a9e1 in_seq=0 out_seq=0 2024-05-14T16:55:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.103+0000 7f9ac9ffb640 10 client.15927 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.103+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/550563538 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c78d289b80 con 0x55c78d32f770 2024-05-14T16:55:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9ac9ffb640 10 client.15927 handle_command_reply: tid=0 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 client.15927 shutdown 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 -- 172.21.15.69:0/550563538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 msgr2=0x55c78d31c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 0x55c78d31c710 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0012750 tx=0x7f9ad0012c10 comp rx=0 tx=0).stop 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 -- 172.21.15.69:0/550563538 shutdown_connections 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c78d32f770 0x55c78d331b60 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d31fad0 0x55c78d31cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 0x55c78d31c710 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/550563538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d3148f0 0x55c78d31f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 -- 172.21.15.69:0/550563538 >> 172.21.15.69:0/550563538 conn(0x55c78d310e00 msgr2=0x55c78d310100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9ac9ffb640 0 client.15927 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 -- 172.21.15.69:0/550563538 shutdown_connections 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 1 -- 172.21.15.69:0/550563538 wait complete. 2024-05-14T16:55:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.104+0000 7f9aeadfd740 20 client.15927 trim_cache size 0 max 16384 2024-05-14T16:55:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.124+0000 7f9aeadfd740 1 Processor -- start 2024-05-14T16:55:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.124+0000 7f9aeadfd740 1 -- start start 2024-05-14T16:55:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.125+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d3148f0 0x55c78d316d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.125+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 0x55c78d31ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.125+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d31fad0 0x55c78d337470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.125+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae01b0f30 con 0x55c78d315210 2024-05-14T16:55:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.125+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae01ae770 con 0x55c78d3148f0 2024-05-14T16:55:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.125+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae01ac5c0 con 0x55c78d31fad0 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.126+0000 7f9ae69e2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d3148f0 0x55c78d316d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.126+0000 7f9ae69e2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d3148f0 0x55c78d316d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:60520/0 (socket says 172.21.15.69:60520) 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.126+0000 7f9ae69e2640 1 -- 172.21.15.69:0/1640080729 learned_addr learned my addr 172.21.15.69:0/1640080729 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.126+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/1640080729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d31fad0 0x55c78d337470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.126+0000 7f9ae59e0640 1 --2- 172.21.15.69:0/1640080729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 0x55c78d31ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.126+0000 7f9ae69e2640 1 -- 172.21.15.69:0/1640080729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d31fad0 msgr2=0x55c78d337470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.126+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/1640080729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d31fad0 0x55c78d337470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.126+0000 7f9ae69e2640 1 -- 172.21.15.69:0/1640080729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 msgr2=0x55c78d31ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.126+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/1640080729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 0x55c78d31ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.126+0000 7f9ae69e2640 1 -- 172.21.15.69:0/1640080729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78d31ac60 con 0x55c78d3148f0 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.127+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/1640080729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d3148f0 0x55c78d316d70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8011dd0 tx=0x7f9ad80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92bb9c9deb6281ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.127+0000 7f9acbfff640 1 -- 172.21.15.69:0/1640080729 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8003df0 con 0x55c78d3148f0 2024-05-14T16:55:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.127+0000 7f9acbfff640 1 -- 172.21.15.69:0/1640080729 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ad8004db0 con 0x55c78d3148f0 2024-05-14T16:55:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.127+0000 7f9acbfff640 1 -- 172.21.15.69:0/1640080729 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad802e6d0 con 0x55c78d3148f0 2024-05-14T16:55:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.128+0000 7f9aeadfd740 1 -- 172.21.15.69:0/1640080729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d3148f0 msgr2=0x55c78d316d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.128+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/1640080729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d3148f0 0x55c78d316d70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8011dd0 tx=0x7f9ad80118d0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.128+0000 7f9aeadfd740 1 -- 172.21.15.69:0/1640080729 shutdown_connections 2024-05-14T16:55:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.128+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/1640080729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d31fad0 0x55c78d337470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.128+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/1640080729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d315210 0x55c78d31ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.128+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/1640080729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d3148f0 0x55c78d316d70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.128+0000 7f9aeadfd740 1 -- 172.21.15.69:0/1640080729 >> 172.21.15.69:0/1640080729 conn(0x55c78d310e00 msgr2=0x55c78d31c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.128+0000 7f9aeadfd740 1 -- 172.21.15.69:0/1640080729 shutdown_connections 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.128+0000 7f9aeadfd740 1 -- 172.21.15.69:0/1640080729 wait complete. 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.128+0000 7f9aeadfd740 1 -- start start 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9aeadfd740 1 Processor -- start 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d328020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d315210 0x55c78d323390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9aeadfd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d31fad0 0x55c78d3239a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae01acc30 con 0x55c78d3148f0 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae01ae0a0 con 0x55c78d315210 2024-05-14T16:55:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9aeadfd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae01ad300 con 0x55c78d31fad0 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae69e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d328020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae69e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d328020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50270/0 (socket says 172.21.15.69:50270) 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae69e2640 1 -- 172.21.15.69:0/2326659954 learned_addr learned my addr 172.21.15.69:0/2326659954 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae59e0640 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d315210 0x55c78d323390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9aeadfd740 10 client.0 _ll_register_callbacks cb 0x7f9ad0009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d31fad0 0x55c78d3239a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae59e0640 1 -- 172.21.15.69:0/2326659954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d31fad0 msgr2=0x55c78d3239a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae59e0640 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d31fad0 0x55c78d3239a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae59e0640 1 -- 172.21.15.69:0/2326659954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 msgr2=0x55c78d328020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae59e0640 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d328020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae59e0640 1 -- 172.21.15.69:0/2326659954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78d324170 con 0x55c78d315210 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d328020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ae59e0640 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d315210 0x55c78d323390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009640 tx=0x7f9ad000ff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a170f2c276e224dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.129+0000 7f9ac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/2326659954 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad001ccc0 con 0x55c78d315210 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2326659954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad8004db0 con 0x55c78d315210 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/2326659954 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f9ad000f920 con 0x55c78d315210 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/2326659954 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9ad0005040 con 0x55c78d315210 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9aeadfd740 10 client.25439 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2326659954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c78d39a140 con 0x55c78d315210 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/2326659954 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ad8004db0 con 0x55c78d315210 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/2326659954 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f9ad0004af0 con 0x55c78d315210 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9aeadfd740 10 client.25439 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9aeadfd740 10 client.25439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9aeadfd740 20 client.25439 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c78d39a920 0x55c78d39cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9aeadfd740 4 client.25439 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.130+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2326659954 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c78d30c630 con 0x55c78d39a920 2024-05-14T16:55:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.131+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c78d39a920 0x55c78d39cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.131+0000 7f9ac9ffb640 10 client.25439 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.131+0000 7f9ae69e2640 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c78d39a920 0x55c78d39cd40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35ef93232fc4967 server_cookie=954249ba10e5e269 in_seq=0 out_seq=0 2024-05-14T16:55:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9ac9ffb640 1 -- 172.21.15.69:0/2326659954 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c78d30c630 con 0x55c78d39a920 2024-05-14T16:55:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9ac9ffb640 10 client.25439 handle_command_reply: tid=0 2024-05-14T16:55:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 client.25439 shutdown 2024-05-14T16:55:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2326659954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d315210 msgr2=0x55c78d323390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d315210 0x55c78d323390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009640 tx=0x7f9ad000ff50 comp rx=0 tx=0).stop 2024-05-14T16:55:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2326659954 shutdown_connections 2024-05-14T16:55:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c78d39a920 0x55c78d39cd40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c78d31fad0 0x55c78d3239a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c78d315210 0x55c78d323390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 --2- 172.21.15.69:0/2326659954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c78d3148f0 0x55c78d328020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9ac9ffb640 0 client.25439 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2326659954 >> 172.21.15.69:0/2326659954 conn(0x55c78d310e00 msgr2=0x55c78d30f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2326659954 shutdown_connections 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 1 -- 172.21.15.69:0/2326659954 wait complete. 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.142+0000 7f9aeadfd740 20 client.25439 trim_cache size 0 max 16384 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:55:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:55:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 -- 172.21.15.69:0/860575083 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9acc0b2a90 msgr2=0x7f9acc0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9acc0b2a90 0x7f9acc0b4f50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9adc1001d0 tx=0x7f9adc104040 comp rx=0 tx=0).stop 2024-05-14T16:55:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 -- 172.21.15.69:0/860575083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0101950 msgr2=0x7f9ae0199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0101950 0x7f9ae0199290 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9adc031440 tx=0x7f9adc004270 comp rx=0 tx=0).stop 2024-05-14T16:55:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 -- 172.21.15.69:0/860575083 shutdown_connections 2024-05-14T16:55:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9acc0b2a90 0x7f9acc0b4f50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ae0107960 0x7f9ae019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae0102310 0x7f9ae01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 --2- 172.21.15.69:0/860575083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0101950 0x7f9ae0199290 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 -- 172.21.15.69:0/860575083 >> 172.21.15.69:0/860575083 conn(0x7f9ae0078a70 msgr2=0x7f9ae0100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 -- 172.21.15.69:0/860575083 shutdown_connections 2024-05-14T16:55:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:01.143+0000 7f9ae8445640 1 -- 172.21.15.69:0/860575083 wait complete. 2024-05-14T16:55:02.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:01 smithi069 ceph-mon[31507]: pgmap v158: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.0 MiB/s wr, 598 op/s 2024-05-14T16:55:02.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:01 smithi203 ceph-mon[35787]: pgmap v158: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.0 MiB/s wr, 598 op/s 2024-05-14T16:55:02.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:01 smithi092 ceph-mon[35763]: pgmap v158: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.0 MiB/s wr, 598 op/s 2024-05-14T16:55:04.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:03 smithi069 ceph-mon[31507]: pgmap v159: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.9 MiB/s wr, 592 op/s 2024-05-14T16:55:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:03 smithi203 ceph-mon[35787]: pgmap v159: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.9 MiB/s wr, 592 op/s 2024-05-14T16:55:04.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:03 smithi092 ceph-mon[35763]: pgmap v159: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.9 MiB/s wr, 592 op/s 2024-05-14T16:55:05.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:04 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:05.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:04 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:05.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:04 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:05.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 275.0 s, 1555.5 tps, lat 64.204 ms stddev 90.796 2024-05-14T16:55:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:05 smithi069 ceph-mon[31507]: pgmap v160: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.9 MiB/s wr, 592 op/s 2024-05-14T16:55:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:05 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:05 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:05 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:55:06.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:05 smithi203 ceph-mon[35787]: pgmap v160: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.9 MiB/s wr, 592 op/s 2024-05-14T16:55:06.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:05 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:06.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:05 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:06.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:05 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:55:06.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:05 smithi092 ceph-mon[35763]: pgmap v160: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.9 MiB/s wr, 592 op/s 2024-05-14T16:55:06.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:05 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:06.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:05 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:06.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:05 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:55:08.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:07 smithi069 ceph-mon[31507]: pgmap v161: 97 pgs: 97 active+clean; 2.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 1.92k op/s 2024-05-14T16:55:08.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:07 smithi203 ceph-mon[35787]: pgmap v161: 97 pgs: 97 active+clean; 2.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 1.92k op/s 2024-05-14T16:55:08.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:07 smithi092 ceph-mon[35763]: pgmap v161: 97 pgs: 97 active+clean; 2.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 1.92k op/s 2024-05-14T16:55:10.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:09 smithi069 ceph-mon[31507]: pgmap v162: 97 pgs: 97 active+clean; 2.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-05-14T16:55:10.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:09 smithi203 ceph-mon[35787]: pgmap v162: 97 pgs: 97 active+clean; 2.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-05-14T16:55:10.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:09 smithi092 ceph-mon[35763]: pgmap v162: 97 pgs: 97 active+clean; 2.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-05-14T16:55:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 280.0 s, 2870.3 tps, lat 34.658 ms stddev 37.403 2024-05-14T16:55:12.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:11 smithi069 ceph-mon[31507]: pgmap v163: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 32 MiB/s wr, 2.05k op/s 2024-05-14T16:55:12.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:11 smithi203 ceph-mon[35787]: pgmap v163: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 32 MiB/s wr, 2.05k op/s 2024-05-14T16:55:12.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:11 smithi092 ceph-mon[35763]: pgmap v163: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 32 MiB/s wr, 2.05k op/s 2024-05-14T16:55:14.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:13 smithi069 ceph-mon[31507]: pgmap v164: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2024-05-14T16:55:14.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:13 smithi203 ceph-mon[35787]: pgmap v164: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2024-05-14T16:55:14.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:13 smithi092 ceph-mon[35763]: pgmap v164: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2024-05-14T16:55:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.273+0000 7f7ae2047640 1 -- 172.21.15.69:0/498043480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7adc101980 msgr2=0x7f7adc101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.273+0000 7f7ae2047640 1 --2- 172.21.15.69:0/498043480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7adc101980 0x7f7adc101e00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009930 tx=0x7f7ac402a420 comp rx=0 tx=0).stop 2024-05-14T16:55:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.273+0000 7f7ae2047640 1 -- 172.21.15.69:0/498043480 shutdown_connections 2024-05-14T16:55:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.273+0000 7f7ae2047640 1 --2- 172.21.15.69:0/498043480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7adc102340 0x7f7adc10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.273+0000 7f7ae2047640 1 --2- 172.21.15.69:0/498043480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7adc101980 0x7f7adc101e00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.273+0000 7f7ae2047640 1 --2- 172.21.15.69:0/498043480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7adc107940 0x7f7adc107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.273+0000 7f7ae2047640 1 -- 172.21.15.69:0/498043480 >> 172.21.15.69:0/498043480 conn(0x7f7adc078ad0 msgr2=0x7f7adc078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.273+0000 7f7ae2047640 1 -- 172.21.15.69:0/498043480 shutdown_connections 2024-05-14T16:55:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.273+0000 7f7ae2047640 1 -- 172.21.15.69:0/498043480 wait complete. 2024-05-14T16:55:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.274+0000 7f7ae2047640 1 Processor -- start 2024-05-14T16:55:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.274+0000 7f7ae2047640 1 -- start start 2024-05-14T16:55:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.275+0000 7f7ae2047640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7adc101980 0x7f7adc198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.275+0000 7f7ae2047640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7adc102340 0x7f7adc1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.275+0000 7f7adb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7adc101980 0x7f7adc198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.275+0000 7f7adb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7adc101980 0x7f7adc198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:34730/0 (socket says 172.21.15.69:34730) 2024-05-14T16:55:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.275+0000 7f7adb7fe640 1 -- 172.21.15.69:0/2946535887 learned_addr learned my addr 172.21.15.69:0/2946535887 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.275+0000 7f7ae2047640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7adc107940 0x7f7adc19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.276+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc111c00 con 0x7f7adc107940 2024-05-14T16:55:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.276+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc111a80 con 0x7f7adc101980 2024-05-14T16:55:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.276+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc111d80 con 0x7f7adc102340 2024-05-14T16:55:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.276+0000 7f7adaffd640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7adc102340 0x7f7adc1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.277+0000 7f7adb7fe640 1 -- 172.21.15.69:0/2946535887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7adc102340 msgr2=0x7f7adc1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.277+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7adc102340 0x7f7adc1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.277+0000 7f7adb7fe640 1 -- 172.21.15.69:0/2946535887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7adc107940 msgr2=0x7f7adc19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.277+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7adc107940 0x7f7adc19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.277+0000 7f7adb7fe640 1 -- 172.21.15.69:0/2946535887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc19d150 con 0x7f7adc101980 2024-05-14T16:55:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.277+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7adc101980 0x7f7adc198160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009d50 tx=0x7f7ad400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.277+0000 7f7ad8ff9640 1 -- 172.21.15.69:0/2946535887 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4003b70 con 0x7f7adc101980 2024-05-14T16:55:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.277+0000 7f7ad8ff9640 1 -- 172.21.15.69:0/2946535887 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ad4003d10 con 0x7f7adc101980 2024-05-14T16:55:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.278+0000 7f7ad8ff9640 1 -- 172.21.15.69:0/2946535887 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4010960 con 0x7f7adc101980 2024-05-14T16:55:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.278+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7adc19d440 con 0x7f7adc101980 2024-05-14T16:55:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.279+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7adc1a4cd0 con 0x7f7adc101980 2024-05-14T16:55:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.282+0000 7f7ad8ff9640 1 -- 172.21.15.69:0/2946535887 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7ad40045f0 con 0x7f7adc101980 2024-05-14T16:55:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.284+0000 7f7ae4d66740 1 Processor -- start 2024-05-14T16:55:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.284+0000 7f7ae4d66740 1 -- start start 2024-05-14T16:55:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.285+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a01c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.285+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a013b30 0x55668a013f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.285+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a027370 0x55668a029760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.285+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556689f86c30 con 0x55668a013b30 2024-05-14T16:55:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.285+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556689f767d0 con 0x55668a013750 2024-05-14T16:55:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.285+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556689f7a810 con 0x55668a027370 2024-05-14T16:55:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.286+0000 7f7adaffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a027370 0x55668a029760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.286+0000 7f7adaffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a027370 0x55668a029760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:50334/0 (socket says 172.21.15.69:50334) 2024-05-14T16:55:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.286+0000 7f7adaffd640 1 -- 172.21.15.69:0/1226151492 learned_addr learned my addr 172.21.15.69:0/1226151492 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.286+0000 7f7adbfff640 1 --2- 172.21.15.69:0/1226151492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a01c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.286+0000 7f7ad8ff9640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7ac80b2a90 0x7f7ac80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.286+0000 7f7adaffd640 1 -- 172.21.15.69:0/1226151492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 msgr2=0x55668a01c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.286+0000 7f7adaffd640 1 --2- 172.21.15.69:0/1226151492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a01c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.286+0000 7f7adaffd640 1 -- 172.21.15.69:0/1226151492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a013b30 msgr2=0x55668a013f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:55:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.286+0000 7f7adbfff640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7ac80b2a90 0x7f7ac80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.287+0000 7f7adaffd640 1 --2- 172.21.15.69:0/1226151492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a013b30 0x55668a013f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.287+0000 7f7adaffd640 1 -- 172.21.15.69:0/1226151492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556689c8b210 con 0x55668a027370 2024-05-14T16:55:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.287+0000 7f7ad8ff9640 1 -- 172.21.15.69:0/2946535887 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7ad40e2d20 con 0x7f7adc101980 2024-05-14T16:55:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.288+0000 7f7adaffd640 1 --2- 172.21.15.69:0/1226151492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a027370 0x55668a029760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4000c00 tx=0x7f7ac4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee1e3a0b4c747036 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.290+0000 7f7adbfff640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7ac80b2a90 0x7f7ac80b4f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0095f0 tx=0x7f7acc005f90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.290+0000 7f7ad1ffb640 1 -- 172.21.15.69:0/1226151492 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4035030 con 0x55668a027370 2024-05-14T16:55:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.291+0000 7f7ad1ffb640 1 -- 172.21.15.69:0/1226151492 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac40054f0 con 0x55668a027370 2024-05-14T16:55:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.291+0000 7f7ad1ffb640 1 -- 172.21.15.69:0/1226151492 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4031c80 con 0x55668a027370 2024-05-14T16:55:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.292+0000 7f7ae4d66740 1 -- 172.21.15.69:0/1226151492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a027370 msgr2=0x55668a029760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.292+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/1226151492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a027370 0x55668a029760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4000c00 tx=0x7f7ac4005c00 comp rx=0 tx=0).stop 2024-05-14T16:55:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.292+0000 7f7ae4d66740 1 -- 172.21.15.69:0/1226151492 shutdown_connections 2024-05-14T16:55:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.292+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/1226151492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a027370 0x55668a029760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.292+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/1226151492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a013b30 0x55668a013f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.292+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/1226151492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a01c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.292+0000 7f7ae4d66740 1 -- 172.21.15.69:0/1226151492 >> 172.21.15.69:0/1226151492 conn(0x55668a00fe00 msgr2=0x55668a0101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.292+0000 7f7ae4d66740 1 -- 172.21.15.69:0/1226151492 shutdown_connections 2024-05-14T16:55:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.292+0000 7f7ae4d66740 1 -- 172.21.15.69:0/1226151492 wait complete. 2024-05-14T16:55:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.292+0000 7f7ae4d66740 1 -- start start 2024-05-14T16:55:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7ae4d66740 1 Processor -- start 2024-05-14T16:55:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a021b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a022070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a026890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556689c5c790 con 0x55668a027370 2024-05-14T16:55:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556689f32890 con 0x55668a013750 2024-05-14T16:55:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55668a01cdd0 con 0x55668a013b30 2024-05-14T16:55:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7ae4d66740 10 client.0 _ll_register_callbacks cb 0x556689f35d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a026890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7adaffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a022070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7adb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a021b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a026890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33120/0 (socket says 172.21.15.69:33120) 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7adaffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a022070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:50346/0 (socket says 172.21.15.69:50346) 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.293+0000 7f7adbfff640 1 -- 172.21.15.69:0/3996299915 learned_addr learned my addr 172.21.15.69:0/3996299915 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7adb7fe640 1 -- 172.21.15.69:0/3996299915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 msgr2=0x55668a022070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a022070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7adb7fe640 1 -- 172.21.15.69:0/3996299915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 msgr2=0x55668a026890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a026890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7adb7fe640 1 -- 172.21.15.69:0/3996299915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556689d57620 con 0x55668a013750 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7adbfff640 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a026890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a021b30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40e0480 tx=0x7f7ad40a6000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e3e30f6512e7ea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7abaffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7abaffd640 1 -- 172.21.15.69:0/3996299915 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40a9b20 con 0x55668a013750 2024-05-14T16:55:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7abaffd640 1 -- 172.21.15.69:0/3996299915 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7ad40e4040 con 0x55668a013750 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7ae4d66740 1 -- 172.21.15.69:0/3996299915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ac40054f0 con 0x55668a013750 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7abaffd640 1 -- 172.21.15.69:0/3996299915 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7ad40aa710 con 0x55668a013750 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7ae4d66740 10 client.25451 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7ae4d66740 1 -- 172.21.15.69:0/3996299915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55668a03e310 con 0x55668a013750 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.294+0000 7f7abaffd640 1 -- 172.21.15.69:0/3996299915 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac40054f0 con 0x55668a013750 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.295+0000 7f7abaffd640 1 -- 172.21.15.69:0/3996299915 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f7ad40e0bd0 con 0x55668a013750 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.295+0000 7f7ae4d66740 10 client.25451 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.295+0000 7f7ae4d66740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.295+0000 7f7ae4d66740 20 client.25451 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.295+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55668a03f810 0x55668a041c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.295+0000 7f7ae4d66740 4 client.25451 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.295+0000 7f7ae4d66740 1 -- 172.21.15.69:0/3996299915 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556689f88b80 con 0x55668a03f810 2024-05-14T16:55:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.295+0000 7f7adaffd640 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55668a03f810 0x55668a041c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.295+0000 7f7abaffd640 10 client.25451 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.295+0000 7f7adaffd640 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55668a03f810 0x55668a041c00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae3998c3d5da559 server_cookie=45fa3941c5a25fcf in_seq=0 out_seq=0 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7abaffd640 1 -- 172.21.15.69:0/3996299915 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556689f88b80 con 0x55668a03f810 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7abaffd640 10 client.25451 handle_command_reply: tid=0 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 client.25451 shutdown 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 -- 172.21.15.69:0/3996299915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 msgr2=0x55668a021b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a021b30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40e0480 tx=0x7f7ad40a6000 comp rx=0 tx=0).stop 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 -- 172.21.15.69:0/3996299915 shutdown_connections 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55668a03f810 0x55668a041c00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a026890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a022070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7abaffd640 0 client.25451 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/3996299915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a021b30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 -- 172.21.15.69:0/3996299915 >> 172.21.15.69:0/3996299915 conn(0x55668a00fe00 msgr2=0x55668a00f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 -- 172.21.15.69:0/3996299915 shutdown_connections 2024-05-14T16:55:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.296+0000 7f7ae4d66740 1 -- 172.21.15.69:0/3996299915 wait complete. 2024-05-14T16:55:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.297+0000 7f7ae4d66740 20 client.25451 trim_cache size 0 max 16384 2024-05-14T16:55:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7ae4d66740 1 Processor -- start 2024-05-14T16:55:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7ae4d66740 1 -- start start 2024-05-14T16:55:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a052d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a0532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a018a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1aee50 con 0x55668a027370 2024-05-14T16:55:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1ac690 con 0x55668a013750 2024-05-14T16:55:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1aa4e0 con 0x55668a013b30 2024-05-14T16:55:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a018a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adaffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a0532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a018a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33126/0 (socket says 172.21.15.69:33126) 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adaffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a0532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:50348/0 (socket says 172.21.15.69:50348) 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adbfff640 1 -- 172.21.15.69:0/1277200732 learned_addr learned my addr 172.21.15.69:0/1277200732 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/1277200732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a052d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adbfff640 1 -- 172.21.15.69:0/1277200732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 msgr2=0x55668a0532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adbfff640 1 --2- 172.21.15.69:0/1277200732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a0532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adbfff640 1 -- 172.21.15.69:0/1277200732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 msgr2=0x55668a052d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adbfff640 1 --2- 172.21.15.69:0/1277200732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a052d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adbfff640 1 -- 172.21.15.69:0/1277200732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668a014d30 con 0x55668a027370 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adaffd640 1 --2- 172.21.15.69:0/1277200732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a0532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.315+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/1277200732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a052d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.316+0000 7f7adbfff640 1 --2- 172.21.15.69:0/1277200732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a018a90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7acc012240 tx=0x7f7acc009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc899c58aacc2f6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.316+0000 7f7ad17fa640 1 -- 172.21.15.69:0/1277200732 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc018070 con 0x55668a027370 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.316+0000 7f7ad17fa640 1 -- 172.21.15.69:0/1277200732 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7acc0035f0 con 0x55668a027370 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.316+0000 7f7ad17fa640 1 -- 172.21.15.69:0/1277200732 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc020610 con 0x55668a027370 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 -- 172.21.15.69:0/1277200732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 msgr2=0x55668a018a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/1277200732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a018a90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7acc012240 tx=0x7f7acc009770 comp rx=0 tx=0).stop 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 -- 172.21.15.69:0/1277200732 shutdown_connections 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/1277200732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a018a90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/1277200732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a0532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/1277200732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a052d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 -- 172.21.15.69:0/1277200732 >> 172.21.15.69:0/1277200732 conn(0x55668a00fe00 msgr2=0x55668a021e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 -- 172.21.15.69:0/1277200732 shutdown_connections 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 -- 172.21.15.69:0/1277200732 wait complete. 2024-05-14T16:55:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 -- start start 2024-05-14T16:55:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.317+0000 7f7ae4d66740 1 Processor -- start 2024-05-14T16:55:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.318+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a09dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a09e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7ae4d66740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a0a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1aab50 con 0x55668a027370 2024-05-14T16:55:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1abfc0 con 0x55668a013750 2024-05-14T16:55:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7ae4d66740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1ab220 con 0x55668a013b30 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7ae4d66740 10 client.0 _ll_register_callbacks cb 0x7f7ad40098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adaffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a09e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adaffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a09e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:50356/0 (socket says 172.21.15.69:50356) 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a0a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adaffd640 1 -- 172.21.15.69:0/2264605398 learned_addr learned my addr 172.21.15.69:0/2264605398 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a09dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adaffd640 1 -- 172.21.15.69:0/2264605398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 msgr2=0x55668a09dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adaffd640 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a09dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adaffd640 1 -- 172.21.15.69:0/2264605398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 msgr2=0x55668a0a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adaffd640 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a0a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adaffd640 1 -- 172.21.15.69:0/2264605398 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668a0a9200 con 0x55668a013b30 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a09dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.319+0000 7f7adbfff640 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a0a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.320+0000 7f7adaffd640 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a09e180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55668a014d00 tx=0x7f7ac402a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3bcc3a5ccb1e9aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.320+0000 7f7abaffd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.320+0000 7f7abaffd640 1 -- 172.21.15.69:0/2264605398 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4035040 con 0x55668a013b30 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.320+0000 7f7abaffd640 1 -- 172.21.15.69:0/2264605398 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7ac4005230 con 0x55668a013b30 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.320+0000 7f7ae4d66740 1 -- 172.21.15.69:0/2264605398 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7acc0035f0 con 0x55668a013b30 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.320+0000 7f7abaffd640 1 -- 172.21.15.69:0/2264605398 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7ac4031e10 con 0x55668a013b30 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.320+0000 7f7ae4d66740 10 client.25399 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.320+0000 7f7ae4d66740 1 -- 172.21.15.69:0/2264605398 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55668a0aa3c0 con 0x55668a013b30 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.321+0000 7f7abaffd640 1 -- 172.21.15.69:0/2264605398 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7acc0035f0 con 0x55668a013b30 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.321+0000 7f7abaffd640 1 -- 172.21.15.69:0/2264605398 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f7ac4051120 con 0x55668a013b30 2024-05-14T16:55:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.321+0000 7f7ae4d66740 10 client.25399 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.321+0000 7f7ae4d66740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.321+0000 7f7ae4d66740 20 client.25399 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.321+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55668a0aa750 0x55668a0acb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.321+0000 7f7ae4d66740 4 client.25399 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:55:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.321+0000 7f7ae4d66740 1 -- 172.21.15.69:0/2264605398 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556689f82360 con 0x55668a0aa750 2024-05-14T16:55:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.321+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55668a0aa750 0x55668a0acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.322+0000 7f7abaffd640 10 client.25399 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.322+0000 7f7adb7fe640 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55668a0aa750 0x55668a0acb40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb479657c4a45153 server_cookie=e3b479602c4af174 in_seq=0 out_seq=0 2024-05-14T16:55:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.335+0000 7f7abaffd640 1 -- 172.21.15.69:0/2264605398 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556689f82360 con 0x55668a0aa750 2024-05-14T16:55:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.335+0000 7f7abaffd640 10 client.25399 handle_command_reply: tid=0 2024-05-14T16:55:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.335+0000 7f7ae4d66740 1 client.25399 shutdown 2024-05-14T16:55:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.336+0000 7f7ae4d66740 1 -- 172.21.15.69:0/2264605398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 msgr2=0x55668a09e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.336+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a09e180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55668a014d00 tx=0x7f7ac402a420 comp rx=0 tx=0).stop 2024-05-14T16:55:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.337+0000 7f7ae4d66740 1 -- 172.21.15.69:0/2264605398 shutdown_connections 2024-05-14T16:55:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.337+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55668a0aa750 0x55668a0acb40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.337+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668a027370 0x55668a0a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.337+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55668a013b30 0x55668a09e180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.337+0000 7f7ae4d66740 1 --2- 172.21.15.69:0/2264605398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55668a013750 0x55668a09dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.337+0000 7f7ae4d66740 1 -- 172.21.15.69:0/2264605398 >> 172.21.15.69:0/2264605398 conn(0x55668a00fe00 msgr2=0x55668a00e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.337+0000 7f7abaffd640 0 client.25399 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.337+0000 7f7ae4d66740 1 -- 172.21.15.69:0/2264605398 shutdown_connections 2024-05-14T16:55:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.337+0000 7f7ae4d66740 1 -- 172.21.15.69:0/2264605398 wait complete. 2024-05-14T16:55:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.337+0000 7f7ae4d66740 20 client.25399 trim_cache size 0 max 16384 2024-05-14T16:55:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:55:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:55:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:55:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.338+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7ac80b2a90 msgr2=0x7f7ac80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.338+0000 7f7ae2047640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7ac80b2a90 0x7f7ac80b4f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0095f0 tx=0x7f7acc005f90 comp rx=0 tx=0).stop 2024-05-14T16:55:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.338+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7adc101980 msgr2=0x7f7adc198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.338+0000 7f7ae2047640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7adc101980 0x7f7adc198160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009d50 tx=0x7f7ad400c6a0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.338+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 shutdown_connections 2024-05-14T16:55:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.338+0000 7f7ae2047640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7ac80b2a90 0x7f7ac80b4f50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.338+0000 7f7ae2047640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7adc107940 0x7f7adc19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.338+0000 7f7ae2047640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7adc102340 0x7f7adc1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.338+0000 7f7ae2047640 1 --2- 172.21.15.69:0/2946535887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7adc101980 0x7f7adc198160 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009d50 tx=0x7f7ad400c6a0 comp rx=0 tx=0).stop 2024-05-14T16:55:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.338+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 >> 172.21.15.69:0/2946535887 conn(0x7f7adc078ad0 msgr2=0x7f7adc104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.339+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 shutdown_connections 2024-05-14T16:55:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:14.339+0000 7f7ae2047640 1 -- 172.21.15.69:0/2946535887 wait complete. 2024-05-14T16:55:15.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 285.0 s, 3697.3 tps, lat 27.218 ms stddev 33.339 2024-05-14T16:55:16.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:15 smithi069 ceph-mon[31507]: pgmap v165: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2024-05-14T16:55:16.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:15 smithi203 ceph-mon[35787]: pgmap v165: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2024-05-14T16:55:16.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:15 smithi092 ceph-mon[35763]: pgmap v165: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2024-05-14T16:55:18.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:17 smithi069 ceph-mon[31507]: pgmap v166: 97 pgs: 97 active+clean; 3.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 MiB/s wr, 2.07k op/s 2024-05-14T16:55:18.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:17 smithi203 ceph-mon[35787]: pgmap v166: 97 pgs: 97 active+clean; 3.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 MiB/s wr, 2.07k op/s 2024-05-14T16:55:18.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:17 smithi092 ceph-mon[35763]: pgmap v166: 97 pgs: 97 active+clean; 3.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 MiB/s wr, 2.07k op/s 2024-05-14T16:55:20.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:19 smithi069 ceph-mon[31507]: pgmap v167: 97 pgs: 97 active+clean; 3.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 742 op/s 2024-05-14T16:55:20.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:19 smithi203 ceph-mon[35787]: pgmap v167: 97 pgs: 97 active+clean; 3.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 742 op/s 2024-05-14T16:55:20.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:19 smithi092 ceph-mon[35763]: pgmap v167: 97 pgs: 97 active+clean; 3.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 742 op/s 2024-05-14T16:55:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 290.0 s, 3428.2 tps, lat 29.163 ms stddev 33.055 2024-05-14T16:55:22.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:21 smithi069 ceph-mon[31507]: pgmap v168: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 MiB/s wr, 1.06k op/s 2024-05-14T16:55:22.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:21 smithi203 ceph-mon[35787]: pgmap v168: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 MiB/s wr, 1.06k op/s 2024-05-14T16:55:22.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:21 smithi092 ceph-mon[35763]: pgmap v168: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 MiB/s wr, 1.06k op/s 2024-05-14T16:55:24.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:23 smithi069 ceph-mon[31507]: pgmap v169: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 664 op/s 2024-05-14T16:55:24.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:23 smithi203 ceph-mon[35787]: pgmap v169: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 664 op/s 2024-05-14T16:55:24.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:23 smithi092 ceph-mon[35763]: pgmap v169: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 664 op/s 2024-05-14T16:55:25.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 295.0 s, 3681.2 tps, lat 27.242 ms stddev 32.865 2024-05-14T16:55:26.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:25 smithi069 ceph-mon[31507]: pgmap v170: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 664 op/s 2024-05-14T16:55:26.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:25 smithi203 ceph-mon[35787]: pgmap v170: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 664 op/s 2024-05-14T16:55:26.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:25 smithi092 ceph-mon[35763]: pgmap v170: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 664 op/s 2024-05-14T16:55:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.483+0000 7fe057df0640 1 -- 172.21.15.69:0/2794002593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe050111240 msgr2=0x7fe050109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.483+0000 7fe057df0640 1 --2- 172.21.15.69:0/2794002593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe050111240 0x7fe050109a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe044009be0 tx=0x7fe04402a790 comp rx=0 tx=0).stop 2024-05-14T16:55:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.483+0000 7fe057df0640 1 -- 172.21.15.69:0/2794002593 shutdown_connections 2024-05-14T16:55:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.483+0000 7fe057df0640 1 --2- 172.21.15.69:0/2794002593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe05011c230 0x7fe05011e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.483+0000 7fe057df0640 1 --2- 172.21.15.69:0/2794002593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050111620 0x7fe050109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.483+0000 7fe057df0640 1 --2- 172.21.15.69:0/2794002593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe050111240 0x7fe050109a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.483+0000 7fe057df0640 1 -- 172.21.15.69:0/2794002593 >> 172.21.15.69:0/2794002593 conn(0x7fe05006db20 msgr2=0x7fe05006df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.490+0000 7fe057df0640 1 -- 172.21.15.69:0/2794002593 shutdown_connections 2024-05-14T16:55:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.491+0000 7fe057df0640 1 -- 172.21.15.69:0/2794002593 wait complete. 2024-05-14T16:55:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.491+0000 7fe057df0640 1 Processor -- start 2024-05-14T16:55:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.494+0000 7fe057df0640 1 -- start start 2024-05-14T16:55:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.495+0000 7fe057df0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050111240 0x7fe0501197a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.495+0000 7fe057df0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05011c230 0x7fe050119ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.495+0000 7fe057df0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050112930 0x7fe050112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.495+0000 7fe057df0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe050121b50 con 0x7fe050112930 2024-05-14T16:55:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.495+0000 7fe057df0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe0501219d0 con 0x7fe050111240 2024-05-14T16:55:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.495+0000 7fe057df0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe050121cd0 con 0x7fe05011c230 2024-05-14T16:55:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.495+0000 7fe055b8c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050111240 0x7fe0501197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.495+0000 7fe055b8c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050111240 0x7fe0501197a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:34076/0 (socket says 172.21.15.69:34076) 2024-05-14T16:55:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.495+0000 7fe055b8c640 1 -- 172.21.15.69:0/2642923862 learned_addr learned my addr 172.21.15.69:0/2642923862 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.497+0000 7fe05638d640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050112930 0x7fe050112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.497+0000 7fe05538b640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05011c230 0x7fe050119ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.497+0000 7fe05638d640 1 -- 172.21.15.69:0/2642923862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05011c230 msgr2=0x7fe050119ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.497+0000 7fe05638d640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05011c230 0x7fe050119ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.497+0000 7fe05638d640 1 -- 172.21.15.69:0/2642923862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050111240 msgr2=0x7fe0501197a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.497+0000 7fe05638d640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050111240 0x7fe0501197a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.497+0000 7fe05638d640 1 -- 172.21.15.69:0/2642923862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0501172c0 con 0x7fe050112930 2024-05-14T16:55:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.498+0000 7fe05638d640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050112930 0x7fe050112de0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe05010ecf0 tx=0x7fe04800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.499+0000 7fe04effd640 1 -- 172.21.15.69:0/2642923862 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04800cc90 con 0x7fe050112930 2024-05-14T16:55:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.499+0000 7fe04effd640 1 -- 172.21.15.69:0/2642923862 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe04800ce30 con 0x7fe050112930 2024-05-14T16:55:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.499+0000 7fe04effd640 1 -- 172.21.15.69:0/2642923862 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048010430 con 0x7fe050112930 2024-05-14T16:55:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.499+0000 7fe057df0640 1 -- 172.21.15.69:0/2642923862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0501175b0 con 0x7fe050112930 2024-05-14T16:55:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.499+0000 7fe057df0640 1 -- 172.21.15.69:0/2642923862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe050117ab0 con 0x7fe050112930 2024-05-14T16:55:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.504+0000 7fe05abe7740 1 Processor -- start 2024-05-14T16:55:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.506+0000 7fe04effd640 1 -- 172.21.15.69:0/2642923862 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe0480035f0 con 0x7fe050112930 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.506+0000 7fe05abe7740 1 -- start start 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 0x559a5ea68cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 0x559a5ea73590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea75ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05abe7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559a5e9dbc30 con 0x559a5ea73ad0 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05abe7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559a5e9cb7d0 con 0x559a5ea688f0 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05abe7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559a5e9cf810 con 0x559a5ea69210 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05538b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 0x559a5ea68cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05538b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 0x559a5ea68cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:34090/0 (socket says 172.21.15.69:34090) 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05538b640 1 -- 172.21.15.69:0/1764344998 learned_addr learned my addr 172.21.15.69:0/1764344998 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05538b640 1 --2- 172.21.15.69:0/1764344998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea75ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05538b640 1 -- 172.21.15.69:0/1764344998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 msgr2=0x559a5ea73590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:55:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05538b640 1 --2- 172.21.15.69:0/1764344998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 0x559a5ea73590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05538b640 1 -- 172.21.15.69:0/1764344998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 msgr2=0x559a5ea75ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05538b640 1 --2- 172.21.15.69:0/1764344998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea75ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.507+0000 7fe05538b640 1 -- 172.21.15.69:0/1764344998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a5e6e0210 con 0x559a5ea688f0 2024-05-14T16:55:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.508+0000 7fe04effd640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe0400b2ba0 0x7fe0400b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.508+0000 7fe04effd640 1 -- 172.21.15.69:0/2642923862 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe0480e2730 con 0x7fe050112930 2024-05-14T16:55:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.508+0000 7fe055b8c640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe0400b2ba0 0x7fe0400b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.509+0000 7fe05538b640 1 --2- 172.21.15.69:0/1764344998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 0x559a5ea68cd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe03c005fd0 tx=0x7fe03c00d360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89f598cb0e3064bc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.509+0000 7fe03bfff640 1 -- 172.21.15.69:0/1764344998 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c005220 con 0x559a5ea688f0 2024-05-14T16:55:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.509+0000 7fe03bfff640 1 -- 172.21.15.69:0/1764344998 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe03c004c50 con 0x559a5ea688f0 2024-05-14T16:55:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.509+0000 7fe03bfff640 1 -- 172.21.15.69:0/1764344998 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c00a8b0 con 0x559a5ea688f0 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.510+0000 7fe05abe7740 1 -- 172.21.15.69:0/1764344998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 msgr2=0x559a5ea68cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.510+0000 7fe05abe7740 1 --2- 172.21.15.69:0/1764344998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 0x559a5ea68cd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe03c005fd0 tx=0x7fe03c00d360 comp rx=0 tx=0).stop 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.510+0000 7fe05abe7740 1 -- 172.21.15.69:0/1764344998 shutdown_connections 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.510+0000 7fe05abe7740 1 --2- 172.21.15.69:0/1764344998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea75ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.510+0000 7fe05abe7740 1 --2- 172.21.15.69:0/1764344998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 0x559a5ea73590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.510+0000 7fe05abe7740 1 --2- 172.21.15.69:0/1764344998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 0x559a5ea68cd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.510+0000 7fe05abe7740 1 -- 172.21.15.69:0/1764344998 >> 172.21.15.69:0/1764344998 conn(0x559a5ea64e00 msgr2=0x559a5ea651e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.511+0000 7fe055b8c640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe0400b2ba0 0x7fe0400b5060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe04402ad80 tx=0x7fe044030680 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.511+0000 7fe05abe7740 1 -- 172.21.15.69:0/1764344998 shutdown_connections 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.511+0000 7fe05abe7740 1 -- 172.21.15.69:0/1764344998 wait complete. 2024-05-14T16:55:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.512+0000 7fe05abe7740 1 -- start start 2024-05-14T16:55:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.512+0000 7fe05abe7740 1 Processor -- start 2024-05-14T16:55:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.513+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.513+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea69210 0x559a5ea706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.513+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea70be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.513+0000 7fe05abe7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559a5e6b1790 con 0x559a5ea73ad0 2024-05-14T16:55:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.513+0000 7fe05abe7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559a5e987890 con 0x559a5ea69210 2024-05-14T16:55:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.513+0000 7fe05abe7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559a5ea767a0 con 0x559a5ea688f0 2024-05-14T16:55:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.513+0000 7fe05abe7740 10 client.0 _ll_register_callbacks cb 0x559a5e98ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.514+0000 7fe055b8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.514+0000 7fe055b8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea733b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:40068/0 (socket says 172.21.15.69:40068) 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.514+0000 7fe055b8c640 1 -- 172.21.15.69:0/2503481914 learned_addr learned my addr 172.21.15.69:0/2503481914 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.514+0000 7fe05638d640 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea70be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.514+0000 7fe05538b640 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea69210 0x559a5ea706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.514+0000 7fe055b8c640 1 -- 172.21.15.69:0/2503481914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea69210 msgr2=0x559a5ea706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.514+0000 7fe055b8c640 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea69210 0x559a5ea706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.514+0000 7fe055b8c640 1 -- 172.21.15.69:0/2503481914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 msgr2=0x559a5ea70be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.514+0000 7fe055b8c640 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea70be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.514+0000 7fe055b8c640 1 -- 172.21.15.69:0/2503481914 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a5e7ac620 con 0x559a5ea688f0 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.515+0000 7fe055b8c640 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea733b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe044003dc0 tx=0x7fe044009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdde41729dc09ac9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.515+0000 7fe0397fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:55:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.516+0000 7fe0397fa640 1 -- 172.21.15.69:0/2503481914 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044041070 con 0x559a5ea688f0 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.516+0000 7fe0397fa640 1 -- 172.21.15.69:0/2503481914 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe044004040 con 0x559a5ea688f0 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.516+0000 7fe05abe7740 1 -- 172.21.15.69:0/2503481914 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe03c004c50 con 0x559a5ea688f0 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.516+0000 7fe0397fa640 1 -- 172.21.15.69:0/2503481914 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe044036040 con 0x559a5ea688f0 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.516+0000 7fe05abe7740 10 client.25405 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.516+0000 7fe05abe7740 1 -- 172.21.15.69:0/2503481914 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559a5ea821e0 con 0x559a5ea688f0 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe0397fa640 1 -- 172.21.15.69:0/2503481914 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe03c004c50 con 0x559a5ea688f0 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe0397fa640 1 -- 172.21.15.69:0/2503481914 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fe0440488c0 con 0x559a5ea688f0 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe05abe7740 10 client.25405 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe05abe7740 10 client.25405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe05abe7740 20 client.25405 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559a5ea83bd0 0x559a5ea86050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe05abe7740 4 client.25405 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe05abe7740 1 -- 172.21.15.69:0/2503481914 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a5e9ddb80 con 0x559a5ea83bd0 2024-05-14T16:55:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe05538b640 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559a5ea83bd0 0x559a5ea86050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe0397fa640 10 client.25405 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.517+0000 7fe05538b640 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559a5ea83bd0 0x559a5ea86050 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0f856003c08169 server_cookie=c927b39276613eac in_seq=0 out_seq=0 2024-05-14T16:55:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.519+0000 7fe0397fa640 1 -- 172.21.15.69:0/2503481914 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a5e9ddb80 con 0x559a5ea83bd0 2024-05-14T16:55:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.519+0000 7fe0397fa640 10 client.25405 handle_command_reply: tid=0 2024-05-14T16:55:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.519+0000 7fe05abe7740 1 client.25405 shutdown 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.519+0000 7fe05abe7740 1 -- 172.21.15.69:0/2503481914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 msgr2=0x559a5ea733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.519+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea733b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe044003dc0 tx=0x7fe044009500 comp rx=0 tx=0).stop 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.520+0000 7fe05abe7740 1 -- 172.21.15.69:0/2503481914 shutdown_connections 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.520+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559a5ea83bd0 0x559a5ea86050 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.520+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea70be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.520+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea69210 0x559a5ea706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.520+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2503481914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea733b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.520+0000 7fe05abe7740 1 -- 172.21.15.69:0/2503481914 >> 172.21.15.69:0/2503481914 conn(0x559a5ea64e00 msgr2=0x559a5ea64100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.520+0000 7fe0397fa640 0 client.25405 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.520+0000 7fe05abe7740 1 -- 172.21.15.69:0/2503481914 shutdown_connections 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.520+0000 7fe05abe7740 1 -- 172.21.15.69:0/2503481914 wait complete. 2024-05-14T16:55:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.520+0000 7fe05abe7740 20 client.25405 trim_cache size 0 max 16384 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.544+0000 7fe05abe7740 1 Processor -- start 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.544+0000 7fe05abe7740 1 -- start start 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.544+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea7f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea69210 0x559a5ea72ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea73ad0 0x559a5ea8b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe05abe7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe034011160 con 0x559a5ea69210 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe05abe7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe03400e9a0 con 0x559a5ea73ad0 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe05abe7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe03400c7f0 con 0x559a5ea688f0 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe055b8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea7f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe055b8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea7f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:40070/0 (socket says 172.21.15.69:40070) 2024-05-14T16:55:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe055b8c640 1 -- 172.21.15.69:0/3706981607 learned_addr learned my addr 172.21.15.69:0/3706981607 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe05538b640 1 --2- 172.21.15.69:0/3706981607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea69210 0x559a5ea72ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe05638d640 1 --2- 172.21.15.69:0/3706981607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea73ad0 0x559a5ea8b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe055b8c640 1 -- 172.21.15.69:0/3706981607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea73ad0 msgr2=0x559a5ea8b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe055b8c640 1 --2- 172.21.15.69:0/3706981607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea73ad0 0x559a5ea8b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe055b8c640 1 -- 172.21.15.69:0/3706981607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea69210 msgr2=0x559a5ea72ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe055b8c640 1 --2- 172.21.15.69:0/3706981607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea69210 0x559a5ea72ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.545+0000 7fe055b8c640 1 -- 172.21.15.69:0/3706981607 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a5ea767a0 con 0x559a5ea688f0 2024-05-14T16:55:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.546+0000 7fe055b8c640 1 --2- 172.21.15.69:0/3706981607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea7f6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe044048ce0 tx=0x7fe044031650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf4497a6c39a975 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.546+0000 7fe03b7fe640 1 -- 172.21.15.69:0/3706981607 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044041040 con 0x559a5ea688f0 2024-05-14T16:55:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.546+0000 7fe03b7fe640 1 -- 172.21.15.69:0/3706981607 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe04404d070 con 0x559a5ea688f0 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.546+0000 7fe03b7fe640 1 -- 172.21.15.69:0/3706981607 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044031c70 con 0x559a5ea688f0 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.547+0000 7fe05abe7740 1 -- 172.21.15.69:0/3706981607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 msgr2=0x559a5ea7f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.547+0000 7fe05abe7740 1 --2- 172.21.15.69:0/3706981607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea7f6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe044048ce0 tx=0x7fe044031650 comp rx=0 tx=0).stop 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.547+0000 7fe05abe7740 1 -- 172.21.15.69:0/3706981607 shutdown_connections 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.547+0000 7fe05abe7740 1 --2- 172.21.15.69:0/3706981607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea73ad0 0x559a5ea8b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.547+0000 7fe05abe7740 1 --2- 172.21.15.69:0/3706981607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea69210 0x559a5ea72ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.547+0000 7fe05abe7740 1 --2- 172.21.15.69:0/3706981607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea688f0 0x559a5ea7f6d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.547+0000 7fe05abe7740 1 -- 172.21.15.69:0/3706981607 >> 172.21.15.69:0/3706981607 conn(0x559a5ea64e00 msgr2=0x559a5ea6ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.547+0000 7fe05abe7740 1 -- 172.21.15.69:0/3706981607 shutdown_connections 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.547+0000 7fe05abe7740 1 -- 172.21.15.69:0/3706981607 wait complete. 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.548+0000 7fe05abe7740 1 -- start start 2024-05-14T16:55:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.548+0000 7fe05abe7740 1 Processor -- start 2024-05-14T16:55:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.549+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 0x559a5ea7ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.549+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 0x559a5ea7bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05abe7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea6ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05538b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 0x559a5ea7bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05538b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 0x559a5ea7bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:40086/0 (socket says 172.21.15.69:40086) 2024-05-14T16:55:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05538b640 1 -- 172.21.15.69:0/2952441656 learned_addr learned my addr 172.21.15.69:0/2952441656 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe03400ce60 con 0x559a5ea73ad0 2024-05-14T16:55:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05638d640 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea6ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe03400e2d0 con 0x559a5ea688f0 2024-05-14T16:55:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe03400d530 con 0x559a5ea69210 2024-05-14T16:55:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe055b8c640 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 0x559a5ea7ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05abe7740 10 client.0 _ll_register_callbacks cb 0x7fe044009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05538b640 1 -- 172.21.15.69:0/2952441656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 msgr2=0x559a5ea7ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05538b640 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 0x559a5ea7ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05538b640 1 -- 172.21.15.69:0/2952441656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 msgr2=0x559a5ea6ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05538b640 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea6ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.550+0000 7fe05538b640 1 -- 172.21.15.69:0/2952441656 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a5ea6d0a0 con 0x559a5ea69210 2024-05-14T16:55:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.551+0000 7fe05538b640 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 0x559a5ea7bf90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe03c012760 tx=0x7fe03c00da40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cecbb4a6046988a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.551+0000 7fe0397fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.551+0000 7fe0397fa640 1 -- 172.21.15.69:0/2952441656 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c012450 con 0x559a5ea69210 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.551+0000 7fe0397fa640 1 -- 172.21.15.69:0/2952441656 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe03c00a620 con 0x559a5ea69210 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.551+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe04404d070 con 0x559a5ea69210 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.551+0000 7fe0397fa640 1 -- 172.21.15.69:0/2952441656 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe03c05c050 con 0x559a5ea69210 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.551+0000 7fe05abe7740 10 client.25411 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.551+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559a5eaeddc0 con 0x559a5ea69210 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.552+0000 7fe0397fa640 1 -- 172.21.15.69:0/2952441656 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe04404d070 con 0x559a5ea69210 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.552+0000 7fe0397fa640 1 -- 172.21.15.69:0/2952441656 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fe03c03d030 con 0x559a5ea69210 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.552+0000 7fe05abe7740 10 client.25411 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.552+0000 7fe05abe7740 10 client.25411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.552+0000 7fe05abe7740 20 client.25411 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.552+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559a5eaee380 0x559a5eaf07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.552+0000 7fe05abe7740 4 client.25411 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:55:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.552+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a5e9d7360 con 0x559a5eaee380 2024-05-14T16:55:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.553+0000 7fe055b8c640 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559a5eaee380 0x559a5eaf07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.554+0000 7fe055b8c640 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559a5eaee380 0x559a5eaf07a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e62ce5c182902cac server_cookie=789bf27faa2d5cf6 in_seq=0 out_seq=0 2024-05-14T16:55:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.554+0000 7fe0397fa640 10 client.25411 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.570+0000 7fe0397fa640 1 -- 172.21.15.69:0/2952441656 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559a5e9d7360 con 0x559a5eaee380 2024-05-14T16:55:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.570+0000 7fe0397fa640 10 client.25411 handle_command_reply: tid=0 2024-05-14T16:55:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.570+0000 7fe05abe7740 1 client.25411 shutdown 2024-05-14T16:55:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.571+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 msgr2=0x559a5ea7bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.571+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 0x559a5ea7bf90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe03c012760 tx=0x7fe03c00da40 comp rx=0 tx=0).stop 2024-05-14T16:55:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.573+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 shutdown_connections 2024-05-14T16:55:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.573+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559a5eaee380 0x559a5eaf07a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.573+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a5ea73ad0 0x559a5ea6ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.573+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559a5ea69210 0x559a5ea7bf90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.573+0000 7fe05abe7740 1 --2- 172.21.15.69:0/2952441656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a5ea688f0 0x559a5ea7ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.573+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 >> 172.21.15.69:0/2952441656 conn(0x559a5ea64e00 msgr2=0x559a5ea6b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.574+0000 7fe0397fa640 0 client.25411 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.575+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 shutdown_connections 2024-05-14T16:55:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.575+0000 7fe05abe7740 1 -- 172.21.15.69:0/2952441656 wait complete. 2024-05-14T16:55:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:55:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:55:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:55:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.575+0000 7fe05abe7740 20 client.25411 trim_cache size 0 max 16384 2024-05-14T16:55:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.577+0000 7fe057df0640 1 -- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe0400b2ba0 msgr2=0x7fe0400b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.577+0000 7fe057df0640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe0400b2ba0 0x7fe0400b5060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe04402ad80 tx=0x7fe044030680 comp rx=0 tx=0).stop 2024-05-14T16:55:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.577+0000 7fe057df0640 1 -- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050112930 msgr2=0x7fe050112de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.577+0000 7fe057df0640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050112930 0x7fe050112de0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe05010ecf0 tx=0x7fe04800ed20 comp rx=0 tx=0).stop 2024-05-14T16:55:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.577+0000 7fe057df0640 1 -- 172.21.15.69:0/2642923862 shutdown_connections 2024-05-14T16:55:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.577+0000 7fe057df0640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe0400b2ba0 0x7fe0400b5060 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.577+0000 7fe057df0640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe050112930 0x7fe050112de0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.577+0000 7fe057df0640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05011c230 0x7fe050119ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.577+0000 7fe057df0640 1 --2- 172.21.15.69:0/2642923862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050111240 0x7fe0501197a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.577+0000 7fe057df0640 1 -- 172.21.15.69:0/2642923862 >> 172.21.15.69:0/2642923862 conn(0x7fe05006db20 msgr2=0x7fe05010ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.578+0000 7fe057df0640 1 -- 172.21.15.69:0/2642923862 shutdown_connections 2024-05-14T16:55:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:27.578+0000 7fe057df0640 1 -- 172.21.15.69:0/2642923862 wait complete. 2024-05-14T16:55:28.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:27 smithi069 ceph-mon[31507]: pgmap v171: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 54 MiB/s wr, 1.19k op/s 2024-05-14T16:55:28.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:27 smithi203 ceph-mon[35787]: pgmap v171: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 54 MiB/s wr, 1.19k op/s 2024-05-14T16:55:28.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:27 smithi092 ceph-mon[35763]: pgmap v171: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 54 MiB/s wr, 1.19k op/s 2024-05-14T16:55:30.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:29 smithi069 ceph-mon[31507]: pgmap v172: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 32 MiB/s wr, 844 op/s 2024-05-14T16:55:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:29 smithi203 ceph-mon[35787]: pgmap v172: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 32 MiB/s wr, 844 op/s 2024-05-14T16:55:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:29 smithi092 ceph-mon[35763]: pgmap v172: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 32 MiB/s wr, 844 op/s 2024-05-14T16:55:30.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 300.0 s, 3607.2 tps, lat 27.644 ms stddev 56.343 2024-05-14T16:55:32.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:31 smithi069 ceph-mon[31507]: pgmap v173: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 64 MiB/s wr, 2.19k op/s 2024-05-14T16:55:32.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:31 smithi203 ceph-mon[35787]: pgmap v173: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 64 MiB/s wr, 2.19k op/s 2024-05-14T16:55:32.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:31 smithi092 ceph-mon[35763]: pgmap v173: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 64 MiB/s wr, 2.19k op/s 2024-05-14T16:55:34.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:33 smithi069 ceph-mon[31507]: pgmap v174: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 48 MiB/s wr, 1.88k op/s 2024-05-14T16:55:34.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:33 smithi203 ceph-mon[35787]: pgmap v174: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 48 MiB/s wr, 1.88k op/s 2024-05-14T16:55:34.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:33 smithi092 ceph-mon[35763]: pgmap v174: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 48 MiB/s wr, 1.88k op/s 2024-05-14T16:55:35.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 305.0 s, 4050.2 tps, lat 24.662 ms stddev 26.273 2024-05-14T16:55:36.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:35 smithi069 ceph-mon[31507]: pgmap v175: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 48 MiB/s wr, 1.88k op/s 2024-05-14T16:55:36.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:35 smithi203 ceph-mon[35787]: pgmap v175: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 48 MiB/s wr, 1.88k op/s 2024-05-14T16:55:36.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:35 smithi092 ceph-mon[35763]: pgmap v175: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 48 MiB/s wr, 1.88k op/s 2024-05-14T16:55:38.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:37 smithi069 ceph-mon[31507]: pgmap v176: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 54 MiB/s wr, 2.21k op/s 2024-05-14T16:55:38.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:37 smithi203 ceph-mon[35787]: pgmap v176: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 54 MiB/s wr, 2.21k op/s 2024-05-14T16:55:38.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:37 smithi092 ceph-mon[35763]: pgmap v176: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 54 MiB/s wr, 2.21k op/s 2024-05-14T16:55:40.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:39 smithi069 ceph-mon[31507]: pgmap v177: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 37 MiB/s wr, 1.68k op/s 2024-05-14T16:55:40.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:39 smithi203 ceph-mon[35787]: pgmap v177: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 37 MiB/s wr, 1.68k op/s 2024-05-14T16:55:40.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:39 smithi092 ceph-mon[35763]: pgmap v177: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 37 MiB/s wr, 1.68k op/s 2024-05-14T16:55:40.395 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 310.0 s, 2322.8 tps, lat 28.640 ms stddev 29.647 2024-05-14T16:55:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.682+0000 7fe33059e640 1 -- 172.21.15.69:0/3256898513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe328107970 msgr2=0x7fe328107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.682+0000 7fe33059e640 1 --2- 172.21.15.69:0/3256898513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe328107970 0x7fe328107d50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe324009900 tx=0x7fe32402a410 comp rx=0 tx=0).stop 2024-05-14T16:55:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.682+0000 7fe33059e640 1 -- 172.21.15.69:0/3256898513 shutdown_connections 2024-05-14T16:55:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.682+0000 7fe33059e640 1 --2- 172.21.15.69:0/3256898513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe328102540 0x7fe32810e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.682+0000 7fe33059e640 1 --2- 172.21.15.69:0/3256898513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe328101ba0 0x7fe328102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.682+0000 7fe33059e640 1 --2- 172.21.15.69:0/3256898513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe328107970 0x7fe328107d50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.682+0000 7fe33059e640 1 -- 172.21.15.69:0/3256898513 >> 172.21.15.69:0/3256898513 conn(0x7fe3280787c0 msgr2=0x7fe328078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.682+0000 7fe33059e640 1 -- 172.21.15.69:0/3256898513 shutdown_connections 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.682+0000 7fe33059e640 1 -- 172.21.15.69:0/3256898513 wait complete. 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe33059e640 1 Processor -- start 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe33059e640 1 -- start start 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe33059e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe328101ba0 0x7fe32810f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe33059e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe328102540 0x7fe32810f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe33059e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe328107970 0x7fe32810fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe33059e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe328112d30 con 0x7fe328101ba0 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe33059e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe328112bb0 con 0x7fe328107970 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe33059e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe328112eb0 con 0x7fe328102540 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32e33a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe328101ba0 0x7fe32810f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32e33a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe328101ba0 0x7fe32810f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43718/0 (socket says 172.21.15.69:43718) 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32e33a640 1 -- 172.21.15.69:0/2062080526 learned_addr learned my addr 172.21.15.69:0/2062080526 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32db39640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe328102540 0x7fe32810f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32e33a640 1 -- 172.21.15.69:0/2062080526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe328102540 msgr2=0x7fe32810f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32e33a640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe328102540 0x7fe32810f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32e33a640 1 -- 172.21.15.69:0/2062080526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe328107970 msgr2=0x7fe32810fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32eb3b640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe328107970 0x7fe32810fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32e33a640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe328107970 0x7fe32810fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32e33a640 1 -- 172.21.15.69:0/2062080526 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3281a76b0 con 0x7fe328101ba0 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32eb3b640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe328107970 0x7fe32810fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.683+0000 7fe32e33a640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe328101ba0 0x7fe32810f250 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe324006010 tx=0x7fe3240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.684+0000 7fe31f7fe640 1 -- 172.21.15.69:0/2062080526 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32403f030 con 0x7fe328101ba0 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.684+0000 7fe33059e640 1 -- 172.21.15.69:0/2062080526 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3281a7940 con 0x7fe328101ba0 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.684+0000 7fe31f7fe640 1 -- 172.21.15.69:0/2062080526 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe324003800 con 0x7fe328101ba0 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.684+0000 7fe31f7fe640 1 -- 172.21.15.69:0/2062080526 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324035070 con 0x7fe328101ba0 2024-05-14T16:55:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.684+0000 7fe33059e640 1 -- 172.21.15.69:0/2062080526 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3281a7e30 con 0x7fe328101ba0 2024-05-14T16:55:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.686+0000 7fe33338b740 1 Processor -- start 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.686+0000 7fe31f7fe640 1 -- 172.21.15.69:0/2062080526 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe324003cb0 con 0x7fe328101ba0 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe33338b740 1 -- start start 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe33338b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe33338b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe33338b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe33338b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560541837c30 con 0x5605418c5210 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe33338b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5605418277d0 con 0x5605418cfad0 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe33338b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56054182b810 con 0x5605418c48f0 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe32e33a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe32e33a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43724/0 (socket says 172.21.15.69:43724) 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe32e33a640 1 -- 172.21.15.69:0/401567328 learned_addr learned my addr 172.21.15.69:0/401567328 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe32eb3b640 1 --2- 172.21.15.69:0/401567328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe32db39640 1 --2- 172.21.15.69:0/401567328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe32e33a640 1 -- 172.21.15.69:0/401567328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 msgr2=0x5605418c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe32e33a640 1 --2- 172.21.15.69:0/401567328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe32e33a640 1 -- 172.21.15.69:0/401567328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 msgr2=0x5605418d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe32e33a640 1 --2- 172.21.15.69:0/401567328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.687+0000 7fe32e33a640 1 -- 172.21.15.69:0/401567328 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56054153c210 con 0x5605418c5210 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.688+0000 7fe31f7fe640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe3140b2a10 0x7fe3140b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.688+0000 7fe32e33a640 1 --2- 172.21.15.69:0/401567328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cf590 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe324100710 tx=0x7fe32410b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc219ce99505e7cf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.688+0000 7fe32eb3b640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe3140b2a10 0x7fe3140b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.688+0000 7fe31cbfe640 1 -- 172.21.15.69:0/401567328 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3240c61f0 con 0x5605418c5210 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.688+0000 7fe31f7fe640 1 -- 172.21.15.69:0/2062080526 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe324103040 con 0x7fe328101ba0 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.688+0000 7fe31cbfe640 1 -- 172.21.15.69:0/401567328 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3240c9e40 con 0x5605418c5210 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.688+0000 7fe31cbfe640 1 -- 172.21.15.69:0/401567328 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3240c9aa0 con 0x5605418c5210 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 -- 172.21.15.69:0/401567328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 msgr2=0x5605418cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe32eb3b640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe3140b2a10 0x7fe3140b4ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe32000f560 tx=0x7fe320007730 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 --2- 172.21.15.69:0/401567328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cf590 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe324100710 tx=0x7fe32410b360 comp rx=0 tx=0).stop 2024-05-14T16:55:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 -- 172.21.15.69:0/401567328 shutdown_connections 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 --2- 172.21.15.69:0/401567328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 --2- 172.21.15.69:0/401567328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cf590 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 --2- 172.21.15.69:0/401567328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 -- 172.21.15.69:0/401567328 >> 172.21.15.69:0/401567328 conn(0x5605418c0e00 msgr2=0x5605418c11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 -- 172.21.15.69:0/401567328 shutdown_connections 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 -- 172.21.15.69:0/401567328 wait complete. 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 -- start start 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 Processor -- start 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56054150d790 con 0x5605418c5210 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5605417e3890 con 0x5605418cfad0 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5605418d24c0 con 0x5605418c48f0 2024-05-14T16:55:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe33338b740 10 client.0 _ll_register_callbacks cb 0x5605417e6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe32db39640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe32db39640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43734/0 (socket says 172.21.15.69:43734) 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe32db39640 1 -- 172.21.15.69:0/3699699121 learned_addr learned my addr 172.21.15.69:0/3699699121 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.689+0000 7fe32e33a640 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe32eb3b640 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe32db39640 1 -- 172.21.15.69:0/3699699121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 msgr2=0x5605418cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe32db39640 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe32db39640 1 -- 172.21.15.69:0/3699699121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 msgr2=0x5605418ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe32db39640 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe32db39640 1 -- 172.21.15.69:0/3699699121 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560541608620 con 0x5605418c5210 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe32eb3b640 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe32db39640 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cc760 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe318011760 tx=0x7fe318011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2807a9857e34477 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe311ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:55:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe311ffb640 1 -- 172.21.15.69:0/3699699121 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31800fe10 con 0x5605418c5210 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe311ffb640 1 -- 172.21.15.69:0/3699699121 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe318005020 con 0x5605418c5210 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe33338b740 1 -- 172.21.15.69:0/3699699121 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3240c9e40 con 0x5605418c5210 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe311ffb640 1 -- 172.21.15.69:0/3699699121 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe318005220 con 0x5605418c5210 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe33338b740 10 client.15975 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe33338b740 1 -- 172.21.15.69:0/3699699121 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5605418ddf40 con 0x5605418c5210 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe311ffb640 1 -- 172.21.15.69:0/3699699121 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3240c9e40 con 0x5605418c5210 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe311ffb640 1 -- 172.21.15.69:0/3699699121 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fe318018030 con 0x5605418c5210 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe33338b740 10 client.15975 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.690+0000 7fe33338b740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.691+0000 7fe33338b740 20 client.15975 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.691+0000 7fe33338b740 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5605418df560 0x5605418e1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.691+0000 7fe33338b740 4 client.15975 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.691+0000 7fe33338b740 1 -- 172.21.15.69:0/3699699121 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560541839b80 con 0x5605418df560 2024-05-14T16:55:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.691+0000 7fe32e33a640 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5605418df560 0x5605418e1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.691+0000 7fe32e33a640 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5605418df560 0x5605418e1980 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7b4a63f2b6646d server_cookie=698bc65088e43947 in_seq=0 out_seq=0 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.691+0000 7fe311ffb640 10 client.15975 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe311ffb640 1 -- 172.21.15.69:0/3699699121 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560541839b80 con 0x5605418df560 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe311ffb640 10 client.15975 handle_command_reply: tid=0 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 client.15975 shutdown 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 -- 172.21.15.69:0/3699699121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 msgr2=0x5605418cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cc760 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe318011760 tx=0x7fe318011c20 comp rx=0 tx=0).stop 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 -- 172.21.15.69:0/3699699121 shutdown_connections 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5605418df560 0x5605418e1980 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe311ffb640 0 client.15975 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cc760 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 --2- 172.21.15.69:0/3699699121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 -- 172.21.15.69:0/3699699121 >> 172.21.15.69:0/3699699121 conn(0x5605418c0e00 msgr2=0x5605418c0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 -- 172.21.15.69:0/3699699121 shutdown_connections 2024-05-14T16:55:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 1 -- 172.21.15.69:0/3699699121 wait complete. 2024-05-14T16:55:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.692+0000 7fe33338b740 20 client.15975 trim_cache size 0 max 16384 2024-05-14T16:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe33338b740 1 Processor -- start 2024-05-14T16:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe33338b740 1 -- start start 2024-05-14T16:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe33338b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe33338b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe33338b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418e7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe33338b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b3180 con 0x5605418c5210 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe33338b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b09c0 con 0x5605418cfad0 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe33338b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281ae810 con 0x5605418c48f0 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe32db39640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe32db39640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43750/0 (socket says 172.21.15.69:43750) 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.707+0000 7fe32db39640 1 -- 172.21.15.69:0/174097047 learned_addr learned my addr 172.21.15.69:0/174097047 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe32eb3b640 1 --2- 172.21.15.69:0/174097047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418e7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe32e33a640 1 --2- 172.21.15.69:0/174097047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe32db39640 1 -- 172.21.15.69:0/174097047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 msgr2=0x5605418db390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe32db39640 1 --2- 172.21.15.69:0/174097047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418db390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe32db39640 1 -- 172.21.15.69:0/174097047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 msgr2=0x5605418e7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe32db39640 1 --2- 172.21.15.69:0/174097047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418e7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe32db39640 1 -- 172.21.15.69:0/174097047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605418cacb0 con 0x5605418c5210 2024-05-14T16:55:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe32db39640 1 --2- 172.21.15.69:0/174097047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cefa0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe31800c7b0 tx=0x7fe318005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9976ebea375925e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe313fff640 1 -- 172.21.15.69:0/174097047 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3180196b0 con 0x5605418c5210 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe313fff640 1 -- 172.21.15.69:0/174097047 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe318019850 con 0x5605418c5210 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.708+0000 7fe313fff640 1 -- 172.21.15.69:0/174097047 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3180112a0 con 0x5605418c5210 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 -- 172.21.15.69:0/174097047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 msgr2=0x5605418cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 --2- 172.21.15.69:0/174097047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cefa0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe31800c7b0 tx=0x7fe318005d20 comp rx=0 tx=0).stop 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 -- 172.21.15.69:0/174097047 shutdown_connections 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 --2- 172.21.15.69:0/174097047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418e7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 --2- 172.21.15.69:0/174097047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418cefa0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 --2- 172.21.15.69:0/174097047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418db390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 -- 172.21.15.69:0/174097047 >> 172.21.15.69:0/174097047 conn(0x5605418c0e00 msgr2=0x5605418c7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 -- 172.21.15.69:0/174097047 shutdown_connections 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 -- 172.21.15.69:0/174097047 wait complete. 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 -- start start 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 Processor -- start 2024-05-14T16:55:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418d3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281aee80 con 0x5605418c5210 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b02f0 con 0x5605418cfad0 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281af550 con 0x5605418c48f0 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe33338b740 10 client.0 _ll_register_callbacks cb 0x7fe31800ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe32db39640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe32db39640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418d3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43754/0 (socket says 172.21.15.69:43754) 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe32db39640 1 -- 172.21.15.69:0/1151123222 learned_addr learned my addr 172.21.15.69:0/1151123222 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe32e33a640 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.709+0000 7fe32eb3b640 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418d3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe32db39640 1 -- 172.21.15.69:0/1151123222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 msgr2=0x5605418d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe32db39640 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe32db39640 1 -- 172.21.15.69:0/1151123222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 msgr2=0x5605418d3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe32db39640 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418d3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe32db39640 1 -- 172.21.15.69:0/1151123222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560541949090 con 0x5605418c5210 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe32db39640 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418d3a10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe3180058c0 tx=0x7fe31800fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe35c6cd008571f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe311ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe311ffb640 1 -- 172.21.15.69:0/1151123222 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318019a00 con 0x5605418c5210 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe311ffb640 1 -- 172.21.15.69:0/1151123222 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe318019ba0 con 0x5605418c5210 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe33338b740 1 -- 172.21.15.69:0/1151123222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe318019850 con 0x5605418c5210 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe311ffb640 1 -- 172.21.15.69:0/1151123222 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe318019da0 con 0x5605418c5210 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe33338b740 10 client.15981 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe33338b740 1 -- 172.21.15.69:0/1151123222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56054194a360 con 0x5605418c5210 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.710+0000 7fe311ffb640 1 -- 172.21.15.69:0/1151123222 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe318019850 con 0x5605418c5210 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.711+0000 7fe311ffb640 1 -- 172.21.15.69:0/1151123222 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fe318018040 con 0x5605418c5210 2024-05-14T16:55:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.711+0000 7fe33338b740 10 client.15981 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.711+0000 7fe33338b740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.711+0000 7fe33338b740 20 client.15981 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.711+0000 7fe33338b740 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56054194a9c0 0x56054194cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.711+0000 7fe33338b740 4 client.15981 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:55:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.711+0000 7fe33338b740 1 -- 172.21.15.69:0/1151123222 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560541833360 con 0x56054194a9c0 2024-05-14T16:55:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.711+0000 7fe32e33a640 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56054194a9c0 0x56054194cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.711+0000 7fe311ffb640 10 client.15981 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.711+0000 7fe32e33a640 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56054194a9c0 0x56054194cde0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e46df39c4b7f7f2 server_cookie=55d0297353637482 in_seq=0 out_seq=0 2024-05-14T16:55:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe311ffb640 1 -- 172.21.15.69:0/1151123222 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560541833360 con 0x56054194a9c0 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe311ffb640 10 client.15981 handle_command_reply: tid=0 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 client.15981 shutdown 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 -- 172.21.15.69:0/1151123222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 msgr2=0x5605418d3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418d3a10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe3180058c0 tx=0x7fe31800fec0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 -- 172.21.15.69:0/1151123222 shutdown_connections 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56054194a9c0 0x56054194cde0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605418cfad0 0x5605418d3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe311ffb640 0 client.15981 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605418c5210 0x5605418d3a10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 --2- 172.21.15.69:0/1151123222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605418c48f0 0x5605418d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 -- 172.21.15.69:0/1151123222 >> 172.21.15.69:0/1151123222 conn(0x5605418c0e00 msgr2=0x5605418da380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 -- 172.21.15.69:0/1151123222 shutdown_connections 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.730+0000 7fe33338b740 1 -- 172.21.15.69:0/1151123222 wait complete. 2024-05-14T16:55:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33338b740 20 client.15981 trim_cache size 0 max 16384 2024-05-14T16:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:55:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 -- 172.21.15.69:0/2062080526 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe3140b2a10 msgr2=0x7fe3140b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe3140b2a10 0x7fe3140b4ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe32000f560 tx=0x7fe320007730 comp rx=0 tx=0).stop 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 -- 172.21.15.69:0/2062080526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe328101ba0 msgr2=0x7fe32810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe328101ba0 0x7fe32810f250 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe324006010 tx=0x7fe3240042f0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 -- 172.21.15.69:0/2062080526 shutdown_connections 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe3140b2a10 0x7fe3140b4ed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe328107970 0x7fe32810fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe328102540 0x7fe32810f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 --2- 172.21.15.69:0/2062080526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe328101ba0 0x7fe32810f250 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 -- 172.21.15.69:0/2062080526 >> 172.21.15.69:0/2062080526 conn(0x7fe3280787c0 msgr2=0x7fe3281007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 -- 172.21.15.69:0/2062080526 shutdown_connections 2024-05-14T16:55:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:40.731+0000 7fe33059e640 1 -- 172.21.15.69:0/2062080526 wait complete. 2024-05-14T16:55:41.834 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.7 2024-05-14T16:55:41.835 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:55:42.096 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:42.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:41 smithi069 ceph-mon[31507]: pgmap v178: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 41 MiB/s wr, 1.94k op/s 2024-05-14T16:55:42.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:41 smithi203 ceph-mon[35787]: pgmap v178: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 41 MiB/s wr, 1.94k op/s 2024-05-14T16:55:42.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:41 smithi092 ceph-mon[35763]: pgmap v178: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 41 MiB/s wr, 1.94k op/s 2024-05-14T16:55:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.730+0000 7f38e256e700 1 -- 172.21.15.69:0/3810924558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38dc1088f0 msgr2=0x7f38dc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.730+0000 7f38e256e700 1 --2- 172.21.15.69:0/3810924558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38dc1088f0 0x7f38dc110470 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f38d400b0d0 tx=0x7f38d400b3e0 comp rx=0 tx=0).stop 2024-05-14T16:55:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.730+0000 7f38e256e700 1 -- 172.21.15.69:0/3810924558 shutdown_connections 2024-05-14T16:55:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.730+0000 7f38e256e700 1 --2- 172.21.15.69:0/3810924558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38dc1088f0 0x7f38dc110470 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.730+0000 7f38e256e700 1 --2- 172.21.15.69:0/3810924558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38dc1037f0 0x7f38dc103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.730+0000 7f38e256e700 1 --2- 172.21.15.69:0/3810924558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc1041a0 0x7f38dc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.730+0000 7f38e256e700 1 -- 172.21.15.69:0/3810924558 >> 172.21.15.69:0/3810924558 conn(0x7f38dc07a6a0 msgr2=0x7f38dc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.730+0000 7f38e256e700 1 -- 172.21.15.69:0/3810924558 shutdown_connections 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.730+0000 7f38e256e700 1 -- 172.21.15.69:0/3810924558 wait complete. 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38e256e700 1 Processor -- start 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38e256e700 1 -- start start 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38e256e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38dc1037f0 0x7f38dc075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38e256e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc1041a0 0x7f38dc075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38e256e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38dc1088f0 0x7f38dc0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38e256e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1136c0 con 0x7f38dc1037f0 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38e256e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc113540 con 0x7f38dc1088f0 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38e256e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc113840 con 0x7f38dc1041a0 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38dc1037f0 0x7f38dc075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38dc1037f0 0x7f38dc075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43782/0 (socket says 172.21.15.69:43782) 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38dbfff700 1 -- 172.21.15.69:0/2618464049 learned_addr learned my addr 172.21.15.69:0/2618464049 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38d3fff700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc1041a0 0x7f38dc075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38e0b0b700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38dc1088f0 0x7f38dc0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38dbfff700 1 -- 172.21.15.69:0/2618464049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc1041a0 msgr2=0x7f38dc075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38dbfff700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc1041a0 0x7f38dc075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38dbfff700 1 -- 172.21.15.69:0/2618464049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38dc1088f0 msgr2=0x7f38dc0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38dbfff700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38dc1088f0 0x7f38dc0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38dbfff700 1 -- 172.21.15.69:0/2618464049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc1a8050 con 0x7f38dc1037f0 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38e0b0b700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38dc1088f0 0x7f38dc0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.731+0000 7f38dbfff700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38dc1037f0 0x7f38dc075840 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00d950 tx=0x7f38cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.732+0000 7f38d9ffb700 1 -- 172.21.15.69:0/2618464049 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc016070 con 0x7f38dc1037f0 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.732+0000 7f38d9ffb700 1 -- 172.21.15.69:0/2618464049 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38cc009df0 con 0x7f38dc1037f0 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.732+0000 7f38e256e700 1 -- 172.21.15.69:0/2618464049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc1a8340 con 0x7f38dc1037f0 2024-05-14T16:55:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.732+0000 7f38d9ffb700 1 -- 172.21.15.69:0/2618464049 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00f3c0 con 0x7f38dc1037f0 2024-05-14T16:55:43.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.732+0000 7f38e256e700 1 -- 172.21.15.69:0/2618464049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc1a8880 con 0x7f38dc1037f0 2024-05-14T16:55:43.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.733+0000 7f38e256e700 1 -- 172.21.15.69:0/2618464049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc06a230 con 0x7f38dc1037f0 2024-05-14T16:55:43.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.735+0000 7f38d9ffb700 1 -- 172.21.15.69:0/2618464049 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f38cc019070 con 0x7f38dc1037f0 2024-05-14T16:55:43.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.735+0000 7f38d9ffb700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f38c80b29f0 0x7f38c80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:43.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.736+0000 7f38d3fff700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f38c80b29f0 0x7f38c80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:43.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.736+0000 7f38d9ffb700 1 -- 172.21.15.69:0/2618464049 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f38cc0e2220 con 0x7f38dc1037f0 2024-05-14T16:55:43.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.736+0000 7f38d3fff700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f38c80b29f0 0x7f38c80b4eb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f38dc103080 tx=0x7f38c40073d0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:43.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.739+0000 7f38d9ffb700 1 -- 172.21.15.69:0/2618464049 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f38cc0a9c10 con 0x7f38dc1037f0 2024-05-14T16:55:43.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.997+0000 7f38e256e700 1 -- 172.21.15.69:0/2618464049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38dc02e720 con 0x7f38dc1037f0 2024-05-14T16:55:43.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:43.998+0000 7f38d9ffb700 1 -- 172.21.15.69:0/2618464049 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11863 (secure 0 0 0) 0x7f38cc0a9510 con 0x7f38dc1037f0 2024-05-14T16:55:43.997 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:55:43.998 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:54:13.610116+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24446,24422,24433]},"id":1}]} 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 -- 172.21.15.69:0/2618464049 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f38c80b29f0 msgr2=0x7f38c80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f38c80b29f0 0x7f38c80b4eb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f38dc103080 tx=0x7f38c40073d0 comp rx=0 tx=0).stop 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 -- 172.21.15.69:0/2618464049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38dc1037f0 msgr2=0x7f38dc075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38dc1037f0 0x7f38dc075840 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00d950 tx=0x7f38cc00dc60 comp rx=0 tx=0).stop 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 -- 172.21.15.69:0/2618464049 shutdown_connections 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f38c80b29f0 0x7f38c80b4eb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38dc1037f0 0x7f38dc075840 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38dc1088f0 0x7f38dc0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 --2- 172.21.15.69:0/2618464049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc1041a0 0x7f38dc075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 -- 172.21.15.69:0/2618464049 >> 172.21.15.69:0/2618464049 conn(0x7f38dc07a6a0 msgr2=0x7f38dc101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 -- 172.21.15.69:0/2618464049 shutdown_connections 2024-05-14T16:55:44.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:44.000+0000 7f38e256e700 1 -- 172.21.15.69:0/2618464049 wait complete. 2024-05-14T16:55:44.001 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 35 2024-05-14T16:55:44.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:43 smithi069 ceph-mon[31507]: pgmap v179: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.2 MiB/s wr, 596 op/s 2024-05-14T16:55:44.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:43 smithi203 ceph-mon[35787]: pgmap v179: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.2 MiB/s wr, 596 op/s 2024-05-14T16:55:44.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:43 smithi092 ceph-mon[35763]: pgmap v179: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.2 MiB/s wr, 596 op/s 2024-05-14T16:55:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:44 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2618464049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:45.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:44 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2618464049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:45.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:44 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2618464049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:45.377 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 133.7 --await-for 45.0' 2024-05-14T16:55:45.377 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 133.7 --await-for 45.0 2024-05-14T16:55:45.398 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 315.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:55:45.618 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:46.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:45 smithi069 ceph-mon[31507]: pgmap v180: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.2 MiB/s wr, 596 op/s 2024-05-14T16:55:46.151 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:45 smithi092 ceph-mon[35763]: pgmap v180: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.2 MiB/s wr, 596 op/s 2024-05-14T16:55:46.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:45 smithi203 ceph-mon[35787]: pgmap v180: 97 pgs: 97 active+clean; 3.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.2 MiB/s wr, 596 op/s 2024-05-14T16:55:47.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.245+0000 7f020a28b700 1 -- 172.21.15.69:0/2926924260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0204101f90 msgr2=0x7f0204102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:47.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.245+0000 7f020a28b700 1 --2- 172.21.15.69:0/2926924260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0204101f90 0x7f0204102410 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009990 tx=0x7f01ec009ca0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.245+0000 7f020a28b700 1 -- 172.21.15.69:0/2926924260 shutdown_connections 2024-05-14T16:55:47.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.245+0000 7f020a28b700 1 --2- 172.21.15.69:0/2926924260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0204102950 0x7f0204110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.245+0000 7f020a28b700 1 --2- 172.21.15.69:0/2926924260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204109740 0x7f0204109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.245+0000 7f020a28b700 1 --2- 172.21.15.69:0/2926924260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0204101f90 0x7f0204102410 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.245+0000 7f020a28b700 1 -- 172.21.15.69:0/2926924260 >> 172.21.15.69:0/2926924260 conn(0x7f0204077710 msgr2=0x7f0204077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:47.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.246+0000 7f020a28b700 1 -- 172.21.15.69:0/2926924260 shutdown_connections 2024-05-14T16:55:47.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.246+0000 7f020a28b700 1 -- 172.21.15.69:0/2926924260 wait complete. 2024-05-14T16:55:47.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.246+0000 7f020a28b700 1 Processor -- start 2024-05-14T16:55:47.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.246+0000 7f020a28b700 1 -- start start 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f020a28b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204101f90 0x7f0204103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f020a28b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0204102950 0x7f02041036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f020a28b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0204109740 0x7f0204103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f020a28b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0204113770 con 0x7f0204109740 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f020a28b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f02041135f0 con 0x7f0204101f90 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f020a28b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f02041138f0 con 0x7f0204102950 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0208828700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0204109740 0x7f0204103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0208828700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0204109740 0x7f0204103bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35852/0 (socket says 172.21.15.69:35852) 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0208828700 1 -- 172.21.15.69:0/3211213727 learned_addr learned my addr 172.21.15.69:0/3211213727 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0203fff700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204101f90 0x7f0204103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f01fbfff700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0204102950 0x7f02041036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0208828700 1 -- 172.21.15.69:0/3211213727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0204102950 msgr2=0x7f02041036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0208828700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0204102950 0x7f02041036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0208828700 1 -- 172.21.15.69:0/3211213727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204101f90 msgr2=0x7f0204103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:47.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0208828700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204101f90 0x7f0204103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0208828700 1 -- 172.21.15.69:0/3211213727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02041a80a0 con 0x7f0204109740 2024-05-14T16:55:47.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f01fbfff700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0204102950 0x7f02041036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:55:47.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0208828700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0204109740 0x7f0204103bf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b970 tx=0x7f01fc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:47.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0201ffb700 1 -- 172.21.15.69:0/3211213727 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc019070 con 0x7f0204109740 2024-05-14T16:55:47.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0201ffb700 1 -- 172.21.15.69:0/3211213727 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01fc00cde0 con 0x7f0204109740 2024-05-14T16:55:47.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f020a28b700 1 -- 172.21.15.69:0/3211213727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02041a8390 con 0x7f0204109740 2024-05-14T16:55:47.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f0201ffb700 1 -- 172.21.15.69:0/3211213727 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc007370 con 0x7f0204109740 2024-05-14T16:55:47.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.247+0000 7f020a28b700 1 -- 172.21.15.69:0/3211213727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041a88d0 con 0x7f0204109740 2024-05-14T16:55:47.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.250+0000 7f0201ffb700 1 -- 172.21.15.69:0/3211213727 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f01fc01c070 con 0x7f0204109740 2024-05-14T16:55:47.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.251+0000 7f0201ffb700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f01f00b2b20 0x7f01f00b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.251+0000 7f0203fff700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f01f00b2b20 0x7f01f00b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.251+0000 7f0201ffb700 1 -- 172.21.15.69:0/3211213727 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f01fc0e19b0 con 0x7f0204109740 2024-05-14T16:55:47.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.251+0000 7f0214b21b80 1 Processor -- start 2024-05-14T16:55:47.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.251+0000 7f0214b21b80 1 -- start start 2024-05-14T16:55:47.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.251+0000 7f0203fff700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f01f00b2b20 0x7f01f00b4fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005950 tx=0x7f01f400b410 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:47.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ace30 0x562e041ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ad750 0x562e041a7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041a7b60 0x562e041a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f0214b21b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e04105e20 con 0x562e041ad750 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f0214b21b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562e040f59c0 con 0x562e041ace30 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f0214b21b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562e040f9a00 con 0x562e041a7b60 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ad750 0x562e041a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ad750 0x562e041a7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35860/0 (socket says 172.21.15.69:35860) 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fbfff700 1 -- 172.21.15.69:0/954782110 learned_addr learned my addr 172.21.15.69:0/954782110 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f0208828700 1 --2- 172.21.15.69:0/954782110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041a7b60 0x562e041a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f0203fff700 1 --2- 172.21.15.69:0/954782110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ace30 0x562e041ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fbfff700 1 -- 172.21.15.69:0/954782110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041a7b60 msgr2=0x562e041a7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fbfff700 1 --2- 172.21.15.69:0/954782110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041a7b60 0x562e041a7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fbfff700 1 -- 172.21.15.69:0/954782110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ace30 msgr2=0x562e041ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fbfff700 1 --2- 172.21.15.69:0/954782110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ace30 0x562e041ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fbfff700 1 -- 172.21.15.69:0/954782110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e04045a60 con 0x562e041ad750 2024-05-14T16:55:47.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f0203fff700 1 --2- 172.21.15.69:0/954782110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ace30 0x562e041ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:47.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fbfff700 1 --2- 172.21.15.69:0/954782110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ad750 0x562e041a7540 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f01ec000c00 tx=0x7f01ec005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9ddb32accff8f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:47.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fa7fc700 1 -- 172.21.15.69:0/954782110 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec026070 con 0x562e041ad750 2024-05-14T16:55:47.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fa7fc700 1 -- 172.21.15.69:0/954782110 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ec009590 con 0x562e041ad750 2024-05-14T16:55:47.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.252+0000 7f01fa7fc700 1 -- 172.21.15.69:0/954782110 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00ba30 con 0x562e041ad750 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.253+0000 7f0214b21b80 1 -- 172.21.15.69:0/954782110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ad750 msgr2=0x562e041a7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.253+0000 7f0214b21b80 1 --2- 172.21.15.69:0/954782110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ad750 0x562e041a7540 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f01ec000c00 tx=0x7f01ec005ef0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.253+0000 7f0214b21b80 1 -- 172.21.15.69:0/954782110 shutdown_connections 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.253+0000 7f0214b21b80 1 --2- 172.21.15.69:0/954782110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041a7b60 0x562e041a7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.253+0000 7f0214b21b80 1 --2- 172.21.15.69:0/954782110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ace30 0x562e041ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.253+0000 7f0214b21b80 1 --2- 172.21.15.69:0/954782110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ad750 0x562e041a7540 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.253+0000 7f0214b21b80 1 -- 172.21.15.69:0/954782110 >> 172.21.15.69:0/954782110 conn(0x562e040d23e0 msgr2=0x562e041a1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.253+0000 7f0214b21b80 1 -- 172.21.15.69:0/954782110 shutdown_connections 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f0214b21b80 1 -- 172.21.15.69:0/954782110 wait complete. 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f0214b21b80 1 -- start start 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f0214b21b80 1 Processor -- start 2024-05-14T16:55:47.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041a7b60 0x562e041b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ad750 0x562e041b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f0214b21b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e03ffc130 con 0x562e041ace30 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f0214b21b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562e04097950 con 0x562e041a7b60 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f0214b21b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562e041b4130 con 0x562e041ad750 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f0214b21b80 10 client.0 _ll_register_callbacks cb 0x562e0408c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.254+0000 7f01fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35870/0 (socket says 172.21.15.69:35870) 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f0203fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041a7b60 0x562e041b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01fbfff700 1 -- 172.21.15.69:0/2838602443 learned_addr learned my addr 172.21.15.69:0/2838602443 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f0208828700 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ad750 0x562e041b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01fbfff700 1 -- 172.21.15.69:0/2838602443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ad750 msgr2=0x562e041b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01fbfff700 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ad750 0x562e041b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01fbfff700 1 -- 172.21.15.69:0/2838602443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041a7b60 msgr2=0x562e041b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:47.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01fbfff700 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041a7b60 0x562e041b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01fbfff700 1 -- 172.21.15.69:0/2838602443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e04094820 con 0x562e041ace30 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f0203fff700 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041a7b60 0x562e041b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01fbfff700 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041b07b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f01ec024040 tx=0x7f01ec003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f202785b386c961 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01e37fe700 1 -- 172.21.15.69:0/2838602443 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec026040 con 0x562e041ace30 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01e37fe700 1 -- 172.21.15.69:0/2838602443 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f01ec00b9d0 con 0x562e041ace30 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f0214b21b80 1 -- 172.21.15.69:0/2838602443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec009590 con 0x562e041ace30 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f01e37fe700 1 -- 172.21.15.69:0/2838602443 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f01ec037050 con 0x562e041ace30 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f0214b21b80 10 client.15999 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.255+0000 7f0214b21b80 1 -- 172.21.15.69:0/2838602443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562e041be7f0 con 0x562e041ace30 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.256+0000 7f01e37fe700 1 -- 172.21.15.69:0/2838602443 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ec009590 con 0x562e041ace30 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.256+0000 7f01e37fe700 1 -- 172.21.15.69:0/2838602443 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f01ec032030 con 0x562e041ace30 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.256+0000 7f0214b21b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.256+0000 7f0214b21b80 10 client.15999 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:55:47.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.256+0000 7f0214b21b80 20 client.15999 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.256+0000 7f0214b21b80 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562e041c0cc0 0x562e041c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.256+0000 7f0214b21b80 4 client.15999 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.256+0000 7f0214b21b80 1 -- 172.21.15.69:0/2838602443 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e041734f0 con 0x562e041c0cc0 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.256+0000 7f0203fff700 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562e041c0cc0 0x562e041c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f0203fff700 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562e041c0cc0 0x562e041c30b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87f6508a589ec1b server_cookie=f48d76e114bd4d06 in_seq=0 out_seq=0 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f01e37fe700 10 client.15999 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f01e37fe700 1 -- 172.21.15.69:0/2838602443 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e041734f0 con 0x562e041c0cc0 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f01e37fe700 10 client.15999 handle_command_reply: tid=0 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f0214b21b80 1 client.15999 shutdown 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f0214b21b80 1 -- 172.21.15.69:0/2838602443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 msgr2=0x562e041b07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f0214b21b80 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041b07b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f01ec024040 tx=0x7f01ec003b10 comp rx=0 tx=0).stop 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f0214b21b80 1 -- 172.21.15.69:0/2838602443 shutdown_connections 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f0214b21b80 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562e041c0cc0 0x562e041c30b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f0214b21b80 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ad750 0x562e041b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f0214b21b80 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041a7b60 0x562e041b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f01e37fe700 0 client.15999 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:47.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f0214b21b80 1 --2- 172.21.15.69:0/2838602443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041b07b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.257+0000 7f0214b21b80 1 -- 172.21.15.69:0/2838602443 >> 172.21.15.69:0/2838602443 conn(0x562e040d23e0 msgr2=0x562e0419c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:47.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.258+0000 7f0214b21b80 1 -- 172.21.15.69:0/2838602443 shutdown_connections 2024-05-14T16:55:47.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.258+0000 7f0214b21b80 1 -- 172.21.15.69:0/2838602443 wait complete. 2024-05-14T16:55:47.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.258+0000 7f0214b21b80 20 client.15999 trim_cache size 0 max 16384 2024-05-14T16:55:47.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0214b21b80 1 Processor -- start 2024-05-14T16:55:47.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0214b21b80 1 -- start start 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041a7b60 0x562e041a7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ace30 0x562e041af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 0x562e0419c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0214b21b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02041b27f0 con 0x562e041a7b60 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0214b21b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f02041afff0 con 0x562e041ad750 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0214b21b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f02041ade40 con 0x562e041ace30 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0203fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041a7b60 0x562e041a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0203fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041a7b60 0x562e041a7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35874/0 (socket says 172.21.15.69:35874) 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.277+0000 7f0203fff700 1 -- 172.21.15.69:0/509782965 learned_addr learned my addr 172.21.15.69:0/509782965 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f0203fff700 1 -- 172.21.15.69:0/509782965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ace30 msgr2=0x562e041af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f01fbfff700 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ace30 0x562e041af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f0203fff700 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ace30 0x562e041af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f0203fff700 1 -- 172.21.15.69:0/509782965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 msgr2=0x562e0419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f0208828700 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 0x562e0419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f0203fff700 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 0x562e0419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f0203fff700 1 -- 172.21.15.69:0/509782965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e0419ceb0 con 0x562e041a7b60 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f01fbfff700 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ace30 0x562e041af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f0208828700 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 0x562e0419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f0203fff700 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041a7b60 0x562e041a7540 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f01f400bb50 tx=0x7f01f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df239855b741803c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f01f9ffb700 1 -- 172.21.15.69:0/509782965 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f402d070 con 0x562e041a7b60 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f01f9ffb700 1 -- 172.21.15.69:0/509782965 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f4028820 con 0x562e041a7b60 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.278+0000 7f01f9ffb700 1 -- 172.21.15.69:0/509782965 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4011cb0 con 0x562e041a7b60 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 -- 172.21.15.69:0/509782965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041a7b60 msgr2=0x562e041a7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041a7b60 0x562e041a7540 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f01f400bb50 tx=0x7f01f4013000 comp rx=0 tx=0).stop 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 -- 172.21.15.69:0/509782965 shutdown_connections 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041a7b60 0x562e041a7540 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 0x562e0419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 --2- 172.21.15.69:0/509782965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041ace30 0x562e041af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 -- 172.21.15.69:0/509782965 >> 172.21.15.69:0/509782965 conn(0x562e040d23e0 msgr2=0x562e041ab920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 -- 172.21.15.69:0/509782965 shutdown_connections 2024-05-14T16:55:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 -- 172.21.15.69:0/509782965 wait complete. 2024-05-14T16:55:47.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 -- start start 2024-05-14T16:55:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.279+0000 7f0214b21b80 1 Processor -- start 2024-05-14T16:55:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041a7b60 0x562e041c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f0214b21b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 0x562e041c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f0214b21b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02041ae4b0 con 0x562e041ace30 2024-05-14T16:55:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f0214b21b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f02041af920 con 0x562e041ad750 2024-05-14T16:55:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f0214b21b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f02041aeb80 con 0x562e041a7b60 2024-05-14T16:55:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f0214b21b80 10 client.0 _ll_register_callbacks cb 0x562e0408c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35878/0 (socket says 172.21.15.69:35878) 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01fbfff700 1 -- 172.21.15.69:0/1493564485 learned_addr learned my addr 172.21.15.69:0/1493564485 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f0203fff700 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041a7b60 0x562e041c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f0208828700 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 0x562e041c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01fbfff700 1 -- 172.21.15.69:0/1493564485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041a7b60 msgr2=0x562e041c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01fbfff700 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041a7b60 0x562e041c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01fbfff700 1 -- 172.21.15.69:0/1493564485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 msgr2=0x562e041c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01fbfff700 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 0x562e041c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01fbfff700 1 -- 172.21.15.69:0/1493564485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e041c5890 con 0x562e041ace30 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01fbfff700 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041c4a60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005d10 tx=0x7f01ec005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d1f71008215602 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01e37fe700 1 -- 172.21.15.69:0/1493564485 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec026040 con 0x562e041ace30 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01e37fe700 1 -- 172.21.15.69:0/1493564485 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f01ec00f460 con 0x562e041ace30 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f0214b21b80 1 -- 172.21.15.69:0/1493564485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01f4028820 con 0x562e041ace30 2024-05-14T16:55:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.280+0000 7f01e37fe700 1 -- 172.21.15.69:0/1493564485 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f01ec00f660 con 0x562e041ace30 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f0214b21b80 10 client.16005 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f0214b21b80 1 -- 172.21.15.69:0/1493564485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562e0422c270 con 0x562e041ace30 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f01e37fe700 1 -- 172.21.15.69:0/1493564485 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f4028820 con 0x562e041ace30 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f01e37fe700 1 -- 172.21.15.69:0/1493564485 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f01ec042050 con 0x562e041ace30 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f0214b21b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f0214b21b80 10 client.16005 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f0214b21b80 20 client.16005 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f0214b21b80 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562e0422c910 0x562e0422ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f0214b21b80 4 client.16005 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.7, "await_for": 45.0}] 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f0214b21b80 1 -- 172.21.15.69:0/1493564485 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.7, "await_for": 45.0}) -- 0x562e04107d70 con 0x562e0422c910 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f0203fff700 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562e0422c910 0x562e0422ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f0203fff700 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562e0422c910 0x562e0422ed30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1312d9e028f4c8c server_cookie=ff5b2d05f868edcd in_seq=0 out_seq=0 2024-05-14T16:55:47.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.281+0000 7f01e37fe700 10 client.16005 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:47.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.532+0000 7f01e37fe700 1 -- 172.21.15.69:0/1493564485 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562e04107d70 con 0x562e0422c910 2024-05-14T16:55:47.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.532+0000 7f01e37fe700 10 client.16005 handle_command_reply: tid=0 2024-05-14T16:55:47.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.532+0000 7f0214b21b80 1 client.16005 shutdown 2024-05-14T16:55:47.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 1 -- 172.21.15.69:0/1493564485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 msgr2=0x562e041c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:47.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041c4a60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005d10 tx=0x7f01ec005900 comp rx=0 tx=0).stop 2024-05-14T16:55:47.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 1 -- 172.21.15.69:0/1493564485 shutdown_connections 2024-05-14T16:55:47.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562e0422c910 0x562e0422ed30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562e041ad750 0x562e041c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f01e37fe700 0 client.16005 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:47.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e041a7b60 0x562e041c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 1 --2- 172.21.15.69:0/1493564485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e041ace30 0x562e041c4a60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 1 -- 172.21.15.69:0/1493564485 >> 172.21.15.69:0/1493564485 conn(0x562e040d23e0 msgr2=0x562e041ac1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:47.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 1 -- 172.21.15.69:0/1493564485 shutdown_connections 2024-05-14T16:55:47.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 1 -- 172.21.15.69:0/1493564485 wait complete. 2024-05-14T16:55:47.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.533+0000 7f0214b21b80 20 client.16005 trim_cache size 0 max 16384 2024-05-14T16:55:47.534 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:55:47.534 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 35, 2024-05-14T16:55:47.534 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:55:47.534 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 18, 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "3c97cd8e": { 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 18, 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 133.7, 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:55:47.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:55:47.536 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:55:47.536 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:55:47.536 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:55:47.536 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:55:47.536 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:55:47.536 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:55:47.536 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:55:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 -- 172.21.15.69:0/3211213727 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f01f00b2b20 msgr2=0x7f01f00b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f01f00b2b20 0x7f01f00b4fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005950 tx=0x7f01f400b410 comp rx=0 tx=0).stop 2024-05-14T16:55:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 -- 172.21.15.69:0/3211213727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0204109740 msgr2=0x7f0204103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0204109740 0x7f0204103bf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b970 tx=0x7f01fc00bc80 comp rx=0 tx=0).stop 2024-05-14T16:55:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 -- 172.21.15.69:0/3211213727 shutdown_connections 2024-05-14T16:55:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f01f00b2b20 0x7f01f00b4fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0204109740 0x7f0204103bf0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204101f90 0x7f0204103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 --2- 172.21.15.69:0/3211213727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0204102950 0x7f02041036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 -- 172.21.15.69:0/3211213727 >> 172.21.15.69:0/3211213727 conn(0x7f0204077710 msgr2=0x7f0204100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 -- 172.21.15.69:0/3211213727 shutdown_connections 2024-05-14T16:55:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:47.534+0000 7f020a28b700 1 -- 172.21.15.69:0/3211213727 wait complete. 2024-05-14T16:55:48.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:47 smithi069 ceph-mon[31507]: pgmap v181: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.6 MiB/s wr, 648 op/s 2024-05-14T16:55:48.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:47 smithi203 ceph-mon[35787]: pgmap v181: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.6 MiB/s wr, 648 op/s 2024-05-14T16:55:48.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:47 smithi092 ceph-mon[35763]: pgmap v181: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.6 MiB/s wr, 648 op/s 2024-05-14T16:55:48.444 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3c97cd8e', quiesce took 6.6 seconds. Will release after: 7.1 2024-05-14T16:55:50.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:49 smithi069 ceph-mon[31507]: pgmap v182: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 3.8 MiB/s wr, 311 op/s 2024-05-14T16:55:50.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:49 smithi203 ceph-mon[35787]: pgmap v182: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 3.8 MiB/s wr, 311 op/s 2024-05-14T16:55:50.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:49 smithi092 ceph-mon[35763]: pgmap v182: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 3.8 MiB/s wr, 311 op/s 2024-05-14T16:55:50.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 320.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:55:52.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:51 smithi069 ceph-mon[31507]: pgmap v183: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-05-14T16:55:52.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:51 smithi203 ceph-mon[35787]: pgmap v183: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-05-14T16:55:52.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:51 smithi092 ceph-mon[35763]: pgmap v183: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-05-14T16:55:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.826+0000 7fde22872640 1 -- 172.21.15.69:0/2881547613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c104600 msgr2=0x7fde1c10cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.826+0000 7fde22872640 1 --2- 172.21.15.69:0/2881547613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c104600 0x7fde1c10cae0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fde04009900 tx=0x7fde0402a510 comp rx=0 tx=0).stop 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.826+0000 7fde22872640 1 -- 172.21.15.69:0/2881547613 shutdown_connections 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.826+0000 7fde22872640 1 --2- 172.21.15.69:0/2881547613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde1c10d020 0x7fde1c10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.826+0000 7fde22872640 1 --2- 172.21.15.69:0/2881547613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c104600 0x7fde1c10cae0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.826+0000 7fde22872640 1 --2- 172.21.15.69:0/2881547613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1c103ce0 0x7fde1c1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.826+0000 7fde22872640 1 -- 172.21.15.69:0/2881547613 >> 172.21.15.69:0/2881547613 conn(0x7fde1c078ad0 msgr2=0x7fde1c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.826+0000 7fde22872640 1 -- 172.21.15.69:0/2881547613 shutdown_connections 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.826+0000 7fde22872640 1 -- 172.21.15.69:0/2881547613 wait complete. 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde22872640 1 Processor -- start 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde22872640 1 -- start start 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde22872640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c103ce0 0x7fde1c101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde22872640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1c104600 0x7fde1c101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde22872640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde1c10d020 0x7fde1c1023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde22872640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c112cd0 con 0x7fde1c103ce0 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde22872640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c112b50 con 0x7fde1c104600 2024-05-14T16:55:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde22872640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c112e50 con 0x7fde1c10d020 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde1bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c103ce0 0x7fde1c101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde1bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c103ce0 0x7fde1c101930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35892/0 (socket says 172.21.15.69:35892) 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde1bfff640 1 -- 172.21.15.69:0/541072811 learned_addr learned my addr 172.21.15.69:0/541072811 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde13fff640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1c104600 0x7fde1c101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.827+0000 7fde20e0f640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde1c10d020 0x7fde1c1023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde1bfff640 1 -- 172.21.15.69:0/541072811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde1c10d020 msgr2=0x7fde1c1023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde1bfff640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde1c10d020 0x7fde1c1023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde1bfff640 1 -- 172.21.15.69:0/541072811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1c104600 msgr2=0x7fde1c101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde1bfff640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1c104600 0x7fde1c101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde1bfff640 1 -- 172.21.15.69:0/541072811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c1a76d0 con 0x7fde1c103ce0 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde1bfff640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c103ce0 0x7fde1c101930 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00d950 tx=0x7fde0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde19ffb640 1 -- 172.21.15.69:0/541072811 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c017070 con 0x7fde1c103ce0 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde22872640 1 -- 172.21.15.69:0/541072811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c1a79c0 con 0x7fde1c103ce0 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde19ffb640 1 -- 172.21.15.69:0/541072811 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde0c009c30 con 0x7fde1c103ce0 2024-05-14T16:55:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde19ffb640 1 -- 172.21.15.69:0/541072811 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00f620 con 0x7fde1c103ce0 2024-05-14T16:55:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.828+0000 7fde22872640 1 -- 172.21.15.69:0/541072811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c1a7ee0 con 0x7fde1c103ce0 2024-05-14T16:55:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde19ffb640 1 -- 172.21.15.69:0/541072811 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fde0c010460 con 0x7fde1c103ce0 2024-05-14T16:55:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde25590740 1 Processor -- start 2024-05-14T16:55:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde25590740 1 -- start start 2024-05-14T16:55:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde25590740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c10b8f0 0x55945c10bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde25590740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 0x55945c116590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde25590740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c116ad0 0x55945c118ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde25590740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55945c07ec30 con 0x55945c10b8f0 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde25590740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55945c06e7d0 con 0x55945c10c210 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde25590740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55945c072810 con 0x55945c116ad0 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde20e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c10b8f0 0x55945c10bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde20e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c10b8f0 0x55945c10bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35902/0 (socket says 172.21.15.69:35902) 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde20e0f640 1 -- 172.21.15.69:0/1723918131 learned_addr learned my addr 172.21.15.69:0/1723918131 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde1bfff640 1 --2- 172.21.15.69:0/1723918131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 0x55945c116590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde20e0f640 1 -- 172.21.15.69:0/1723918131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c116ad0 msgr2=0x55945c118ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde20e0f640 1 --2- 172.21.15.69:0/1723918131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c116ad0 0x55945c118ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde20e0f640 1 -- 172.21.15.69:0/1723918131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 msgr2=0x55945c116590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde20e0f640 1 --2- 172.21.15.69:0/1723918131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 0x55945c116590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde20e0f640 1 -- 172.21.15.69:0/1723918131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945bd83210 con 0x55945c10b8f0 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde1bfff640 1 --2- 172.21.15.69:0/1723918131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 0x55945c116590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.831+0000 7fde19ffb640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fde080b2a10 0x7fde080b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.832+0000 7fde1bfff640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fde080b2a10 0x7fde080b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.832+0000 7fde20e0f640 1 --2- 172.21.15.69:0/1723918131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c10b8f0 0x55945c10bcd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fde14009e60 tx=0x7fde14010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757f0a0578db76db server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.832+0000 7fde19ffb640 1 -- 172.21.15.69:0/541072811 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fde0c0e2240 con 0x7fde1c103ce0 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.832+0000 7fde127fc640 1 -- 172.21.15.69:0/1723918131 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14016070 con 0x55945c10b8f0 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.832+0000 7fde127fc640 1 -- 172.21.15.69:0/1723918131 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde14004d20 con 0x55945c10b8f0 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.832+0000 7fde127fc640 1 -- 172.21.15.69:0/1723918131 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14012740 con 0x55945c10b8f0 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.832+0000 7fde25590740 1 -- 172.21.15.69:0/1723918131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c10b8f0 msgr2=0x55945c10bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.832+0000 7fde1bfff640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fde080b2a10 0x7fde080b4ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0dfa50 tx=0x7fde0c010cb0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.832+0000 7fde25590740 1 --2- 172.21.15.69:0/1723918131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c10b8f0 0x55945c10bcd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fde14009e60 tx=0x7fde14010740 comp rx=0 tx=0).stop 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 -- 172.21.15.69:0/1723918131 shutdown_connections 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 --2- 172.21.15.69:0/1723918131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c116ad0 0x55945c118ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 --2- 172.21.15.69:0/1723918131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 0x55945c116590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 --2- 172.21.15.69:0/1723918131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c10b8f0 0x55945c10bcd0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 -- 172.21.15.69:0/1723918131 >> 172.21.15.69:0/1723918131 conn(0x55945c107e00 msgr2=0x55945c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 -- 172.21.15.69:0/1723918131 shutdown_connections 2024-05-14T16:55:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 -- 172.21.15.69:0/1723918131 wait complete. 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 -- start start 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 Processor -- start 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10b8f0 0x55945c116470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10c210 0x55945c113760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c113d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55945bd54790 con 0x55945c116ad0 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55945c02a890 con 0x55945c10b8f0 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55945c1197a0 con 0x55945c10c210 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.833+0000 7fde25590740 10 client.0 _ll_register_callbacks cb 0x55945c02dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde20e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c113d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde20e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c113d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35914/0 (socket says 172.21.15.69:35914) 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde20e0f640 1 -- 172.21.15.69:0/501738067 learned_addr learned my addr 172.21.15.69:0/501738067 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde20e0f640 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10b8f0 0x55945c116470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde13fff640 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10c210 0x55945c113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde20e0f640 1 -- 172.21.15.69:0/501738067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10c210 msgr2=0x55945c113760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde20e0f640 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10c210 0x55945c113760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde20e0f640 1 -- 172.21.15.69:0/501738067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10b8f0 msgr2=0x55945c116470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde20e0f640 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10b8f0 0x55945c116470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde20e0f640 1 -- 172.21.15.69:0/501738067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945be4f620 con 0x55945c116ad0 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde20e0f640 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c113d00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fde14005b40 tx=0x7fde14005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb4bb38c60cfdb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fddfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fddfb7fe640 1 -- 172.21.15.69:0/501738067 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14016040 con 0x55945c116ad0 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fddfb7fe640 1 -- 172.21.15.69:0/501738067 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fde14013b30 con 0x55945c116ad0 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde25590740 1 -- 172.21.15.69:0/501738067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde14004d20 con 0x55945c116ad0 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fddfb7fe640 1 -- 172.21.15.69:0/501738067 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fde14030050 con 0x55945c116ad0 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde25590740 10 client.16017 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.834+0000 7fde25590740 1 -- 172.21.15.69:0/501738067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55945c125200 con 0x55945c116ad0 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.835+0000 7fddfb7fe640 1 -- 172.21.15.69:0/501738067 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde14004d20 con 0x55945c116ad0 2024-05-14T16:55:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.835+0000 7fddfb7fe640 1 -- 172.21.15.69:0/501738067 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fde1400f040 con 0x55945c116ad0 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.835+0000 7fde25590740 10 client.16017 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.835+0000 7fde25590740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.835+0000 7fde25590740 20 client.16017 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.835+0000 7fde25590740 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55945c1265b0 0x55945c1289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.835+0000 7fde25590740 4 client.16017 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.835+0000 7fde25590740 1 -- 172.21.15.69:0/501738067 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55945c080b80 con 0x55945c1265b0 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.835+0000 7fde13fff640 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55945c1265b0 0x55945c1289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.836+0000 7fddfb7fe640 10 client.16017 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.836+0000 7fde13fff640 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55945c1265b0 0x55945c1289a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e88c5220869a1f91 server_cookie=7ab837b3bdc2fc28 in_seq=0 out_seq=0 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.836+0000 7fddfb7fe640 1 -- 172.21.15.69:0/501738067 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55945c080b80 con 0x55945c1265b0 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.836+0000 7fddfb7fe640 10 client.16017 handle_command_reply: tid=0 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.836+0000 7fde25590740 1 client.16017 shutdown 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.836+0000 7fde25590740 1 -- 172.21.15.69:0/501738067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 msgr2=0x55945c113d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.836+0000 7fde25590740 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c113d00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fde14005b40 tx=0x7fde14005c40 comp rx=0 tx=0).stop 2024-05-14T16:55:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.837+0000 7fde25590740 1 -- 172.21.15.69:0/501738067 shutdown_connections 2024-05-14T16:55:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.837+0000 7fde25590740 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55945c1265b0 0x55945c1289a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.837+0000 7fde25590740 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c113d00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.837+0000 7fde25590740 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10c210 0x55945c113760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.837+0000 7fddfb7fe640 0 client.16017 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.837+0000 7fde25590740 1 --2- 172.21.15.69:0/501738067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10b8f0 0x55945c116470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.837+0000 7fde25590740 1 -- 172.21.15.69:0/501738067 >> 172.21.15.69:0/501738067 conn(0x55945c107e00 msgr2=0x55945c107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.837+0000 7fde25590740 1 -- 172.21.15.69:0/501738067 shutdown_connections 2024-05-14T16:55:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.837+0000 7fde25590740 1 -- 172.21.15.69:0/501738067 wait complete. 2024-05-14T16:55:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.837+0000 7fde25590740 20 client.16017 trim_cache size 0 max 16384 2024-05-14T16:55:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde25590740 1 Processor -- start 2024-05-14T16:55:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde25590740 1 -- start start 2024-05-14T16:55:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde25590740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10b8f0 0x55945c122670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde25590740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10c210 0x55945c115fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde25590740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c12dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde25590740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1b3210 con 0x55945c116ad0 2024-05-14T16:55:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde25590740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1b0a50 con 0x55945c10b8f0 2024-05-14T16:55:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde25590740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1ae8a0 con 0x55945c10c210 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde20e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c12dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde20e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c12dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35918/0 (socket says 172.21.15.69:35918) 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde20e0f640 1 -- 172.21.15.69:0/2565327406 learned_addr learned my addr 172.21.15.69:0/2565327406 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde20e0f640 1 --2- 172.21.15.69:0/2565327406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10b8f0 0x55945c122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde20e0f640 1 -- 172.21.15.69:0/2565327406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10c210 msgr2=0x55945c115fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde20e0f640 1 --2- 172.21.15.69:0/2565327406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10c210 0x55945c115fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde20e0f640 1 -- 172.21.15.69:0/2565327406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10b8f0 msgr2=0x55945c122670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde20e0f640 1 --2- 172.21.15.69:0/2565327406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10b8f0 0x55945c122670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde20e0f640 1 -- 172.21.15.69:0/2565327406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945c10b6b0 con 0x55945c116ad0 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.852+0000 7fde20e0f640 1 --2- 172.21.15.69:0/2565327406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c12dfd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fde14005d70 tx=0x7fde14010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f22b6b00385e495 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde11ffb640 1 -- 172.21.15.69:0/2565327406 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14016040 con 0x55945c116ad0 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde11ffb640 1 -- 172.21.15.69:0/2565327406 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde14013e50 con 0x55945c116ad0 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde11ffb640 1 -- 172.21.15.69:0/2565327406 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14012cc0 con 0x55945c116ad0 2024-05-14T16:55:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde25590740 1 -- 172.21.15.69:0/2565327406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 msgr2=0x55945c12dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde25590740 1 --2- 172.21.15.69:0/2565327406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c12dfd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fde14005d70 tx=0x7fde14010740 comp rx=0 tx=0).stop 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde25590740 1 -- 172.21.15.69:0/2565327406 shutdown_connections 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde25590740 1 --2- 172.21.15.69:0/2565327406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c12dfd0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde25590740 1 --2- 172.21.15.69:0/2565327406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10c210 0x55945c115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde25590740 1 --2- 172.21.15.69:0/2565327406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10b8f0 0x55945c122670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde25590740 1 -- 172.21.15.69:0/2565327406 >> 172.21.15.69:0/2565327406 conn(0x55945c107e00 msgr2=0x55945c10db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde25590740 1 -- 172.21.15.69:0/2565327406 shutdown_connections 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde25590740 1 -- 172.21.15.69:0/2565327406 wait complete. 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.853+0000 7fde25590740 1 -- start start 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde25590740 1 Processor -- start 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde25590740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10b8f0 0x55945c11a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde25590740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 0x55945c11a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde25590740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c11af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde25590740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1aef10 con 0x55945c116ad0 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde25590740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1b0380 con 0x55945c10c210 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde25590740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1af5e0 con 0x55945c10b8f0 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde25590740 10 client.0 _ll_register_callbacks cb 0x7fde140124c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde20e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c11af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde20e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c11af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35922/0 (socket says 172.21.15.69:35922) 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde20e0f640 1 -- 172.21.15.69:0/1792548146 learned_addr learned my addr 172.21.15.69:0/1792548146 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde13fff640 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 0x55945c11a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde20e0f640 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10b8f0 0x55945c11a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde20e0f640 1 -- 172.21.15.69:0/1792548146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10b8f0 msgr2=0x55945c11a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde20e0f640 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10b8f0 0x55945c11a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde20e0f640 1 -- 172.21.15.69:0/1792548146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 msgr2=0x55945c11a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde20e0f640 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 0x55945c11a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.854+0000 7fde20e0f640 1 -- 172.21.15.69:0/1792548146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945c18fb80 con 0x55945c116ad0 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde20e0f640 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c11af10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fde14009db0 tx=0x7fde14009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2addb27fd69859f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fddfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fddfb7fe640 1 -- 172.21.15.69:0/1792548146 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14016070 con 0x55945c116ad0 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fddfb7fe640 1 -- 172.21.15.69:0/1792548146 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fde14004ca0 con 0x55945c116ad0 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde25590740 1 -- 172.21.15.69:0/1792548146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde14013e50 con 0x55945c116ad0 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fddfb7fe640 1 -- 172.21.15.69:0/1792548146 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fde14005070 con 0x55945c116ad0 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde25590740 10 client.16023 fetch_fsmap learned FSMap version 35 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde25590740 1 -- 172.21.15.69:0/1792548146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55945c190d70 con 0x55945c116ad0 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fddfb7fe640 1 -- 172.21.15.69:0/1792548146 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde14013e50 con 0x55945c116ad0 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fddfb7fe640 1 -- 172.21.15.69:0/1792548146 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fde140128a0 con 0x55945c116ad0 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde25590740 10 client.16023 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde25590740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde25590740 20 client.16023 populate_metadata read hostname 'smithi069' 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde25590740 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55945c191320 0x55945c193710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde25590740 4 client.16023 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde25590740 1 -- 172.21.15.69:0/1792548146 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55945c07a360 con 0x55945c191320 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.855+0000 7fde13fff640 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55945c191320 0x55945c193710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.856+0000 7fddfb7fe640 10 client.16023 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.856+0000 7fde13fff640 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55945c191320 0x55945c193710 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f363afaf5dbb6306 server_cookie=6bc59a904538146a in_seq=0 out_seq=0 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fddfb7fe640 1 -- 172.21.15.69:0/1792548146 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55945c07a360 con 0x55945c191320 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fddfb7fe640 10 client.16023 handle_command_reply: tid=0 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 client.16023 shutdown 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 -- 172.21.15.69:0/1792548146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 msgr2=0x55945c11af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c11af10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fde14009db0 tx=0x7fde14009600 comp rx=0 tx=0).stop 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 -- 172.21.15.69:0/1792548146 shutdown_connections 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55945c191320 0x55945c193710 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55945c116ad0 0x55945c11af10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55945c10c210 0x55945c11a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fddfb7fe640 0 client.16023 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:55:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 --2- 172.21.15.69:0/1792548146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55945c10b8f0 0x55945c11a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 -- 172.21.15.69:0/1792548146 >> 172.21.15.69:0/1792548146 conn(0x55945c107e00 msgr2=0x55945c10e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:53.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 -- 172.21.15.69:0/1792548146 shutdown_connections 2024-05-14T16:55:53.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.873+0000 7fde25590740 1 -- 172.21.15.69:0/1792548146 wait complete. 2024-05-14T16:55:53.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde25590740 20 client.16023 trim_cache size 0 max 16384 2024-05-14T16:55:53.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:55:53.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:55:53.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:55:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 -- 172.21.15.69:0/541072811 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fde080b2a10 msgr2=0x7fde080b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fde080b2a10 0x7fde080b4ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0dfa50 tx=0x7fde0c010cb0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 -- 172.21.15.69:0/541072811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c103ce0 msgr2=0x7fde1c101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c103ce0 0x7fde1c101930 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00d950 tx=0x7fde0c00de10 comp rx=0 tx=0).stop 2024-05-14T16:55:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 -- 172.21.15.69:0/541072811 shutdown_connections 2024-05-14T16:55:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fde080b2a10 0x7fde080b4ed0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde1c10d020 0x7fde1c1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1c104600 0x7fde1c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 --2- 172.21.15.69:0/541072811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c103ce0 0x7fde1c101930 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 -- 172.21.15.69:0/541072811 >> 172.21.15.69:0/541072811 conn(0x7fde1c078ad0 msgr2=0x7fde1c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 -- 172.21.15.69:0/541072811 shutdown_connections 2024-05-14T16:55:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:55:53.874+0000 7fde22872640 1 -- 172.21.15.69:0/541072811 wait complete. 2024-05-14T16:55:54.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:53 smithi069 ceph-mon[31507]: pgmap v184: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 573 KiB/s wr, 78 op/s 2024-05-14T16:55:54.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:53 smithi203 ceph-mon[35787]: pgmap v184: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 573 KiB/s wr, 78 op/s 2024-05-14T16:55:54.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:53 smithi092 ceph-mon[35763]: pgmap v184: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 573 KiB/s wr, 78 op/s 2024-05-14T16:55:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 325.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:55:55.545 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:55:55.804 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:56.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:55 smithi069 ceph-mon[31507]: pgmap v185: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 573 KiB/s wr, 78 op/s 2024-05-14T16:55:56.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:55 smithi203 ceph-mon[35787]: pgmap v185: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 573 KiB/s wr, 78 op/s 2024-05-14T16:55:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:55 smithi092 ceph-mon[35763]: pgmap v185: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 573 KiB/s wr, 78 op/s 2024-05-14T16:55:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.752+0000 7f798f3db700 1 -- 172.21.15.69:0/399991854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7988102e30 msgr2=0x7f798810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.752+0000 7f798f3db700 1 --2- 172.21.15.69:0/399991854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7988102e30 0x7f798810f2f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f798000b0a0 tx=0x7f798000b3b0 comp rx=0 tx=0).stop 2024-05-14T16:55:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 -- 172.21.15.69:0/399991854 shutdown_connections 2024-05-14T16:55:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 --2- 172.21.15.69:0/399991854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7988102e30 0x7f798810f2f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 --2- 172.21.15.69:0/399991854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79881084b0 0x7f7988108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 --2- 172.21.15.69:0/399991854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7988102470 0x7f79881028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 -- 172.21.15.69:0/399991854 >> 172.21.15.69:0/399991854 conn(0x7f79880793f0 msgr2=0x7f7988079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 -- 172.21.15.69:0/399991854 shutdown_connections 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 -- 172.21.15.69:0/399991854 wait complete. 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 Processor -- start 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 -- start start 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7988102470 0x7f798819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.753+0000 7f798f3db700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7988102e30 0x7f798819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798f3db700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79881084b0 0x7f79881a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798f3db700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7988112640 con 0x7f79881084b0 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798f3db700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f79881124c0 con 0x7f7988102470 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798f3db700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79881127c0 con 0x7f7988102e30 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798d978700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79881084b0 0x7f79881a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798d978700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79881084b0 0x7f79881a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35650/0 (socket says 172.21.15.69:35650) 2024-05-14T16:55:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798d978700 1 -- 172.21.15.69:0/393905962 learned_addr learned my addr 172.21.15.69:0/393905962 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798c976700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7988102e30 0x7f798819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798d177700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7988102470 0x7f798819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798d978700 1 -- 172.21.15.69:0/393905962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7988102e30 msgr2=0x7f798819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798d978700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7988102e30 0x7f798819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798d978700 1 -- 172.21.15.69:0/393905962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7988102470 msgr2=0x7f798819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798d978700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7988102470 0x7f798819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798d978700 1 -- 172.21.15.69:0/393905962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79881a5f80 con 0x7f79881084b0 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798d978700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79881084b0 0x7f79881a5a40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f79800062a0 tx=0x7f7980004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f797a7fc700 1 -- 172.21.15.69:0/393905962 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980021070 con 0x7f79881084b0 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f797a7fc700 1 -- 172.21.15.69:0/393905962 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7980007c80 con 0x7f79881084b0 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798f3db700 1 -- 172.21.15.69:0/393905962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79881a6150 con 0x7f79881084b0 2024-05-14T16:55:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f797a7fc700 1 -- 172.21.15.69:0/393905962 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798001d450 con 0x7f79881084b0 2024-05-14T16:55:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.754+0000 7f798f3db700 1 -- 172.21.15.69:0/393905962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79881a6650 con 0x7f79881084b0 2024-05-14T16:55:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.755+0000 7f798f3db700 1 -- 172.21.15.69:0/393905962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f798806a230 con 0x7f79881084b0 2024-05-14T16:55:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.757+0000 7f797a7fc700 1 -- 172.21.15.69:0/393905962 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7980004020 con 0x7f79881084b0 2024-05-14T16:55:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.757+0000 7f797a7fc700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79740b2a70 0x7f79740b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:57.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.757+0000 7f798d177700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79740b2a70 0x7f79740b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:57.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.758+0000 7f797a7fc700 1 -- 172.21.15.69:0/393905962 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f798000f020 con 0x7f79881084b0 2024-05-14T16:55:57.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.758+0000 7f798d177700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79740b2a70 0x7f79740b4f30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f797c006fd0 tx=0x7f797c008040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:57.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:57.762+0000 7f797a7fc700 1 -- 172.21.15.69:0/393905962 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f798002a020 con 0x7f79881084b0 2024-05-14T16:55:57.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:57 smithi069 ceph-mon[31507]: pgmap v186: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-14T16:55:58.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.021+0000 7f798f3db700 1 -- 172.21.15.69:0/393905962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f798802e560 con 0x7f79881084b0 2024-05-14T16:55:58.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.022+0000 7f797a7fc700 1 -- 172.21.15.69:0/393905962 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11863 (secure 0 0 0) 0x7f79800ab6e0 con 0x7f79881084b0 2024-05-14T16:55:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:55:58.022 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:54:13.610116+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24446,24422,24433]},"id":1}]} 2024-05-14T16:55:58.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 -- 172.21.15.69:0/393905962 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79740b2a70 msgr2=0x7f79740b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:58.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79740b2a70 0x7f79740b4f30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f797c006fd0 tx=0x7f797c008040 comp rx=0 tx=0).stop 2024-05-14T16:55:58.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 -- 172.21.15.69:0/393905962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79881084b0 msgr2=0x7f79881a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:58.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79881084b0 0x7f79881a5a40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f79800062a0 tx=0x7f7980004e20 comp rx=0 tx=0).stop 2024-05-14T16:55:58.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 -- 172.21.15.69:0/393905962 shutdown_connections 2024-05-14T16:55:58.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79740b2a70 0x7f79740b4f30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:58.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79881084b0 0x7f79881a5a40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:58.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7988102470 0x7f798819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:58.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 --2- 172.21.15.69:0/393905962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7988102e30 0x7f798819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:58.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 -- 172.21.15.69:0/393905962 >> 172.21.15.69:0/393905962 conn(0x7f79880793f0 msgr2=0x7f7988100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:58.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 -- 172.21.15.69:0/393905962 shutdown_connections 2024-05-14T16:55:58.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:55:58.024+0000 7f798f3db700 1 -- 172.21.15.69:0/393905962 wait complete. 2024-05-14T16:55:58.025 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 35 2024-05-14T16:55:58.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:57 smithi203 ceph-mon[35787]: pgmap v186: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-14T16:55:58.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:57 smithi092 ceph-mon[35763]: pgmap v186: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-14T16:55:58.671 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id 3c97cd8e --release --await' 2024-05-14T16:55:58.671 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id 3c97cd8e --release --await 2024-05-14T16:55:58.916 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:58.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:58 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/393905962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:59.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:58 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/393905962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:59.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:58 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/393905962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:00.153 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:55:59 smithi069 ceph-mon[31507]: pgmap v187: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-14T16:56:00.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:55:59 smithi203 ceph-mon[35787]: pgmap v187: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-14T16:56:00.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:55:59 smithi092 ceph-mon[35763]: pgmap v187: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-14T16:56:00.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 330.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.532+0000 7f03f5478700 1 -- 172.21.15.69:0/3114837303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f01077d0 msgr2=0x7f03f010f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.532+0000 7f03f5478700 1 --2- 172.21.15.69:0/3114837303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f01077d0 0x7f03f010f370 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f03e800b0d0 tx=0x7f03e800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.532+0000 7f03f5478700 1 -- 172.21.15.69:0/3114837303 shutdown_connections 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.532+0000 7f03f5478700 1 --2- 172.21.15.69:0/3114837303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f01077d0 0x7f03f010f370 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.532+0000 7f03f5478700 1 --2- 172.21.15.69:0/3114837303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03f0102a80 0x7f03f0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.532+0000 7f03f5478700 1 --2- 172.21.15.69:0/3114837303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03f0103430 0x7f03f01070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.532+0000 7f03f5478700 1 -- 172.21.15.69:0/3114837303 >> 172.21.15.69:0/3114837303 conn(0x7f03f00793f0 msgr2=0x7f03f0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.532+0000 7f03f5478700 1 -- 172.21.15.69:0/3114837303 shutdown_connections 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.533+0000 7f03f5478700 1 -- 172.21.15.69:0/3114837303 wait complete. 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.533+0000 7f03f5478700 1 Processor -- start 2024-05-14T16:56:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.533+0000 7f03f5478700 1 -- start start 2024-05-14T16:56:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.533+0000 7f03f5478700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0102a80 0x7f03f019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03f5478700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03f0103430 0x7f03f019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03f5478700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03f01077d0 0x7f03f01a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03f5478700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03f0112660 con 0x7f03f0102a80 2024-05-14T16:56:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03f5478700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03f01124e0 con 0x7f03f0103430 2024-05-14T16:56:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03f5478700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03f01127e0 con 0x7f03f01077d0 2024-05-14T16:56:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03eeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0102a80 0x7f03f019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03eeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0102a80 0x7f03f019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35668/0 (socket says 172.21.15.69:35668) 2024-05-14T16:56:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03eeffd700 1 -- 172.21.15.69:0/3546799830 learned_addr learned my addr 172.21.15.69:0/3546799830 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03ee7fc700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03f0103430 0x7f03f019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03ef7fe700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03f01077d0 0x7f03f01a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03eeffd700 1 -- 172.21.15.69:0/3546799830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03f01077d0 msgr2=0x7f03f01a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03eeffd700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03f01077d0 0x7f03f01a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03eeffd700 1 -- 172.21.15.69:0/3546799830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03f0103430 msgr2=0x7f03f019afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03eeffd700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03f0103430 0x7f03f019afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03eeffd700 1 -- 172.21.15.69:0/3546799830 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f01a6020 con 0x7f03f0102a80 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03eeffd700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0102a80 0x7f03f019aa60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f03d800e9a0 tx=0x7f03d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03e7fff700 1 -- 172.21.15.69:0/3546799830 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d800cbd0 con 0x7f03f0102a80 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03f5478700 1 -- 172.21.15.69:0/3546799830 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f01a6250 con 0x7f03f0102a80 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03e7fff700 1 -- 172.21.15.69:0/3546799830 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d800cd70 con 0x7f03f0102a80 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03e7fff700 1 -- 172.21.15.69:0/3546799830 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8018680 con 0x7f03f0102a80 2024-05-14T16:56:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.534+0000 7f03f5478700 1 -- 172.21.15.69:0/3546799830 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f01a6790 con 0x7f03f0102a80 2024-05-14T16:56:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.538+0000 7f03e7fff700 1 -- 172.21.15.69:0/3546799830 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f03d8010430 con 0x7f03f0102a80 2024-05-14T16:56:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.538+0000 7f03e7fff700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03dc0b29f0 0x7f03dc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.538+0000 7f03ee7fc700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03dc0b29f0 0x7f03dc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.538+0000 7f03e7fff700 1 -- 172.21.15.69:0/3546799830 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f03d80e2890 con 0x7f03f0102a80 2024-05-14T16:56:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ffd0eb80 1 Processor -- start 2024-05-14T16:56:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ffd0eb80 1 -- start start 2024-05-14T16:56:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ee7fc700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03dc0b29f0 0x7f03dc0b4eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f03f01070e0 tx=0x7f03e000b590 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 0x5599c5a8c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8c6c0 0x5599c5a864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a86f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5599c59e4d90 con 0x5599c5a86ad0 2024-05-14T16:56:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5599c59d4930 con 0x5599c5a8c6c0 2024-05-14T16:56:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599c59d8970 con 0x5599c5a8bda0 2024-05-14T16:56:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ef7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a86f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ef7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a86f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35670/0 (socket says 172.21.15.69:35670) 2024-05-14T16:56:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ef7fe700 1 -- 172.21.15.69:0/3636463904 learned_addr learned my addr 172.21.15.69:0/3636463904 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03ee7fc700 1 --2- 172.21.15.69:0/3636463904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8c6c0 0x5599c5a864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.539+0000 7f03eeffd700 1 --2- 172.21.15.69:0/3636463904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 0x5599c5a8c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.540+0000 7f03ef7fe700 1 -- 172.21.15.69:0/3636463904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 msgr2=0x5599c5a8c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.540+0000 7f03ef7fe700 1 --2- 172.21.15.69:0/3636463904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 0x5599c5a8c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.540+0000 7f03ef7fe700 1 -- 172.21.15.69:0/3636463904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8c6c0 msgr2=0x5599c5a864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.540+0000 7f03ef7fe700 1 --2- 172.21.15.69:0/3636463904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8c6c0 0x5599c5a864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.540+0000 7f03ef7fe700 1 -- 172.21.15.69:0/3636463904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599c59249d0 con 0x5599c5a86ad0 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.540+0000 7f03eeffd700 1 --2- 172.21.15.69:0/3636463904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 0x5599c5a8c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.540+0000 7f03ef7fe700 1 --2- 172.21.15.69:0/3636463904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a86f50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009ce0 tx=0x7f03e8009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb48676ccff3980 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.540+0000 7f03e57fa700 1 -- 172.21.15.69:0/3636463904 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e80093e0 con 0x5599c5a86ad0 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.540+0000 7f03e57fa700 1 -- 172.21.15.69:0/3636463904 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e8020070 con 0x5599c5a86ad0 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.540+0000 7f03e57fa700 1 -- 172.21.15.69:0/3636463904 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e80055d0 con 0x5599c5a86ad0 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3636463904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 msgr2=0x5599c5a86f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/3636463904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a86f50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009ce0 tx=0x7f03e8009d10 comp rx=0 tx=0).stop 2024-05-14T16:56:00.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3636463904 shutdown_connections 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/3636463904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a86f50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/3636463904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 0x5599c5a8c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/3636463904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8c6c0 0x5599c5a864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3636463904 >> 172.21.15.69:0/3636463904 conn(0x5599c59b1350 msgr2=0x5599c5a80ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3636463904 shutdown_connections 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3636463904 wait complete. 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 -- start start 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 Processor -- start 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a86ad0 0x5599c5a923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8bda0 0x5599c5a8f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8c6c0 0x5599c5a8fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5599c58db0a0 con 0x5599c5a8bda0 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5599c59768c0 con 0x5599c5a8c6c0 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.541+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599c5a93040 con 0x5599c5a86ad0 2024-05-14T16:56:00.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ffd0eb80 10 client.0 _ll_register_callbacks cb 0x5599c596b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ee7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8bda0 0x5599c5a8f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ee7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8bda0 0x5599c5a8f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35676/0 (socket says 172.21.15.69:35676) 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ee7fc700 1 -- 172.21.15.69:0/3362341811 learned_addr learned my addr 172.21.15.69:0/3362341811 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03eeffd700 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a86ad0 0x5599c5a923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ef7fe700 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8c6c0 0x5599c5a8fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ee7fc700 1 -- 172.21.15.69:0/3362341811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a86ad0 msgr2=0x5599c5a923f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ee7fc700 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a86ad0 0x5599c5a923f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ee7fc700 1 -- 172.21.15.69:0/3362341811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8c6c0 msgr2=0x5599c5a8fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ee7fc700 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8c6c0 0x5599c5a8fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ee7fc700 1 -- 172.21.15.69:0/3362341811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599c5973790 con 0x5599c5a8bda0 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ee7fc700 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8bda0 0x5599c5a8f790 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f03e000fe70 tx=0x7f03e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae1287f5cd046c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03d67fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:56:00.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03d67fc700 1 -- 172.21.15.69:0/3362341811 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e000fea0 con 0x5599c5a8bda0 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3362341811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e8020070 con 0x5599c5a8bda0 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03d67fc700 1 -- 172.21.15.69:0/3362341811 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f03e0028470 con 0x5599c5a8bda0 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03d67fc700 1 -- 172.21.15.69:0/3362341811 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f03e0028670 con 0x5599c5a8bda0 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ffd0eb80 10 client.16041 fetch_fsmap learned FSMap version 35 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.542+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3362341811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599c5a9d7d0 con 0x5599c5a8bda0 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.543+0000 7f03d67fc700 1 -- 172.21.15.69:0/3362341811 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e8020070 con 0x5599c5a8bda0 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.543+0000 7f03d67fc700 1 -- 172.21.15.69:0/3362341811 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f03e001f990 con 0x5599c5a8bda0 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.543+0000 7f03ffd0eb80 10 client.16041 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.543+0000 7f03ffd0eb80 10 client.16041 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.543+0000 7f03ffd0eb80 20 client.16041 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.543+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5599c5a9fe60 0x5599c5aa2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.543+0000 7f03ffd0eb80 4 client.16041 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.543+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3362341811 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599c5a52460 con 0x5599c5a9fe60 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.543+0000 7f03eeffd700 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5599c5a9fe60 0x5599c5aa2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03d67fc700 10 client.16041 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:00.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03eeffd700 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5599c5a9fe60 0x5599c5aa2250 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eff45fc6e9aa4ccb server_cookie=de82ce21414e2eea in_seq=0 out_seq=0 2024-05-14T16:56:00.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03d67fc700 1 -- 172.21.15.69:0/3362341811 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5599c5a52460 con 0x5599c5a9fe60 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03d67fc700 10 client.16041 handle_command_reply: tid=0 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03ffd0eb80 1 client.16041 shutdown 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3362341811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8bda0 msgr2=0x5599c5a8f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8bda0 0x5599c5a8f790 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f03e000fe70 tx=0x7f03e0004750 comp rx=0 tx=0).stop 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3362341811 shutdown_connections 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5599c5a9fe60 0x5599c5aa2250 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8c6c0 0x5599c5a8fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a86ad0 0x5599c5a923f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/3362341811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8bda0 0x5599c5a8f790 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3362341811 >> 172.21.15.69:0/3362341811 conn(0x5599c59b1350 msgr2=0x5599c5a7b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.544+0000 7f03d67fc700 0 client.16041 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.545+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3362341811 shutdown_connections 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.545+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/3362341811 wait complete. 2024-05-14T16:56:00.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.545+0000 7f03ffd0eb80 20 client.16041 trim_cache size 0 max 16384 2024-05-14T16:56:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.564+0000 7f03ffd0eb80 1 Processor -- start 2024-05-14T16:56:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03ffd0eb80 1 -- start start 2024-05-14T16:56:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8bda0 0x5599c5a9bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8c6c0 0x5599c5a7b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03f01b0730 con 0x5599c5a86ad0 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03f01adf30 con 0x5599c5a8bda0 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03f01abd80 con 0x5599c5a8c6c0 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03eeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03eeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a864b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35686/0 (socket says 172.21.15.69:35686) 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03eeffd700 1 -- 172.21.15.69:0/1877831205 learned_addr learned my addr 172.21.15.69:0/1877831205 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03ef7fe700 1 --2- 172.21.15.69:0/1877831205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8c6c0 0x5599c5a7b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03ee7fc700 1 --2- 172.21.15.69:0/1877831205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8bda0 0x5599c5a9bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03eeffd700 1 -- 172.21.15.69:0/1877831205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8c6c0 msgr2=0x5599c5a7b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03eeffd700 1 --2- 172.21.15.69:0/1877831205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8c6c0 0x5599c5a7b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03eeffd700 1 -- 172.21.15.69:0/1877831205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8bda0 msgr2=0x5599c5a9bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03eeffd700 1 --2- 172.21.15.69:0/1877831205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8bda0 0x5599c5a9bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03eeffd700 1 -- 172.21.15.69:0/1877831205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599c5a8c3d0 con 0x5599c5a86ad0 2024-05-14T16:56:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.565+0000 7f03ee7fc700 1 --2- 172.21.15.69:0/1877831205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8bda0 0x5599c5a9bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.566+0000 7f03eeffd700 1 --2- 172.21.15.69:0/1877831205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a864b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f03d80a5c00 tx=0x7f03d8010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccaacb868fbb963 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.566+0000 7f03e4ff9700 1 -- 172.21.15.69:0/1877831205 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d808e070 con 0x5599c5a86ad0 2024-05-14T16:56:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.566+0000 7f03e4ff9700 1 -- 172.21.15.69:0/1877831205 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d80aa280 con 0x5599c5a86ad0 2024-05-14T16:56:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.566+0000 7f03e4ff9700 1 -- 172.21.15.69:0/1877831205 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d80aa040 con 0x5599c5a86ad0 2024-05-14T16:56:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.566+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1877831205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 msgr2=0x5599c5a864b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.566+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/1877831205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a864b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f03d80a5c00 tx=0x7f03d8010cb0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1877831205 shutdown_connections 2024-05-14T16:56:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/1877831205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a86ad0 0x5599c5a864b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/1877831205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8c6c0 0x5599c5a7b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/1877831205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a8bda0 0x5599c5a9bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1877831205 >> 172.21.15.69:0/1877831205 conn(0x5599c59b1350 msgr2=0x5599c5a8b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1877831205 shutdown_connections 2024-05-14T16:56:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1877831205 wait complete. 2024-05-14T16:56:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 -- start start 2024-05-14T16:56:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 Processor -- start 2024-05-14T16:56:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a86ad0 0x5599c5a98750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 0x5599c5a952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8c6c0 0x5599c5a957e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03f01ac3f0 con 0x5599c5a8c6c0 2024-05-14T16:56:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03f01ad860 con 0x5599c5a86ad0 2024-05-14T16:56:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03f01acac0 con 0x5599c5a8bda0 2024-05-14T16:56:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ffd0eb80 10 client.0 _ll_register_callbacks cb 0x5599c596b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ef7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8c6c0 0x5599c5a957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03eeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a86ad0 0x5599c5a98750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.567+0000 7f03ef7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8c6c0 0x5599c5a957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35694/0 (socket says 172.21.15.69:35694) 2024-05-14T16:56:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ef7fe700 1 -- 172.21.15.69:0/1941080155 learned_addr learned my addr 172.21.15.69:0/1941080155 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ee7fc700 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 0x5599c5a952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ef7fe700 1 -- 172.21.15.69:0/1941080155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 msgr2=0x5599c5a952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ef7fe700 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 0x5599c5a952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ef7fe700 1 -- 172.21.15.69:0/1941080155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a86ad0 msgr2=0x5599c5a98750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ef7fe700 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a86ad0 0x5599c5a98750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ef7fe700 1 -- 172.21.15.69:0/1941080155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599c5a95f20 con 0x5599c5a8c6c0 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ef7fe700 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8c6c0 0x5599c5a957e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009fd0 tx=0x7f03e80051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b487999d92d0c665 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03d67fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03d67fc700 1 -- 172.21.15.69:0/1941080155 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8033060 con 0x5599c5a8c6c0 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03d67fc700 1 -- 172.21.15.69:0/1941080155 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f03e8005220 con 0x5599c5a8c6c0 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1941080155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d80aa280 con 0x5599c5a8c6c0 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03d67fc700 1 -- 172.21.15.69:0/1941080155 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f03e8005420 con 0x5599c5a8c6c0 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ffd0eb80 10 client.16047 fetch_fsmap learned FSMap version 35 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1941080155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599c5b0b670 con 0x5599c5a8c6c0 2024-05-14T16:56:00.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.568+0000 7f03d67fc700 1 -- 172.21.15.69:0/1941080155 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d80aa280 con 0x5599c5a8c6c0 2024-05-14T16:56:00.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.569+0000 7f03d67fc700 1 -- 172.21.15.69:0/1941080155 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f03e8011070 con 0x5599c5a8c6c0 2024-05-14T16:56:00.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.569+0000 7f03ffd0eb80 10 client.16047 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:56:00.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.569+0000 7f03ffd0eb80 10 client.16047 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:56:00.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.569+0000 7f03ffd0eb80 20 client.16047 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:00.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.569+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5599c5b0b8f0 0x5599c5b0dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:00.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.569+0000 7f03ffd0eb80 4 client.16047 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3c97cd8e", "release": true, "await": true}] 2024-05-14T16:56:00.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.569+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1941080155 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3c97cd8e", "release": true, "await": true}) -- 0x5599c59e6ce0 con 0x5599c5b0b8f0 2024-05-14T16:56:00.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.569+0000 7f03eeffd700 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5599c5b0b8f0 0x5599c5b0dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:00.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.569+0000 7f03d67fc700 10 client.16047 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:00.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.569+0000 7f03eeffd700 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5599c5b0b8f0 0x5599c5b0dce0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab42c28caaed06e server_cookie=25b49fa553e83eab in_seq=0 out_seq=0 2024-05-14T16:56:00.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03d67fc700 1 -- 172.21.15.69:0/1941080155 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5599c59e6ce0 con 0x5599c5b0b8f0 2024-05-14T16:56:00.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03d67fc700 10 client.16047 handle_command_reply: tid=0 2024-05-14T16:56:00.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03ffd0eb80 1 client.16047 shutdown 2024-05-14T16:56:00.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1941080155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8c6c0 msgr2=0x5599c5a957e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:00.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8c6c0 0x5599c5a957e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009fd0 tx=0x7f03e80051b0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1941080155 shutdown_connections 2024-05-14T16:56:00.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5599c5b0b8f0 0x5599c5b0dce0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599c5a8c6c0 0x5599c5a957e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5599c5a86ad0 0x5599c5a98750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03ffd0eb80 1 --2- 172.21.15.69:0/1941080155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599c5a8bda0 0x5599c5a952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.577+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1941080155 >> 172.21.15.69:0/1941080155 conn(0x5599c59b1350 msgr2=0x5599c5a8f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.578+0000 7f03d67fc700 0 client.16047 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.578+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1941080155 shutdown_connections 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.578+0000 7f03ffd0eb80 1 -- 172.21.15.69:0/1941080155 wait complete. 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.578+0000 7f03ffd0eb80 20 client.16047 trim_cache size 0 max 16384 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 35, 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 20, 2024-05-14T16:56:00.579 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "3c97cd8e": { 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 20, 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 133.7, 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:56:00.580 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:56:00.581 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:00.581 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:00.581 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:00.581 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:00.581 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:00.581 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.579+0000 7f03f5478700 1 -- 172.21.15.69:0/3546799830 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03dc0b29f0 msgr2=0x7f03dc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.579+0000 7f03f5478700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03dc0b29f0 0x7f03dc0b4eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f03f01070e0 tx=0x7f03e000b590 comp rx=0 tx=0).stop 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.579+0000 7f03f5478700 1 -- 172.21.15.69:0/3546799830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0102a80 msgr2=0x7f03f019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.579+0000 7f03f5478700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0102a80 0x7f03f019aa60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f03d800e9a0 tx=0x7f03d800ecb0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.579+0000 7f03f5478700 1 -- 172.21.15.69:0/3546799830 shutdown_connections 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.579+0000 7f03f5478700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03dc0b29f0 0x7f03dc0b4eb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.579+0000 7f03f5478700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0102a80 0x7f03f019aa60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.579+0000 7f03f5478700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03f01077d0 0x7f03f01a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.579+0000 7f03f5478700 1 --2- 172.21.15.69:0/3546799830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03f0103430 0x7f03f019afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.579+0000 7f03f5478700 1 -- 172.21.15.69:0/3546799830 >> 172.21.15.69:0/3546799830 conn(0x7f03f00793f0 msgr2=0x7f03f0100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.580+0000 7f03f5478700 1 -- 172.21.15.69:0/3546799830 shutdown_connections 2024-05-14T16:56:00.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:00.580+0000 7f03f5478700 1 -- 172.21.15.69:0/3546799830 wait complete. 2024-05-14T16:56:01.993 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3c97cd8e', total seconds elapsed: 20.2 2024-05-14T16:56:02.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:01 smithi069 ceph-mon[31507]: pgmap v188: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-05-14T16:56:02.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:01 smithi203 ceph-mon[35787]: pgmap v188: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-05-14T16:56:02.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:01 smithi092 ceph-mon[35763]: pgmap v188: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-05-14T16:56:04.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:03 smithi069 ceph-mon[31507]: pgmap v189: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-05-14T16:56:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:03 smithi203 ceph-mon[35787]: pgmap v189: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-05-14T16:56:04.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:03 smithi092 ceph-mon[35763]: pgmap v189: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-05-14T16:56:05.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 335.0 s, 2692.7 tps, lat 198.382 ms stddev 1881.362 2024-05-14T16:56:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:05 smithi069 ceph-mon[31507]: pgmap v190: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-05-14T16:56:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:05 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:06.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:05 smithi203 ceph-mon[35787]: pgmap v190: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-05-14T16:56:06.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:05 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:06.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:05 smithi092 ceph-mon[35763]: pgmap v190: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-05-14T16:56:06.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:05 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.007+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3209436176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8102a60 msgr2=0x7fc2d8102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.007+0000 7fc2df1b6640 1 --2- 172.21.15.69:0/3209436176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8102a60 0x7fc2d8102ee0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009930 tx=0x7fc2c402a420 comp rx=0 tx=0).stop 2024-05-14T16:56:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.008+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3209436176 shutdown_connections 2024-05-14T16:56:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.008+0000 7fc2df1b6640 1 --2- 172.21.15.69:0/3209436176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d8103420 0x7fc2d810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.008+0000 7fc2df1b6640 1 --2- 172.21.15.69:0/3209436176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8102a60 0x7fc2d8102ee0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.008+0000 7fc2df1b6640 1 --2- 172.21.15.69:0/3209436176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d8108a70 0x7fc2d8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.008+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3209436176 >> 172.21.15.69:0/3209436176 conn(0x7fc2d80fe530 msgr2=0x7fc2d8100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.008+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3209436176 shutdown_connections 2024-05-14T16:56:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.009+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3209436176 wait complete. 2024-05-14T16:56:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.009+0000 7fc2df1b6640 1 Processor -- start 2024-05-14T16:56:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2df1b6640 1 -- start start 2024-05-14T16:56:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2df1b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8102a60 0x7fc2d819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2df1b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d8103420 0x7fc2d819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2df1b6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d8108a70 0x7fc2d819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2df1b6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8112d80 con 0x7fc2d8102a60 2024-05-14T16:56:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2df1b6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8112c00 con 0x7fc2d8103420 2024-05-14T16:56:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2df1b6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8112f00 con 0x7fc2d8108a70 2024-05-14T16:56:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2dd753640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d8108a70 0x7fc2d819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2dd753640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d8108a70 0x7fc2d819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:56268/0 (socket says 172.21.15.69:56268) 2024-05-14T16:56:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2dd753640 1 -- 172.21.15.69:0/3883625562 learned_addr learned my addr 172.21.15.69:0/3883625562 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8102a60 0x7fc2d819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2d7fff640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d8103420 0x7fc2d819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2dd753640 1 -- 172.21.15.69:0/3883625562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d8103420 msgr2=0x7fc2d819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2dd753640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d8103420 0x7fc2d819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2dd753640 1 -- 172.21.15.69:0/3883625562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8102a60 msgr2=0x7fc2d819b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2dd753640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8102a60 0x7fc2d819b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.010+0000 7fc2dd753640 1 -- 172.21.15.69:0/3883625562 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d81a0470 con 0x7fc2d8108a70 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.011+0000 7fc2dd753640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d8108a70 0x7fc2d819fd40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000eb10 tx=0x7fc2d000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.011+0000 7fc2d5ffb640 1 -- 172.21.15.69:0/3883625562 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0004b20 con 0x7fc2d8108a70 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.011+0000 7fc2d5ffb640 1 -- 172.21.15.69:0/3883625562 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d0004cc0 con 0x7fc2d8108a70 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.011+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3883625562 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d81a0760 con 0x7fc2d8108a70 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.011+0000 7fc2d5ffb640 1 -- 172.21.15.69:0/3883625562 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0010670 con 0x7fc2d8108a70 2024-05-14T16:56:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.011+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3883625562 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d81a7ff0 con 0x7fc2d8108a70 2024-05-14T16:56:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.014+0000 7fc2d5ffb640 1 -- 172.21.15.69:0/3883625562 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc2d0005020 con 0x7fc2d8108a70 2024-05-14T16:56:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.014+0000 7fc2d5ffb640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc2c80b2b10 0x7fc2c80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.014+0000 7fc2d5ffb640 1 -- 172.21.15.69:0/3883625562 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc2d00e2b20 con 0x7fc2d8108a70 2024-05-14T16:56:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc2c80b2b10 0x7fc2c80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2e1fad740 1 Processor -- start 2024-05-14T16:56:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2e1fad740 1 -- start start 2024-05-14T16:56:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e3645cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e3646210 0x55a7e3650590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 0x55a7e3652ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a7e35b8c30 con 0x55a7e3650ad0 2024-05-14T16:56:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc2c80b2b10 0x7fc2c80b4fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009b50 tx=0x7fc2cc005f90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a7e35a87d0 con 0x55a7e36458f0 2024-05-14T16:56:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a7e35ac810 con 0x55a7e3646210 2024-05-14T16:56:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2dcf52640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e3645cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2dcf52640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e3645cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:50000/0 (socket says 172.21.15.69:50000) 2024-05-14T16:56:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.015+0000 7fc2dcf52640 1 -- 172.21.15.69:0/2758148424 learned_addr learned my addr 172.21.15.69:0/2758148424 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.016+0000 7fc2dd753640 1 --2- 172.21.15.69:0/2758148424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 0x55a7e3652ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.016+0000 7fc2dcf52640 1 -- 172.21.15.69:0/2758148424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e3646210 msgr2=0x55a7e3650590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:56:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.016+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/2758148424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e3646210 0x55a7e3650590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.016+0000 7fc2dcf52640 1 -- 172.21.15.69:0/2758148424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 msgr2=0x55a7e3652ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.016+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/2758148424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 0x55a7e3652ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.016+0000 7fc2dcf52640 1 -- 172.21.15.69:0/2758148424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e32bd210 con 0x55a7e36458f0 2024-05-14T16:56:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.017+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/2758148424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e3645cd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc005bc0 tx=0x7fc2cc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2143b49145ae74 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.018+0000 7fc2c2ffd640 1 -- 172.21.15.69:0/2758148424 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc025bc0 con 0x55a7e36458f0 2024-05-14T16:56:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.018+0000 7fc2c2ffd640 1 -- 172.21.15.69:0/2758148424 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2cc029070 con 0x55a7e36458f0 2024-05-14T16:56:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.018+0000 7fc2c2ffd640 1 -- 172.21.15.69:0/2758148424 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc00e610 con 0x55a7e36458f0 2024-05-14T16:56:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.019+0000 7fc2e1fad740 1 -- 172.21.15.69:0/2758148424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 msgr2=0x55a7e3645cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.019+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/2758148424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e3645cd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc005bc0 tx=0x7fc2cc014000 comp rx=0 tx=0).stop 2024-05-14T16:56:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.019+0000 7fc2e1fad740 1 -- 172.21.15.69:0/2758148424 shutdown_connections 2024-05-14T16:56:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.020+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/2758148424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 0x55a7e3652ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.020+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/2758148424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e3646210 0x55a7e3650590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.020+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/2758148424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e3645cd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.020+0000 7fc2e1fad740 1 -- 172.21.15.69:0/2758148424 >> 172.21.15.69:0/2758148424 conn(0x55a7e3641e00 msgr2=0x55a7e36421e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.020+0000 7fc2e1fad740 1 -- 172.21.15.69:0/2758148424 shutdown_connections 2024-05-14T16:56:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.020+0000 7fc2e1fad740 1 -- 172.21.15.69:0/2758148424 wait complete. 2024-05-14T16:56:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.020+0000 7fc2e1fad740 1 -- start start 2024-05-14T16:56:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.020+0000 7fc2e1fad740 1 Processor -- start 2024-05-14T16:56:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 0x55a7e3650510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e364d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 0x55a7e364df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a7e328e790 con 0x55a7e3646210 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a7e3564890 con 0x55a7e3650ad0 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a7e36537a0 con 0x55a7e36458f0 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e364d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e364d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52546/0 (socket says 172.21.15.69:52546) 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2d7fff640 1 -- 172.21.15.69:0/143951142 learned_addr learned my addr 172.21.15.69:0/143951142 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2e1fad740 10 client.0 _ll_register_callbacks cb 0x55a7e3567d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2dd753640 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 0x55a7e364df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 0x55a7e3650510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2d7fff640 1 -- 172.21.15.69:0/143951142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 msgr2=0x55a7e3650510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2d7fff640 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 0x55a7e3650510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2d7fff640 1 -- 172.21.15.69:0/143951142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 msgr2=0x55a7e364df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2d7fff640 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 0x55a7e364df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2d7fff640 1 -- 172.21.15.69:0/143951142 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e3389620 con 0x55a7e3646210 2024-05-14T16:56:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.021+0000 7fc2dd753640 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 0x55a7e364df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.022+0000 7fc2d7fff640 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e364d800 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4006010 tx=0x7fc2c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aeba068bb47af1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.022+0000 7fc2b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.022+0000 7fc2b7fff640 1 -- 172.21.15.69:0/143951142 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4035040 con 0x55a7e3646210 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.022+0000 7fc2b7fff640 1 -- 172.21.15.69:0/143951142 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc2c4005450 con 0x55a7e3646210 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.022+0000 7fc2e1fad740 1 -- 172.21.15.69:0/143951142 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2cc029070 con 0x55a7e3646210 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.022+0000 7fc2b7fff640 1 -- 172.21.15.69:0/143951142 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc2c4005650 con 0x55a7e3646210 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.022+0000 7fc2e1fad740 10 client.16059 fetch_fsmap learned FSMap version 35 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.022+0000 7fc2e1fad740 1 -- 172.21.15.69:0/143951142 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a7e365f260 con 0x55a7e3646210 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.022+0000 7fc2b7fff640 1 -- 172.21.15.69:0/143951142 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c40044a0 con 0x55a7e3646210 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.022+0000 7fc2b7fff640 1 -- 172.21.15.69:0/143951142 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fc2c4003af0 con 0x55a7e3646210 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.023+0000 7fc2e1fad740 10 client.16059 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.023+0000 7fc2e1fad740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.023+0000 7fc2e1fad740 20 client.16059 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.023+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a7e36607a0 0x55a7e3662c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.023+0000 7fc2e1fad740 4 client.16059 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.023+0000 7fc2e1fad740 1 -- 172.21.15.69:0/143951142 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc2cc029070 con 0x55a7e36607a0 2024-05-14T16:56:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.024+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a7e36607a0 0x55a7e3662c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.025+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a7e36607a0 0x55a7e3662c00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe3916d6b33d6fc server_cookie=d866e99337492c0 in_seq=0 out_seq=0 2024-05-14T16:56:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.025+0000 7fc2b7fff640 10 client.16059 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.026+0000 7fc2b7fff640 1 -- 172.21.15.69:0/143951142 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fc2cc029070 con 0x55a7e36607a0 2024-05-14T16:56:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.026+0000 7fc2b7fff640 10 client.16059 handle_command_reply: tid=0 2024-05-14T16:56:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.026+0000 7fc2e1fad740 1 client.16059 shutdown 2024-05-14T16:56:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.026+0000 7fc2e1fad740 1 -- 172.21.15.69:0/143951142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 msgr2=0x55a7e364d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.026+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e364d800 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4006010 tx=0x7fc2c4004270 comp rx=0 tx=0).stop 2024-05-14T16:56:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.026+0000 7fc2e1fad740 1 -- 172.21.15.69:0/143951142 shutdown_connections 2024-05-14T16:56:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.026+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a7e36607a0 0x55a7e3662c00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.027+0000 7fc2b7fff640 0 client.16059 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.026+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 0x55a7e364df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.027+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e364d800 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.027+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/143951142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 0x55a7e3650510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.027+0000 7fc2e1fad740 1 -- 172.21.15.69:0/143951142 >> 172.21.15.69:0/143951142 conn(0x55a7e3641e00 msgr2=0x55a7e3641100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.027+0000 7fc2e1fad740 1 -- 172.21.15.69:0/143951142 shutdown_connections 2024-05-14T16:56:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.027+0000 7fc2e1fad740 1 -- 172.21.15.69:0/143951142 wait complete. 2024-05-14T16:56:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.027+0000 7fc2e1fad740 20 client.16059 trim_cache size 0 max 16384 2024-05-14T16:56:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2e1fad740 1 Processor -- start 2024-05-14T16:56:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2e1fad740 1 -- start start 2024-05-14T16:56:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e365c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e3646210 0x55a7e364c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 0x55a7e364c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81aeee0 con 0x55a7e3650ad0 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81ac720 con 0x55a7e36458f0 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81aa590 con 0x55a7e3646210 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2dd753640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 0x55a7e364c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2dd753640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 0x55a7e364c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52552/0 (socket says 172.21.15.69:52552) 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2dd753640 1 -- 172.21.15.69:0/2691283143 learned_addr learned my addr 172.21.15.69:0/2691283143 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.049+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e365c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.050+0000 7fc2d7fff640 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e3646210 0x55a7e364c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.050+0000 7fc2dd753640 1 -- 172.21.15.69:0/2691283143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e3646210 msgr2=0x55a7e364c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.050+0000 7fc2dd753640 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e3646210 0x55a7e364c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.050+0000 7fc2dd753640 1 -- 172.21.15.69:0/2691283143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 msgr2=0x55a7e365c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.050+0000 7fc2dd753640 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e365c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.050+0000 7fc2dd753640 1 -- 172.21.15.69:0/2691283143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e363d830 con 0x55a7e3650ad0 2024-05-14T16:56:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.050+0000 7fc2d7fff640 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e3646210 0x55a7e364c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.050+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e365c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.050+0000 7fc2dd753640 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 0x55a7e364c610 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00e0280 tx=0x7fc2d000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c345727cfbd04ec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.050+0000 7fc2c27fc640 1 -- 172.21.15.69:0/2691283143 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00c9070 con 0x55a7e3650ad0 2024-05-14T16:56:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.051+0000 7fc2c27fc640 1 -- 172.21.15.69:0/2691283143 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d00a98c0 con 0x55a7e3650ad0 2024-05-14T16:56:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.051+0000 7fc2c27fc640 1 -- 172.21.15.69:0/2691283143 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00aa010 con 0x55a7e3650ad0 2024-05-14T16:56:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.051+0000 7fc2e1fad740 1 -- 172.21.15.69:0/2691283143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 msgr2=0x55a7e364c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.051+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 0x55a7e364c610 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00e0280 tx=0x7fc2d000cee0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.052+0000 7fc2e1fad740 1 -- 172.21.15.69:0/2691283143 shutdown_connections 2024-05-14T16:56:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.052+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3650ad0 0x55a7e364c610 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.052+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e3646210 0x55a7e364c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.052+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/2691283143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e36458f0 0x55a7e365c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.052+0000 7fc2e1fad740 1 -- 172.21.15.69:0/2691283143 >> 172.21.15.69:0/2691283143 conn(0x55a7e3641e00 msgr2=0x55a7e363cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.052+0000 7fc2e1fad740 1 -- 172.21.15.69:0/2691283143 shutdown_connections 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.052+0000 7fc2e1fad740 1 -- 172.21.15.69:0/2691283143 wait complete. 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.052+0000 7fc2e1fad740 1 -- start start 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.052+0000 7fc2e1fad740 1 Processor -- start 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 0x55a7e3658bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e36590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2e1fad740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 0x55a7e3656120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81aabe0 con 0x55a7e3646210 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81ac050 con 0x55a7e3650ad0 2024-05-14T16:56:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e36590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2e1fad740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81ab2b0 con 0x55a7e36458f0 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e36590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52568/0 (socket says 172.21.15.69:52568) 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2d7fff640 1 -- 172.21.15.69:0/1779240589 learned_addr learned my addr 172.21.15.69:0/1779240589 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 0x55a7e3658bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2e1fad740 10 client.0 _ll_register_callbacks cb 0x7fc2d00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2dd753640 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 0x55a7e3656120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2dcf52640 1 -- 172.21.15.69:0/1779240589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 msgr2=0x55a7e3656120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 0x55a7e3656120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2dcf52640 1 -- 172.21.15.69:0/1779240589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 msgr2=0x55a7e36590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e36590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2dcf52640 1 -- 172.21.15.69:0/1779240589 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e3656920 con 0x55a7e36458f0 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2d7fff640 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e36590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2dd753640 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 0x55a7e3656120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2dcf52640 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 0x55a7e3658bb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc02edd0 tx=0x7fc2cc004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf75a8f08d7a843 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.053+0000 7fc2b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2b7fff640 1 -- 172.21.15.69:0/1779240589 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc03d070 con 0x55a7e36458f0 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2e1fad740 1 -- 172.21.15.69:0/1779240589 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2d00a98c0 con 0x55a7e36458f0 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2b7fff640 1 -- 172.21.15.69:0/1779240589 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc2cc0395c0 con 0x55a7e36458f0 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2b7fff640 1 -- 172.21.15.69:0/1779240589 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc2cc049050 con 0x55a7e36458f0 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2e1fad740 10 client.25489 fetch_fsmap learned FSMap version 35 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2e1fad740 1 -- 172.21.15.69:0/1779240589 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a7e36cacc0 con 0x55a7e36458f0 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2b7fff640 1 -- 172.21.15.69:0/1779240589 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d00a98c0 con 0x55a7e36458f0 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2b7fff640 1 -- 172.21.15.69:0/1779240589 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fc2cc042440 con 0x55a7e36458f0 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2e1fad740 10 client.25489 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:56:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2e1fad740 10 client.25489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2e1fad740 20 client.25489 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a7e36607a0 0x55a7e36cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2e1fad740 4 client.25489 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:56:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2e1fad740 1 -- 172.21.15.69:0/1779240589 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7e35bab80 con 0x55a7e36607a0 2024-05-14T16:56:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.054+0000 7fc2d7fff640 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a7e36607a0 0x55a7e36cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.055+0000 7fc2b7fff640 10 client.25489 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.055+0000 7fc2d7fff640 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a7e36607a0 0x55a7e36cd120 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62b5756f638d6c94 server_cookie=2fea100d9fe040d9 in_seq=0 out_seq=0 2024-05-14T16:56:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.065+0000 7fc2b7fff640 1 -- 172.21.15.69:0/1779240589 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a7e35bab80 con 0x55a7e36607a0 2024-05-14T16:56:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.065+0000 7fc2b7fff640 10 client.25489 handle_command_reply: tid=0 2024-05-14T16:56:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.065+0000 7fc2e1fad740 1 client.25489 shutdown 2024-05-14T16:56:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.066+0000 7fc2e1fad740 1 -- 172.21.15.69:0/1779240589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 msgr2=0x55a7e3658bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.066+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 0x55a7e3658bb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc02edd0 tx=0x7fc2cc004560 comp rx=0 tx=0).stop 2024-05-14T16:56:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.066+0000 7fc2e1fad740 1 -- 172.21.15.69:0/1779240589 shutdown_connections 2024-05-14T16:56:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.066+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a7e36607a0 0x55a7e36cd120 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.066+0000 7fc2b7fff640 0 client.25489 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.066+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a7e3650ad0 0x55a7e3656120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.066+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a7e3646210 0x55a7e36590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.066+0000 7fc2e1fad740 1 --2- 172.21.15.69:0/1779240589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7e36458f0 0x55a7e3658bb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.067+0000 7fc2e1fad740 1 -- 172.21.15.69:0/1779240589 >> 172.21.15.69:0/1779240589 conn(0x55a7e3641e00 msgr2=0x55a7e364c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.067+0000 7fc2e1fad740 1 -- 172.21.15.69:0/1779240589 shutdown_connections 2024-05-14T16:56:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.067+0000 7fc2e1fad740 1 -- 172.21.15.69:0/1779240589 wait complete. 2024-05-14T16:56:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.067+0000 7fc2e1fad740 20 client.25489 trim_cache size 0 max 16384 2024-05-14T16:56:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:56:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:56:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:56:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3883625562 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc2c80b2b10 msgr2=0x7fc2c80b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc2c80b2b10 0x7fc2c80b4fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009b50 tx=0x7fc2cc005f90 comp rx=0 tx=0).stop 2024-05-14T16:56:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3883625562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d8108a70 msgr2=0x7fc2d819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d8108a70 0x7fc2d819fd40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000eb10 tx=0x7fc2d000c2d0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3883625562 shutdown_connections 2024-05-14T16:56:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc2c80b2b10 0x7fc2c80b4fd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d8108a70 0x7fc2d819fd40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d8103420 0x7fc2d819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 --2- 172.21.15.69:0/3883625562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8102a60 0x7fc2d819b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3883625562 >> 172.21.15.69:0/3883625562 conn(0x7fc2d80fe530 msgr2=0x7fc2d80fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3883625562 shutdown_connections 2024-05-14T16:56:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:07.068+0000 7fc2df1b6640 1 -- 172.21.15.69:0/3883625562 wait complete. 2024-05-14T16:56:07.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:06 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:07.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:06 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:07.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:06 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:56:07.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:06 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:07.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:06 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:07.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:06 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:56:07.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:06 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:07.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:06 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:07.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:06 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:56:08.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:07 smithi069 ceph-mon[31507]: pgmap v191: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.46k op/s 2024-05-14T16:56:08.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:07 smithi203 ceph-mon[35787]: pgmap v191: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.46k op/s 2024-05-14T16:56:08.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:07 smithi092 ceph-mon[35763]: pgmap v191: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.46k op/s 2024-05-14T16:56:10.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:09 smithi069 ceph-mon[31507]: pgmap v192: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 1.39k op/s 2024-05-14T16:56:10.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:09 smithi069 ceph-mon[31507]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T16:56:10.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:09 smithi203 ceph-mon[35787]: pgmap v192: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 1.39k op/s 2024-05-14T16:56:10.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:09 smithi203 ceph-mon[35787]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T16:56:10.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:09 smithi092 ceph-mon[35763]: pgmap v192: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 1.39k op/s 2024-05-14T16:56:10.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:09 smithi092 ceph-mon[35763]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T16:56:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 340.0 s, 2806.4 tps, lat 35.475 ms stddev 38.078 2024-05-14T16:56:12.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:11 smithi069 ceph-mon[31507]: pgmap v194: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 1.99k op/s 2024-05-14T16:56:12.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:11 smithi203 ceph-mon[35787]: pgmap v194: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 1.99k op/s 2024-05-14T16:56:12.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:11 smithi092 ceph-mon[35763]: pgmap v194: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 1.99k op/s 2024-05-14T16:56:14.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:13 smithi069 ceph-mon[31507]: pgmap v195: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 1.99k op/s 2024-05-14T16:56:14.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:13 smithi069 ceph-mon[31507]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T16:56:14.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:13 smithi203 ceph-mon[35787]: pgmap v195: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 1.99k op/s 2024-05-14T16:56:14.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:13 smithi203 ceph-mon[35787]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T16:56:14.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:13 smithi092 ceph-mon[35763]: pgmap v195: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 1.99k op/s 2024-05-14T16:56:14.255 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:13 smithi092 ceph-mon[35763]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T16:56:15.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 345.0 s, 3166.4 tps, lat 31.628 ms stddev 34.773 2024-05-14T16:56:16.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:15 smithi069 ceph-mon[31507]: pgmap v197: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 434 op/s 2024-05-14T16:56:16.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:15 smithi203 ceph-mon[35787]: pgmap v197: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 434 op/s 2024-05-14T16:56:16.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:15 smithi092 ceph-mon[35763]: pgmap v197: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 434 op/s 2024-05-14T16:56:18.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:17 smithi069 ceph-mon[31507]: pgmap v198: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 900 op/s 2024-05-14T16:56:18.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:17 smithi203 ceph-mon[35787]: pgmap v198: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 900 op/s 2024-05-14T16:56:18.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:17 smithi092 ceph-mon[35763]: pgmap v198: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 900 op/s 2024-05-14T16:56:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.190+0000 7fb45826b640 1 -- 172.21.15.69:0/2925059015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb450107970 msgr2=0x7fb450107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.190+0000 7fb45826b640 1 --2- 172.21.15.69:0/2925059015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb450107970 0x7fb450107d50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009930 tx=0x7fb44c02a420 comp rx=0 tx=0).stop 2024-05-14T16:56:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.191+0000 7fb45826b640 1 -- 172.21.15.69:0/2925059015 shutdown_connections 2024-05-14T16:56:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.191+0000 7fb45826b640 1 --2- 172.21.15.69:0/2925059015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb450102320 0x7fb45010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.191+0000 7fb45826b640 1 --2- 172.21.15.69:0/2925059015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb450101960 0x7fb450101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.191+0000 7fb45826b640 1 --2- 172.21.15.69:0/2925059015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb450107970 0x7fb450107d50 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.191+0000 7fb45826b640 1 -- 172.21.15.69:0/2925059015 >> 172.21.15.69:0/2925059015 conn(0x7fb450078b60 msgr2=0x7fb450078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:20.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:19 smithi203 ceph-mon[35787]: pgmap v199: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.0 MiB/s wr, 734 op/s 2024-05-14T16:56:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.192+0000 7fb45826b640 1 -- 172.21.15.69:0/2925059015 shutdown_connections 2024-05-14T16:56:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.192+0000 7fb45826b640 1 -- 172.21.15.69:0/2925059015 wait complete. 2024-05-14T16:56:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.192+0000 7fb45826b640 1 Processor -- start 2024-05-14T16:56:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.192+0000 7fb45826b640 1 -- start start 2024-05-14T16:56:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb45826b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb450101960 0x7fb4501991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb45826b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb450102320 0x7fb450199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb45826b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb450107970 0x7fb45019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb45826b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb450111c30 con 0x7fb450107970 2024-05-14T16:56:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb45826b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb450111ab0 con 0x7fb450102320 2024-05-14T16:56:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb45826b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb450111db0 con 0x7fb450101960 2024-05-14T16:56:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb456808640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb450107970 0x7fb45019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb456007640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb450101960 0x7fb4501991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb456808640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb450107970 0x7fb45019daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56104/0 (socket says 172.21.15.69:56104) 2024-05-14T16:56:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb456808640 1 -- 172.21.15.69:0/4107080319 learned_addr learned my addr 172.21.15.69:0/4107080319 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb455806640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb450102320 0x7fb450199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb455806640 1 -- 172.21.15.69:0/4107080319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb450101960 msgr2=0x7fb4501991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb455806640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb450101960 0x7fb4501991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb455806640 1 -- 172.21.15.69:0/4107080319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb450107970 msgr2=0x7fb45019daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb455806640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb450107970 0x7fb45019daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb455806640 1 -- 172.21.15.69:0/4107080319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45019e1d0 con 0x7fb450102320 2024-05-14T16:56:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.193+0000 7fb456808640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb450107970 0x7fb45019daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.194+0000 7fb455806640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb450102320 0x7fb450199720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb44000e9f0 tx=0x7fb44000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.194+0000 7fb4477fe640 1 -- 172.21.15.69:0/4107080319 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44000ccc0 con 0x7fb450102320 2024-05-14T16:56:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.194+0000 7fb4477fe640 1 -- 172.21.15.69:0/4107080319 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4400108e0 con 0x7fb450102320 2024-05-14T16:56:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.194+0000 7fb45826b640 1 -- 172.21.15.69:0/4107080319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45019e4c0 con 0x7fb450102320 2024-05-14T16:56:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.194+0000 7fb45826b640 1 -- 172.21.15.69:0/4107080319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4500fe820 con 0x7fb450102320 2024-05-14T16:56:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.194+0000 7fb4477fe640 1 -- 172.21.15.69:0/4107080319 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440018880 con 0x7fb450102320 2024-05-14T16:56:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.198+0000 7fb45af88740 1 Processor -- start 2024-05-14T16:56:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.198+0000 7fb45af88740 1 -- start start 2024-05-14T16:56:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.198+0000 7fb45af88740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fed4f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.198+0000 7fb45af88740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.198+0000 7fb45af88740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 0x5598fed65140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.198+0000 7fb45af88740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5598fecb9c30 con 0x5598fed62d50 2024-05-14T16:56:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.198+0000 7fb45af88740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5598feca97d0 con 0x5598fed4f900 2024-05-14T16:56:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.198+0000 7fb45af88740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5598fecad810 con 0x5598fed4efe0 2024-05-14T16:56:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb456007640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb456808640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fed4f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb456007640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:46276/0 (socket says 172.21.15.69:46276) 2024-05-14T16:56:20.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb456007640 1 -- 172.21.15.69:0/109049284 learned_addr learned my addr 172.21.15.69:0/109049284 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb456808640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fed4f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:42748/0 (socket says 172.21.15.69:42748) 2024-05-14T16:56:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb4477fe640 1 -- 172.21.15.69:0/4107080319 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb440010430 con 0x7fb450102320 2024-05-14T16:56:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb456007640 1 -- 172.21.15.69:0/109049284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 msgr2=0x5598fed4f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb456007640 1 --2- 172.21.15.69:0/109049284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fed4f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb456007640 1 -- 172.21.15.69:0/109049284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 msgr2=0x5598fed65140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb455806640 1 --2- 172.21.15.69:0/109049284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 0x5598fed65140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb456007640 1 --2- 172.21.15.69:0/109049284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 0x5598fed65140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb456007640 1 -- 172.21.15.69:0/109049284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598fe9be210 con 0x5598fed4f900 2024-05-14T16:56:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.199+0000 7fb455806640 1 --2- 172.21.15.69:0/109049284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 0x5598fed65140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:56:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.200+0000 7fb456007640 1 --2- 172.21.15.69:0/109049284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009930 tx=0x7fb44c005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e7b3df08052abff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.200+0000 7fb444bfe640 1 -- 172.21.15.69:0/109049284 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c035040 con 0x5598fed4f900 2024-05-14T16:56:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.200+0000 7fb444bfe640 1 -- 172.21.15.69:0/109049284 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c0316a0 con 0x5598fed4f900 2024-05-14T16:56:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.200+0000 7fb444bfe640 1 -- 172.21.15.69:0/109049284 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0047a0 con 0x5598fed4f900 2024-05-14T16:56:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.200+0000 7fb4477fe640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb43c0b2b50 0x7fb43c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.200+0000 7fb456808640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb43c0b2b50 0x7fb43c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.200+0000 7fb4477fe640 1 -- 172.21.15.69:0/4107080319 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fb4400e28f0 con 0x7fb450102320 2024-05-14T16:56:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.201+0000 7fb45af88740 1 -- 172.21.15.69:0/109049284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 msgr2=0x5598fed4a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.201+0000 7fb45af88740 1 --2- 172.21.15.69:0/109049284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009930 tx=0x7fb44c005be0 comp rx=0 tx=0).stop 2024-05-14T16:56:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.201+0000 7fb456808640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb43c0b2b50 0x7fb43c0b5010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb4480049d0 tx=0x7fb448009500 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.201+0000 7fb45af88740 1 -- 172.21.15.69:0/109049284 shutdown_connections 2024-05-14T16:56:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.201+0000 7fb45af88740 1 --2- 172.21.15.69:0/109049284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 0x5598fed65140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.202+0000 7fb45af88740 1 --2- 172.21.15.69:0/109049284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a830 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.202+0000 7fb45af88740 1 --2- 172.21.15.69:0/109049284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fed4f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.202+0000 7fb45af88740 1 -- 172.21.15.69:0/109049284 >> 172.21.15.69:0/109049284 conn(0x5598fed4b610 msgr2=0x5598fed4b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.202+0000 7fb45af88740 1 -- 172.21.15.69:0/109049284 shutdown_connections 2024-05-14T16:56:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.202+0000 7fb45af88740 1 -- 172.21.15.69:0/109049284 wait complete. 2024-05-14T16:56:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.202+0000 7fb45af88740 1 -- start start 2024-05-14T16:56:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb45af88740 1 Processor -- start 2024-05-14T16:56:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb45af88740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4efe0 0x5598fed4a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb45af88740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb45af88740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed62d50 0x5598fed47f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:21.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb45af88740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5598fe98f790 con 0x5598fed4efe0 2024-05-14T16:56:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb45af88740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5598fec65890 con 0x5598fed4f900 2024-05-14T16:56:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb45af88740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5598fed65780 con 0x5598fed62d50 2024-05-14T16:56:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb45af88740 10 client.0 _ll_register_callbacks cb 0x5598fec68d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb456007640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4efe0 0x5598fed4a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb456808640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed62d50 0x5598fed47f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb456007640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4efe0 0x5598fed4a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56130/0 (socket says 172.21.15.69:56130) 2024-05-14T16:56:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb456808640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed62d50 0x5598fed47f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:42764/0 (socket says 172.21.15.69:42764) 2024-05-14T16:56:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb456007640 1 -- 172.21.15.69:0/1179158561 learned_addr learned my addr 172.21.15.69:0/1179158561 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.203+0000 7fb455806640 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:21.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:19 smithi069 ceph-mon[31507]: pgmap v199: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.0 MiB/s wr, 734 op/s 2024-05-14T16:56:21.210 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:19 smithi092 ceph-mon[35763]: pgmap v199: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.0 MiB/s wr, 734 op/s 2024-05-14T16:56:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb455806640 1 -- 172.21.15.69:0/1179158561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed62d50 msgr2=0x5598fed47f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb455806640 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed62d50 0x5598fed47f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb455806640 1 -- 172.21.15.69:0/1179158561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4efe0 msgr2=0x5598fed4a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb455806640 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4efe0 0x5598fed4a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb455806640 1 -- 172.21.15.69:0/1179158561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598fea8a620 con 0x5598fed4f900 2024-05-14T16:56:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb456808640 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed62d50 0x5598fed47f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb456007640 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4efe0 0x5598fed4a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb455806640 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed479e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb4400dffd0 tx=0x7fb4400a9b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cccd41883c0348d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb439ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:56:21.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb439ffb640 1 -- 172.21.15.69:0/1179158561 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400a9c10 con 0x5598fed4f900 2024-05-14T16:56:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb439ffb640 1 -- 172.21.15.69:0/1179158561 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fb4400e0250 con 0x5598fed4f900 2024-05-14T16:56:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb45af88740 1 -- 172.21.15.69:0/1179158561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb44c0316a0 con 0x5598fed4f900 2024-05-14T16:56:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb439ffb640 1 -- 172.21.15.69:0/1179158561 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb4400e0450 con 0x5598fed4f900 2024-05-14T16:56:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb45af88740 10 client.25568 fetch_fsmap learned FSMap version 35 2024-05-14T16:56:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb45af88740 1 -- 172.21.15.69:0/1179158561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5598fed711b0 con 0x5598fed4f900 2024-05-14T16:56:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.204+0000 7fb439ffb640 1 -- 172.21.15.69:0/1179158561 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c0316a0 con 0x5598fed4f900 2024-05-14T16:56:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.205+0000 7fb439ffb640 1 -- 172.21.15.69:0/1179158561 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fb4400e2ce0 con 0x5598fed4f900 2024-05-14T16:56:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.205+0000 7fb45af88740 10 client.25568 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:56:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.205+0000 7fb45af88740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.205+0000 7fb45af88740 20 client.25568 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.205+0000 7fb45af88740 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5598fed72620 0x5598fed74a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.205+0000 7fb45af88740 4 client.25568 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.205+0000 7fb45af88740 1 -- 172.21.15.69:0/1179158561 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598fecbbb80 con 0x5598fed72620 2024-05-14T16:56:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.205+0000 7fb456007640 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5598fed72620 0x5598fed74a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.206+0000 7fb439ffb640 10 client.25568 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.206+0000 7fb456007640 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5598fed72620 0x5598fed74a50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ade7e6f3f050ef3 server_cookie=6ca3f9acd02b6ab1 in_seq=0 out_seq=0 2024-05-14T16:56:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb439ffb640 1 -- 172.21.15.69:0/1179158561 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598fecbbb80 con 0x5598fed72620 2024-05-14T16:56:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb439ffb640 10 client.25568 handle_command_reply: tid=0 2024-05-14T16:56:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 client.25568 shutdown 2024-05-14T16:56:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 -- 172.21.15.69:0/1179158561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 msgr2=0x5598fed479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed479e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb4400dffd0 tx=0x7fb4400a9b10 comp rx=0 tx=0).stop 2024-05-14T16:56:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 -- 172.21.15.69:0/1179158561 shutdown_connections 2024-05-14T16:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5598fed72620 0x5598fed74a50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed62d50 0x5598fed47f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed479e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 --2- 172.21.15.69:0/1179158561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4efe0 0x5598fed4a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb439ffb640 0 client.25568 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 -- 172.21.15.69:0/1179158561 >> 172.21.15.69:0/1179158561 conn(0x5598fed4b610 msgr2=0x5598fed51480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 -- 172.21.15.69:0/1179158561 shutdown_connections 2024-05-14T16:56:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 1 -- 172.21.15.69:0/1179158561 wait complete. 2024-05-14T16:56:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.207+0000 7fb45af88740 20 client.25568 trim_cache size 0 max 16384 2024-05-14T16:56:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb45af88740 1 Processor -- start 2024-05-14T16:56:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb45af88740 1 -- start start 2024-05-14T16:56:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb45af88740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fed6e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb45af88740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb45af88740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 0x5598fed462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb45af88740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4501aee20 con 0x5598fed62d50 2024-05-14T16:56:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb45af88740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4501ac660 con 0x5598fed4f900 2024-05-14T16:56:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb45af88740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb4501aa4d0 con 0x5598fed4efe0 2024-05-14T16:56:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb455806640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb455806640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:46286/0 (socket says 172.21.15.69:46286) 2024-05-14T16:56:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb455806640 1 -- 172.21.15.69:0/1035326758 learned_addr learned my addr 172.21.15.69:0/1035326758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb456007640 1 --2- 172.21.15.69:0/1035326758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fed6e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.229+0000 7fb456808640 1 --2- 172.21.15.69:0/1035326758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 0x5598fed462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.230+0000 7fb455806640 1 -- 172.21.15.69:0/1035326758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 msgr2=0x5598fed6e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.230+0000 7fb455806640 1 --2- 172.21.15.69:0/1035326758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fed6e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.230+0000 7fb455806640 1 -- 172.21.15.69:0/1035326758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 msgr2=0x5598fed462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.230+0000 7fb455806640 1 --2- 172.21.15.69:0/1035326758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 0x5598fed462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.230+0000 7fb455806640 1 -- 172.21.15.69:0/1035326758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598fed4f730 con 0x5598fed4f900 2024-05-14T16:56:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.230+0000 7fb456007640 1 --2- 172.21.15.69:0/1035326758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fed6e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.230+0000 7fb455806640 1 --2- 172.21.15.69:0/1035326758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb440000c00 tx=0x7fb4400a9380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1421e688aac4d342 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.230+0000 7fb43bfff640 1 -- 172.21.15.69:0/1035326758 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400adf10 con 0x5598fed4f900 2024-05-14T16:56:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.230+0000 7fb43bfff640 1 -- 172.21.15.69:0/1035326758 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4400b29c0 con 0x5598fed4f900 2024-05-14T16:56:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.230+0000 7fb43bfff640 1 -- 172.21.15.69:0/1035326758 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400a57e0 con 0x5598fed4f900 2024-05-14T16:56:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.231+0000 7fb45af88740 1 -- 172.21.15.69:0/1035326758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 msgr2=0x5598fed4a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.231+0000 7fb45af88740 1 --2- 172.21.15.69:0/1035326758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb440000c00 tx=0x7fb4400a9380 comp rx=0 tx=0).stop 2024-05-14T16:56:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.231+0000 7fb45af88740 1 -- 172.21.15.69:0/1035326758 shutdown_connections 2024-05-14T16:56:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.231+0000 7fb45af88740 1 --2- 172.21.15.69:0/1035326758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed62d50 0x5598fed462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.231+0000 7fb45af88740 1 --2- 172.21.15.69:0/1035326758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed4f900 0x5598fed4a220 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.231+0000 7fb45af88740 1 --2- 172.21.15.69:0/1035326758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fed6e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.231+0000 7fb45af88740 1 -- 172.21.15.69:0/1035326758 >> 172.21.15.69:0/1035326758 conn(0x5598fed4b610 msgr2=0x5598fed41870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.231+0000 7fb45af88740 1 -- 172.21.15.69:0/1035326758 shutdown_connections 2024-05-14T16:56:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.231+0000 7fb45af88740 1 -- 172.21.15.69:0/1035326758 wait complete. 2024-05-14T16:56:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.232+0000 7fb45af88740 1 -- start start 2024-05-14T16:56:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.232+0000 7fb45af88740 1 Processor -- start 2024-05-14T16:56:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb45af88740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fedd5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb45af88740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4f900 0x5598fedd57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb45af88740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed62d50 0x5598fed65da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb45af88740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4501aab20 con 0x5598fed4f900 2024-05-14T16:56:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb45af88740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4501abf90 con 0x5598fed62d50 2024-05-14T16:56:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb45af88740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb4501ab1f0 con 0x5598fed4efe0 2024-05-14T16:56:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb45af88740 10 client.0 _ll_register_callbacks cb 0x7fb4400a9450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb455806640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4f900 0x5598fedd57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456808640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed62d50 0x5598fed65da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456808640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed62d50 0x5598fed65da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:46288/0 (socket says 172.21.15.69:46288) 2024-05-14T16:56:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb455806640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4f900 0x5598fedd57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56154/0 (socket says 172.21.15.69:56154) 2024-05-14T16:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456007640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fedd5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456808640 1 -- 172.21.15.69:0/4246122108 learned_addr learned my addr 172.21.15.69:0/4246122108 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456808640 1 -- 172.21.15.69:0/4246122108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 msgr2=0x5598fedd5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456808640 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fedd5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456808640 1 -- 172.21.15.69:0/4246122108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4f900 msgr2=0x5598fedd57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:22.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456808640 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4f900 0x5598fedd57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456808640 1 -- 172.21.15.69:0/4246122108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598fed66490 con 0x5598fed62d50 2024-05-14T16:56:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456007640 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fedd5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb455806640 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4f900 0x5598fedd57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb456808640 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed62d50 0x5598fed65da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb44800fef0 tx=0x7fb448004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bedf12b55315760 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.233+0000 7fb439ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:56:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.234+0000 7fb439ffb640 1 -- 172.21.15.69:0/4246122108 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44802f060 con 0x5598fed62d50 2024-05-14T16:56:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.234+0000 7fb439ffb640 1 -- 172.21.15.69:0/4246122108 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fb448011bd0 con 0x5598fed62d50 2024-05-14T16:56:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.234+0000 7fb45af88740 1 -- 172.21.15.69:0/4246122108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4400b29c0 con 0x5598fed62d50 2024-05-14T16:56:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.234+0000 7fb439ffb640 1 -- 172.21.15.69:0/4246122108 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb448011dd0 con 0x5598fed62d50 2024-05-14T16:56:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.234+0000 7fb45af88740 10 client.25574 fetch_fsmap learned FSMap version 35 2024-05-14T16:56:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.235+0000 7fb45af88740 1 -- 172.21.15.69:0/4246122108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5598feddca50 con 0x5598fed62d50 2024-05-14T16:56:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.235+0000 7fb439ffb640 1 -- 172.21.15.69:0/4246122108 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4400b29c0 con 0x5598fed62d50 2024-05-14T16:56:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.235+0000 7fb439ffb640 1 -- 172.21.15.69:0/4246122108 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7fb44801e530 con 0x5598fed62d50 2024-05-14T16:56:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.235+0000 7fb45af88740 10 client.25574 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:56:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.235+0000 7fb45af88740 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.235+0000 7fb45af88740 20 client.25574 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.235+0000 7fb45af88740 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5598feddd230 0x5598feddf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.235+0000 7fb45af88740 4 client.25574 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:56:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.235+0000 7fb45af88740 1 -- 172.21.15.69:0/4246122108 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598fecb5360 con 0x5598feddd230 2024-05-14T16:56:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.235+0000 7fb456007640 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5598feddd230 0x5598feddf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.236+0000 7fb439ffb640 10 client.25574 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.236+0000 7fb456007640 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5598feddd230 0x5598feddf650 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c6f4fe88e35cb4 server_cookie=e14bf023fbd98fa4 in_seq=0 out_seq=0 2024-05-14T16:56:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb439ffb640 1 -- 172.21.15.69:0/4246122108 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5598fecb5360 con 0x5598feddd230 2024-05-14T16:56:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb439ffb640 10 client.25574 handle_command_reply: tid=0 2024-05-14T16:56:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 client.25574 shutdown 2024-05-14T16:56:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 -- 172.21.15.69:0/4246122108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed62d50 msgr2=0x5598fed65da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed62d50 0x5598fed65da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb44800fef0 tx=0x7fb448004a40 comp rx=0 tx=0).stop 2024-05-14T16:56:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 -- 172.21.15.69:0/4246122108 shutdown_connections 2024-05-14T16:56:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5598feddd230 0x5598feddf650 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598fed62d50 0x5598fed65da0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598fed4f900 0x5598fedd57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 --2- 172.21.15.69:0/4246122108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598fed4efe0 0x5598fedd5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 -- 172.21.15.69:0/4246122108 >> 172.21.15.69:0/4246122108 conn(0x5598fed4b610 msgr2=0x5598fed46710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb439ffb640 0 client.25574 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 -- 172.21.15.69:0/4246122108 shutdown_connections 2024-05-14T16:56:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 1 -- 172.21.15.69:0/4246122108 wait complete. 2024-05-14T16:56:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.246+0000 7fb45af88740 20 client.25574 trim_cache size 0 max 16384 2024-05-14T16:56:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:56:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:56:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:56:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 -- 172.21.15.69:0/4107080319 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb43c0b2b50 msgr2=0x7fb43c0b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb43c0b2b50 0x7fb43c0b5010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb4480049d0 tx=0x7fb448009500 comp rx=0 tx=0).stop 2024-05-14T16:56:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 -- 172.21.15.69:0/4107080319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb450102320 msgr2=0x7fb450199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb450102320 0x7fb450199720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb44000e9f0 tx=0x7fb44000eeb0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 -- 172.21.15.69:0/4107080319 shutdown_connections 2024-05-14T16:56:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb43c0b2b50 0x7fb43c0b5010 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb450107970 0x7fb45019daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb450102320 0x7fb450199720 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 --2- 172.21.15.69:0/4107080319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb450101960 0x7fb4501991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 -- 172.21.15.69:0/4107080319 >> 172.21.15.69:0/4107080319 conn(0x7fb450078b60 msgr2=0x7fb45006b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.247+0000 7fb45826b640 1 -- 172.21.15.69:0/4107080319 shutdown_connections 2024-05-14T16:56:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:20.248+0000 7fb45826b640 1 -- 172.21.15.69:0/4107080319 wait complete. 2024-05-14T16:56:22.631 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 350.0 s, 3674.0 tps, lat 27.308 ms stddev 28.727 2024-05-14T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:21 smithi069 ceph-mon[31507]: pgmap v200: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 MiB/s wr, 574 op/s 2024-05-14T16:56:22.650 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:21 smithi092 ceph-mon[35763]: pgmap v200: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 MiB/s wr, 574 op/s 2024-05-14T16:56:22.657 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:21 smithi203 ceph-mon[35787]: pgmap v200: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 MiB/s wr, 574 op/s 2024-05-14T16:56:24.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:23 smithi069 ceph-mon[31507]: pgmap v201: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 755 op/s 2024-05-14T16:56:24.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:23 smithi203 ceph-mon[35787]: pgmap v201: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 755 op/s 2024-05-14T16:56:24.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:23 smithi092 ceph-mon[35763]: pgmap v201: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 755 op/s 2024-05-14T16:56:25.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:24 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:25.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:24 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:25.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:24 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:25.672 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 355.0 s, 2808.4 tps, lat 35.414 ms stddev 40.128 2024-05-14T16:56:26.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:25 smithi069 ceph-mon[31507]: pgmap v202: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 682 op/s 2024-05-14T16:56:26.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:25 smithi203 ceph-mon[35787]: pgmap v202: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 682 op/s 2024-05-14T16:56:26.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:25 smithi092 ceph-mon[35763]: pgmap v202: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 682 op/s 2024-05-14T16:56:27.437 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.4 2024-05-14T16:56:27.443 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:56:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:28.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:27 smithi069 ceph-mon[31507]: pgmap v203: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-05-14T16:56:28.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:27 smithi203 ceph-mon[35787]: pgmap v203: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-05-14T16:56:28.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:27 smithi092 ceph-mon[35763]: pgmap v203: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-05-14T16:56:30.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:29 smithi069 ceph-mon[31507]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.3 MiB/s wr, 714 op/s 2024-05-14T16:56:30.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:29 smithi203 ceph-mon[35787]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.3 MiB/s wr, 714 op/s 2024-05-14T16:56:30.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:29 smithi092 ceph-mon[35763]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.3 MiB/s wr, 714 op/s 2024-05-14T16:56:30.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 360.0 s, 4011.0 tps, lat 25.021 ms stddev 29.000 2024-05-14T16:56:30.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.504+0000 7f879a39b700 1 -- 172.21.15.69:0/2674601915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8794102c40 msgr2=0x7f87941030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:30.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.504+0000 7f879a39b700 1 --2- 172.21.15.69:0/2674601915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8794102c40 0x7f87941030c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f877c009990 tx=0x7f877c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.504+0000 7f879a39b700 1 -- 172.21.15.69:0/2674601915 shutdown_connections 2024-05-14T16:56:30.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.504+0000 7f879a39b700 1 --2- 172.21.15.69:0/2674601915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87941084c0 0x7f87941088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.504+0000 7f879a39b700 1 --2- 172.21.15.69:0/2674601915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8794103600 0x7f879410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.504+0000 7f879a39b700 1 --2- 172.21.15.69:0/2674601915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8794102c40 0x7f87941030c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.504+0000 7f879a39b700 1 -- 172.21.15.69:0/2674601915 >> 172.21.15.69:0/2674601915 conn(0x7f8794078ab0 msgr2=0x7f87940ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.504+0000 7f879a39b700 1 -- 172.21.15.69:0/2674601915 shutdown_connections 2024-05-14T16:56:30.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.504+0000 7f879a39b700 1 -- 172.21.15.69:0/2674601915 wait complete. 2024-05-14T16:56:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.505+0000 7f879a39b700 1 Processor -- start 2024-05-14T16:56:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.505+0000 7f879a39b700 1 -- start start 2024-05-14T16:56:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.505+0000 7f879a39b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8794102c40 0x7f879419bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.505+0000 7f879a39b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8794103600 0x7f879419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f879a39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87941084c0 0x7f87941a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f878bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8794103600 0x7f879419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f879a39b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8794112670 con 0x7f8794103600 2024-05-14T16:56:30.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f879a39b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f87941124f0 con 0x7f87941084c0 2024-05-14T16:56:30.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f879a39b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f87941127f0 con 0x7f8794102c40 2024-05-14T16:56:30.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f878bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8794103600 0x7f879419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42278/0 (socket says 172.21.15.69:42278) 2024-05-14T16:56:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f878bfff700 1 -- 172.21.15.69:0/463463974 learned_addr learned my addr 172.21.15.69:0/463463974 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:30.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f8793fff700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8794102c40 0x7f879419bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f8798938700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87941084c0 0x7f87941a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f8793fff700 1 -- 172.21.15.69:0/463463974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87941084c0 msgr2=0x7f87941a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:30.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f8793fff700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87941084c0 0x7f87941a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f8793fff700 1 -- 172.21.15.69:0/463463974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8794103600 msgr2=0x7f879419c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:30.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f8793fff700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8794103600 0x7f879419c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.506+0000 7f8793fff700 1 -- 172.21.15.69:0/463463974 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87941a0eb0 con 0x7f8794102c40 2024-05-14T16:56:30.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.507+0000 7f8793fff700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8794102c40 0x7f879419bef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f878400e9c0 tx=0x7f878400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:30.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.507+0000 7f8791ffb700 1 -- 172.21.15.69:0/463463974 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878400cb70 con 0x7f8794102c40 2024-05-14T16:56:30.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.507+0000 7f8791ffb700 1 -- 172.21.15.69:0/463463974 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878400cd10 con 0x7f8794102c40 2024-05-14T16:56:30.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.507+0000 7f8791ffb700 1 -- 172.21.15.69:0/463463974 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784018680 con 0x7f8794102c40 2024-05-14T16:56:30.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.507+0000 7f879a39b700 1 -- 172.21.15.69:0/463463974 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87941a11a0 con 0x7f8794102c40 2024-05-14T16:56:30.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.507+0000 7f879a39b700 1 -- 172.21.15.69:0/463463974 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87941a8a20 con 0x7f8794102c40 2024-05-14T16:56:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.508+0000 7f879a39b700 1 -- 172.21.15.69:0/463463974 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8794103c50 con 0x7f8794102c40 2024-05-14T16:56:30.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.510+0000 7f8791ffb700 1 -- 172.21.15.69:0/463463974 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f878401f020 con 0x7f8794102c40 2024-05-14T16:56:30.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.511+0000 7f8791ffb700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f87800b2bc0 0x7f87800b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.511+0000 7f8791ffb700 1 -- 172.21.15.69:0/463463974 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f87840e2470 con 0x7f8794102c40 2024-05-14T16:56:30.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.512+0000 7f878bfff700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f87800b2bc0 0x7f87800b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.512+0000 7f878bfff700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f87800b2bc0 0x7f87800b5080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8794102c10 tx=0x7f877c00b260 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.516+0000 7f8791ffb700 1 -- 172.21.15.69:0/463463974 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87840a9de0 con 0x7f8794102c40 2024-05-14T16:56:30.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.929+0000 7f879a39b700 1 -- 172.21.15.69:0/463463974 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87941a8bf0 con 0x7f8794102c40 2024-05-14T16:56:30.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.930+0000 7f8791ffb700 1 -- 172.21.15.69:0/463463974 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11861 (secure 0 0 0) 0x7f87840a96e0 con 0x7f8794102c40 2024-05-14T16:56:30.954 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:56:30.959 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:56:23.611697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24433,24446,24422]},"id":1}]} 2024-05-14T16:56:30.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.933+0000 7f879a39b700 1 -- 172.21.15.69:0/463463974 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f87800b2bc0 msgr2=0x7f87800b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:30.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.933+0000 7f879a39b700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f87800b2bc0 0x7f87800b5080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8794102c10 tx=0x7f877c00b260 comp rx=0 tx=0).stop 2024-05-14T16:56:30.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.933+0000 7f879a39b700 1 -- 172.21.15.69:0/463463974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8794102c40 msgr2=0x7f879419bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:30.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.933+0000 7f879a39b700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8794102c40 0x7f879419bef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f878400e9c0 tx=0x7f878400ecd0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.933+0000 7f879a39b700 1 -- 172.21.15.69:0/463463974 shutdown_connections 2024-05-14T16:56:31.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.934+0000 7f879a39b700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87941084c0 0x7f87941a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.934+0000 7f879a39b700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8794102c40 0x7f879419bef0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.934+0000 7f879a39b700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f87800b2bc0 0x7f87800b5080 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.934+0000 7f879a39b700 1 --2- 172.21.15.69:0/463463974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8794103600 0x7f879419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.934+0000 7f879a39b700 1 -- 172.21.15.69:0/463463974 >> 172.21.15.69:0/463463974 conn(0x7f8794078ab0 msgr2=0x7f8794101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:31.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.934+0000 7f879a39b700 1 -- 172.21.15.69:0/463463974 shutdown_connections 2024-05-14T16:56:31.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:30.934+0000 7f879a39b700 1 -- 172.21.15.69:0/463463974 wait complete. 2024-05-14T16:56:31.092 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 36 2024-05-14T16:56:31.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:30 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/463463974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:31.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:30 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/463463974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:31.255 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:30 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/463463974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:31.601 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 136.4 --await-for 45.0' 2024-05-14T16:56:31.612 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 136.4 --await-for 45.0 2024-05-14T16:56:31.980 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:32.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:31 smithi092 ceph-mon[35763]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 11 MiB/s wr, 908 op/s 2024-05-14T16:56:32.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:31 smithi069 ceph-mon[31507]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 11 MiB/s wr, 908 op/s 2024-05-14T16:56:32.468 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:31 smithi203 ceph-mon[35787]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 11 MiB/s wr, 908 op/s 2024-05-14T16:56:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.370+0000 7f80db845640 1 -- 172.21.15.69:0/655597833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d4109c60 msgr2=0x7f80d410dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.370+0000 7f80db845640 1 --2- 172.21.15.69:0/655597833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d4109c60 0x7f80d410dca0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009900 tx=0x7f80c402a410 comp rx=0 tx=0).stop 2024-05-14T16:56:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.370+0000 7f80db845640 1 -- 172.21.15.69:0/655597833 shutdown_connections 2024-05-14T16:56:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.370+0000 7f80db845640 1 --2- 172.21.15.69:0/655597833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80d410e380 0x7f80d4115f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.370+0000 7f80db845640 1 --2- 172.21.15.69:0/655597833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d4109c60 0x7f80d410dca0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.370+0000 7f80db845640 1 --2- 172.21.15.69:0/655597833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d41092b0 0x7f80d4109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.370+0000 7f80db845640 1 -- 172.21.15.69:0/655597833 >> 172.21.15.69:0/655597833 conn(0x7f80d4076fe0 msgr2=0x7f80d4079400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.371+0000 7f80db845640 1 -- 172.21.15.69:0/655597833 shutdown_connections 2024-05-14T16:56:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.371+0000 7f80db845640 1 -- 172.21.15.69:0/655597833 wait complete. 2024-05-14T16:56:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.371+0000 7f80db845640 1 Processor -- start 2024-05-14T16:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.371+0000 7f80db845640 1 -- start start 2024-05-14T16:56:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.371+0000 7f80db845640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d41092b0 0x7f80d41a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80db845640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d410e380 0x7f80d41a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80db845640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80d419f6f0 0x7f80d419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80db845640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4119310 con 0x7f80d41092b0 2024-05-14T16:56:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80db845640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4119190 con 0x7f80d419f6f0 2024-05-14T16:56:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80db845640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4119490 con 0x7f80d410e380 2024-05-14T16:56:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d95e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d41092b0 0x7f80d41a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d95e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d41092b0 0x7f80d41a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42296/0 (socket says 172.21.15.69:42296) 2024-05-14T16:56:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d95e1640 1 -- 172.21.15.69:0/2745079742 learned_addr learned my addr 172.21.15.69:0/2745079742 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d9de2640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80d419f6f0 0x7f80d419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d8de0640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d410e380 0x7f80d41a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d95e1640 1 -- 172.21.15.69:0/2745079742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d410e380 msgr2=0x7f80d41a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d95e1640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d410e380 0x7f80d41a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d95e1640 1 -- 172.21.15.69:0/2745079742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80d419f6f0 msgr2=0x7f80d419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d95e1640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80d419f6f0 0x7f80d419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d95e1640 1 -- 172.21.15.69:0/2745079742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d41a02f0 con 0x7f80d41092b0 2024-05-14T16:56:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d9de2640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80d419f6f0 0x7f80d419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d8de0640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d410e380 0x7f80d41a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80d95e1640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d41092b0 0x7f80d41a55e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f80d000e9f0 tx=0x7f80d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.372+0000 7f80ca7fc640 1 -- 172.21.15.69:0/2745079742 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d000cd80 con 0x7f80d41092b0 2024-05-14T16:56:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.373+0000 7f80db845640 1 -- 172.21.15.69:0/2745079742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d41ac380 con 0x7f80d41092b0 2024-05-14T16:56:33.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.373+0000 7f80ca7fc640 1 -- 172.21.15.69:0/2745079742 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80d00054d0 con 0x7f80d41092b0 2024-05-14T16:56:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.373+0000 7f80ca7fc640 1 -- 172.21.15.69:0/2745079742 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0010690 con 0x7f80d41092b0 2024-05-14T16:56:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.373+0000 7f80db845640 1 -- 172.21.15.69:0/2745079742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d41ac730 con 0x7f80d41092b0 2024-05-14T16:56:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80de637740 1 Processor -- start 2024-05-14T16:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80de637740 1 -- start start 2024-05-14T16:56:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80de637740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b58648cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80de637740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b58653590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80de637740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 0x559b58655ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80de637740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559b585bbc30 con 0x559b586488f0 2024-05-14T16:56:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80de637740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559b585ab7d0 con 0x559b58649210 2024-05-14T16:56:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80de637740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559b585af810 con 0x559b58653ad0 2024-05-14T16:56:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80d9de2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b58648cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80d9de2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b58648cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42310/0 (socket says 172.21.15.69:42310) 2024-05-14T16:56:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.377+0000 7f80d9de2640 1 -- 172.21.15.69:0/810253369 learned_addr learned my addr 172.21.15.69:0/810253369 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.378+0000 7f80d8de0640 1 --2- 172.21.15.69:0/810253369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 0x559b58655ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.378+0000 7f80d9de2640 1 -- 172.21.15.69:0/810253369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 msgr2=0x559b58655ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.378+0000 7f80d9de2640 1 --2- 172.21.15.69:0/810253369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 0x559b58655ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.378+0000 7f80d9de2640 1 -- 172.21.15.69:0/810253369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 msgr2=0x559b58653590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:56:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.378+0000 7f80d9de2640 1 --2- 172.21.15.69:0/810253369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b58653590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.378+0000 7f80d9de2640 1 -- 172.21.15.69:0/810253369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b582c0210 con 0x559b586488f0 2024-05-14T16:56:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.378+0000 7f80d9de2640 1 --2- 172.21.15.69:0/810253369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b58648cd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f80cc011790 tx=0x7f80cc011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83add0b2024361d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.379+0000 7f80ca7fc640 1 -- 172.21.15.69:0/2745079742 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f80d0004e90 con 0x7f80d41092b0 2024-05-14T16:56:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.379+0000 7f80bf7fe640 1 -- 172.21.15.69:0/810253369 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc003c30 con 0x559b586488f0 2024-05-14T16:56:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.379+0000 7f80bf7fe640 1 -- 172.21.15.69:0/810253369 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80cc00fe50 con 0x559b586488f0 2024-05-14T16:56:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.379+0000 7f80bf7fe640 1 -- 172.21.15.69:0/810253369 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0193e0 con 0x559b586488f0 2024-05-14T16:56:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80ca7fc640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80c00b2a10 0x7f80c00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80de637740 1 -- 172.21.15.69:0/810253369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 msgr2=0x559b58648cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80de637740 1 --2- 172.21.15.69:0/810253369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b58648cd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f80cc011790 tx=0x7f80cc011c50 comp rx=0 tx=0).stop 2024-05-14T16:56:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80de637740 1 -- 172.21.15.69:0/810253369 shutdown_connections 2024-05-14T16:56:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80de637740 1 --2- 172.21.15.69:0/810253369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 0x559b58655ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80de637740 1 --2- 172.21.15.69:0/810253369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b58653590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80de637740 1 --2- 172.21.15.69:0/810253369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b58648cd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80de637740 1 -- 172.21.15.69:0/810253369 >> 172.21.15.69:0/810253369 conn(0x559b58644e00 msgr2=0x559b586451e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80de637740 1 -- 172.21.15.69:0/810253369 shutdown_connections 2024-05-14T16:56:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80ca7fc640 1 -- 172.21.15.69:0/2745079742 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f80d00e2a80 con 0x7f80d41092b0 2024-05-14T16:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80d95e1640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80c00b2a10 0x7f80c00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.380+0000 7f80de637740 1 -- 172.21.15.69:0/810253369 wait complete. 2024-05-14T16:56:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.381+0000 7f80de637740 1 -- start start 2024-05-14T16:56:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.381+0000 7f80d95e1640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80c00b2a10 0x7f80c00b4ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x559b58649e90 tx=0x7f80d0009310 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.381+0000 7f80de637740 1 Processor -- start 2024-05-14T16:56:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.381+0000 7f80de637740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b586488f0 0x559b58653470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.381+0000 7f80de637740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b58650760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.381+0000 7f80de637740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b58653ad0 0x559b58650d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.381+0000 7f80de637740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559b58291790 con 0x559b58653ad0 2024-05-14T16:56:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.381+0000 7f80de637740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559b58567890 con 0x559b58649210 2024-05-14T16:56:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.381+0000 7f80de637740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559b586567a0 con 0x559b586488f0 2024-05-14T16:56:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.381+0000 7f80de637740 10 client.0 _ll_register_callbacks cb 0x559b5856ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d9de2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b58653ad0 0x559b58650d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d9de2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b58653ad0 0x559b58650d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42324/0 (socket says 172.21.15.69:42324) 2024-05-14T16:56:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d8de0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b58650760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d9de2640 1 -- 172.21.15.69:0/830652963 learned_addr learned my addr 172.21.15.69:0/830652963 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d9de2640 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b586488f0 0x559b58653470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d8de0640 1 -- 172.21.15.69:0/830652963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b586488f0 msgr2=0x559b58653470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d8de0640 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b586488f0 0x559b58653470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d8de0640 1 -- 172.21.15.69:0/830652963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b58653ad0 msgr2=0x559b58650d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d8de0640 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b58653ad0 0x559b58650d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d8de0640 1 -- 172.21.15.69:0/830652963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b5838c620 con 0x559b58649210 2024-05-14T16:56:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80d8de0640 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b58650760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f80c402a930 tx=0x7f80c4005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b9032c081a8f422 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:56:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80bcff9640 1 -- 172.21.15.69:0/830652963 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4035040 con 0x559b58649210 2024-05-14T16:56:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80bcff9640 1 -- 172.21.15.69:0/830652963 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f80c4004970 con 0x559b58649210 2024-05-14T16:56:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.382+0000 7f80de637740 1 -- 172.21.15.69:0/830652963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80cc00fe50 con 0x559b58649210 2024-05-14T16:56:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.383+0000 7f80bcff9640 1 -- 172.21.15.69:0/830652963 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f80c404d050 con 0x559b58649210 2024-05-14T16:56:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.383+0000 7f80de637740 10 client.25586 fetch_fsmap learned FSMap version 36 2024-05-14T16:56:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.383+0000 7f80de637740 1 -- 172.21.15.69:0/830652963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559b58662200 con 0x559b58649210 2024-05-14T16:56:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.383+0000 7f80bcff9640 1 -- 172.21.15.69:0/830652963 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80cc00fe50 con 0x559b58649210 2024-05-14T16:56:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.384+0000 7f80bcff9640 1 -- 172.21.15.69:0/830652963 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f80c4044070 con 0x559b58649210 2024-05-14T16:56:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.384+0000 7f80de637740 10 client.25586 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:56:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.384+0000 7f80de637740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.384+0000 7f80de637740 20 client.25586 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.384+0000 7f80de637740 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b58663760 0x559b58665b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.384+0000 7f80de637740 4 client.25586 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.384+0000 7f80de637740 1 -- 172.21.15.69:0/830652963 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b585bdb80 con 0x559b58663760 2024-05-14T16:56:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.387+0000 7f80d9de2640 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b58663760 0x559b58665b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.393+0000 7f80d9de2640 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b58663760 0x559b58665b50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f83696c5afcb69 server_cookie=26a1341a28430870 in_seq=0 out_seq=0 2024-05-14T16:56:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.393+0000 7f80bcff9640 10 client.25586 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.396+0000 7f80bcff9640 1 -- 172.21.15.69:0/830652963 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b585bdb80 con 0x559b58663760 2024-05-14T16:56:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.396+0000 7f80bcff9640 10 client.25586 handle_command_reply: tid=0 2024-05-14T16:56:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.397+0000 7f80de637740 1 client.25586 shutdown 2024-05-14T16:56:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.397+0000 7f80de637740 1 -- 172.21.15.69:0/830652963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 msgr2=0x559b58650760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.397+0000 7f80de637740 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b58650760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f80c402a930 tx=0x7f80c4005270 comp rx=0 tx=0).stop 2024-05-14T16:56:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.399+0000 7f80de637740 1 -- 172.21.15.69:0/830652963 shutdown_connections 2024-05-14T16:56:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.399+0000 7f80de637740 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b58663760 0x559b58665b50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.400+0000 7f80bcff9640 0 client.25586 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.400+0000 7f80de637740 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b58653ad0 0x559b58650d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.400+0000 7f80de637740 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b58650760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.400+0000 7f80de637740 1 --2- 172.21.15.69:0/830652963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b586488f0 0x559b58653470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.400+0000 7f80de637740 1 -- 172.21.15.69:0/830652963 >> 172.21.15.69:0/830652963 conn(0x559b58644e00 msgr2=0x559b58644100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.400+0000 7f80de637740 1 -- 172.21.15.69:0/830652963 shutdown_connections 2024-05-14T16:56:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.400+0000 7f80de637740 1 -- 172.21.15.69:0/830652963 wait complete. 2024-05-14T16:56:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.401+0000 7f80de637740 20 client.25586 trim_cache size 0 max 16384 2024-05-14T16:56:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.433+0000 7f80de637740 1 Processor -- start 2024-05-14T16:56:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.433+0000 7f80de637740 1 -- start start 2024-05-14T16:56:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.434+0000 7f80de637740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b5865f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.434+0000 7f80de637740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58649210 0x559b58652fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.434+0000 7f80de637740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58653ad0 0x559b5866b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.434+0000 7f80d9de2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b5865f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.434+0000 7f80de637740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b7660 con 0x559b586488f0 2024-05-14T16:56:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.434+0000 7f80d9de2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b5865f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42334/0 (socket says 172.21.15.69:42334) 2024-05-14T16:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.434+0000 7f80d9de2640 1 -- 172.21.15.69:0/2590385149 learned_addr learned my addr 172.21.15.69:0/2590385149 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.434+0000 7f80de637740 1 -- 172.21.15.69:0/2590385149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b4ed0 con 0x559b58653ad0 2024-05-14T16:56:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.434+0000 7f80de637740 1 -- 172.21.15.69:0/2590385149 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b2d40 con 0x559b58649210 2024-05-14T16:56:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.435+0000 7f80d9de2640 1 --2- 172.21.15.69:0/2590385149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58653ad0 0x559b5866b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.435+0000 7f80d9de2640 1 -- 172.21.15.69:0/2590385149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58649210 msgr2=0x559b58652fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:56:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.435+0000 7f80d9de2640 1 --2- 172.21.15.69:0/2590385149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58649210 0x559b58652fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.435+0000 7f80d9de2640 1 -- 172.21.15.69:0/2590385149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 msgr2=0x559b5865f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.435+0000 7f80d9de2640 1 --2- 172.21.15.69:0/2590385149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b5865f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.435+0000 7f80d9de2640 1 -- 172.21.15.69:0/2590385149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b58651070 con 0x559b58653ad0 2024-05-14T16:56:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.436+0000 7f80d9de2640 1 --2- 172.21.15.69:0/2590385149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58653ad0 0x559b5866b440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80cc011790 tx=0x7f80cc005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf83f0d6809958d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.436+0000 7f80beffd640 1 -- 172.21.15.69:0/2590385149 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00fc30 con 0x559b58653ad0 2024-05-14T16:56:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.436+0000 7f80beffd640 1 -- 172.21.15.69:0/2590385149 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80cc00fdd0 con 0x559b58653ad0 2024-05-14T16:56:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.436+0000 7f80beffd640 1 -- 172.21.15.69:0/2590385149 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc01a510 con 0x559b58653ad0 2024-05-14T16:56:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.437+0000 7f80de637740 1 -- 172.21.15.69:0/2590385149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58653ad0 msgr2=0x559b5866b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.437+0000 7f80de637740 1 --2- 172.21.15.69:0/2590385149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58653ad0 0x559b5866b440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80cc011790 tx=0x7f80cc005bd0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.438+0000 7f80de637740 1 -- 172.21.15.69:0/2590385149 shutdown_connections 2024-05-14T16:56:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.438+0000 7f80de637740 1 --2- 172.21.15.69:0/2590385149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58653ad0 0x559b5866b440 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.438+0000 7f80de637740 1 --2- 172.21.15.69:0/2590385149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58649210 0x559b58652fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.438+0000 7f80de637740 1 --2- 172.21.15.69:0/2590385149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b5865f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.438+0000 7f80de637740 1 -- 172.21.15.69:0/2590385149 >> 172.21.15.69:0/2590385149 conn(0x559b58644e00 msgr2=0x559b5864bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.438+0000 7f80de637740 1 -- 172.21.15.69:0/2590385149 shutdown_connections 2024-05-14T16:56:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.438+0000 7f80de637740 1 -- 172.21.15.69:0/2590385149 wait complete. 2024-05-14T16:56:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.438+0000 7f80de637740 1 -- start start 2024-05-14T16:56:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.438+0000 7f80de637740 1 Processor -- start 2024-05-14T16:56:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80de637740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b586c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80de637740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b586c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80de637740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 0x559b58657d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80de637740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b3390 con 0x559b586488f0 2024-05-14T16:56:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80de637740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b4800 con 0x559b58649210 2024-05-14T16:56:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80de637740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b3a60 con 0x559b58653ad0 2024-05-14T16:56:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80d9de2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b586c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80d9de2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b586c7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42350/0 (socket says 172.21.15.69:42350) 2024-05-14T16:56:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80d9de2640 1 -- 172.21.15.69:0/1008537601 learned_addr learned my addr 172.21.15.69:0/1008537601 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80de637740 10 client.0 _ll_register_callbacks cb 0x7f80c40043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80d8de0640 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b586c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.439+0000 7f80d9de2640 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 0x559b58657d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.440+0000 7f80d8de0640 1 -- 172.21.15.69:0/1008537601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 msgr2=0x559b58657d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.440+0000 7f80d8de0640 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 0x559b58657d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.440+0000 7f80d8de0640 1 -- 172.21.15.69:0/1008537601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 msgr2=0x559b586c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.440+0000 7f80d8de0640 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b586c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.440+0000 7f80d8de0640 1 -- 172.21.15.69:0/1008537601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b58658480 con 0x559b58649210 2024-05-14T16:56:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.440+0000 7f80d9de2640 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 0x559b58657d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.441+0000 7f80d8de0640 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b586c78b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x559b58651c10 tx=0x7f80c40044a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7381df6458dcbf47 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.441+0000 7f80bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:56:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.442+0000 7f80bcff9640 1 -- 172.21.15.69:0/1008537601 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c404a070 con 0x559b58649210 2024-05-14T16:56:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.442+0000 7f80bcff9640 1 -- 172.21.15.69:0/1008537601 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f80c4003a00 con 0x559b58649210 2024-05-14T16:56:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.442+0000 7f80de637740 1 -- 172.21.15.69:0/1008537601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80cc00fdd0 con 0x559b58649210 2024-05-14T16:56:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.442+0000 7f80bcff9640 1 -- 172.21.15.69:0/1008537601 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f80c4059050 con 0x559b58649210 2024-05-14T16:56:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.442+0000 7f80de637740 10 client.25592 fetch_fsmap learned FSMap version 36 2024-05-14T16:56:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.442+0000 7f80de637740 1 -- 172.21.15.69:0/1008537601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559b586ceab0 con 0x559b58649210 2024-05-14T16:56:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.442+0000 7f80bcff9640 1 -- 172.21.15.69:0/1008537601 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80cc00fdd0 con 0x559b58649210 2024-05-14T16:56:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.442+0000 7f80bcff9640 1 -- 172.21.15.69:0/1008537601 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f80c4044040 con 0x559b58649210 2024-05-14T16:56:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.443+0000 7f80de637740 10 client.25592 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:56:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.443+0000 7f80de637740 10 client.25592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.443+0000 7f80de637740 20 client.25592 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.443+0000 7f80de637740 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b58663760 0x559b586d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.443+0000 7f80de637740 4 client.25592 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:56:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.443+0000 7f80de637740 1 -- 172.21.15.69:0/1008537601 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b585b7360 con 0x559b58663760 2024-05-14T16:56:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.443+0000 7f80d9de2640 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b58663760 0x559b586d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.443+0000 7f80bcff9640 10 client.25592 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.443+0000 7f80d9de2640 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b58663760 0x559b586d1240 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8782a8dd085c175 server_cookie=ca24a493aed26cf9 in_seq=0 out_seq=0 2024-05-14T16:56:35.374 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:33 smithi203 ceph-mon[35787]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 11 MiB/s wr, 887 op/s 2024-05-14T16:56:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:33 smithi092 ceph-mon[35763]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 11 MiB/s wr, 887 op/s 2024-05-14T16:56:35.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:34 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80bcff9640 1 -- 172.21.15.69:0/1008537601 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559b585b7360 con 0x559b58663760 2024-05-14T16:56:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80bcff9640 10 client.25592 handle_command_reply: tid=0 2024-05-14T16:56:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 client.25592 shutdown 2024-05-14T16:56:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 -- 172.21.15.69:0/1008537601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 msgr2=0x559b586c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b586c78b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x559b58651c10 tx=0x7f80c40044a0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 -- 172.21.15.69:0/1008537601 shutdown_connections 2024-05-14T16:56:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x559b58663760 0x559b586d1240 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b58653ad0 0x559b58657d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b58649210 0x559b586c78b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 --2- 172.21.15.69:0/1008537601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b586488f0 0x559b586c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 -- 172.21.15.69:0/1008537601 >> 172.21.15.69:0/1008537601 conn(0x559b58644e00 msgr2=0x559b5864b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80bcff9640 0 client.25592 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 -- 172.21.15.69:0/1008537601 shutdown_connections 2024-05-14T16:56:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.459+0000 7f80de637740 1 -- 172.21.15.69:0/1008537601 wait complete. 2024-05-14T16:56:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.460+0000 7f80de637740 20 client.25592 trim_cache size 0 max 16384 2024-05-14T16:56:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:56:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:56:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:56:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.461+0000 7f80db845640 1 -- 172.21.15.69:0/2745079742 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80c00b2a10 msgr2=0x7f80c00b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.461+0000 7f80db845640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80c00b2a10 0x7f80c00b4ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x559b58649e90 tx=0x7f80d0009310 comp rx=0 tx=0).stop 2024-05-14T16:56:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.461+0000 7f80db845640 1 -- 172.21.15.69:0/2745079742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d41092b0 msgr2=0x7f80d41a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.461+0000 7f80db845640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d41092b0 0x7f80d41a55e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f80d000e9f0 tx=0x7f80d000edf0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.462+0000 7f80db845640 1 -- 172.21.15.69:0/2745079742 shutdown_connections 2024-05-14T16:56:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.462+0000 7f80db845640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80c00b2a10 0x7f80c00b4ed0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.462+0000 7f80db845640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80d419f6f0 0x7f80d419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.462+0000 7f80db845640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d410e380 0x7f80d41a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.462+0000 7f80db845640 1 --2- 172.21.15.69:0/2745079742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d41092b0 0x7f80d41a55e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.462+0000 7f80db845640 1 -- 172.21.15.69:0/2745079742 >> 172.21.15.69:0/2745079742 conn(0x7f80d4076fe0 msgr2=0x7f80d4079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.462+0000 7f80db845640 1 -- 172.21.15.69:0/2745079742 shutdown_connections 2024-05-14T16:56:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:33.462+0000 7f80db845640 1 -- 172.21.15.69:0/2745079742 wait complete. 2024-05-14T16:56:35.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:33 smithi069 ceph-mon[31507]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 11 MiB/s wr, 887 op/s 2024-05-14T16:56:35.800 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:34 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:35.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.994+0000 7f3d462ff700 1 -- 172.21.15.69:0/69636965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d40102440 msgr2=0x7f3d401028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:35.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.994+0000 7f3d462ff700 1 --2- 172.21.15.69:0/69636965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d40102440 0x7f3d401028c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3d340098f0 tx=0x7f3d34009c00 comp rx=0 tx=0).stop 2024-05-14T16:56:35.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.994+0000 7f3d462ff700 1 -- 172.21.15.69:0/69636965 shutdown_connections 2024-05-14T16:56:35.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.994+0000 7f3d462ff700 1 --2- 172.21.15.69:0/69636965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d401084c0 0x7f3d401088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.994+0000 7f3d462ff700 1 --2- 172.21.15.69:0/69636965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d40102e00 0x7f3d4010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.994+0000 7f3d462ff700 1 --2- 172.21.15.69:0/69636965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d40102440 0x7f3d401028c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:35.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.994+0000 7f3d462ff700 1 -- 172.21.15.69:0/69636965 >> 172.21.15.69:0/69636965 conn(0x7f3d400793e0 msgr2=0x7f3d400797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:35.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.994+0000 7f3d462ff700 1 -- 172.21.15.69:0/69636965 shutdown_connections 2024-05-14T16:56:35.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.994+0000 7f3d462ff700 1 -- 172.21.15.69:0/69636965 wait complete. 2024-05-14T16:56:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.995+0000 7f3d462ff700 1 Processor -- start 2024-05-14T16:56:35.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.996+0000 7f3d462ff700 1 -- start start 2024-05-14T16:56:35.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d462ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d40102440 0x7f3d4019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:35.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d462ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d40102e00 0x7f3d4019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:35.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d462ff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d401084c0 0x7f3d40198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:35.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d462ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3d40112630 con 0x7f3d40102440 2024-05-14T16:56:35.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d462ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401124b0 con 0x7f3d40102e00 2024-05-14T16:56:35.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d462ff700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401127b0 con 0x7f3d401084c0 2024-05-14T16:56:35.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d40102440 0x7f3d4019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:35.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d40102440 0x7f3d4019e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50570/0 (socket says 172.21.15.69:50570) 2024-05-14T16:56:35.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d3ffff700 1 -- 172.21.15.69:0/1234934857 learned_addr learned my addr 172.21.15.69:0/1234934857 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d40102e00 0x7f3d4019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:35.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.997+0000 7f3d4489c700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d401084c0 0x7f3d40198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:35.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.998+0000 7f3d3ffff700 1 -- 172.21.15.69:0/1234934857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d401084c0 msgr2=0x7f3d40198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:36.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.998+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d401084c0 0x7f3d40198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:36.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.998+0000 7f3d3ffff700 1 -- 172.21.15.69:0/1234934857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d40102e00 msgr2=0x7f3d4019ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:36.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.998+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d40102e00 0x7f3d4019ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:36.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.998+0000 7f3d3ffff700 1 -- 172.21.15.69:0/1234934857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d40199090 con 0x7f3d40102440 2024-05-14T16:56:36.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.998+0000 7f3d4489c700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d401084c0 0x7f3d40198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:36.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.998+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d40102440 0x7f3d4019e770 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000e9a0 tx=0x7f3d3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:36.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.999+0000 7f3d3d7fa700 1 -- 172.21.15.69:0/1234934857 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3000cbd0 con 0x7f3d40102440 2024-05-14T16:56:36.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.999+0000 7f3d3d7fa700 1 -- 172.21.15.69:0/1234934857 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3000cd70 con 0x7f3d40102440 2024-05-14T16:56:36.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.999+0000 7f3d3d7fa700 1 -- 172.21.15.69:0/1234934857 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30018680 con 0x7f3d40102440 2024-05-14T16:56:36.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:34.999+0000 7f3d462ff700 1 -- 172.21.15.69:0/1234934857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d40199380 con 0x7f3d40102440 2024-05-14T16:56:36.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.000+0000 7f3d462ff700 1 -- 172.21.15.69:0/1234934857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d401a5640 con 0x7f3d40102440 2024-05-14T16:56:36.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.004+0000 7f3d3d7fa700 1 -- 172.21.15.69:0/1234934857 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3d30010430 con 0x7f3d40102440 2024-05-14T16:56:36.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.006+0000 7f3d3d7fa700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3d280b2be0 0x7f3d280b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:36.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.006+0000 7f3d50b95b80 1 Processor -- start 2024-05-14T16:56:36.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.006+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3d280b2be0 0x7f3d280b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:36.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.006+0000 7f3d3d7fa700 1 -- 172.21.15.69:0/1234934857 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f3d300e2750 con 0x7f3d40102440 2024-05-14T16:56:36.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d50b95b80 1 -- start start 2024-05-14T16:56:36.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3d280b2be0 0x7f3d280b50a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3d340098c0 tx=0x7f3d34019040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:36.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d74200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:36.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 0x5614c1d729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:36.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d82bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:36.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5614c1cd0e20 con 0x5614c1d72fe0 2024-05-14T16:56:36.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5614c1cc09c0 con 0x5614c1d72540 2024-05-14T16:56:36.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5614c1cc4a00 con 0x5614c1d73e20 2024-05-14T16:56:36.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d4489c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d82bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:36.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d74200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:36.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d74200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:36710/0 (socket says 172.21.15.69:36710) 2024-05-14T16:56:36.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3ffff700 1 -- 172.21.15.69:0/838269331 learned_addr learned my addr 172.21.15.69:0/838269331 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/838269331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 0x5614c1d729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:36.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3ffff700 1 -- 172.21.15.69:0/838269331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 msgr2=0x5614c1d729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:36.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/838269331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 0x5614c1d729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:36.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3ffff700 1 -- 172.21.15.69:0/838269331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 msgr2=0x5614c1d82bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:36.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/838269331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d82bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:36.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3ffff700 1 -- 172.21.15.69:0/838269331 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614c1c10a60 con 0x5614c1d73e20 2024-05-14T16:56:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/838269331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 0x5614c1d729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:36.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.007+0000 7f3d4489c700 1 --2- 172.21.15.69:0/838269331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d82bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:36.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.008+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/838269331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d74200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3d300a99b0 tx=0x7f3d300a5230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7949ea4be276642 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:36.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.008+0000 7f3d267fc700 1 -- 172.21.15.69:0/838269331 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d300b21d0 con 0x5614c1d73e20 2024-05-14T16:56:36.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.008+0000 7f3d267fc700 1 -- 172.21.15.69:0/838269331 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d300aa120 con 0x5614c1d73e20 2024-05-14T16:56:36.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.008+0000 7f3d267fc700 1 -- 172.21.15.69:0/838269331 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30010720 con 0x5614c1d73e20 2024-05-14T16:56:36.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.009+0000 7f3d50b95b80 1 -- 172.21.15.69:0/838269331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 msgr2=0x5614c1d74200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.009+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/838269331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d74200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3d300a99b0 tx=0x7f3d300a5230 comp rx=0 tx=0).stop 2024-05-14T16:56:36.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.011+0000 7f3d50b95b80 1 -- 172.21.15.69:0/838269331 shutdown_connections 2024-05-14T16:56:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.011+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/838269331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d82bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:36.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.011+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/838269331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d74200 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:36.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.011+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/838269331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 0x5614c1d729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:36.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.011+0000 7f3d50b95b80 1 -- 172.21.15.69:0/838269331 >> 172.21.15.69:0/838269331 conn(0x5614c1c9d3e0 msgr2=0x5614c1d6cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:36.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.011+0000 7f3d50b95b80 1 -- 172.21.15.69:0/838269331 shutdown_connections 2024-05-14T16:56:36.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.011+0000 7f3d50b95b80 1 -- 172.21.15.69:0/838269331 wait complete. 2024-05-14T16:56:36.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.011+0000 7f3d50b95b80 1 -- start start 2024-05-14T16:56:36.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.012+0000 7f3d50b95b80 1 Processor -- start 2024-05-14T16:56:36.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.012+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d72540 0x5614c1d828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:36.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.012+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d7fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:36.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.012+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d73e20 0x5614c1d80190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:36.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.012+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5614c1bc7130 con 0x5614c1d72fe0 2024-05-14T16:56:36.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.012+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5614c1c62950 con 0x5614c1d73e20 2024-05-14T16:56:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.012+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5614c1d83570 con 0x5614c1d72540 2024-05-14T16:56:36.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d50b95b80 10 client.0 _ll_register_callbacks cb 0x5614c1c57710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:36.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d72540 0x5614c1d828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:36.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d72540 0x5614c1d828b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:36722/0 (socket says 172.21.15.69:36722) 2024-05-14T16:56:36.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3ffff700 1 -- 172.21.15.69:0/340254491 learned_addr learned my addr 172.21.15.69:0/340254491 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d4489c700 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d73e20 0x5614c1d80190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:36.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d7fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3ffff700 1 -- 172.21.15.69:0/340254491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d73e20 msgr2=0x5614c1d80190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d73e20 0x5614c1d80190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3ffff700 1 -- 172.21.15.69:0/340254491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 msgr2=0x5614c1d7fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d7fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3ffff700 1 -- 172.21.15.69:0/340254491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614c1c5f820 con 0x5614c1d72540 2024-05-14T16:56:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d7fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d72540 0x5614c1d828b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d300a96d0 tx=0x7f3d3000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8e47e34bea4131b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:36.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.013+0000 7f3d1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:56:36.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.014+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/340254491 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d300a9a20 con 0x5614c1d72540 2024-05-14T16:56:36.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.014+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/340254491 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f3d30010930 con 0x5614c1d72540 2024-05-14T16:56:36.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.014+0000 7f3d50b95b80 1 -- 172.21.15.69:0/340254491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d300aa120 con 0x5614c1d72540 2024-05-14T16:56:36.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.014+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/340254491 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3d300addd0 con 0x5614c1d72540 2024-05-14T16:56:36.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.014+0000 7f3d50b95b80 10 client.25537 fetch_fsmap learned FSMap version 37 2024-05-14T16:56:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.014+0000 7f3d50b95b80 1 -- 172.21.15.69:0/340254491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5614c1d8dcc0 con 0x5614c1d72540 2024-05-14T16:56:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.014+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/340254491 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d300b2880 con 0x5614c1d72540 2024-05-14T16:56:36.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.014+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/340254491 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f3d300a5590 con 0x5614c1d72540 2024-05-14T16:56:36.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.015+0000 7f3d50b95b80 10 client.25537 fetch_fsmap finished waiting for FSMap version 37 2024-05-14T16:56:36.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.015+0000 7f3d50b95b80 10 client.25537 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:56:36.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.015+0000 7f3d50b95b80 20 client.25537 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:36.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.015+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5614c1d90190 0x5614c1d92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:36.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.015+0000 7f3d50b95b80 4 client.25537 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:36.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.015+0000 7f3d50b95b80 1 -- 172.21.15.69:0/340254491 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3d300aa120 con 0x5614c1d90190 2024-05-14T16:56:36.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.015+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5614c1d90190 0x5614c1d92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:36.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.015+0000 7f3d1f7fe700 10 client.25537 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:36.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.015+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5614c1d90190 0x5614c1d92580 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7590e40d20dd5d server_cookie=6967081030ea38fc in_seq=0 out_seq=0 2024-05-14T16:56:36.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.016+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/340254491 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f3d300aa120 con 0x5614c1d90190 2024-05-14T16:56:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.016+0000 7f3d1f7fe700 10 client.25537 handle_command_reply: tid=0 2024-05-14T16:56:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.016+0000 7f3d50b95b80 1 client.25537 shutdown 2024-05-14T16:56:37.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 1 -- 172.21.15.69:0/340254491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d72540 msgr2=0x5614c1d828b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:37.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d72540 0x5614c1d828b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d300a96d0 tx=0x7f3d3000cf20 comp rx=0 tx=0).stop 2024-05-14T16:56:37.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 1 -- 172.21.15.69:0/340254491 shutdown_connections 2024-05-14T16:56:37.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5614c1d90190 0x5614c1d92580 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d1f7fe700 0 client.25537 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:37.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d73e20 0x5614c1d80190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d72540 0x5614c1d828b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/340254491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d7fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 1 -- 172.21.15.69:0/340254491 >> 172.21.15.69:0/340254491 conn(0x5614c1c9d3e0 msgr2=0x5614c1d67860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:37.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 1 -- 172.21.15.69:0/340254491 shutdown_connections 2024-05-14T16:56:37.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 1 -- 172.21.15.69:0/340254491 wait complete. 2024-05-14T16:56:37.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.017+0000 7f3d50b95b80 20 client.25537 trim_cache size 0 max 16384 2024-05-14T16:56:37.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.047+0000 7f3d50b95b80 1 Processor -- start 2024-05-14T16:56:37.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.047+0000 7f3d50b95b80 1 -- start start 2024-05-14T16:56:37.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.047+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 0x5614c1d67970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:37.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.047+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d7cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:37.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.047+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d7d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:37.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.047+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401ae4f0 con 0x5614c1d72fe0 2024-05-14T16:56:37.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.047+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401abcf0 con 0x5614c1d72540 2024-05-14T16:56:37.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.047+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401a9b60 con 0x5614c1d73e20 2024-05-14T16:56:37.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d4489c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d7d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d4489c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d7d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:36724/0 (socket says 172.21.15.69:36724) 2024-05-14T16:56:37.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d4489c700 1 -- 172.21.15.69:0/3317339786 learned_addr learned my addr 172.21.15.69:0/3317339786 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:37.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/3317339786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 0x5614c1d67970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:37.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/3317339786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d7cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:37.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d3ffff700 1 -- 172.21.15.69:0/3317339786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 msgr2=0x5614c1d7d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:37.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/3317339786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d7d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d3ffff700 1 -- 172.21.15.69:0/3317339786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 msgr2=0x5614c1d7cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:37.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/3317339786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d7cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d3ffff700 1 -- 172.21.15.69:0/3317339786 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614c1d72ce0 con 0x5614c1d72540 2024-05-14T16:56:37.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/3317339786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d7cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:37.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.048+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/3317339786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 0x5614c1d67970 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000c5b0 tx=0x7f3d3000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e4cc01ee881c378 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:37.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.049+0000 7f3d25ffb700 1 -- 172.21.15.69:0/3317339786 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d300a9530 con 0x5614c1d72540 2024-05-14T16:56:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.049+0000 7f3d25ffb700 1 -- 172.21.15.69:0/3317339786 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d300aa120 con 0x5614c1d72540 2024-05-14T16:56:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.049+0000 7f3d25ffb700 1 -- 172.21.15.69:0/3317339786 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d300af0f0 con 0x5614c1d72540 2024-05-14T16:56:37.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.050+0000 7f3d50b95b80 1 -- 172.21.15.69:0/3317339786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 msgr2=0x5614c1d67970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:37.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.050+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/3317339786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 0x5614c1d67970 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000c5b0 tx=0x7f3d3000cf90 comp rx=0 tx=0).stop 2024-05-14T16:56:37.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.050+0000 7f3d50b95b80 1 -- 172.21.15.69:0/3317339786 shutdown_connections 2024-05-14T16:56:37.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.050+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/3317339786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d7d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.050+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/3317339786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72540 0x5614c1d67970 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.050+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/3317339786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72fe0 0x5614c1d7cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.050+0000 7f3d50b95b80 1 -- 172.21.15.69:0/3317339786 >> 172.21.15.69:0/3317339786 conn(0x5614c1c9d3e0 msgr2=0x5614c1d7ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:37.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.050+0000 7f3d50b95b80 1 -- 172.21.15.69:0/3317339786 shutdown_connections 2024-05-14T16:56:37.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.050+0000 7f3d50b95b80 1 -- 172.21.15.69:0/3317339786 wait complete. 2024-05-14T16:56:37.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.050+0000 7f3d50b95b80 1 -- start start 2024-05-14T16:56:37.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.051+0000 7f3d50b95b80 1 Processor -- start 2024-05-14T16:56:37.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72540 0x5614c1d90570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:37.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72fe0 0x5614c1d90ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:37.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d50b95b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d90ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:37.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401aa1b0 con 0x5614c1d72540 2024-05-14T16:56:37.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401ab620 con 0x5614c1d72fe0 2024-05-14T16:56:37.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d50b95b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401aa880 con 0x5614c1d73e20 2024-05-14T16:56:37.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d50b95b80 10 client.0 _ll_register_callbacks cb 0x5614c1c57710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:37.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d4489c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d90ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:37.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d3f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72fe0 0x5614c1d90ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:37.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d4489c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d90ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:36730/0 (socket says 172.21.15.69:36730) 2024-05-14T16:56:37.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72540 0x5614c1d90570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:37.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d4489c700 1 -- 172.21.15.69:0/2737782576 learned_addr learned my addr 172.21.15.69:0/2737782576 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:37.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d3f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72fe0 0x5614c1d90ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:56730/0 (socket says 172.21.15.69:56730) 2024-05-14T16:56:37.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d4489c700 1 -- 172.21.15.69:0/2737782576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72fe0 msgr2=0x5614c1d90ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:37.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d4489c700 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72fe0 0x5614c1d90ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d4489c700 1 -- 172.21.15.69:0/2737782576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72540 msgr2=0x5614c1d90570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:37.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d4489c700 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72540 0x5614c1d90570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.052+0000 7f3d4489c700 1 -- 172.21.15.69:0/2737782576 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614c1df66d0 con 0x5614c1d73e20 2024-05-14T16:56:37.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d3f7fe700 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72fe0 0x5614c1d90ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:37.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d4489c700 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d90ff0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007fc0 tx=0x7f3d38009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67cbee705bf7ef16 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:37.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:56:37.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/2737782576 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38003e10 con 0x5614c1d73e20 2024-05-14T16:56:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/2737782576 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f3d38017820 con 0x5614c1d73e20 2024-05-14T16:56:37.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d50b95b80 1 -- 172.21.15.69:0/2737782576 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d300aa120 con 0x5614c1d73e20 2024-05-14T16:56:37.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/2737782576 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3d38017a20 con 0x5614c1d73e20 2024-05-14T16:56:37.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d50b95b80 10 client.25543 fetch_fsmap learned FSMap version 37 2024-05-14T16:56:37.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d50b95b80 1 -- 172.21.15.69:0/2737782576 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5614c1df7a10 con 0x5614c1d73e20 2024-05-14T16:56:37.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/2737782576 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d300aa120 con 0x5614c1d73e20 2024-05-14T16:56:37.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.053+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/2737782576 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f3d38021470 con 0x5614c1d73e20 2024-05-14T16:56:37.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.054+0000 7f3d50b95b80 10 client.25543 fetch_fsmap finished waiting for FSMap version 37 2024-05-14T16:56:37.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.054+0000 7f3d50b95b80 10 client.25543 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:56:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.054+0000 7f3d50b95b80 20 client.25543 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:37.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.054+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5614c1d90190 0x5614c1dfa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:37.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.054+0000 7f3d50b95b80 4 client.25543 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.4, "await_for": 45.0}] 2024-05-14T16:56:37.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.054+0000 7f3d50b95b80 1 -- 172.21.15.69:0/2737782576 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.4, "await_for": 45.0}) -- 0x5614c1d3e4f0 con 0x5614c1d90190 2024-05-14T16:56:37.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.054+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5614c1d90190 0x5614c1dfa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:37.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.054+0000 7f3d1f7fe700 10 client.25543 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:37.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.054+0000 7f3d3ffff700 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5614c1d90190 0x5614c1dfa260 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8ae4c8bd3b7aa8 server_cookie=16a04b5f10e7bf52 in_seq=0 out_seq=0 2024-05-14T16:56:37.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d1f7fe700 1 -- 172.21.15.69:0/2737782576 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5614c1d3e4f0 con 0x5614c1d90190 2024-05-14T16:56:37.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d1f7fe700 10 client.25543 handle_command_reply: tid=0 2024-05-14T16:56:37.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 client.25543 shutdown 2024-05-14T16:56:37.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:34 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:37.819 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:35 smithi203 ceph-mon[35787]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 8.9 MiB/s wr, 736 op/s 2024-05-14T16:56:37.833 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:35 smithi092 ceph-mon[35763]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 8.9 MiB/s wr, 736 op/s 2024-05-14T16:56:37.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 -- 172.21.15.69:0/2737782576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 msgr2=0x5614c1d90ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:37.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d90ff0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007fc0 tx=0x7f3d38009300 comp rx=0 tx=0).stop 2024-05-14T16:56:37.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 -- 172.21.15.69:0/2737782576 shutdown_connections 2024-05-14T16:56:37.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5614c1d90190 0x5614c1dfa260 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614c1d72540 0x5614c1d90570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d1f7fe700 0 client.25543 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:37.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614c1d73e20 0x5614c1d90ff0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 --2- 172.21.15.69:0/2737782576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614c1d72fe0 0x5614c1d90ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 -- 172.21.15.69:0/2737782576 >> 172.21.15.69:0/2737782576 conn(0x5614c1c9d3e0 msgr2=0x5614c1d8b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:37.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 -- 172.21.15.69:0/2737782576 shutdown_connections 2024-05-14T16:56:37.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 1 -- 172.21.15.69:0/2737782576 wait complete. 2024-05-14T16:56:37.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d50b95b80 20 client.25543 trim_cache size 0 max 16384 2024-05-14T16:56:37.975 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:56:37.991 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 37, 2024-05-14T16:56:37.997 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:56:38.002 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 22, 2024-05-14T16:56:38.007 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:56:38.014 INFO:teuthology.orchestra.run.smithi069.stdout: "78c5781c": { 2024-05-14T16:56:38.019 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 22, 2024-05-14T16:56:38.024 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:56:38.029 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:56:38.035 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:56:38.044 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:56:38.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:56:38.055 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:56:38.090 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 136.4, 2024-05-14T16:56:38.096 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:56:38.125 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:56:38.131 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:56:38.145 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:56:38.167 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:56:38.173 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:56:38.178 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:38.184 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:38.190 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:38.197 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:38.202 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:38.207 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:56:38.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d462ff700 1 -- 172.21.15.69:0/1234934857 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3d280b2be0 msgr2=0x7f3d280b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:38.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d462ff700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3d280b2be0 0x7f3d280b50a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3d340098c0 tx=0x7f3d34019040 comp rx=0 tx=0).stop 2024-05-14T16:56:38.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d462ff700 1 -- 172.21.15.69:0/1234934857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d40102440 msgr2=0x7f3d4019e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:38.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.302+0000 7f3d462ff700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d40102440 0x7f3d4019e770 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000e9a0 tx=0x7f3d3000ecb0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.303+0000 7f3d462ff700 1 -- 172.21.15.69:0/1234934857 shutdown_connections 2024-05-14T16:56:38.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.303+0000 7f3d462ff700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3d280b2be0 0x7f3d280b50a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.303+0000 7f3d462ff700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d40102440 0x7f3d4019e770 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.303+0000 7f3d462ff700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d401084c0 0x7f3d40198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.303+0000 7f3d462ff700 1 --2- 172.21.15.69:0/1234934857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d40102e00 0x7f3d4019ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.303+0000 7f3d462ff700 1 -- 172.21.15.69:0/1234934857 >> 172.21.15.69:0/1234934857 conn(0x7f3d400793e0 msgr2=0x7f3d40100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:38.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.303+0000 7f3d462ff700 1 -- 172.21.15.69:0/1234934857 shutdown_connections 2024-05-14T16:56:38.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:35.303+0000 7f3d462ff700 1 -- 172.21.15.69:0/1234934857 wait complete. 2024-05-14T16:56:38.373 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 365.0 s, 2527.4 tps, lat 36.933 ms stddev 42.270 2024-05-14T16:56:38.381 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:35 smithi069 ceph-mon[31507]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 8.9 MiB/s wr, 736 op/s 2024-05-14T16:56:38.387 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '78c5781c', quiesce took 11.0 seconds. Will release after: 5.399999999999999 2024-05-14T16:56:38.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:37 smithi092 ceph-mon[35763]: pgmap v208: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-05-14T16:56:38.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:37 smithi069 ceph-mon[31507]: pgmap v208: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-05-14T16:56:38.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:37 smithi203 ceph-mon[35787]: pgmap v208: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-05-14T16:56:40.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:39 smithi092 ceph-mon[35763]: pgmap v209: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.1 MiB/s wr, 609 op/s 2024-05-14T16:56:40.400 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 370.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:56:40.824 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:39 smithi203 ceph-mon[35787]: pgmap v209: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.1 MiB/s wr, 609 op/s 2024-05-14T16:56:40.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:39 smithi069 ceph-mon[31507]: pgmap v209: 97 pgs: 97 active+clean; 3.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.1 MiB/s wr, 609 op/s 2024-05-14T16:56:42.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:41 smithi092 ceph-mon[35763]: pgmap v210: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2024-05-14T16:56:42.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:41 smithi069 ceph-mon[31507]: pgmap v210: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2024-05-14T16:56:42.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:41 smithi203 ceph-mon[35787]: pgmap v210: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2024-05-14T16:56:43.793 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:56:44.056 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:44.172 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:43 smithi069 ceph-mon[31507]: pgmap v211: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 35 MiB/s wr, 1.20k op/s 2024-05-14T16:56:44.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:43 smithi092 ceph-mon[35763]: pgmap v211: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 35 MiB/s wr, 1.20k op/s 2024-05-14T16:56:44.260 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:43 smithi203 ceph-mon[35787]: pgmap v211: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 35 MiB/s wr, 1.20k op/s 2024-05-14T16:56:45.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 375.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:56:45.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.754+0000 7f53026eb700 1 -- 172.21.15.69:0/1779654841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc102140 msgr2=0x7f52fc102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:45.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.754+0000 7f53026eb700 1 --2- 172.21.15.69:0/1779654841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc102140 0x7f52fc102520 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0099c0 tx=0x7f52ec009cd0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.754+0000 7f53026eb700 1 -- 172.21.15.69:0/1779654841 shutdown_connections 2024-05-14T16:56:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.754+0000 7f53026eb700 1 --2- 172.21.15.69:0/1779654841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc102140 0x7f52fc102520 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.754+0000 7f53026eb700 1 --2- 172.21.15.69:0/1779654841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52fc10bd80 0x7f52fc112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.754+0000 7f53026eb700 1 --2- 172.21.15.69:0/1779654841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52fc102af0 0x7f52fc10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.754+0000 7f53026eb700 1 -- 172.21.15.69:0/1779654841 >> 172.21.15.69:0/1779654841 conn(0x7f52fc0777e0 msgr2=0x7f52fc077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:45.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.754+0000 7f53026eb700 1 -- 172.21.15.69:0/1779654841 shutdown_connections 2024-05-14T16:56:45.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.754+0000 7f53026eb700 1 -- 172.21.15.69:0/1779654841 wait complete. 2024-05-14T16:56:45.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f53026eb700 1 Processor -- start 2024-05-14T16:56:45.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f53026eb700 1 -- start start 2024-05-14T16:56:45.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f53026eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52fc102140 0x7f52fc19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:45.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f53026eb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52fc102af0 0x7f52fc19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:45.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f53026eb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc10bd80 0x7f52fc1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:45.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f53026eb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1159b0 con 0x7f52fc10bd80 2024-05-14T16:56:45.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f53026eb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc115830 con 0x7f52fc102140 2024-05-14T16:56:45.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f53026eb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc115b30 con 0x7f52fc102af0 2024-05-14T16:56:45.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f5300c88700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc10bd80 0x7f52fc1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:45.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f5300c88700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc10bd80 0x7f52fc1a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39632/0 (socket says 172.21.15.69:39632) 2024-05-14T16:56:45.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f5300c88700 1 -- 172.21.15.69:0/3992367304 learned_addr learned my addr 172.21.15.69:0/3992367304 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:46.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f52f3fff700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52fc102af0 0x7f52fc19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:46.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f52fbfff700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52fc102140 0x7f52fc19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f5300c88700 1 -- 172.21.15.69:0/3992367304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52fc102af0 msgr2=0x7f52fc19c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f5300c88700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52fc102af0 0x7f52fc19c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f5300c88700 1 -- 172.21.15.69:0/3992367304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52fc102140 msgr2=0x7f52fc19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:46.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f5300c88700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52fc102140 0x7f52fc19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f5300c88700 1 -- 172.21.15.69:0/3992367304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc1a0e70 con 0x7f52fc10bd80 2024-05-14T16:56:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.755+0000 7f5300c88700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc10bd80 0x7f52fc1a0730 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f52f400ada0 tx=0x7f52f400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.756+0000 7f52f9ffb700 1 -- 172.21.15.69:0/3992367304 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4017020 con 0x7f52fc10bd80 2024-05-14T16:56:46.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.756+0000 7f52f9ffb700 1 -- 172.21.15.69:0/3992367304 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f400ce30 con 0x7f52fc10bd80 2024-05-14T16:56:46.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.756+0000 7f52f9ffb700 1 -- 172.21.15.69:0/3992367304 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4005350 con 0x7f52fc10bd80 2024-05-14T16:56:46.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.756+0000 7f53026eb700 1 -- 172.21.15.69:0/3992367304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52fc1a1160 con 0x7f52fc10bd80 2024-05-14T16:56:46.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.756+0000 7f53026eb700 1 -- 172.21.15.69:0/3992367304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52fc1a89e0 con 0x7f52fc10bd80 2024-05-14T16:56:46.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.757+0000 7f53026eb700 1 -- 172.21.15.69:0/3992367304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52fc06a230 con 0x7f52fc10bd80 2024-05-14T16:56:46.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.758+0000 7f52f9ffb700 1 -- 172.21.15.69:0/3992367304 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f52f400c980 con 0x7f52fc10bd80 2024-05-14T16:56:46.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.759+0000 7f52f9ffb700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e80b2a70 0x7f52e80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:46.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.759+0000 7f52fbfff700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e80b2a70 0x7f52e80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:46.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.759+0000 7f52f9ffb700 1 -- 172.21.15.69:0/3992367304 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f52f40e22b0 con 0x7f52fc10bd80 2024-05-14T16:56:46.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.760+0000 7f52fbfff700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e80b2a70 0x7f52e80b4f30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009750 tx=0x7f52ec01a040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:46.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:45.763+0000 7f52f9ffb700 1 -- 172.21.15.69:0/3992367304 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52f4021020 con 0x7f52fc10bd80 2024-05-14T16:56:46.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.021+0000 7f53026eb700 1 -- 172.21.15.69:0/3992367304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52fc02e560 con 0x7f52fc10bd80 2024-05-14T16:56:46.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.022+0000 7f52f9ffb700 1 -- 172.21.15.69:0/3992367304 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11863 (secure 0 0 0) 0x7f52f40a9d10 con 0x7f52fc10bd80 2024-05-14T16:56:46.165 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:56:46.170 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:56:33.614815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:56:46.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 -- 172.21.15.69:0/3992367304 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e80b2a70 msgr2=0x7f52e80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:46.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e80b2a70 0x7f52e80b4f30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009750 tx=0x7f52ec01a040 comp rx=0 tx=0).stop 2024-05-14T16:56:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 -- 172.21.15.69:0/3992367304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc10bd80 msgr2=0x7f52fc1a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc10bd80 0x7f52fc1a0730 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f52f400ada0 tx=0x7f52f400c330 comp rx=0 tx=0).stop 2024-05-14T16:56:46.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 -- 172.21.15.69:0/3992367304 shutdown_connections 2024-05-14T16:56:46.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52e80b2a70 0x7f52e80b4f30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc10bd80 0x7f52fc1a0730 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52fc102140 0x7f52fc19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 --2- 172.21.15.69:0/3992367304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52fc102af0 0x7f52fc19c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 -- 172.21.15.69:0/3992367304 >> 172.21.15.69:0/3992367304 conn(0x7f52fc0777e0 msgr2=0x7f52fc109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:46.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.024+0000 7f53026eb700 1 -- 172.21.15.69:0/3992367304 shutdown_connections 2024-05-14T16:56:46.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:46.025+0000 7f53026eb700 1 -- 172.21.15.69:0/3992367304 wait complete. 2024-05-14T16:56:46.293 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 37 2024-05-14T16:56:46.300 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:45 smithi092 ceph-mon[35763]: pgmap v212: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 33 MiB/s wr, 1.05k op/s 2024-05-14T16:56:46.388 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:45 smithi069 ceph-mon[31507]: pgmap v212: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 33 MiB/s wr, 1.05k op/s 2024-05-14T16:56:46.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:45 smithi203 ceph-mon[35787]: pgmap v212: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 33 MiB/s wr, 1.05k op/s 2024-05-14T16:56:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 -- 172.21.15.69:0/657735689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f784c106e90 msgr2=0x7f784c107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 --2- 172.21.15.69:0/657735689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f784c106e90 0x7f784c107310 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7844009cd0 tx=0x7f784402a970 comp rx=0 tx=0).stop 2024-05-14T16:56:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 -- 172.21.15.69:0/657735689 shutdown_connections 2024-05-14T16:56:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 --2- 172.21.15.69:0/657735689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f784c106e90 0x7f784c107310 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 --2- 172.21.15.69:0/657735689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f784c1023b0 0x7f784c106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 --2- 172.21.15.69:0/657735689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f784c101a90 0x7f784c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 -- 172.21.15.69:0/657735689 >> 172.21.15.69:0/657735689 conn(0x7f784c078b60 msgr2=0x7f784c078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 -- 172.21.15.69:0/657735689 shutdown_connections 2024-05-14T16:56:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 -- 172.21.15.69:0/657735689 wait complete. 2024-05-14T16:56:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 Processor -- start 2024-05-14T16:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.557+0000 7f78540fc640 1 -- start start 2024-05-14T16:56:46.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f78540fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f784c101a90 0x7f784c1a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f78540fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f784c1023b0 0x7f784c1a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f78540fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f784c106e90 0x7f784c19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f78540fc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f784c1192c0 con 0x7f784c106e90 2024-05-14T16:56:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f78540fc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f784c119140 con 0x7f784c101a90 2024-05-14T16:56:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f78540fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f784c119440 con 0x7f784c1023b0 2024-05-14T16:56:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7852699640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f784c106e90 0x7f784c19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7852699640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f784c106e90 0x7f784c19f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39658/0 (socket says 172.21.15.69:39658) 2024-05-14T16:56:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7852699640 1 -- 172.21.15.69:0/3539584179 learned_addr learned my addr 172.21.15.69:0/3539584179 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7851e98640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f784c101a90 0x7f784c1a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7851697640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f784c1023b0 0x7f784c1a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7852699640 1 -- 172.21.15.69:0/3539584179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f784c1023b0 msgr2=0x7f784c1a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7852699640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f784c1023b0 0x7f784c1a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7852699640 1 -- 172.21.15.69:0/3539584179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f784c101a90 msgr2=0x7f784c1a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7852699640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f784c101a90 0x7f784c1a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7852699640 1 -- 172.21.15.69:0/3539584179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c19fd20 con 0x7f784c106e90 2024-05-14T16:56:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7852699640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f784c106e90 0x7f784c19f5a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7844009df0 tx=0x7f78440093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7842ffd640 1 -- 172.21.15.69:0/3539584179 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844047070 con 0x7f784c106e90 2024-05-14T16:56:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7842ffd640 1 -- 172.21.15.69:0/3539584179 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784402f830 con 0x7f784c106e90 2024-05-14T16:56:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f78540fc640 1 -- 172.21.15.69:0/3539584179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c19ffb0 con 0x7f784c106e90 2024-05-14T16:56:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f7842ffd640 1 -- 172.21.15.69:0/3539584179 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844034070 con 0x7f784c106e90 2024-05-14T16:56:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.558+0000 7f78540fc640 1 -- 172.21.15.69:0/3539584179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c1ac2b0 con 0x7f784c106e90 2024-05-14T16:56:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7842ffd640 1 -- 172.21.15.69:0/3539584179 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f784404a070 con 0x7f784c106e90 2024-05-14T16:56:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7856e13740 1 Processor -- start 2024-05-14T16:56:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7856e13740 1 -- start start 2024-05-14T16:56:46.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7856e13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e94a8f0 0x55aa7e94acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7856e13740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e955590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7856e13740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e955ad0 0x55aa7e957ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7856e13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e8bdc30 con 0x55aa7e94a8f0 2024-05-14T16:56:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7856e13740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e8ad7d0 con 0x55aa7e955ad0 2024-05-14T16:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7856e13740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e8b1810 con 0x55aa7e94b210 2024-05-14T16:56:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7851697640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e94a8f0 0x55aa7e94acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7851697640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e94a8f0 0x55aa7e94acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39664/0 (socket says 172.21.15.69:39664) 2024-05-14T16:56:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.561+0000 7f7851697640 1 -- 172.21.15.69:0/1485430227 learned_addr learned my addr 172.21.15.69:0/1485430227 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7851e98640 1 --2- 172.21.15.69:0/1485430227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e955590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7851697640 1 --2- 172.21.15.69:0/1485430227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e955ad0 0x55aa7e957ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7851697640 1 -- 172.21.15.69:0/1485430227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 msgr2=0x55aa7e955590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7851697640 1 --2- 172.21.15.69:0/1485430227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e955590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7851697640 1 -- 172.21.15.69:0/1485430227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e955ad0 msgr2=0x55aa7e957ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7851697640 1 --2- 172.21.15.69:0/1485430227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e955ad0 0x55aa7e957ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7851697640 1 -- 172.21.15.69:0/1485430227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7e5c2210 con 0x55aa7e94a8f0 2024-05-14T16:56:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7842ffd640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f78380b2a10 0x7f78380b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7851e98640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f78380b2a10 0x7f78380b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7851697640 1 --2- 172.21.15.69:0/1485430227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e94a8f0 0x55aa7e94acd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f783c0058a0 tx=0x7f783c00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db24d71d32e2641 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7842ffd640 1 -- 172.21.15.69:0/3539584179 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f7844102b30 con 0x7f784c106e90 2024-05-14T16:56:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7837fff640 1 -- 172.21.15.69:0/1485430227 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c004030 con 0x55aa7e94a8f0 2024-05-14T16:56:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7837fff640 1 -- 172.21.15.69:0/1485430227 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c0054d0 con 0x55aa7e94a8f0 2024-05-14T16:56:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.562+0000 7f7837fff640 1 -- 172.21.15.69:0/1485430227 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c004a90 con 0x55aa7e94a8f0 2024-05-14T16:56:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 -- 172.21.15.69:0/1485430227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e94a8f0 msgr2=0x55aa7e94acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7851e98640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f78380b2a10 0x7f78380b4ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55aa7e94be90 tx=0x7f784800a2d0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 --2- 172.21.15.69:0/1485430227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e94a8f0 0x55aa7e94acd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f783c0058a0 tx=0x7f783c00b780 comp rx=0 tx=0).stop 2024-05-14T16:56:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 -- 172.21.15.69:0/1485430227 shutdown_connections 2024-05-14T16:56:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 --2- 172.21.15.69:0/1485430227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e955ad0 0x55aa7e957ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 --2- 172.21.15.69:0/1485430227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e955590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 --2- 172.21.15.69:0/1485430227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e94a8f0 0x55aa7e94acd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 -- 172.21.15.69:0/1485430227 >> 172.21.15.69:0/1485430227 conn(0x55aa7e946e00 msgr2=0x55aa7e9471e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 -- 172.21.15.69:0/1485430227 shutdown_connections 2024-05-14T16:56:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 -- 172.21.15.69:0/1485430227 wait complete. 2024-05-14T16:56:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 -- start start 2024-05-14T16:56:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.563+0000 7f7856e13740 1 Processor -- start 2024-05-14T16:56:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7856e13740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94a8f0 0x55aa7e955420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7856e13740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94b210 0x55aa7e952710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7856e13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e952c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7856e13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e593790 con 0x55aa7e955ad0 2024-05-14T16:56:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7856e13740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e869890 con 0x55aa7e94b210 2024-05-14T16:56:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7856e13740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e9587a0 con 0x55aa7e94a8f0 2024-05-14T16:56:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7856e13740 10 client.0 _ll_register_callbacks cb 0x55aa7e86cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7852699640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e952c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7852699640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e952c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39670/0 (socket says 172.21.15.69:39670) 2024-05-14T16:56:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7852699640 1 -- 172.21.15.69:0/2037060250 learned_addr learned my addr 172.21.15.69:0/2037060250 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7851697640 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94b210 0x55aa7e952710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7851e98640 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94a8f0 0x55aa7e955420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7852699640 1 -- 172.21.15.69:0/2037060250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94a8f0 msgr2=0x55aa7e955420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7852699640 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94a8f0 0x55aa7e955420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7852699640 1 -- 172.21.15.69:0/2037060250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94b210 msgr2=0x55aa7e952710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7852699640 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94b210 0x55aa7e952710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7852699640 1 -- 172.21.15.69:0/2037060250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa7e68e620 con 0x55aa7e955ad0 2024-05-14T16:56:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f7852699640 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e952c70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f78440c9ed0 tx=0x7f78440c99c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7dd323a76a7d1e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f78357fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:56:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.564+0000 7f78357fa640 1 -- 172.21.15.69:0/2037060250 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440c9a50 con 0x55aa7e955ad0 2024-05-14T16:56:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f78357fa640 1 -- 172.21.15.69:0/2037060250 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f784402f380 con 0x55aa7e955ad0 2024-05-14T16:56:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f7856e13740 1 -- 172.21.15.69:0/2037060250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f783c0054d0 con 0x55aa7e955ad0 2024-05-14T16:56:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f78357fa640 1 -- 172.21.15.69:0/2037060250 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f78440c9f40 con 0x55aa7e955ad0 2024-05-14T16:56:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f7856e13740 10 client.16140 fetch_fsmap learned FSMap version 37 2024-05-14T16:56:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f7856e13740 1 -- 172.21.15.69:0/2037060250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55aa7e964230 con 0x55aa7e955ad0 2024-05-14T16:56:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f78357fa640 1 -- 172.21.15.69:0/2037060250 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c0054d0 con 0x55aa7e955ad0 2024-05-14T16:56:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f78357fa640 1 -- 172.21.15.69:0/2037060250 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f7844041030 con 0x55aa7e955ad0 2024-05-14T16:56:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f7856e13740 10 client.16140 fetch_fsmap finished waiting for FSMap version 37 2024-05-14T16:56:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f7856e13740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f7856e13740 20 client.16140 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f7856e13740 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55aa7e9657b0 0x55aa7e967bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f7856e13740 4 client.16140 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f7856e13740 1 -- 172.21.15.69:0/2037060250 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa7e8bfb80 con 0x55aa7e9657b0 2024-05-14T16:56:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.565+0000 7f7851e98640 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55aa7e9657b0 0x55aa7e967bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.566+0000 7f7851e98640 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55aa7e9657b0 0x55aa7e967bf0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c4c9af461994ea server_cookie=a2e13e2fd7f0fc7e in_seq=0 out_seq=0 2024-05-14T16:56:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.566+0000 7f78357fa640 10 client.16140 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.566+0000 7f78357fa640 1 -- 172.21.15.69:0/2037060250 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa7e8bfb80 con 0x55aa7e9657b0 2024-05-14T16:56:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.566+0000 7f78357fa640 10 client.16140 handle_command_reply: tid=0 2024-05-14T16:56:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.566+0000 7f7856e13740 1 client.16140 shutdown 2024-05-14T16:56:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 1 -- 172.21.15.69:0/2037060250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 msgr2=0x55aa7e952c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e952c70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f78440c9ed0 tx=0x7f78440c99c0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 1 -- 172.21.15.69:0/2037060250 shutdown_connections 2024-05-14T16:56:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55aa7e9657b0 0x55aa7e967bf0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e952c70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94b210 0x55aa7e952710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f78357fa640 0 client.16140 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 1 --2- 172.21.15.69:0/2037060250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94a8f0 0x55aa7e955420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 1 -- 172.21.15.69:0/2037060250 >> 172.21.15.69:0/2037060250 conn(0x55aa7e946e00 msgr2=0x55aa7e946100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 1 -- 172.21.15.69:0/2037060250 shutdown_connections 2024-05-14T16:56:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 1 -- 172.21.15.69:0/2037060250 wait complete. 2024-05-14T16:56:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.567+0000 7f7856e13740 20 client.16140 trim_cache size 0 max 16384 2024-05-14T16:56:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7856e13740 1 Processor -- start 2024-05-14T16:56:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7856e13740 1 -- start start 2024-05-14T16:56:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7856e13740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 0x55aa7e9616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7856e13740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e96d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7856e13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e96da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7856e13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f784c1b7400 con 0x55aa7e955ad0 2024-05-14T16:56:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7856e13740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f784c1b4c40 con 0x55aa7e94a8f0 2024-05-14T16:56:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7856e13740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f784c1b2ab0 con 0x55aa7e94b210 2024-05-14T16:56:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7852699640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e96da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7852699640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e96da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39684/0 (socket says 172.21.15.69:39684) 2024-05-14T16:56:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7852699640 1 -- 172.21.15.69:0/1088710671 learned_addr learned my addr 172.21.15.69:0/1088710671 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7851697640 1 --2- 172.21.15.69:0/1088710671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e96d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7851e98640 1 --2- 172.21.15.69:0/1088710671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 0x55aa7e9616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7852699640 1 -- 172.21.15.69:0/1088710671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 msgr2=0x55aa7e96d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7852699640 1 --2- 172.21.15.69:0/1088710671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e96d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7852699640 1 -- 172.21.15.69:0/1088710671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 msgr2=0x55aa7e9616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7852699640 1 --2- 172.21.15.69:0/1088710671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 0x55aa7e9616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7852699640 1 -- 172.21.15.69:0/1088710671 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7e9516c0 con 0x55aa7e955ad0 2024-05-14T16:56:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7851697640 1 --2- 172.21.15.69:0/1088710671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e96d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.582+0000 7f7852699640 1 --2- 172.21.15.69:0/1088710671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e96da00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7844032000 tx=0x7f784402a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd96b94f8e6a6e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f78377fe640 1 -- 172.21.15.69:0/1088710671 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440c9a50 con 0x55aa7e955ad0 2024-05-14T16:56:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f78377fe640 1 -- 172.21.15.69:0/1088710671 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78440ce240 con 0x55aa7e955ad0 2024-05-14T16:56:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f78377fe640 1 -- 172.21.15.69:0/1088710671 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440c9400 con 0x55aa7e955ad0 2024-05-14T16:56:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f7856e13740 1 -- 172.21.15.69:0/1088710671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 msgr2=0x55aa7e96da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f7856e13740 1 --2- 172.21.15.69:0/1088710671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e96da00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7844032000 tx=0x7f784402a970 comp rx=0 tx=0).stop 2024-05-14T16:56:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f7856e13740 1 -- 172.21.15.69:0/1088710671 shutdown_connections 2024-05-14T16:56:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f7856e13740 1 --2- 172.21.15.69:0/1088710671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e96da00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f7856e13740 1 --2- 172.21.15.69:0/1088710671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e96d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f7856e13740 1 --2- 172.21.15.69:0/1088710671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 0x55aa7e9616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f7856e13740 1 -- 172.21.15.69:0/1088710671 >> 172.21.15.69:0/1088710671 conn(0x55aa7e946e00 msgr2=0x55aa7e946100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f7856e13740 1 -- 172.21.15.69:0/1088710671 shutdown_connections 2024-05-14T16:56:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.583+0000 7f7856e13740 1 -- 172.21.15.69:0/1088710671 wait complete. 2024-05-14T16:56:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7856e13740 1 -- start start 2024-05-14T16:56:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7856e13740 1 Processor -- start 2024-05-14T16:56:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7856e13740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 0x55aa7e95b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7856e13740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e95bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7856e13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e95c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7856e13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f784c1b3100 con 0x55aa7e955ad0 2024-05-14T16:56:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7856e13740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f784c1b4570 con 0x55aa7e94a8f0 2024-05-14T16:56:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7856e13740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f784c1b37d0 con 0x55aa7e94b210 2024-05-14T16:56:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7856e13740 10 client.0 _ll_register_callbacks cb 0x7f78440c9d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7852699640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e95c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7852699640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e95c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39690/0 (socket says 172.21.15.69:39690) 2024-05-14T16:56:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7852699640 1 -- 172.21.15.69:0/2300620389 learned_addr learned my addr 172.21.15.69:0/2300620389 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7851e98640 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 0x55aa7e95b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7851697640 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e95bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7852699640 1 -- 172.21.15.69:0/2300620389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 msgr2=0x55aa7e95bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7852699640 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e95bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7852699640 1 -- 172.21.15.69:0/2300620389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 msgr2=0x55aa7e95b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7852699640 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 0x55aa7e95b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7852699640 1 -- 172.21.15.69:0/2300620389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa7e9cf150 con 0x55aa7e955ad0 2024-05-14T16:56:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7851697640 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e95bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:56:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.584+0000 7f7851e98640 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 0x55aa7e95b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:56:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f7852699640 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e95c150 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f78440c9c40 tx=0x7f7844100320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ca9b7b5221781a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f78357fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:56:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f78357fa640 1 -- 172.21.15.69:0/2300620389 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440c9a50 con 0x55aa7e955ad0 2024-05-14T16:56:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f78357fa640 1 -- 172.21.15.69:0/2300620389 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f7844004d50 con 0x55aa7e955ad0 2024-05-14T16:56:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f7856e13740 1 -- 172.21.15.69:0/2300620389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78440ce240 con 0x55aa7e955ad0 2024-05-14T16:56:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f78357fa640 1 -- 172.21.15.69:0/2300620389 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f78440c91a0 con 0x55aa7e955ad0 2024-05-14T16:56:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f7856e13740 10 client.16146 fetch_fsmap learned FSMap version 37 2024-05-14T16:56:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f7856e13740 1 -- 172.21.15.69:0/2300620389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55aa7e9cf150 con 0x55aa7e955ad0 2024-05-14T16:56:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f78357fa640 1 -- 172.21.15.69:0/2300620389 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78440ce240 con 0x55aa7e955ad0 2024-05-14T16:56:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f78357fa640 1 -- 172.21.15.69:0/2300620389 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f7844041070 con 0x55aa7e955ad0 2024-05-14T16:56:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f7856e13740 10 client.16146 fetch_fsmap finished waiting for FSMap version 37 2024-05-14T16:56:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f7856e13740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f7856e13740 20 client.16146 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f7856e13740 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55aa7e9d0790 0x55aa7e9d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f7856e13740 4 client.16146 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:56:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.585+0000 7f7856e13740 1 -- 172.21.15.69:0/2300620389 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa7e8b9360 con 0x55aa7e9d0790 2024-05-14T16:56:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.586+0000 7f7851e98640 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55aa7e9d0790 0x55aa7e9d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.586+0000 7f78357fa640 10 client.16146 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.586+0000 7f7851e98640 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55aa7e9d0790 0x55aa7e9d2bb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8f70995c15cbee server_cookie=e89b7c41cd8b302a in_seq=0 out_seq=0 2024-05-14T16:56:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.599+0000 7f78357fa640 1 -- 172.21.15.69:0/2300620389 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55aa7e8b9360 con 0x55aa7e9d0790 2024-05-14T16:56:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.599+0000 7f78357fa640 10 client.16146 handle_command_reply: tid=0 2024-05-14T16:56:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.599+0000 7f7856e13740 1 client.16146 shutdown 2024-05-14T16:56:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.599+0000 7f7856e13740 1 -- 172.21.15.69:0/2300620389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 msgr2=0x55aa7e95c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.599+0000 7f7856e13740 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e95c150 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f78440c9c40 tx=0x7f7844100320 comp rx=0 tx=0).stop 2024-05-14T16:56:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f7856e13740 1 -- 172.21.15.69:0/2300620389 shutdown_connections 2024-05-14T16:56:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f7856e13740 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55aa7e9d0790 0x55aa7e9d2bb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f7856e13740 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e955ad0 0x55aa7e95c150 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f7856e13740 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aa7e94b210 0x55aa7e95bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78357fa640 0 client.16146 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f7856e13740 1 --2- 172.21.15.69:0/2300620389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aa7e94a8f0 0x55aa7e95b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f7856e13740 1 -- 172.21.15.69:0/2300620389 >> 172.21.15.69:0/2300620389 conn(0x55aa7e946e00 msgr2=0x55aa7e960690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f7856e13740 1 -- 172.21.15.69:0/2300620389 shutdown_connections 2024-05-14T16:56:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f7856e13740 1 -- 172.21.15.69:0/2300620389 wait complete. 2024-05-14T16:56:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f7856e13740 20 client.16146 trim_cache size 0 max 16384 2024-05-14T16:56:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:56:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:56:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:56:48.903 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:47 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3992367304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:48.926 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:48 smithi092 ceph-mon[35763]: pgmap v213: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 MiB/s wr, 1.06k op/s 2024-05-14T16:56:48.931 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:47 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3992367304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:48.937 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:48 smithi203 ceph-mon[35787]: pgmap v213: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 MiB/s wr, 1.06k op/s 2024-05-14T16:56:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 -- 172.21.15.69:0/3539584179 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f78380b2a10 msgr2=0x7f78380b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f78380b2a10 0x7f78380b4ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55aa7e94be90 tx=0x7f784800a2d0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 -- 172.21.15.69:0/3539584179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f784c106e90 msgr2=0x7f784c19f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f784c106e90 0x7f784c19f5a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7844009df0 tx=0x7f78440093d0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 -- 172.21.15.69:0/3539584179 shutdown_connections 2024-05-14T16:56:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f78380b2a10 0x7f78380b4ed0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f784c106e90 0x7f784c19f5a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f784c1023b0 0x7f784c1a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 --2- 172.21.15.69:0/3539584179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f784c101a90 0x7f784c1a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 -- 172.21.15.69:0/3539584179 >> 172.21.15.69:0/3539584179 conn(0x7f784c078b60 msgr2=0x7f784c104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 -- 172.21.15.69:0/3539584179 shutdown_connections 2024-05-14T16:56:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:46.600+0000 7f78540fc640 1 -- 172.21.15.69:0/3539584179 wait complete. 2024-05-14T16:56:49.097 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id 78c5781c --release --await' 2024-05-14T16:56:49.109 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id 78c5781c --release --await 2024-05-14T16:56:49.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:47 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3992367304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:49.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:48 smithi069 ceph-mon[31507]: pgmap v213: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 MiB/s wr, 1.06k op/s 2024-05-14T16:56:49.393 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:50.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:50 smithi069 ceph-mon[31507]: pgmap v214: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 788 op/s 2024-05-14T16:56:50.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:50 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:50.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 380.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:56:50.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:50 smithi203 ceph-mon[35787]: pgmap v214: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 788 op/s 2024-05-14T16:56:50.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:50 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:50.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:50 smithi092 ceph-mon[35763]: pgmap v214: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 788 op/s 2024-05-14T16:56:50.508 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:50 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:51.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.002+0000 7f4016afa700 1 -- 172.21.15.69:0/4143457666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f401006bab0 msgr2=0x7f401006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:51.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.002+0000 7f4016afa700 1 --2- 172.21.15.69:0/4143457666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f401006bab0 0x7f401006bf30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009990 tx=0x7f3ffc009ca0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.002+0000 7f4016afa700 1 -- 172.21.15.69:0/4143457666 shutdown_connections 2024-05-14T16:56:51.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.002+0000 7f4016afa700 1 --2- 172.21.15.69:0/4143457666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f401006c470 0x7f40101103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.002+0000 7f4016afa700 1 --2- 172.21.15.69:0/4143457666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40101095a0 0x7f4010109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.002+0000 7f4016afa700 1 --2- 172.21.15.69:0/4143457666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f401006bab0 0x7f401006bf30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.002+0000 7f4016afa700 1 -- 172.21.15.69:0/4143457666 >> 172.21.15.69:0/4143457666 conn(0x7f40100793b0 msgr2=0x7f40100797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:51.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.002+0000 7f4016afa700 1 -- 172.21.15.69:0/4143457666 shutdown_connections 2024-05-14T16:56:51.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.002+0000 7f4016afa700 1 -- 172.21.15.69:0/4143457666 wait complete. 2024-05-14T16:56:51.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.003+0000 7f4016afa700 1 Processor -- start 2024-05-14T16:56:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.003+0000 7f4016afa700 1 -- start start 2024-05-14T16:56:51.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.003+0000 7f4016afa700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f401006bab0 0x7f4010102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.003+0000 7f4016afa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f401006c470 0x7f4010102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.003+0000 7f4016afa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40101095a0 0x7f4010102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:51.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.003+0000 7f4016afa700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f40101137b0 con 0x7f401006bab0 2024-05-14T16:56:51.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.003+0000 7f4016afa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4010113630 con 0x7f401006c470 2024-05-14T16:56:51.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.003+0000 7f4016afa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4010113930 con 0x7f40101095a0 2024-05-14T16:56:51.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.003+0000 7f4014896700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f401006bab0 0x7f4010102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:51.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f4014896700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f401006bab0 0x7f4010102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39702/0 (socket says 172.21.15.69:39702) 2024-05-14T16:56:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f4014896700 1 -- 172.21.15.69:0/923789739 learned_addr learned my addr 172.21.15.69:0/923789739 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:51.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f400ffff700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f401006c470 0x7f4010102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:51.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f4015097700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40101095a0 0x7f4010102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:51.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f4014896700 1 -- 172.21.15.69:0/923789739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40101095a0 msgr2=0x7f4010102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:51.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f4014896700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40101095a0 0x7f4010102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f4014896700 1 -- 172.21.15.69:0/923789739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f401006c470 msgr2=0x7f4010102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:51.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f4014896700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f401006c470 0x7f4010102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f4014896700 1 -- 172.21.15.69:0/923789739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40101a8290 con 0x7f401006bab0 2024-05-14T16:56:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f4014896700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f401006bab0 0x7f4010102500 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f400400b530 tx=0x7f400400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:51.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f400dffb700 1 -- 172.21.15.69:0/923789739 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004015020 con 0x7f401006bab0 2024-05-14T16:56:51.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f400dffb700 1 -- 172.21.15.69:0/923789739 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4004011ce0 con 0x7f401006bab0 2024-05-14T16:56:51.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f4016afa700 1 -- 172.21.15.69:0/923789739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40101a8580 con 0x7f401006bab0 2024-05-14T16:56:51.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.004+0000 7f400dffb700 1 -- 172.21.15.69:0/923789739 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004010e40 con 0x7f401006bab0 2024-05-14T16:56:51.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.005+0000 7f4016afa700 1 -- 172.21.15.69:0/923789739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40101a8ae0 con 0x7f401006bab0 2024-05-14T16:56:51.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.008+0000 7f400dffb700 1 -- 172.21.15.69:0/923789739 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4004018020 con 0x7f401006bab0 2024-05-14T16:56:51.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.008+0000 7f400dffb700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40000b2a70 0x7f40000b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:51.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.008+0000 7f400ffff700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40000b2a70 0x7f40000b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:51.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.008+0000 7f400dffb700 1 -- 172.21.15.69:0/923789739 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f40040e1cc0 con 0x7f401006bab0 2024-05-14T16:56:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.009+0000 7f400ffff700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40000b2a70 0x7f40000b4f30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00b5c0 tx=0x7f3ffc005960 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:51.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.009+0000 7f4021390b80 1 Processor -- start 2024-05-14T16:56:51.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.009+0000 7f4021390b80 1 -- start start 2024-05-14T16:56:51.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.009+0000 7f4021390b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a95824420 0x560a95824800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:51.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.009+0000 7f4021390b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9581fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:51.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.009+0000 7f4021390b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a958202c0 0x560a95820720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:51.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.009+0000 7f4021390b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a9577de70 con 0x560a95824d40 2024-05-14T16:56:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.009+0000 7f4021390b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560a9576da10 con 0x560a958202c0 2024-05-14T16:56:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f4021390b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560a95771a50 con 0x560a95824420 2024-05-14T16:56:51.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f400ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9581fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:51.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f400ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9581fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39704/0 (socket says 172.21.15.69:39704) 2024-05-14T16:56:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f400ffff700 1 -- 172.21.15.69:0/922665392 learned_addr learned my addr 172.21.15.69:0/922665392 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:51.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f4014896700 1 --2- 172.21.15.69:0/922665392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a95824420 0x560a95824800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:51.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f4015097700 1 --2- 172.21.15.69:0/922665392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a958202c0 0x560a95820720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:51.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f400ffff700 1 -- 172.21.15.69:0/922665392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a95824420 msgr2=0x560a95824800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f400ffff700 1 --2- 172.21.15.69:0/922665392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a95824420 0x560a95824800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f400ffff700 1 -- 172.21.15.69:0/922665392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a958202c0 msgr2=0x560a95820720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:51.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f400ffff700 1 --2- 172.21.15.69:0/922665392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a958202c0 0x560a95820720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f400ffff700 1 -- 172.21.15.69:0/922665392 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a957170b0 con 0x560a95824d40 2024-05-14T16:56:51.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.010+0000 7f400ffff700 1 --2- 172.21.15.69:0/922665392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9581fca0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc015000 tx=0x7f3ffc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3700d2b9529289 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:51.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.012+0000 7f3ffaffd700 1 -- 172.21.15.69:0/922665392 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc034070 con 0x560a95824d40 2024-05-14T16:56:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.012+0000 7f3ffaffd700 1 -- 172.21.15.69:0/922665392 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ffc024930 con 0x560a95824d40 2024-05-14T16:56:51.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.012+0000 7f3ffaffd700 1 -- 172.21.15.69:0/922665392 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc02fb10 con 0x560a95824d40 2024-05-14T16:56:51.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 -- 172.21.15.69:0/922665392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 msgr2=0x560a9581fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:51.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 --2- 172.21.15.69:0/922665392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9581fca0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc015000 tx=0x7f3ffc01f000 comp rx=0 tx=0).stop 2024-05-14T16:56:51.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 -- 172.21.15.69:0/922665392 shutdown_connections 2024-05-14T16:56:51.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 --2- 172.21.15.69:0/922665392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a958202c0 0x560a95820720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 --2- 172.21.15.69:0/922665392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a95824420 0x560a95824800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 --2- 172.21.15.69:0/922665392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9581fca0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 -- 172.21.15.69:0/922665392 >> 172.21.15.69:0/922665392 conn(0x560a9574ceb0 msgr2=0x560a9581a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:51.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 -- 172.21.15.69:0/922665392 shutdown_connections 2024-05-14T16:56:51.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 -- 172.21.15.69:0/922665392 wait complete. 2024-05-14T16:56:51.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 -- start start 2024-05-14T16:56:51.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.013+0000 7f4021390b80 1 Processor -- start 2024-05-14T16:56:51.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f4021390b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a9582bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:51.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f4021390b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824420 0x560a95828ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f4021390b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824d40 0x560a95829400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:51.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f4021390b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a956740a0 con 0x560a95824420 2024-05-14T16:56:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f4021390b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560a956d5ea0 con 0x560a95824d40 2024-05-14T16:56:51.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f4021390b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560a9582c820 con 0x560a958202c0 2024-05-14T16:56:51.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f4021390b80 10 client.0 _ll_register_callbacks cb 0x560a95708a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:51.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f400ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824420 0x560a95828ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f400ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824420 0x560a95828ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39718/0 (socket says 172.21.15.69:39718) 2024-05-14T16:56:51.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f400ffff700 1 -- 172.21.15.69:0/1097641994 learned_addr learned my addr 172.21.15.69:0/1097641994 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:51.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f4015097700 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824d40 0x560a95829400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:51.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f4014896700 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a9582bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:51.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f400ffff700 1 -- 172.21.15.69:0/1097641994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 msgr2=0x560a9582bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:51.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f400ffff700 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a9582bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f400ffff700 1 -- 172.21.15.69:0/1097641994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824d40 msgr2=0x560a95829400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:51.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f400ffff700 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824d40 0x560a95829400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:51.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f400ffff700 1 -- 172.21.15.69:0/1097641994 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a956bd9d0 con 0x560a95824420 2024-05-14T16:56:51.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f400ffff700 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824420 0x560a95828ec0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009350 tx=0x7f3ffc024d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5f2fcb3cf9b69a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f3feffff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:56:51.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f3feffff700 1 -- 172.21.15.69:0/1097641994 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc03c070 con 0x560a95824420 2024-05-14T16:56:52.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.014+0000 7f4021390b80 1 -- 172.21.15.69:0/1097641994 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ffc024930 con 0x560a95824420 2024-05-14T16:56:52.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f3feffff700 1 -- 172.21.15.69:0/1097641994 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f3ffc02f750 con 0x560a95824420 2024-05-14T16:56:52.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f3feffff700 1 -- 172.21.15.69:0/1097641994 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3ffc045050 con 0x560a95824420 2024-05-14T16:56:52.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f4021390b80 10 client.16158 fetch_fsmap learned FSMap version 38 2024-05-14T16:56:52.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f4021390b80 1 -- 172.21.15.69:0/1097641994 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560a95727860 con 0x560a95824420 2024-05-14T16:56:52.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f3feffff700 1 -- 172.21.15.69:0/1097641994 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ffc024930 con 0x560a95824420 2024-05-14T16:56:52.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f3feffff700 1 -- 172.21.15.69:0/1097641994 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f3ffc03f020 con 0x560a95824420 2024-05-14T16:56:52.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f4021390b80 10 client.16158 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T16:56:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f4021390b80 10 client.16158 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:56:52.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f4021390b80 20 client.16158 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:52.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f4021390b80 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560a95839260 0x560a9583b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f4021390b80 4 client.16158 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:52.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f4021390b80 1 -- 172.21.15.69:0/1097641994 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a9577fdc0 con 0x560a95839260 2024-05-14T16:56:52.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.015+0000 7f4014896700 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560a95839260 0x560a9583b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:52.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.016+0000 7f3feffff700 10 client.16158 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:52.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.016+0000 7f4014896700 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560a95839260 0x560a9583b650 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f01d853e84e9e2 server_cookie=c9cf71c5fd3294bb in_seq=0 out_seq=0 2024-05-14T16:56:52.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.016+0000 7f3feffff700 1 -- 172.21.15.69:0/1097641994 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a9577fdc0 con 0x560a95839260 2024-05-14T16:56:52.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.016+0000 7f3feffff700 10 client.16158 handle_command_reply: tid=0 2024-05-14T16:56:52.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.016+0000 7f4021390b80 1 client.16158 shutdown 2024-05-14T16:56:52.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 1 -- 172.21.15.69:0/1097641994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824420 msgr2=0x560a95828ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:52.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824420 0x560a95828ec0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009350 tx=0x7f3ffc024d10 comp rx=0 tx=0).stop 2024-05-14T16:56:52.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 1 -- 172.21.15.69:0/1097641994 shutdown_connections 2024-05-14T16:56:52.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560a95839260 0x560a9583b650 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824d40 0x560a95829400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f3feffff700 0 client.16158 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:52.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a9582bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 1 --2- 172.21.15.69:0/1097641994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824420 0x560a95828ec0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 1 -- 172.21.15.69:0/1097641994 >> 172.21.15.69:0/1097641994 conn(0x560a9574ceb0 msgr2=0x560a95814fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:52.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 1 -- 172.21.15.69:0/1097641994 shutdown_connections 2024-05-14T16:56:52.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 1 -- 172.21.15.69:0/1097641994 wait complete. 2024-05-14T16:56:52.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.017+0000 7f4021390b80 20 client.16158 trim_cache size 0 max 16384 2024-05-14T16:56:52.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.036+0000 7f4021390b80 1 Processor -- start 2024-05-14T16:56:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.036+0000 7f4021390b80 1 -- start start 2024-05-14T16:56:52.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4021390b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a958150d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4021390b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824420 0x560a9582abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4021390b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9582b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4021390b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f40101b2a30 con 0x560a95824d40 2024-05-14T16:56:52.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4021390b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f40101b0260 con 0x560a95824420 2024-05-14T16:56:52.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4021390b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ae0d0 con 0x560a958202c0 2024-05-14T16:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4015097700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9582b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:52.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4015097700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9582b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39730/0 (socket says 172.21.15.69:39730) 2024-05-14T16:56:52.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4015097700 1 -- 172.21.15.69:0/2045895255 learned_addr learned my addr 172.21.15.69:0/2045895255 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:52.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f400ffff700 1 --2- 172.21.15.69:0/2045895255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824420 0x560a9582abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:52.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4014896700 1 --2- 172.21.15.69:0/2045895255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a958150d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:52.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4015097700 1 -- 172.21.15.69:0/2045895255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 msgr2=0x560a958150d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:52.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4015097700 1 --2- 172.21.15.69:0/2045895255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a958150d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4015097700 1 -- 172.21.15.69:0/2045895255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824420 msgr2=0x560a9582abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:52.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4015097700 1 --2- 172.21.15.69:0/2045895255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824420 0x560a9582abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4015097700 1 -- 172.21.15.69:0/2045895255 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a95815610 con 0x560a95824d40 2024-05-14T16:56:52.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f4015097700 1 --2- 172.21.15.69:0/2045895255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9582b110 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f400800a5c0 tx=0x7f40080097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ae855270af3019 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:52.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f3ffa7fc700 1 -- 172.21.15.69:0/2045895255 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080177f0 con 0x560a95824d40 2024-05-14T16:56:52.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f3ffa7fc700 1 -- 172.21.15.69:0/2045895255 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f400801d070 con 0x560a95824d40 2024-05-14T16:56:52.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.037+0000 7f3ffa7fc700 1 -- 172.21.15.69:0/2045895255 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008016340 con 0x560a95824d40 2024-05-14T16:56:52.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 -- 172.21.15.69:0/2045895255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 msgr2=0x560a9582b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 --2- 172.21.15.69:0/2045895255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9582b110 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f400800a5c0 tx=0x7f40080097b0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 -- 172.21.15.69:0/2045895255 shutdown_connections 2024-05-14T16:56:52.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 --2- 172.21.15.69:0/2045895255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9582b110 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 --2- 172.21.15.69:0/2045895255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a958150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 --2- 172.21.15.69:0/2045895255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824420 0x560a9582abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 -- 172.21.15.69:0/2045895255 >> 172.21.15.69:0/2045895255 conn(0x560a9574ceb0 msgr2=0x560a9582a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:52.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 -- 172.21.15.69:0/2045895255 shutdown_connections 2024-05-14T16:56:52.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 -- 172.21.15.69:0/2045895255 wait complete. 2024-05-14T16:56:52.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 -- start start 2024-05-14T16:56:52.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.038+0000 7f4021390b80 1 Processor -- start 2024-05-14T16:56:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4021390b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a9583ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4021390b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824420 0x560a9583cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4021390b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9583d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4021390b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ae720 con 0x560a95824d40 2024-05-14T16:56:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4021390b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f40101afb90 con 0x560a95824420 2024-05-14T16:56:52.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4021390b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f40101aedf0 con 0x560a958202c0 2024-05-14T16:56:52.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4021390b80 10 client.0 _ll_register_callbacks cb 0x560a95708a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4015097700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9583d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:52.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4015097700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9583d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39734/0 (socket says 172.21.15.69:39734) 2024-05-14T16:56:52.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4015097700 1 -- 172.21.15.69:0/2094829525 learned_addr learned my addr 172.21.15.69:0/2094829525 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4015097700 1 -- 172.21.15.69:0/2094829525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 msgr2=0x560a9583ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4015097700 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a9583ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4015097700 1 -- 172.21.15.69:0/2094829525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824420 msgr2=0x560a9583cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:52.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4015097700 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824420 0x560a9583cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4015097700 1 -- 172.21.15.69:0/2094829525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a9583dd90 con 0x560a95824d40 2024-05-14T16:56:52.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4015097700 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9583d550 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4008009620 tx=0x7f40080187e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763771108ae6725 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f3feffff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:56:52.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f3feffff700 1 -- 172.21.15.69:0/2094829525 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008025070 con 0x560a95824d40 2024-05-14T16:56:52.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f3feffff700 1 -- 172.21.15.69:0/2094829525 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f4008017a00 con 0x560a95824d40 2024-05-14T16:56:52.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.039+0000 7f4021390b80 1 -- 172.21.15.69:0/2094829525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f400801d070 con 0x560a95824d40 2024-05-14T16:56:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f3feffff700 1 -- 172.21.15.69:0/2094829525 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4008017c20 con 0x560a95824d40 2024-05-14T16:56:52.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f4021390b80 10 client.16164 fetch_fsmap learned FSMap version 38 2024-05-14T16:56:52.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f4021390b80 1 -- 172.21.15.69:0/2094829525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560a958a48a0 con 0x560a95824d40 2024-05-14T16:56:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f3feffff700 1 -- 172.21.15.69:0/2094829525 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f400801d070 con 0x560a95824d40 2024-05-14T16:56:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f3feffff700 1 -- 172.21.15.69:0/2094829525 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f40080213f0 con 0x560a95824d40 2024-05-14T16:56:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f4021390b80 10 client.16164 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T16:56:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f4021390b80 10 client.16164 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:56:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f4021390b80 20 client.16164 populate_metadata read hostname 'smithi069' 2024-05-14T16:56:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f4021390b80 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560a958a4f00 0x560a958a7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f4021390b80 4 client.16164 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "78c5781c", "release": true, "await": true}] 2024-05-14T16:56:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f4021390b80 1 -- 172.21.15.69:0/2094829525 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "78c5781c", "release": true, "await": true}) -- 0x560a957795a0 con 0x560a958a4f00 2024-05-14T16:56:52.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.040+0000 7f4014896700 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560a958a4f00 0x560a958a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:52.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.041+0000 7f3feffff700 10 client.16164 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:52.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.041+0000 7f4014896700 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560a958a4f00 0x560a958a7380 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3beeaafa74b7163 server_cookie=81e60581c1512cb8 in_seq=0 out_seq=0 2024-05-14T16:56:52.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f3feffff700 1 -- 172.21.15.69:0/2094829525 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560a957795a0 con 0x560a958a4f00 2024-05-14T16:56:52.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f3feffff700 10 client.16164 handle_command_reply: tid=0 2024-05-14T16:56:52.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 client.16164 shutdown 2024-05-14T16:56:52.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 -- 172.21.15.69:0/2094829525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 msgr2=0x560a9583d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:52.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9583d550 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4008009620 tx=0x7f40080187e0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 -- 172.21.15.69:0/2094829525 shutdown_connections 2024-05-14T16:56:52.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560a958a4f00 0x560a958a7380 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a95824d40 0x560a9583d550 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560a958202c0 0x560a9583ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 --2- 172.21.15.69:0/2094829525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a95824420 0x560a9583cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 -- 172.21.15.69:0/2094829525 >> 172.21.15.69:0/2094829525 conn(0x560a9574ceb0 msgr2=0x560a9582ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:52.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f3feffff700 0 client.16164 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:56:52.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 -- 172.21.15.69:0/2094829525 shutdown_connections 2024-05-14T16:56:52.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 1 -- 172.21.15.69:0/2094829525 wait complete. 2024-05-14T16:56:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.046+0000 7f4021390b80 20 client.16164 trim_cache size 0 max 16384 2024-05-14T16:56:52.926 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:56:52.945 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 38, 2024-05-14T16:56:52.956 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:56:52.962 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 24, 2024-05-14T16:56:52.967 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:56:52.973 INFO:teuthology.orchestra.run.smithi069.stdout: "78c5781c": { 2024-05-14T16:56:52.995 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 24, 2024-05-14T16:56:53.000 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:56:53.006 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:56:53.011 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:56:53.017 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:56:53.023 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:56:53.029 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:56:53.034 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 136.4, 2024-05-14T16:56:53.040 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:56:53.045 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:56:53.051 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:56:53.057 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:56:53.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:56:53.090 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:56:53.096 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:53.101 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:53.107 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:53.128 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:53.134 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:56:53.140 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:56:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 -- 172.21.15.69:0/923789739 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40000b2a70 msgr2=0x7f40000b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:53.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40000b2a70 0x7f40000b4f30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00b5c0 tx=0x7f3ffc005960 comp rx=0 tx=0).stop 2024-05-14T16:56:53.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 -- 172.21.15.69:0/923789739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f401006bab0 msgr2=0x7f4010102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:53.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f401006bab0 0x7f4010102500 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f400400b530 tx=0x7f400400b840 comp rx=0 tx=0).stop 2024-05-14T16:56:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 -- 172.21.15.69:0/923789739 shutdown_connections 2024-05-14T16:56:53.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f40000b2a70 0x7f40000b4f30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f401006bab0 0x7f4010102500 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40101095a0 0x7f4010102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 --2- 172.21.15.69:0/923789739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f401006c470 0x7f4010102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 -- 172.21.15.69:0/923789739 >> 172.21.15.69:0/923789739 conn(0x7f40100793b0 msgr2=0x7f4010101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 -- 172.21.15.69:0/923789739 shutdown_connections 2024-05-14T16:56:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:56:51.047+0000 7f4016afa700 1 -- 172.21.15.69:0/923789739 wait complete. 2024-05-14T16:56:53.250 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:52 smithi069 ceph-mon[31507]: pgmap v215: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 29 MiB/s wr, 794 op/s 2024-05-14T16:56:53.257 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '78c5781c', total seconds elapsed: 25.8 2024-05-14T16:56:53.280 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:52 smithi092 ceph-mon[35763]: pgmap v215: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 29 MiB/s wr, 794 op/s 2024-05-14T16:56:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:52 smithi203 ceph-mon[35787]: pgmap v215: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 29 MiB/s wr, 794 op/s 2024-05-14T16:56:54.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:54 smithi069 ceph-mon[31507]: pgmap v216: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 336 op/s 2024-05-14T16:56:54.464 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:54 smithi203 ceph-mon[35787]: pgmap v216: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 336 op/s 2024-05-14T16:56:54.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:54 smithi092 ceph-mon[35763]: pgmap v216: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 336 op/s 2024-05-14T16:56:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 385.0 s, 2627.6 tps, lat 154.740 ms stddev 1392.985 2024-05-14T16:56:56.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:56 smithi069 ceph-mon[31507]: pgmap v217: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-14T16:56:56.706 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:56 smithi092 ceph-mon[35763]: pgmap v217: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-14T16:56:56.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:56 smithi203 ceph-mon[35787]: pgmap v217: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-14T16:56:57.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:57 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:57.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:57 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:57.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:57 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:58.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:58 smithi069 ceph-mon[31507]: pgmap v218: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.6 MiB/s wr, 406 op/s 2024-05-14T16:56:58.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:58 smithi203 ceph-mon[35787]: pgmap v218: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.6 MiB/s wr, 406 op/s 2024-05-14T16:56:58.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:58 smithi092 ceph-mon[35763]: pgmap v218: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.6 MiB/s wr, 406 op/s 2024-05-14T16:56:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.724+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/521330693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a0410f6d0 msgr2=0x7f7a04111ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.724+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/521330693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a0410f6d0 0x7f7a04111ac0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009c10 tx=0x7f79fc02a6e0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.724+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/521330693 shutdown_connections 2024-05-14T16:56:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.724+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/521330693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a0410f6d0 0x7f7a04111ac0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.724+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/521330693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a04101490 0x7f7a041018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.724+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/521330693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a04103b50 0x7f7a04100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.724+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/521330693 >> 172.21.15.69:0/521330693 conn(0x7f7a04077ee0 msgr2=0x7f7a040ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.725+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/521330693 shutdown_connections 2024-05-14T16:56:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.725+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/521330693 wait complete. 2024-05-14T16:56:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.725+0000 7f7a0b1c1640 1 Processor -- start 2024-05-14T16:56:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.725+0000 7f7a0b1c1640 1 -- start start 2024-05-14T16:56:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.725+0000 7f7a0b1c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a04101490 0x7f7a0419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.726+0000 7f7a08f5d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a04101490 0x7f7a0419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.726+0000 7f7a08f5d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a04101490 0x7f7a0419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:56092/0 (socket says 172.21.15.69:56092) 2024-05-14T16:56:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.726+0000 7f7a08f5d640 1 -- 172.21.15.69:0/1451592525 learned_addr learned my addr 172.21.15.69:0/1451592525 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:56:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.726+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a04103b50 0x7f7a0419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.726+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a0410f6d0 0x7f7a0419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.726+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041150a0 con 0x7f7a04103b50 2024-05-14T16:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.726+0000 7f7a0975e640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a0410f6d0 0x7f7a0419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.726+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7a04114f20 con 0x7f7a04101490 2024-05-14T16:56:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.726+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7a04115220 con 0x7f7a0410f6d0 2024-05-14T16:56:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.726+0000 7f7a03fff640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a04103b50 0x7f7a0419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.727+0000 7f7a08f5d640 1 -- 172.21.15.69:0/1451592525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a0410f6d0 msgr2=0x7f7a0419fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.727+0000 7f7a08f5d640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a0410f6d0 0x7f7a0419fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.727+0000 7f7a08f5d640 1 -- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a04103b50 msgr2=0x7f7a0419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.727+0000 7f7a08f5d640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a04103b50 0x7f7a0419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.727+0000 7f7a08f5d640 1 -- 172.21.15.69:0/1451592525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a041a0590 con 0x7f7a04101490 2024-05-14T16:56:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.727+0000 7f7a08f5d640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a04101490 0x7f7a0419b5a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f79f800e9f0 tx=0x7f79f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.727+0000 7f7a01ffb640 1 -- 172.21.15.69:0/1451592525 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f800cd80 con 0x7f7a04101490 2024-05-14T16:57:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.727+0000 7f7a01ffb640 1 -- 172.21.15.69:0/1451592525 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79f80108e0 con 0x7f7a04101490 2024-05-14T16:57:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.727+0000 7f7a01ffb640 1 -- 172.21.15.69:0/1451592525 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8017920 con 0x7f7a04101490 2024-05-14T16:57:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.727+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a041a0880 con 0x7f7a04101490 2024-05-14T16:57:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.728+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a041a8110 con 0x7f7a04101490 2024-05-14T16:57:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.731+0000 7f7a01ffb640 1 -- 172.21.15.69:0/1451592525 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f79f8010430 con 0x7f7a04101490 2024-05-14T16:57:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.731+0000 7f7a0dfba740 1 Processor -- start 2024-05-14T16:57:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.732+0000 7f7a01ffb640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79f40b2ae0 0x7f79f40b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.732+0000 7f7a01ffb640 1 -- 172.21.15.69:0/1451592525 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f79f80e2b50 con 0x7f7a04101490 2024-05-14T16:57:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.732+0000 7f7a0dfba740 1 -- start start 2024-05-14T16:57:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.732+0000 7f7a0975e640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79f40b2ae0 0x7f79f40b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.732+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6660830 0x555fc6660c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.732+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6661150 0x555fc6673da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.732+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc66742e0 0x555fc66766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.732+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555fc65d3c30 con 0x555fc6660830 2024-05-14T16:57:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.732+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555fc65c37d0 con 0x555fc66742e0 2024-05-14T16:57:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.732+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555fc65c7810 con 0x555fc6661150 2024-05-14T16:57:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.733+0000 7f7a08f5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6660830 0x555fc6660c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.733+0000 7f7a08f5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6660830 0x555fc6660c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47364/0 (socket says 172.21.15.69:47364) 2024-05-14T16:57:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.733+0000 7f7a08f5d640 1 -- 172.21.15.69:0/2411348750 learned_addr learned my addr 172.21.15.69:0/2411348750 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.733+0000 7f7a03fff640 1 --2- 172.21.15.69:0/2411348750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6661150 0x555fc6673da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.733+0000 7f7a03fff640 1 -- 172.21.15.69:0/2411348750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc66742e0 msgr2=0x555fc66766d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:57:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.733+0000 7f7a03fff640 1 --2- 172.21.15.69:0/2411348750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc66742e0 0x555fc66766d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.733+0000 7f7a03fff640 1 -- 172.21.15.69:0/2411348750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6660830 msgr2=0x555fc6660c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.733+0000 7f7a0975e640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79f40b2ae0 0x7f79f40b4fa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f79fc006010 tx=0x7f79fc0098e0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.734+0000 7f7a03fff640 1 --2- 172.21.15.69:0/2411348750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6660830 0x555fc6660c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.734+0000 7f7a03fff640 1 -- 172.21.15.69:0/2411348750 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fc62d8210 con 0x555fc6661150 2024-05-14T16:57:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.734+0000 7f7a08f5d640 1 --2- 172.21.15.69:0/2411348750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6660830 0x555fc6660c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.734+0000 7f7a03fff640 1 --2- 172.21.15.69:0/2411348750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6661150 0x555fc6673da0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f79f0005fd0 tx=0x7f79f00113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8161a0a25b6eed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.734+0000 7f79eeffd640 1 -- 172.21.15.69:0/2411348750 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0011e10 con 0x555fc6661150 2024-05-14T16:57:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.734+0000 7f79eeffd640 1 -- 172.21.15.69:0/2411348750 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79f00054d0 con 0x555fc6661150 2024-05-14T16:57:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.734+0000 7f79eeffd640 1 -- 172.21.15.69:0/2411348750 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f00126b0 con 0x555fc6661150 2024-05-14T16:57:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.735+0000 7f7a0dfba740 1 -- 172.21.15.69:0/2411348750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6661150 msgr2=0x555fc6673da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.735+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/2411348750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6661150 0x555fc6673da0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f79f0005fd0 tx=0x7f79f00113e0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.735+0000 7f7a0dfba740 1 -- 172.21.15.69:0/2411348750 shutdown_connections 2024-05-14T16:57:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.735+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/2411348750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc66742e0 0x555fc66766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.735+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/2411348750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6661150 0x555fc6673da0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.735+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/2411348750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6660830 0x555fc6660c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.735+0000 7f7a0dfba740 1 -- 172.21.15.69:0/2411348750 >> 172.21.15.69:0/2411348750 conn(0x555fc665ce00 msgr2=0x555fc665d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.735+0000 7f7a0dfba740 1 -- 172.21.15.69:0/2411348750 shutdown_connections 2024-05-14T16:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.735+0000 7f7a0dfba740 1 -- 172.21.15.69:0/2411348750 wait complete. 2024-05-14T16:57:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.736+0000 7f7a0dfba740 1 -- start start 2024-05-14T16:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.736+0000 7f7a0dfba740 1 Processor -- start 2024-05-14T16:57:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.736+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6660830 0x555fc6673ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.736+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 0x555fc6670f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.736+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc66715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.736+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555fc62a9790 con 0x555fc66742e0 2024-05-14T16:57:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.736+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555fc657f890 con 0x555fc6661150 2024-05-14T16:57:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.736+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555fc6676f50 con 0x555fc6660830 2024-05-14T16:57:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.736+0000 7f7a0dfba740 10 client.0 _ll_register_callbacks cb 0x555fc6582d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a08f5d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6660830 0x555fc6673ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a03fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 0x555fc6670f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a03fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 0x555fc6670f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:56094/0 (socket says 172.21.15.69:56094) 2024-05-14T16:57:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a03fff640 1 -- 172.21.15.69:0/777167906 learned_addr learned my addr 172.21.15.69:0/777167906 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a0975e640 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc66715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a03fff640 1 -- 172.21.15.69:0/777167906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6660830 msgr2=0x555fc6673ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a03fff640 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6660830 0x555fc6673ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a03fff640 1 -- 172.21.15.69:0/777167906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 msgr2=0x555fc66715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a03fff640 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc66715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a03fff640 1 -- 172.21.15.69:0/777167906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fc63a4620 con 0x555fc6661150 2024-05-14T16:57:00.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a0975e640 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc66715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f7a03fff640 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 0x555fc6670f90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f79f0004cc0 tx=0x7f79f0011ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7758dba8f8526f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.737+0000 7f79e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:57:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f79e3fff640 1 -- 172.21.15.69:0/777167906 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0028020 con 0x555fc6661150 2024-05-14T16:57:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f7a0dfba740 1 -- 172.21.15.69:0/777167906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79f00054d0 con 0x555fc6661150 2024-05-14T16:57:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f79e3fff640 1 -- 172.21.15.69:0/777167906 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f79f00050c0 con 0x555fc6661150 2024-05-14T16:57:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f79e3fff640 1 -- 172.21.15.69:0/777167906 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79f0033050 con 0x555fc6661150 2024-05-14T16:57:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f7a0dfba740 10 client.25646 fetch_fsmap learned FSMap version 39 2024-05-14T16:57:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f7a0dfba740 1 -- 172.21.15.69:0/777167906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555fc6682a30 con 0x555fc6661150 2024-05-14T16:57:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f79e3fff640 1 -- 172.21.15.69:0/777167906 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79f00054d0 con 0x555fc6661150 2024-05-14T16:57:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f79e3fff640 1 -- 172.21.15.69:0/777167906 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f79f0024410 con 0x555fc6661150 2024-05-14T16:57:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f7a0dfba740 10 client.25646 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T16:57:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f7a0dfba740 10 client.25646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f7a0dfba740 20 client.25646 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555fc6684330 0x555fc6686770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f7a0dfba740 4 client.25646 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.738+0000 7f7a0dfba740 1 -- 172.21.15.69:0/777167906 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fc65d5b80 con 0x555fc6684330 2024-05-14T16:57:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.739+0000 7f7a08f5d640 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555fc6684330 0x555fc6686770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.739+0000 7f79e3fff640 10 client.25646 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.739+0000 7f7a08f5d640 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555fc6684330 0x555fc6686770 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1cf88ec791c959 server_cookie=9be9f421c990634d in_seq=0 out_seq=0 2024-05-14T16:57:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f79e3fff640 1 -- 172.21.15.69:0/777167906 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555fc65d5b80 con 0x555fc6684330 2024-05-14T16:57:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f79e3fff640 10 client.25646 handle_command_reply: tid=0 2024-05-14T16:57:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 client.25646 shutdown 2024-05-14T16:57:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 -- 172.21.15.69:0/777167906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 msgr2=0x555fc6670f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 0x555fc6670f90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f79f0004cc0 tx=0x7f79f0011ec0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 -- 172.21.15.69:0/777167906 shutdown_connections 2024-05-14T16:57:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555fc6684330 0x555fc6686770 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc66715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 0x555fc6670f90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/777167906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6660830 0x555fc6673ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 -- 172.21.15.69:0/777167906 >> 172.21.15.69:0/777167906 conn(0x555fc665ce00 msgr2=0x555fc665c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f79e3fff640 0 client.25646 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 -- 172.21.15.69:0/777167906 shutdown_connections 2024-05-14T16:57:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 1 -- 172.21.15.69:0/777167906 wait complete. 2024-05-14T16:57:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.740+0000 7f7a0dfba740 20 client.25646 trim_cache size 0 max 16384 2024-05-14T16:57:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.760+0000 7f7a0dfba740 1 Processor -- start 2024-05-14T16:57:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a0dfba740 1 -- start start 2024-05-14T16:57:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6660830 0x555fc667fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6661150 0x555fc666c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc66742e0 0x555fc66737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041aeea0 con 0x555fc6661150 2024-05-14T16:57:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041ac710 con 0x555fc6660830 2024-05-14T16:57:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041aa5c0 con 0x555fc66742e0 2024-05-14T16:57:01.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a03fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6661150 0x555fc666c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a03fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6661150 0x555fc666c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47378/0 (socket says 172.21.15.69:47378) 2024-05-14T16:57:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a0975e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc66742e0 0x555fc66737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a03fff640 1 -- 172.21.15.69:0/2055889258 learned_addr learned my addr 172.21.15.69:0/2055889258 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.761+0000 7f7a08f5d640 1 --2- 172.21.15.69:0/2055889258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6660830 0x555fc667fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.762+0000 7f7a0975e640 1 -- 172.21.15.69:0/2055889258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6660830 msgr2=0x555fc667fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.762+0000 7f7a0975e640 1 --2- 172.21.15.69:0/2055889258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6660830 0x555fc667fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.762+0000 7f7a0975e640 1 -- 172.21.15.69:0/2055889258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6661150 msgr2=0x555fc666c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.762+0000 7f7a0975e640 1 --2- 172.21.15.69:0/2055889258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6661150 0x555fc666c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.762+0000 7f7a0975e640 1 -- 172.21.15.69:0/2055889258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fc6671910 con 0x555fc66742e0 2024-05-14T16:57:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.762+0000 7f7a03fff640 1 --2- 172.21.15.69:0/2055889258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6661150 0x555fc666c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.762+0000 7f7a0975e640 1 --2- 172.21.15.69:0/2055889258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc66742e0 0x555fc66737d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f79fc034040 tx=0x7f79fc03d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=654cf6032a245605 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.762+0000 7f79ee7fc640 1 -- 172.21.15.69:0/2055889258 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc036040 con 0x555fc66742e0 2024-05-14T16:57:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.762+0000 7f79ee7fc640 1 -- 172.21.15.69:0/2055889258 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79fc050070 con 0x555fc66742e0 2024-05-14T16:57:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.762+0000 7f79ee7fc640 1 -- 172.21.15.69:0/2055889258 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc03d460 con 0x555fc66742e0 2024-05-14T16:57:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.763+0000 7f7a0dfba740 1 -- 172.21.15.69:0/2055889258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc66742e0 msgr2=0x555fc66737d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.763+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/2055889258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc66742e0 0x555fc66737d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f79fc034040 tx=0x7f79fc03d000 comp rx=0 tx=0).stop 2024-05-14T16:57:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.763+0000 7f7a0dfba740 1 -- 172.21.15.69:0/2055889258 shutdown_connections 2024-05-14T16:57:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.763+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/2055889258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc66742e0 0x555fc66737d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.763+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/2055889258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc6661150 0x555fc666c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.763+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/2055889258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6660830 0x555fc667fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.763+0000 7f7a0dfba740 1 -- 172.21.15.69:0/2055889258 >> 172.21.15.69:0/2055889258 conn(0x555fc665ce00 msgr2=0x555fc666bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.764+0000 7f7a0dfba740 1 -- 172.21.15.69:0/2055889258 shutdown_connections 2024-05-14T16:57:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.764+0000 7f7a0dfba740 1 -- 172.21.15.69:0/2055889258 wait complete. 2024-05-14T16:57:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.764+0000 7f7a0dfba740 1 -- start start 2024-05-14T16:57:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.764+0000 7f7a0dfba740 1 Processor -- start 2024-05-14T16:57:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6660830 0x555fc667c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 0x555fc66771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0dfba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc6677820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041aac10 con 0x555fc66742e0 2024-05-14T16:57:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041ac040 con 0x555fc6661150 2024-05-14T16:57:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0dfba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7a041ab2a0 con 0x555fc6660830 2024-05-14T16:57:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0dfba740 10 client.0 _ll_register_callbacks cb 0x7f79f0005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0975e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc6677820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0975e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc6677820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47384/0 (socket says 172.21.15.69:47384) 2024-05-14T16:57:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0975e640 1 -- 172.21.15.69:0/3601731990 learned_addr learned my addr 172.21.15.69:0/3601731990 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a03fff640 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 0x555fc66771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0975e640 1 -- 172.21.15.69:0/3601731990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6660830 msgr2=0x555fc667c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0975e640 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6660830 0x555fc667c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0975e640 1 -- 172.21.15.69:0/3601731990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 msgr2=0x555fc66771a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0975e640 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 0x555fc66771a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.765+0000 7f7a0975e640 1 -- 172.21.15.69:0/3601731990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fc66fdbf0 con 0x555fc66742e0 2024-05-14T16:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.766+0000 7f7a0975e640 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc6677820 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f79fc032f90 tx=0x7f79fc030800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b2c3d473e51222 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.766+0000 7f79e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:57:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.766+0000 7f79e3fff640 1 -- 172.21.15.69:0/3601731990 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc036070 con 0x555fc66742e0 2024-05-14T16:57:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.766+0000 7f7a0dfba740 1 -- 172.21.15.69:0/3601731990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79fc050070 con 0x555fc66742e0 2024-05-14T16:57:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.766+0000 7f79e3fff640 1 -- 172.21.15.69:0/3601731990 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f79fc003f80 con 0x555fc66742e0 2024-05-14T16:57:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.766+0000 7f79e3fff640 1 -- 172.21.15.69:0/3601731990 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79fc004180 con 0x555fc66742e0 2024-05-14T16:57:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.766+0000 7f7a0dfba740 10 client.16182 fetch_fsmap learned FSMap version 39 2024-05-14T16:57:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.766+0000 7f7a0dfba740 1 -- 172.21.15.69:0/3601731990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555fc66fed40 con 0x555fc66742e0 2024-05-14T16:57:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.767+0000 7f79e3fff640 1 -- 172.21.15.69:0/3601731990 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79fc050070 con 0x555fc66742e0 2024-05-14T16:57:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.767+0000 7f79e3fff640 1 -- 172.21.15.69:0/3601731990 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f79fc03da70 con 0x555fc66742e0 2024-05-14T16:57:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.767+0000 7f7a0dfba740 10 client.16182 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T16:57:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.767+0000 7f7a0dfba740 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.767+0000 7f7a0dfba740 20 client.16182 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.767+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555fc66ff4a0 0x555fc67018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.767+0000 7f7a0dfba740 4 client.16182 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:57:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.767+0000 7f7a0dfba740 1 -- 172.21.15.69:0/3601731990 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555fc65cf360 con 0x555fc66ff4a0 2024-05-14T16:57:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.767+0000 7f7a08f5d640 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555fc66ff4a0 0x555fc67018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.768+0000 7f79e3fff640 10 client.16182 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.768+0000 7f7a08f5d640 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555fc66ff4a0 0x555fc67018c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48e789988e5820f6 server_cookie=25f6d7eb49d8552b in_seq=0 out_seq=0 2024-05-14T16:57:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f79e3fff640 1 -- 172.21.15.69:0/3601731990 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555fc65cf360 con 0x555fc66ff4a0 2024-05-14T16:57:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f79e3fff640 10 client.16182 handle_command_reply: tid=0 2024-05-14T16:57:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 client.16182 shutdown 2024-05-14T16:57:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 -- 172.21.15.69:0/3601731990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 msgr2=0x555fc6677820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc6677820 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f79fc032f90 tx=0x7f79fc030800 comp rx=0 tx=0).stop 2024-05-14T16:57:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 -- 172.21.15.69:0/3601731990 shutdown_connections 2024-05-14T16:57:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555fc66ff4a0 0x555fc67018c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fc66742e0 0x555fc6677820 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555fc6661150 0x555fc66771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 --2- 172.21.15.69:0/3601731990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555fc6660830 0x555fc667c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 -- 172.21.15.69:0/3601731990 >> 172.21.15.69:0/3601731990 conn(0x555fc665ce00 msgr2=0x555fc666c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f79e3fff640 0 client.16182 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 -- 172.21.15.69:0/3601731990 shutdown_connections 2024-05-14T16:57:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 1 -- 172.21.15.69:0/3601731990 wait complete. 2024-05-14T16:57:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.785+0000 7f7a0dfba740 20 client.16182 trim_cache size 0 max 16384 2024-05-14T16:57:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:57:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:57:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:57:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79f40b2ae0 msgr2=0x7f79f40b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79f40b2ae0 0x7f79f40b4fa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f79fc006010 tx=0x7f79fc0098e0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a04101490 msgr2=0x7f7a0419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a04101490 0x7f7a0419b5a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f79f800e9f0 tx=0x7f79f800eeb0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 shutdown_connections 2024-05-14T16:57:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f79f40b2ae0 0x7f79f40b4fa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a0410f6d0 0x7f7a0419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a04103b50 0x7f7a0419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 --2- 172.21.15.69:0/1451592525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a04101490 0x7f7a0419b5a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 >> 172.21.15.69:0/1451592525 conn(0x7f7a04077ee0 msgr2=0x7f7a04110c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 shutdown_connections 2024-05-14T16:57:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:56:59.786+0000 7f7a0b1c1640 1 -- 172.21.15.69:0/1451592525 wait complete. 2024-05-14T16:57:01.981 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:56:59 smithi069 ceph-mon[31507]: pgmap v219: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.6 MiB/s wr, 402 op/s 2024-05-14T16:57:02.001 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 390.0 s, 3102.6 tps, lat 32.391 ms stddev 34.913 2024-05-14T16:57:02.006 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:56:59 smithi092 ceph-mon[35763]: pgmap v219: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.6 MiB/s wr, 402 op/s 2024-05-14T16:57:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:56:59 smithi203 ceph-mon[35787]: pgmap v219: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.6 MiB/s wr, 402 op/s 2024-05-14T16:57:02.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:01 smithi069 ceph-mon[31507]: pgmap v220: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-05-14T16:57:02.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:01 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:02.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:01 smithi203 ceph-mon[35787]: pgmap v220: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-05-14T16:57:02.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:01 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:02.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:01 smithi092 ceph-mon[35763]: pgmap v220: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-05-14T16:57:02.261 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:01 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:04.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:03 smithi069 ceph-mon[31507]: pgmap v221: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.7 MiB/s wr, 709 op/s 2024-05-14T16:57:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:03 smithi203 ceph-mon[35787]: pgmap v221: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.7 MiB/s wr, 709 op/s 2024-05-14T16:57:04.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:03 smithi092 ceph-mon[35763]: pgmap v221: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.7 MiB/s wr, 709 op/s 2024-05-14T16:57:05.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 395.0 s, 829.8 tps, lat 119.276 ms stddev 148.447 2024-05-14T16:57:06.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:05 smithi069 ceph-mon[31507]: pgmap v222: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.7 MiB/s wr, 708 op/s 2024-05-14T16:57:06.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:05 smithi203 ceph-mon[35787]: pgmap v222: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.7 MiB/s wr, 708 op/s 2024-05-14T16:57:06.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:05 smithi092 ceph-mon[35763]: pgmap v222: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.7 MiB/s wr, 708 op/s 2024-05-14T16:57:07.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:06 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:07.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:06 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:07.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:06 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:08.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:07 smithi069 ceph-mon[31507]: pgmap v223: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 25 MiB/s wr, 2.00k op/s 2024-05-14T16:57:08.252 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:07 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:07 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:08.265 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:07 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:57:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:07 smithi092 ceph-mon[35763]: pgmap v223: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 25 MiB/s wr, 2.00k op/s 2024-05-14T16:57:08.278 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:07 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:08.284 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:07 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:08.290 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:07 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:57:08.295 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:07 smithi203 ceph-mon[35787]: pgmap v223: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 25 MiB/s wr, 2.00k op/s 2024-05-14T16:57:08.301 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:07 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:08.307 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:07 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:08.313 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:07 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:57:10.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:09 smithi069 ceph-mon[31507]: pgmap v224: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2024-05-14T16:57:10.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:09 smithi069 ceph-mon[31507]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T16:57:10.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:09 smithi203 ceph-mon[35787]: pgmap v224: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2024-05-14T16:57:10.293 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:09 smithi203 ceph-mon[35787]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T16:57:10.299 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:09 smithi092 ceph-mon[35763]: pgmap v224: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2024-05-14T16:57:10.304 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:09 smithi092 ceph-mon[35763]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T16:57:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 400.0 s, 2632.6 tps, lat 37.971 ms stddev 45.880 2024-05-14T16:57:12.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:11 smithi069 ceph-mon[31507]: pgmap v226: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2024-05-14T16:57:12.599 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:11 smithi092 ceph-mon[35763]: pgmap v226: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2024-05-14T16:57:12.605 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:11 smithi203 ceph-mon[35787]: pgmap v226: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2024-05-14T16:57:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.911+0000 7f34e819d640 1 -- 172.21.15.69:0/2066352814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34e0111620 msgr2=0x7f34e0109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.911+0000 7f34e819d640 1 --2- 172.21.15.69:0/2066352814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34e0111620 0x7f34e0109f90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009cd0 tx=0x7f34d802a9a0 comp rx=0 tx=0).stop 2024-05-14T16:57:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.915+0000 7f34e819d640 1 -- 172.21.15.69:0/2066352814 shutdown_connections 2024-05-14T16:57:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.915+0000 7f34e819d640 1 --2- 172.21.15.69:0/2066352814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e011c230 0x7f34e011e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.915+0000 7f34e819d640 1 --2- 172.21.15.69:0/2066352814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34e0111620 0x7f34e0109f90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.915+0000 7f34e819d640 1 --2- 172.21.15.69:0/2066352814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0111240 0x7f34e0109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.915+0000 7f34e819d640 1 -- 172.21.15.69:0/2066352814 >> 172.21.15.69:0/2066352814 conn(0x7f34e006db20 msgr2=0x7f34e006df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.916+0000 7f34e819d640 1 -- 172.21.15.69:0/2066352814 shutdown_connections 2024-05-14T16:57:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.916+0000 7f34e819d640 1 -- 172.21.15.69:0/2066352814 wait complete. 2024-05-14T16:57:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.917+0000 7f34e819d640 1 Processor -- start 2024-05-14T16:57:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.917+0000 7f34e819d640 1 -- start start 2024-05-14T16:57:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.917+0000 7f34e819d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34e0111240 0x7f34e0119810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.917+0000 7f34e819d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e011c230 0x7f34e01128e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.918+0000 7f34e5f39640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34e0111240 0x7f34e0119810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.918+0000 7f34e5f39640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34e0111240 0x7f34e0119810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49970/0 (socket says 172.21.15.69:49970) 2024-05-14T16:57:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.918+0000 7f34e5738640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e011c230 0x7f34e01128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.918+0000 7f34e5738640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e011c230 0x7f34e01128e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:49894/0 (socket says 172.21.15.69:49894) 2024-05-14T16:57:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.918+0000 7f34e819d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0112e20 0x7f34e01132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.918+0000 7f34e5738640 1 -- 172.21.15.69:0/3273339750 learned_addr learned my addr 172.21.15.69:0/3273339750 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.918+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0121bd0 con 0x7f34e0111240 2024-05-14T16:57:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.918+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0121a50 con 0x7f34e0112e20 2024-05-14T16:57:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.918+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0121d50 con 0x7f34e011c230 2024-05-14T16:57:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.918+0000 7f34e673a640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0112e20 0x7f34e01132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.919+0000 7f34e673a640 1 -- 172.21.15.69:0/3273339750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e011c230 msgr2=0x7f34e01128e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.919+0000 7f34e673a640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e011c230 0x7f34e01128e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.919+0000 7f34e673a640 1 -- 172.21.15.69:0/3273339750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34e0111240 msgr2=0x7f34e0119810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.919+0000 7f34e673a640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34e0111240 0x7f34e0119810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.919+0000 7f34e673a640 1 -- 172.21.15.69:0/3273339750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e0117330 con 0x7f34e0112e20 2024-05-14T16:57:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.919+0000 7f34e673a640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0112e20 0x7f34e01132d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34e010ecf0 tx=0x7f34d000cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.919+0000 7f34d6ffd640 1 -- 172.21.15.69:0/3273339750 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0004220 con 0x7f34e0112e20 2024-05-14T16:57:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.920+0000 7f34e5738640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e011c230 0x7f34e01128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:57:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.920+0000 7f34e5f39640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34e0111240 0x7f34e0119810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:57:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.920+0000 7f34d6ffd640 1 -- 172.21.15.69:0/3273339750 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d00043c0 con 0x7f34e0112e20 2024-05-14T16:57:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.920+0000 7f34d6ffd640 1 -- 172.21.15.69:0/3273339750 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0003dc0 con 0x7f34e0112e20 2024-05-14T16:57:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.920+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e0117620 con 0x7f34e0112e20 2024-05-14T16:57:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.920+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e0117ac0 con 0x7f34e0112e20 2024-05-14T16:57:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.924+0000 7f34eaf93740 1 Processor -- start 2024-05-14T16:57:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.927+0000 7f34eaf93740 1 -- start start 2024-05-14T16:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.927+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da673388f0 0x55da67338cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.927+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67339210 0x55da67343590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.927+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da67345ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.927+0000 7f34eaf93740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55da672abc30 con 0x55da673388f0 2024-05-14T16:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.927+0000 7f34eaf93740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55da6729b7d0 con 0x55da67339210 2024-05-14T16:57:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.927+0000 7f34eaf93740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55da6729f810 con 0x55da67343ad0 2024-05-14T16:57:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.927+0000 7f34d6ffd640 1 -- 172.21.15.69:0/3273339750 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f34d00077a0 con 0x7f34e0112e20 2024-05-14T16:57:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.928+0000 7f34e5738640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da673388f0 0x55da67338cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.928+0000 7f34e5738640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da673388f0 0x55da67338cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49980/0 (socket says 172.21.15.69:49980) 2024-05-14T16:57:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.928+0000 7f34e5738640 1 -- 172.21.15.69:0/1950601137 learned_addr learned my addr 172.21.15.69:0/1950601137 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.928+0000 7f34e5f39640 1 --2- 172.21.15.69:0/1950601137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67339210 0x55da67343590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.928+0000 7f34e5738640 1 --2- 172.21.15.69:0/1950601137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da67345ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.929+0000 7f34e5738640 1 -- 172.21.15.69:0/1950601137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67339210 msgr2=0x55da67343590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.929+0000 7f34e5738640 1 --2- 172.21.15.69:0/1950601137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67339210 0x55da67343590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.929+0000 7f34e5738640 1 -- 172.21.15.69:0/1950601137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da673388f0 msgr2=0x55da67338cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.929+0000 7f34e5738640 1 --2- 172.21.15.69:0/1950601137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da673388f0 0x55da67338cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.929+0000 7f34e5738640 1 -- 172.21.15.69:0/1950601137 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da66fb0210 con 0x55da67343ad0 2024-05-14T16:57:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.929+0000 7f34d6ffd640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34cc0b2b60 0x7f34cc0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.929+0000 7f34e5738640 1 --2- 172.21.15.69:0/1950601137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da67345ec0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009bf0 tx=0x7f34d802a9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc0477a1228bb22 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.929+0000 7f34d6ffd640 1 -- 172.21.15.69:0/3273339750 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8586+0+0 (secure 0 0 0) 0x7f34d00e22f0 con 0x7f34e0112e20 2024-05-14T16:57:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.930+0000 7f34cbfff640 1 -- 172.21.15.69:0/1950601137 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8034040 con 0x55da67343ad0 2024-05-14T16:57:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.930+0000 7f34cbfff640 1 -- 172.21.15.69:0/1950601137 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d802fda0 con 0x55da67343ad0 2024-05-14T16:57:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.930+0000 7f34e5f39640 1 --2- 172.21.15.69:0/1950601137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67339210 0x55da67343590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.930+0000 7f34cbfff640 1 -- 172.21.15.69:0/1950601137 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8030a60 con 0x55da67343ad0 2024-05-14T16:57:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.931+0000 7f34eaf93740 1 -- 172.21.15.69:0/1950601137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 msgr2=0x55da67345ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.931+0000 7f34eaf93740 1 --2- 172.21.15.69:0/1950601137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da67345ec0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009bf0 tx=0x7f34d802a9a0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.931+0000 7f34eaf93740 1 -- 172.21.15.69:0/1950601137 shutdown_connections 2024-05-14T16:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.931+0000 7f34eaf93740 1 --2- 172.21.15.69:0/1950601137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da67345ec0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.931+0000 7f34eaf93740 1 --2- 172.21.15.69:0/1950601137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67339210 0x55da67343590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.931+0000 7f34eaf93740 1 --2- 172.21.15.69:0/1950601137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da673388f0 0x55da67338cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.931+0000 7f34eaf93740 1 -- 172.21.15.69:0/1950601137 >> 172.21.15.69:0/1950601137 conn(0x55da67334e00 msgr2=0x55da673351e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.931+0000 7f34e5f39640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34cc0b2b60 0x7f34cc0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.932+0000 7f34eaf93740 1 -- 172.21.15.69:0/1950601137 shutdown_connections 2024-05-14T16:57:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.932+0000 7f34eaf93740 1 -- 172.21.15.69:0/1950601137 wait complete. 2024-05-14T16:57:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.932+0000 7f34eaf93740 1 -- start start 2024-05-14T16:57:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.933+0000 7f34eaf93740 1 Processor -- start 2024-05-14T16:57:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.933+0000 7f34e5f39640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34cc0b2b60 0x7f34cc0b5020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55da67339e90 tx=0x7f34dc008040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.933+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da67343430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.933+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 0x55da67340720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.933+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da67340c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.933+0000 7f34eaf93740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55da66f81790 con 0x55da67339210 2024-05-14T16:57:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.933+0000 7f34eaf93740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55da67257890 con 0x55da673388f0 2024-05-14T16:57:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.933+0000 7f34eaf93740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55da67346740 con 0x55da67343ad0 2024-05-14T16:57:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.933+0000 7f34eaf93740 10 client.0 _ll_register_callbacks cb 0x55da6725ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5f39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da67343430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5f39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da67343430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:40740/0 (socket says 172.21.15.69:40740) 2024-05-14T16:57:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5f39640 1 -- 172.21.15.69:0/3325148568 learned_addr learned my addr 172.21.15.69:0/3325148568 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5738640 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 0x55da67340720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e673a640 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da67340c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5f39640 1 -- 172.21.15.69:0/3325148568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 msgr2=0x55da67340c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5f39640 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da67340c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5f39640 1 -- 172.21.15.69:0/3325148568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 msgr2=0x55da67340720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5f39640 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 0x55da67340720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5f39640 1 -- 172.21.15.69:0/3325148568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da6707c620 con 0x55da673388f0 2024-05-14T16:57:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5738640 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 0x55da67340720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34e5f39640 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da67343430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34dc006a30 tx=0x7f34dc0046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff6b4249b828854 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.934+0000 7f34c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:57:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34c97fa640 1 -- 172.21.15.69:0/3325148568 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc01a050 con 0x55da673388f0 2024-05-14T16:57:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34eaf93740 1 -- 172.21.15.69:0/3325148568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d802fda0 con 0x55da673388f0 2024-05-14T16:57:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34c97fa640 1 -- 172.21.15.69:0/3325148568 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8586+0+0 (secure 0 0 0) 0x7f34dc014750 con 0x55da673388f0 2024-05-14T16:57:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34c97fa640 1 -- 172.21.15.69:0/3325148568 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f34dc005970 con 0x55da673388f0 2024-05-14T16:57:14.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34eaf93740 10 client.25661 fetch_fsmap learned FSMap version 40 2024-05-14T16:57:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34eaf93740 1 -- 172.21.15.69:0/3325148568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55da673522a0 con 0x55da673388f0 2024-05-14T16:57:14.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34c97fa640 1 -- 172.21.15.69:0/3325148568 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d802fda0 con 0x55da673388f0 2024-05-14T16:57:14.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34c97fa640 1 -- 172.21.15.69:0/3325148568 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f34dc006160 con 0x55da673388f0 2024-05-14T16:57:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34eaf93740 10 client.25661 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T16:57:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34eaf93740 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34eaf93740 20 client.25661 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:14.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da67353780 0x55da67355bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:14.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34eaf93740 4 client.25661 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34eaf93740 1 -- 172.21.15.69:0/3325148568 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da672adb80 con 0x55da67353780 2024-05-14T16:57:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.935+0000 7f34e5738640 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da67353780 0x55da67355bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:14.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.936+0000 7f34c97fa640 10 client.25661 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.936+0000 7f34e5738640 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da67353780 0x55da67355bb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaee009eb3210c6e server_cookie=fdb3332288fb45d7 in_seq=0 out_seq=0 2024-05-14T16:57:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34c97fa640 1 -- 172.21.15.69:0/3325148568 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da672adb80 con 0x55da67353780 2024-05-14T16:57:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34c97fa640 10 client.25661 handle_command_reply: tid=0 2024-05-14T16:57:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34eaf93740 1 client.25661 shutdown 2024-05-14T16:57:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34eaf93740 1 -- 172.21.15.69:0/3325148568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 msgr2=0x55da67343430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da67343430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34dc006a30 tx=0x7f34dc0046e0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34eaf93740 1 -- 172.21.15.69:0/3325148568 shutdown_connections 2024-05-14T16:57:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da67353780 0x55da67355bb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da67340c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 0x55da67340720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3325148568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da67343430 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34c97fa640 0 client.25661 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.937+0000 7f34eaf93740 1 -- 172.21.15.69:0/3325148568 >> 172.21.15.69:0/3325148568 conn(0x55da67334e00 msgr2=0x55da67334100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.938+0000 7f34eaf93740 1 -- 172.21.15.69:0/3325148568 shutdown_connections 2024-05-14T16:57:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.938+0000 7f34eaf93740 1 -- 172.21.15.69:0/3325148568 wait complete. 2024-05-14T16:57:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.938+0000 7f34eaf93740 20 client.25661 trim_cache size 0 max 16384 2024-05-14T16:57:14.727 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:13 smithi092 ceph-mon[35763]: pgmap v227: 97 pgs: 97 active+clean; 3.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 MiB/s wr, 1.91k op/s 2024-05-14T16:57:14.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:13 smithi092 ceph-mon[35763]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T16:57:14.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:13 smithi203 ceph-mon[35787]: pgmap v227: 97 pgs: 97 active+clean; 3.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 MiB/s wr, 1.91k op/s 2024-05-14T16:57:14.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:13 smithi203 ceph-mon[35787]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T16:57:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.960+0000 7f34eaf93740 1 Processor -- start 2024-05-14T16:57:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.960+0000 7f34eaf93740 1 -- start start 2024-05-14T16:57:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da6734f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 0x55da67342f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da6735b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34eaf93740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01c87e0 con 0x55da67339210 2024-05-14T16:57:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34eaf93740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01c6050 con 0x55da673388f0 2024-05-14T16:57:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34eaf93740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01c3ec0 con 0x55da67343ad0 2024-05-14T16:57:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e5f39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da6734f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e5f39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da6734f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:40748/0 (socket says 172.21.15.69:40748) 2024-05-14T16:57:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e5f39640 1 -- 172.21.15.69:0/2201494531 learned_addr learned my addr 172.21.15.69:0/2201494531 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e673a640 1 --2- 172.21.15.69:0/2201494531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da6735b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e5738640 1 --2- 172.21.15.69:0/2201494531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 0x55da67342f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e673a640 1 -- 172.21.15.69:0/2201494531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 msgr2=0x55da6734f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e673a640 1 --2- 172.21.15.69:0/2201494531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da6734f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e673a640 1 -- 172.21.15.69:0/2201494531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 msgr2=0x55da67342f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e673a640 1 --2- 172.21.15.69:0/2201494531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 0x55da67342f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e673a640 1 -- 172.21.15.69:0/2201494531 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da67253590 con 0x55da67343ad0 2024-05-14T16:57:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.961+0000 7f34e5738640 1 --2- 172.21.15.69:0/2201494531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 0x55da67342f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.962+0000 7f34e673a640 1 --2- 172.21.15.69:0/2201494531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da6735b510 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34d00df9a0 tx=0x7f34d00a9d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ad3e51d6f801943 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.962+0000 7f34cb7fe640 1 -- 172.21.15.69:0/2201494531 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d001d030 con 0x55da67343ad0 2024-05-14T16:57:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.962+0000 7f34cb7fe640 1 -- 172.21.15.69:0/2201494531 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d00910e0 con 0x55da67343ad0 2024-05-14T16:57:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.962+0000 7f34cb7fe640 1 -- 172.21.15.69:0/2201494531 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d00b2160 con 0x55da67343ad0 2024-05-14T16:57:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.962+0000 7f34eaf93740 1 -- 172.21.15.69:0/2201494531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 msgr2=0x55da6735b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.962+0000 7f34eaf93740 1 --2- 172.21.15.69:0/2201494531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da6735b510 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34d00df9a0 tx=0x7f34d00a9d00 comp rx=0 tx=0).stop 2024-05-14T16:57:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.963+0000 7f34eaf93740 1 -- 172.21.15.69:0/2201494531 shutdown_connections 2024-05-14T16:57:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.963+0000 7f34eaf93740 1 --2- 172.21.15.69:0/2201494531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67343ad0 0x55da6735b510 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.963+0000 7f34eaf93740 1 --2- 172.21.15.69:0/2201494531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da67339210 0x55da67342f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.963+0000 7f34eaf93740 1 --2- 172.21.15.69:0/2201494531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da673388f0 0x55da6734f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.963+0000 7f34eaf93740 1 -- 172.21.15.69:0/2201494531 >> 172.21.15.69:0/2201494531 conn(0x55da67334e00 msgr2=0x55da6733ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.963+0000 7f34eaf93740 1 -- 172.21.15.69:0/2201494531 shutdown_connections 2024-05-14T16:57:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.963+0000 7f34eaf93740 1 -- 172.21.15.69:0/2201494531 wait complete. 2024-05-14T16:57:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.963+0000 7f34eaf93740 1 -- start start 2024-05-14T16:57:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.963+0000 7f34eaf93740 1 Processor -- start 2024-05-14T16:57:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da673388f0 0x55da6734c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67339210 0x55da67347420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34eaf93740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67343ad0 0x55da67347b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:15.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34eaf93740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01c4510 con 0x55da673388f0 2024-05-14T16:57:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34eaf93740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01c5980 con 0x55da67343ad0 2024-05-14T16:57:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34eaf93740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01c4be0 con 0x55da67339210 2024-05-14T16:57:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34eaf93740 10 client.0 _ll_register_callbacks cb 0x7f34dc00eef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34e673a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67343ad0 0x55da67347b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34e5738640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67339210 0x55da67347420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34e673a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67343ad0 0x55da67347b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:40752/0 (socket says 172.21.15.69:40752) 2024-05-14T16:57:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34e673a640 1 -- 172.21.15.69:0/3714069694 learned_addr learned my addr 172.21.15.69:0/3714069694 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34e5738640 1 -- 172.21.15.69:0/3714069694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67343ad0 msgr2=0x55da67347b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34e5738640 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67343ad0 0x55da67347b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34e5738640 1 -- 172.21.15.69:0/3714069694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da673388f0 msgr2=0x55da6734c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34e5738640 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da673388f0 0x55da6734c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.964+0000 7f34e5738640 1 -- 172.21.15.69:0/3714069694 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da673bd0c0 con 0x55da67339210 2024-05-14T16:57:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34e5738640 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67339210 0x55da67347420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f34d8002230 tx=0x7f34d8030660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=223f0ec2aac87226 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:57:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34c97fa640 1 -- 172.21.15.69:0/3714069694 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8055060 con 0x55da67339210 2024-05-14T16:57:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34c97fa640 1 -- 172.21.15.69:0/3714069694 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8586+0+0 (secure 0 0 0) 0x7f34d8044040 con 0x55da67339210 2024-05-14T16:57:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34eaf93740 1 -- 172.21.15.69:0/3714069694 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da672a7360 con 0x55da67339210 2024-05-14T16:57:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34c97fa640 1 -- 172.21.15.69:0/3714069694 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f34d8034050 con 0x55da67339210 2024-05-14T16:57:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34eaf93740 10 client.25594 fetch_fsmap learned FSMap version 40 2024-05-14T16:57:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34eaf93740 1 -- 172.21.15.69:0/3714069694 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55da673be250 con 0x55da67339210 2024-05-14T16:57:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34c97fa640 1 -- 172.21.15.69:0/3714069694 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55da672a7360 con 0x55da67339210 2024-05-14T16:57:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34c97fa640 1 -- 172.21.15.69:0/3714069694 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f34d8050ce0 con 0x55da67339210 2024-05-14T16:57:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34eaf93740 10 client.25594 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T16:57:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34eaf93740 10 client.25594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34eaf93740 20 client.25594 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da673be8a0 0x55da673c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34eaf93740 4 client.25594 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:57:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.965+0000 7f34eaf93740 1 -- 172.21.15.69:0/3714069694 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da672215f0 con 0x55da673be8a0 2024-05-14T16:57:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.966+0000 7f34e5f39640 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da673be8a0 0x55da673c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.966+0000 7f34c97fa640 10 client.25594 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.966+0000 7f34e5f39640 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da673be8a0 0x55da673c0c90 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93af55ec69a93ee5 server_cookie=448fb00f58a1de42 in_seq=0 out_seq=0 2024-05-14T16:57:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34c97fa640 1 -- 172.21.15.69:0/3714069694 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55da672215f0 con 0x55da673be8a0 2024-05-14T16:57:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34c97fa640 10 client.25594 handle_command_reply: tid=0 2024-05-14T16:57:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34eaf93740 1 client.25594 shutdown 2024-05-14T16:57:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34eaf93740 1 -- 172.21.15.69:0/3714069694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67339210 msgr2=0x55da67347420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67339210 0x55da67347420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f34d8002230 tx=0x7f34d8030660 comp rx=0 tx=0).stop 2024-05-14T16:57:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34eaf93740 1 -- 172.21.15.69:0/3714069694 shutdown_connections 2024-05-14T16:57:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55da673be8a0 0x55da673c0c90 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da67343ad0 0x55da67347b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55da67339210 0x55da67347420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34c97fa640 0 client.25594 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34eaf93740 1 --2- 172.21.15.69:0/3714069694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da673388f0 0x55da6734c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.983+0000 7f34eaf93740 1 -- 172.21.15.69:0/3714069694 >> 172.21.15.69:0/3714069694 conn(0x55da67334e00 msgr2=0x55da6734e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34eaf93740 1 -- 172.21.15.69:0/3714069694 shutdown_connections 2024-05-14T16:57:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34eaf93740 1 -- 172.21.15.69:0/3714069694 wait complete. 2024-05-14T16:57:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34eaf93740 20 client.25594 trim_cache size 0 max 16384 2024-05-14T16:57:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:57:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:57:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:57:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34cc0b2b60 msgr2=0x7f34cc0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34cc0b2b60 0x7f34cc0b5020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55da67339e90 tx=0x7f34dc008040 comp rx=0 tx=0).stop 2024-05-14T16:57:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0112e20 msgr2=0x7f34e01132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0112e20 0x7f34e01132d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34e010ecf0 tx=0x7f34d000cba0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 shutdown_connections 2024-05-14T16:57:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34cc0b2b60 0x7f34cc0b5020 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0112e20 0x7f34e01132d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e011c230 0x7f34e01128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 --2- 172.21.15.69:0/3273339750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34e0111240 0x7f34e0119810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 >> 172.21.15.69:0/3273339750 conn(0x7f34e006db20 msgr2=0x7f34e010ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 shutdown_connections 2024-05-14T16:57:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:12.984+0000 7f34e819d640 1 -- 172.21.15.69:0/3273339750 wait complete. 2024-05-14T16:57:15.722 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:13 smithi069 ceph-mon[31507]: pgmap v227: 97 pgs: 97 active+clean; 3.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 MiB/s wr, 1.91k op/s 2024-05-14T16:57:15.728 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:13 smithi069 ceph-mon[31507]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T16:57:15.733 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 405.0 s, 3246.8 tps, lat 31.143 ms stddev 55.352 2024-05-14T16:57:16.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:15 smithi069 ceph-mon[31507]: pgmap v229: 97 pgs: 97 active+clean; 3.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 444 op/s 2024-05-14T16:57:16.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:15 smithi203 ceph-mon[35787]: pgmap v229: 97 pgs: 97 active+clean; 3.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 444 op/s 2024-05-14T16:57:16.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:15 smithi092 ceph-mon[35763]: pgmap v229: 97 pgs: 97 active+clean; 3.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 444 op/s 2024-05-14T16:57:18.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:17 smithi069 ceph-mon[31507]: pgmap v230: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.6 MiB/s wr, 948 op/s 2024-05-14T16:57:18.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:17 smithi203 ceph-mon[35787]: pgmap v230: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.6 MiB/s wr, 948 op/s 2024-05-14T16:57:18.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:17 smithi092 ceph-mon[35763]: pgmap v230: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.6 MiB/s wr, 948 op/s 2024-05-14T16:57:20.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:19 smithi069 ceph-mon[31507]: pgmap v231: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 765 op/s 2024-05-14T16:57:20.213 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:19 smithi203 ceph-mon[35787]: pgmap v231: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 765 op/s 2024-05-14T16:57:20.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:19 smithi092 ceph-mon[35763]: pgmap v231: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 765 op/s 2024-05-14T16:57:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 410.0 s, 2289.6 tps, lat 43.592 ms stddev 51.013 2024-05-14T16:57:22.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:21 smithi069 ceph-mon[31507]: pgmap v232: 97 pgs: 97 active+clean; 3.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.5 MiB/s wr, 812 op/s 2024-05-14T16:57:22.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:21 smithi092 ceph-mon[35763]: pgmap v232: 97 pgs: 97 active+clean; 3.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.5 MiB/s wr, 812 op/s 2024-05-14T16:57:22.278 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:21 smithi203 ceph-mon[35787]: pgmap v232: 97 pgs: 97 active+clean; 3.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.5 MiB/s wr, 812 op/s 2024-05-14T16:57:23.737 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 31.3 2024-05-14T16:57:23.771 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:57:24.148 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:57:24.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:23 smithi069 ceph-mon[31507]: pgmap v233: 97 pgs: 97 active+clean; 3.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-14T16:57:24.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:23 smithi203 ceph-mon[35787]: pgmap v233: 97 pgs: 97 active+clean; 3.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-14T16:57:24.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:23 smithi092 ceph-mon[35763]: pgmap v233: 97 pgs: 97 active+clean; 3.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-14T16:57:25.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 415.0 s, 2938.8 tps, lat 33.830 ms stddev 37.481 2024-05-14T16:57:25.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:25 smithi069 ceph-mon[31507]: pgmap v234: 97 pgs: 97 active+clean; 3.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-05-14T16:57:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.115+0000 7fe50ffff640 1 -- 172.21.15.69:0/2213035597 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0318a0 con 0x7fe510102a40 2024-05-14T16:57:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.116+0000 7fe517672640 1 -- 172.21.15.69:0/2213035597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe510102a40 msgr2=0x7fe510102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.116+0000 7fe517672640 1 --2- 172.21.15.69:0/2213035597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe510102a40 0x7fe510102ec0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009930 tx=0x7fe4fc02a420 comp rx=0 tx=0).stop 2024-05-14T16:57:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.116+0000 7fe517672640 1 -- 172.21.15.69:0/2213035597 shutdown_connections 2024-05-14T16:57:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.116+0000 7fe517672640 1 --2- 172.21.15.69:0/2213035597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe510103400 0x7fe51010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.116+0000 7fe517672640 1 --2- 172.21.15.69:0/2213035597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe510102a40 0x7fe510102ec0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.116+0000 7fe517672640 1 --2- 172.21.15.69:0/2213035597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe510108a50 0x7fe510108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.116+0000 7fe517672640 1 -- 172.21.15.69:0/2213035597 >> 172.21.15.69:0/2213035597 conn(0x7fe5100fe510 msgr2=0x7fe510100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.116+0000 7fe517672640 1 -- 172.21.15.69:0/2213035597 shutdown_connections 2024-05-14T16:57:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.116+0000 7fe517672640 1 -- 172.21.15.69:0/2213035597 wait complete. 2024-05-14T16:57:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe517672640 1 Processor -- start 2024-05-14T16:57:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe517672640 1 -- start start 2024-05-14T16:57:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe517672640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe510102a40 0x7fe51019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe517672640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe510103400 0x7fe51019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe517672640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe510108a50 0x7fe51019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe517672640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe510112d60 con 0x7fe510103400 2024-05-14T16:57:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe517672640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe510112be0 con 0x7fe510102a40 2024-05-14T16:57:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe517672640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe510112ee0 con 0x7fe510108a50 2024-05-14T16:57:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe514c0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe510103400 0x7fe51019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe51540e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe510102a40 0x7fe51019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe51540e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe510102a40 0x7fe51019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:44302/0 (socket says 172.21.15.69:44302) 2024-05-14T16:57:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe51540e640 1 -- 172.21.15.69:0/4270494935 learned_addr learned my addr 172.21.15.69:0/4270494935 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe515c0f640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe510108a50 0x7fe51019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe514c0d640 1 -- 172.21.15.69:0/4270494935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe510108a50 msgr2=0x7fe51019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe514c0d640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe510108a50 0x7fe51019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe514c0d640 1 -- 172.21.15.69:0/4270494935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe510102a40 msgr2=0x7fe51019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe514c0d640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe510102a40 0x7fe51019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe514c0d640 1 -- 172.21.15.69:0/4270494935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5101a0410 con 0x7fe510103400 2024-05-14T16:57:26.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe515c0f640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe510108a50 0x7fe51019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.117+0000 7fe51540e640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe510102a40 0x7fe51019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.118+0000 7fe514c0d640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe510103400 0x7fe51019b960 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc031440 tx=0x7fe4fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:26.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.118+0000 7fe50e7fc640 1 -- 172.21.15.69:0/4270494935 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc03f070 con 0x7fe510103400 2024-05-14T16:57:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.118+0000 7fe517672640 1 -- 172.21.15.69:0/4270494935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5101a06a0 con 0x7fe510103400 2024-05-14T16:57:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.118+0000 7fe50e7fc640 1 -- 172.21.15.69:0/4270494935 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4fc003aa0 con 0x7fe510103400 2024-05-14T16:57:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.118+0000 7fe50e7fc640 1 -- 172.21.15.69:0/4270494935 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc035070 con 0x7fe510103400 2024-05-14T16:57:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.118+0000 7fe517672640 1 -- 172.21.15.69:0/4270494935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5101a7f90 con 0x7fe510103400 2024-05-14T16:57:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.122+0000 7fe50e7fc640 1 -- 172.21.15.69:0/4270494935 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe4fc03b460 con 0x7fe510103400 2024-05-14T16:57:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.123+0000 7fe50e7fc640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe5000b2a10 0x7fe5000b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.123+0000 7fe50e7fc640 1 -- 172.21.15.69:0/4270494935 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fe4fc102af0 con 0x7fe510103400 2024-05-14T16:57:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.123+0000 7fe51540e640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe5000b2a10 0x7fe5000b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.123+0000 7fe51a393740 1 Processor -- start 2024-05-14T16:57:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.123+0000 7fe51a393740 1 -- start start 2024-05-14T16:57:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe51540e640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe5000b2a10 0x7fe5000b4ed0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe5040049d0 tx=0x7fe504009500 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe51a393740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d08f0 0x5565f75d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe51a393740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d1210 0x5565f75db590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe51a393740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75dbad0 0x5565f75ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe51a393740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5565f7543c30 con 0x5565f75d1210 2024-05-14T16:57:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe51a393740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5565f75337d0 con 0x5565f75d08f0 2024-05-14T16:57:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe51a393740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5565f7537810 con 0x5565f75dbad0 2024-05-14T16:57:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe514c0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d1210 0x5565f75db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe514c0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d1210 0x5565f75db590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37956/0 (socket says 172.21.15.69:37956) 2024-05-14T16:57:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe514c0d640 1 -- 172.21.15.69:0/4046268264 learned_addr learned my addr 172.21.15.69:0/4046268264 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe515c0f640 1 --2- 172.21.15.69:0/4046268264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75dbad0 0x5565f75ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe51540e640 1 --2- 172.21.15.69:0/4046268264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d08f0 0x5565f75d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe514c0d640 1 -- 172.21.15.69:0/4046268264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75dbad0 msgr2=0x5565f75ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe514c0d640 1 --2- 172.21.15.69:0/4046268264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75dbad0 0x5565f75ddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe514c0d640 1 -- 172.21.15.69:0/4046268264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d08f0 msgr2=0x5565f75d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe514c0d640 1 --2- 172.21.15.69:0/4046268264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d08f0 0x5565f75d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe514c0d640 1 -- 172.21.15.69:0/4046268264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f7248210 con 0x5565f75d1210 2024-05-14T16:57:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe514c0d640 1 --2- 172.21.15.69:0/4046268264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d1210 0x5565f75db590 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0c5e40 tx=0x7fe4fc0c95b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12edec3b38d31f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe4fb7fe640 1 -- 172.21.15.69:0/4046268264 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc048040 con 0x5565f75d1210 2024-05-14T16:57:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe4fb7fe640 1 -- 172.21.15.69:0/4046268264 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4fc0ce170 con 0x5565f75d1210 2024-05-14T16:57:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.124+0000 7fe4fb7fe640 1 -- 172.21.15.69:0/4046268264 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0c9020 con 0x5565f75d1210 2024-05-14T16:57:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.125+0000 7fe51a393740 1 -- 172.21.15.69:0/4046268264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d1210 msgr2=0x5565f75db590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.125+0000 7fe51a393740 1 --2- 172.21.15.69:0/4046268264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d1210 0x5565f75db590 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0c5e40 tx=0x7fe4fc0c95b0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.125+0000 7fe51a393740 1 -- 172.21.15.69:0/4046268264 shutdown_connections 2024-05-14T16:57:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.125+0000 7fe51a393740 1 --2- 172.21.15.69:0/4046268264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75dbad0 0x5565f75ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.125+0000 7fe51a393740 1 --2- 172.21.15.69:0/4046268264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d1210 0x5565f75db590 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.125+0000 7fe51a393740 1 --2- 172.21.15.69:0/4046268264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d08f0 0x5565f75d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.125+0000 7fe51a393740 1 -- 172.21.15.69:0/4046268264 >> 172.21.15.69:0/4046268264 conn(0x5565f75cce00 msgr2=0x5565f75cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.125+0000 7fe51a393740 1 -- 172.21.15.69:0/4046268264 shutdown_connections 2024-05-14T16:57:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.125+0000 7fe51a393740 1 -- 172.21.15.69:0/4046268264 wait complete. 2024-05-14T16:57:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51a393740 1 -- start start 2024-05-14T16:57:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51a393740 1 Processor -- start 2024-05-14T16:57:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51a393740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75db500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51a393740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 0x5565f75d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51a393740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75dbad0 0x5565f75d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51a393740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5565f7219790 con 0x5565f75d08f0 2024-05-14T16:57:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51a393740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5565f74ef890 con 0x5565f75dbad0 2024-05-14T16:57:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51a393740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5565f75de4c0 con 0x5565f75d1210 2024-05-14T16:57:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51a393740 10 client.0 _ll_register_callbacks cb 0x5565f74f2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51540e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75db500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.126+0000 7fe51540e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75db500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37966/0 (socket says 172.21.15.69:37966) 2024-05-14T16:57:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe51540e640 1 -- 172.21.15.69:0/2825096361 learned_addr learned my addr 172.21.15.69:0/2825096361 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:26.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe514c0d640 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 0x5565f75d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe515c0f640 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75dbad0 0x5565f75d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe51540e640 1 -- 172.21.15.69:0/2825096361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 msgr2=0x5565f75d87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe51540e640 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 0x5565f75d87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe51540e640 1 -- 172.21.15.69:0/2825096361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75dbad0 msgr2=0x5565f75d8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe51540e640 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75dbad0 0x5565f75d8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe51540e640 1 -- 172.21.15.69:0/2825096361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f7314620 con 0x5565f75d08f0 2024-05-14T16:57:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe514c0d640 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 0x5565f75d87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe51540e640 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75db500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe50401b8d0 tx=0x7fe504013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853b4582d7b6336d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.127+0000 7fe4f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:57:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.128+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/2825096361 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50402a060 con 0x5565f75d08f0 2024-05-14T16:57:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.128+0000 7fe51a393740 1 -- 172.21.15.69:0/2825096361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4fc0ce170 con 0x5565f75d08f0 2024-05-14T16:57:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.129+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/2825096361 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fe50400e930 con 0x5565f75d08f0 2024-05-14T16:57:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.129+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/2825096361 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe50400eb70 con 0x5565f75d08f0 2024-05-14T16:57:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.129+0000 7fe51a393740 10 client.16206 fetch_fsmap learned FSMap version 40 2024-05-14T16:57:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.129+0000 7fe51a393740 1 -- 172.21.15.69:0/2825096361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5565f75ea040 con 0x5565f75d08f0 2024-05-14T16:57:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.130+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/2825096361 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4fc0ce170 con 0x5565f75d08f0 2024-05-14T16:57:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.130+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/2825096361 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fe504018040 con 0x5565f75d08f0 2024-05-14T16:57:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.130+0000 7fe51a393740 10 client.16206 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T16:57:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.130+0000 7fe51a393740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.130+0000 7fe51a393740 20 client.16206 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.130+0000 7fe51a393740 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f75eb8f0 0x5565f75edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.130+0000 7fe51a393740 4 client.16206 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.130+0000 7fe51a393740 1 -- 172.21.15.69:0/2825096361 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565f7545b80 con 0x5565f75eb8f0 2024-05-14T16:57:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.130+0000 7fe514c0d640 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f75eb8f0 0x5565f75edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.131+0000 7fe4f8ff9640 10 client.16206 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.131+0000 7fe514c0d640 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f75eb8f0 0x5565f75edd90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781416cbfd7abe62 server_cookie=980cd074826246af in_seq=0 out_seq=0 2024-05-14T16:57:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.131+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/2825096361 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5565f7545b80 con 0x5565f75eb8f0 2024-05-14T16:57:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.131+0000 7fe4f8ff9640 10 client.16206 handle_command_reply: tid=0 2024-05-14T16:57:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.131+0000 7fe51a393740 1 client.16206 shutdown 2024-05-14T16:57:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 1 -- 172.21.15.69:0/2825096361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 msgr2=0x5565f75db500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75db500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe50401b8d0 tx=0x7fe504013040 comp rx=0 tx=0).stop 2024-05-14T16:57:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 1 -- 172.21.15.69:0/2825096361 shutdown_connections 2024-05-14T16:57:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f75eb8f0 0x5565f75edd90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75dbad0 0x5565f75d8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 0x5565f75d87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 1 --2- 172.21.15.69:0/2825096361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75db500 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 1 -- 172.21.15.69:0/2825096361 >> 172.21.15.69:0/2825096361 conn(0x5565f75cce00 msgr2=0x5565f75cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe4f8ff9640 0 client.16206 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 1 -- 172.21.15.69:0/2825096361 shutdown_connections 2024-05-14T16:57:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 1 -- 172.21.15.69:0/2825096361 wait complete. 2024-05-14T16:57:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.132+0000 7fe51a393740 20 client.16206 trim_cache size 0 max 16384 2024-05-14T16:57:28.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:25 smithi092 ceph-mon[35763]: pgmap v234: 97 pgs: 97 active+clean; 3.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-05-14T16:57:28.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:26 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/541437598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:57:28.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:27 smithi092 ceph-mon[35763]: pgmap v235: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 11 MiB/s wr, 965 op/s 2024-05-14T16:57:28.396 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:25 smithi203 ceph-mon[35787]: pgmap v234: 97 pgs: 97 active+clean; 3.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-05-14T16:57:28.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:26 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/541437598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:57:28.408 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:27 smithi203 ceph-mon[35787]: pgmap v235: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 11 MiB/s wr, 965 op/s 2024-05-14T16:57:28.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.135+0000 7fd81d798700 1 -- 172.21.15.69:0/2415973346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818108510 msgr2=0x7fd8181088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:28.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.135+0000 7fd81d798700 1 --2- 172.21.15.69:0/2415973346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818108510 0x7fd8181088f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd808009990 tx=0x7fd808009ca0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.135+0000 7fd81d798700 1 -- 172.21.15.69:0/2415973346 shutdown_connections 2024-05-14T16:57:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.135+0000 7fd81d798700 1 --2- 172.21.15.69:0/2415973346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818108510 0x7fd8181088f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.135+0000 7fd81d798700 1 --2- 172.21.15.69:0/2415973346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd818102df0 0x7fd81810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.135+0000 7fd81d798700 1 --2- 172.21.15.69:0/2415973346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd818102430 0x7fd8181028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.135+0000 7fd81d798700 1 -- 172.21.15.69:0/2415973346 >> 172.21.15.69:0/2415973346 conn(0x7fd8180793b0 msgr2=0x7fd8180797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.135+0000 7fd81d798700 1 -- 172.21.15.69:0/2415973346 shutdown_connections 2024-05-14T16:57:28.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.135+0000 7fd81d798700 1 -- 172.21.15.69:0/2415973346 wait complete. 2024-05-14T16:57:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.136+0000 7fd81d798700 1 Processor -- start 2024-05-14T16:57:28.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.136+0000 7fd81d798700 1 -- start start 2024-05-14T16:57:28.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.136+0000 7fd81d798700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818102430 0x7fd81819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.136+0000 7fd81d798700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd818102df0 0x7fd81819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.136+0000 7fd81d798700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd818108510 0x7fd818199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.136+0000 7fd81d798700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd818112610 con 0x7fd818102430 2024-05-14T16:57:28.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.136+0000 7fd81d798700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd818112490 con 0x7fd818102df0 2024-05-14T16:57:28.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.136+0000 7fd81d798700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd818112790 con 0x7fd818108510 2024-05-14T16:57:28.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd816ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818102430 0x7fd81819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd816ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818102430 0x7fd81819f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37974/0 (socket says 172.21.15.69:37974) 2024-05-14T16:57:28.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd816ffd700 1 -- 172.21.15.69:0/541437598 learned_addr learned my addr 172.21.15.69:0/541437598 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:28.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd8177fe700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd818108510 0x7fd818199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd8167fc700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd818102df0 0x7fd81819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd8177fe700 1 -- 172.21.15.69:0/541437598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd818102df0 msgr2=0x7fd81819fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:28.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd8177fe700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd818102df0 0x7fd81819fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd8177fe700 1 -- 172.21.15.69:0/541437598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818102430 msgr2=0x7fd81819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:28.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd8177fe700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818102430 0x7fd81819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd8177fe700 1 -- 172.21.15.69:0/541437598 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81819a1e0 con 0x7fd818108510 2024-05-14T16:57:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd816ffd700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818102430 0x7fd81819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:28.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd8167fc700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd818102df0 0x7fd81819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:28.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd8177fe700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd818108510 0x7fd818199a80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd810010c10 tx=0x7fd810010f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:28.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd7fffff700 1 -- 172.21.15.69:0/541437598 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810004bb0 con 0x7fd818108510 2024-05-14T16:57:28.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd81d798700 1 -- 172.21.15.69:0/541437598 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81819a4d0 con 0x7fd818108510 2024-05-14T16:57:28.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd7fffff700 1 -- 172.21.15.69:0/541437598 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd810004d50 con 0x7fd818108510 2024-05-14T16:57:28.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.137+0000 7fd7fffff700 1 -- 172.21.15.69:0/541437598 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8100057b0 con 0x7fd818108510 2024-05-14T16:57:28.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.138+0000 7fd81d798700 1 -- 172.21.15.69:0/541437598 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd818103fa0 con 0x7fd818108510 2024-05-14T16:57:28.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.139+0000 7fd81d798700 1 -- 172.21.15.69:0/541437598 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81806a230 con 0x7fd818108510 2024-05-14T16:57:28.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.142+0000 7fd7fffff700 1 -- 172.21.15.69:0/541437598 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd81004ab30 con 0x7fd818108510 2024-05-14T16:57:28.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.142+0000 7fd7fffff700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8000b29f0 0x7fd8000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.143+0000 7fd7fffff700 1 -- 172.21.15.69:0/541437598 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fd8100e5a80 con 0x7fd818108510 2024-05-14T16:57:28.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.143+0000 7fd816ffd700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8000b29f0 0x7fd8000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.143+0000 7fd816ffd700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8000b29f0 0x7fd8000b4eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd808006010 tx=0x7fd80800b560 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:28.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.145+0000 7fd7fffff700 1 -- 172.21.15.69:0/541437598 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8100ad450 con 0x7fd818108510 2024-05-14T16:57:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.153+0000 7fe51a393740 1 Processor -- start 2024-05-14T16:57:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.153+0000 7fe51a393740 1 -- start start 2024-05-14T16:57:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.153+0000 7fe51a393740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75e7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51a393740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 0x5565f75db030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51540e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75e7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51540e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75e7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37982/0 (socket says 172.21.15.69:37982) 2024-05-14T16:57:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51540e640 1 -- 172.21.15.69:0/1184957789 learned_addr learned my addr 172.21.15.69:0/1184957789 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51a393740 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75dbad0 0x5565f75db570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51a393740 1 -- 172.21.15.69:0/1184957789 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101b3290 con 0x5565f75d08f0 2024-05-14T16:57:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51a393740 1 -- 172.21.15.69:0/1184957789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101b0b00 con 0x5565f75dbad0 2024-05-14T16:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51a393740 1 -- 172.21.15.69:0/1184957789 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101ae970 con 0x5565f75d1210 2024-05-14T16:57:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe514c0d640 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 0x5565f75db030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe515c0f640 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75dbad0 0x5565f75db570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51540e640 1 -- 172.21.15.69:0/1184957789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 msgr2=0x5565f75db030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51540e640 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 0x5565f75db030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51540e640 1 -- 172.21.15.69:0/1184957789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75dbad0 msgr2=0x5565f75db570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51540e640 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75dbad0 0x5565f75db570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51540e640 1 -- 172.21.15.69:0/1184957789 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f75d92a0 con 0x5565f75d08f0 2024-05-14T16:57:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe514c0d640 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 0x5565f75db030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.154+0000 7fe51540e640 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75e7390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe504025540 tx=0x7fe50400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5590ea18e3afd498 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.155+0000 7fe4faffd640 1 -- 172.21.15.69:0/1184957789 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50400e980 con 0x5565f75d08f0 2024-05-14T16:57:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.155+0000 7fe4faffd640 1 -- 172.21.15.69:0/1184957789 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe504025be0 con 0x5565f75d08f0 2024-05-14T16:57:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.155+0000 7fe4faffd640 1 -- 172.21.15.69:0/1184957789 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50402ebd0 con 0x5565f75d08f0 2024-05-14T16:57:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.155+0000 7fe51a393740 1 -- 172.21.15.69:0/1184957789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 msgr2=0x5565f75e7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.155+0000 7fe51a393740 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75e7390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe504025540 tx=0x7fe50400ef20 comp rx=0 tx=0).stop 2024-05-14T16:57:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.155+0000 7fe51a393740 1 -- 172.21.15.69:0/1184957789 shutdown_connections 2024-05-14T16:57:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.155+0000 7fe51a393740 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75dbad0 0x5565f75db570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.155+0000 7fe51a393740 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d1210 0x5565f75db030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.155+0000 7fe51a393740 1 --2- 172.21.15.69:0/1184957789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75d08f0 0x5565f75e7390 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.155+0000 7fe51a393740 1 -- 172.21.15.69:0/1184957789 >> 172.21.15.69:0/1184957789 conn(0x5565f75cce00 msgr2=0x5565f75d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 1 -- 172.21.15.69:0/1184957789 shutdown_connections 2024-05-14T16:57:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 1 -- 172.21.15.69:0/1184957789 wait complete. 2024-05-14T16:57:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 1 -- start start 2024-05-14T16:57:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 1 Processor -- start 2024-05-14T16:57:29.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d08f0 0x5565f75dec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d1210 0x5565f75df1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75dbad0 0x5565f764e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101aefc0 con 0x5565f75dbad0 2024-05-14T16:57:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101b0430 con 0x5565f75d1210 2024-05-14T16:57:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101af690 con 0x5565f75d08f0 2024-05-14T16:57:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe51a393740 10 client.0 _ll_register_callbacks cb 0x7fe50401b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.156+0000 7fe515c0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75dbad0 0x5565f764e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe514c0d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d1210 0x5565f75df1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe514c0d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d1210 0x5565f75df1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:44326/0 (socket says 172.21.15.69:44326) 2024-05-14T16:57:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe514c0d640 1 -- 172.21.15.69:0/406419622 learned_addr learned my addr 172.21.15.69:0/406419622 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe51540e640 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d08f0 0x5565f75dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe515c0f640 1 -- 172.21.15.69:0/406419622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d08f0 msgr2=0x5565f75dec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe515c0f640 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d08f0 0x5565f75dec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe515c0f640 1 -- 172.21.15.69:0/406419622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d1210 msgr2=0x5565f75df1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe515c0f640 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d1210 0x5565f75df1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe515c0f640 1 -- 172.21.15.69:0/406419622 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f764e800 con 0x5565f75dbad0 2024-05-14T16:57:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe515c0f640 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75dbad0 0x5565f764e210 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x5565f75cdf90 tx=0x7fe508012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9da699a7d66847 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe514c0d640 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d1210 0x5565f75df1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe4f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:57:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/406419622 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508011bb0 con 0x5565f75dbad0 2024-05-14T16:57:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/406419622 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fe508011d50 con 0x5565f75dbad0 2024-05-14T16:57:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.157+0000 7fe51a393740 1 -- 172.21.15.69:0/406419622 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe504025be0 con 0x5565f75dbad0 2024-05-14T16:57:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.158+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/406419622 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe50802b050 con 0x5565f75dbad0 2024-05-14T16:57:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.158+0000 7fe51a393740 10 client.16218 fetch_fsmap learned FSMap version 40 2024-05-14T16:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.158+0000 7fe51a393740 1 -- 172.21.15.69:0/406419622 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5565f7655bf0 con 0x5565f75dbad0 2024-05-14T16:57:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.159+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/406419622 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe504025be0 con 0x5565f75dbad0 2024-05-14T16:57:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.159+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/406419622 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fe50801d540 con 0x5565f75dbad0 2024-05-14T16:57:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.159+0000 7fe51a393740 10 client.16218 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T16:57:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.159+0000 7fe51a393740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.159+0000 7fe51a393740 20 client.16218 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.159+0000 7fe51a393740 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f7656450 0x5565f7658870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.159+0000 7fe51a393740 4 client.16218 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:57:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.159+0000 7fe51a393740 1 -- 172.21.15.69:0/406419622 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565f753f360 con 0x5565f7656450 2024-05-14T16:57:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.159+0000 7fe51540e640 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f7656450 0x5565f7658870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.160+0000 7fe51540e640 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f7656450 0x5565f7658870 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3440b303b64f840e server_cookie=b79921b26e84de9d in_seq=0 out_seq=0 2024-05-14T16:57:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.160+0000 7fe4f8ff9640 10 client.16218 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe4f8ff9640 1 -- 172.21.15.69:0/406419622 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5565f753f360 con 0x5565f7656450 2024-05-14T16:57:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe4f8ff9640 10 client.16218 handle_command_reply: tid=0 2024-05-14T16:57:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe51a393740 1 client.16218 shutdown 2024-05-14T16:57:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe51a393740 1 -- 172.21.15.69:0/406419622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75dbad0 msgr2=0x5565f764e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe51a393740 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75dbad0 0x5565f764e210 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x5565f75cdf90 tx=0x7fe508012d70 comp rx=0 tx=0).stop 2024-05-14T16:57:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe51a393740 1 -- 172.21.15.69:0/406419622 shutdown_connections 2024-05-14T16:57:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe51a393740 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5565f7656450 0x5565f7658870 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe51a393740 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5565f75dbad0 0x5565f764e210 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe51a393740 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565f75d1210 0x5565f75df1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe51a393740 1 --2- 172.21.15.69:0/406419622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565f75d08f0 0x5565f75dec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe51a393740 1 -- 172.21.15.69:0/406419622 >> 172.21.15.69:0/406419622 conn(0x5565f75cce00 msgr2=0x5565f75d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.187+0000 7fe4f8ff9640 0 client.16218 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.188+0000 7fe51a393740 1 -- 172.21.15.69:0/406419622 shutdown_connections 2024-05-14T16:57:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.188+0000 7fe51a393740 1 -- 172.21.15.69:0/406419622 wait complete. 2024-05-14T16:57:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.189+0000 7fe51a393740 20 client.16218 trim_cache size 0 max 16384 2024-05-14T16:57:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:57:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:57:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:57:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 -- 172.21.15.69:0/4270494935 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe5000b2a10 msgr2=0x7fe5000b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe5000b2a10 0x7fe5000b4ed0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe5040049d0 tx=0x7fe504009500 comp rx=0 tx=0).stop 2024-05-14T16:57:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 -- 172.21.15.69:0/4270494935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe510103400 msgr2=0x7fe51019b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe510103400 0x7fe51019b960 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc031440 tx=0x7fe4fc004270 comp rx=0 tx=0).stop 2024-05-14T16:57:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 -- 172.21.15.69:0/4270494935 shutdown_connections 2024-05-14T16:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe5000b2a10 0x7fe5000b4ed0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe510108a50 0x7fe51019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe510103400 0x7fe51019b960 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 --2- 172.21.15.69:0/4270494935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe510102a40 0x7fe51019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 -- 172.21.15.69:0/4270494935 >> 172.21.15.69:0/4270494935 conn(0x7fe5100fe510 msgr2=0x7fe51010e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 -- 172.21.15.69:0/4270494935 shutdown_connections 2024-05-14T16:57:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:26.190+0000 7fe517672640 1 -- 172.21.15.69:0/4270494935 wait complete. 2024-05-14T16:57:29.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.543+0000 7fd81d798700 1 -- 172.21.15.69:0/541437598 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd818104200 con 0x7fd818108510 2024-05-14T16:57:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.544+0000 7fd7fffff700 1 -- 172.21.15.69:0/541437598 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11864 (secure 0 0 0) 0x7fd81004ae20 con 0x7fd818108510 2024-05-14T16:57:29.958 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:57:29.964 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:57:00.741417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:57:29.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.547+0000 7fd81d798700 1 -- 172.21.15.69:0/541437598 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8000b29f0 msgr2=0x7fd8000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:29.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.547+0000 7fd81d798700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8000b29f0 0x7fd8000b4eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd808006010 tx=0x7fd80800b560 comp rx=0 tx=0).stop 2024-05-14T16:57:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.547+0000 7fd81d798700 1 -- 172.21.15.69:0/541437598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd818108510 msgr2=0x7fd818199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.547+0000 7fd81d798700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd818108510 0x7fd818199a80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd810010c10 tx=0x7fd810010f20 comp rx=0 tx=0).stop 2024-05-14T16:57:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.548+0000 7fd81d798700 1 -- 172.21.15.69:0/541437598 shutdown_connections 2024-05-14T16:57:30.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.548+0000 7fd81d798700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd8000b29f0 0x7fd8000b4eb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.548+0000 7fd81d798700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818102430 0x7fd81819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.548+0000 7fd81d798700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd818108510 0x7fd818199a80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.548+0000 7fd81d798700 1 --2- 172.21.15.69:0/541437598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd818102df0 0x7fd81819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.548+0000 7fd81d798700 1 -- 172.21.15.69:0/541437598 >> 172.21.15.69:0/541437598 conn(0x7fd8180793b0 msgr2=0x7fd81806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:30.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.548+0000 7fd81d798700 1 -- 172.21.15.69:0/541437598 shutdown_connections 2024-05-14T16:57:30.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:26.548+0000 7fd81d798700 1 -- 172.21.15.69:0/541437598 wait complete. 2024-05-14T16:57:30.102 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 40 2024-05-14T16:57:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:26 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/541437598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:57:30.115 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 151.3 --await-for 45.0' 2024-05-14T16:57:30.121 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 151.3 --await-for 45.0 2024-05-14T16:57:30.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:27 smithi069 ceph-mon[31507]: pgmap v235: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 11 MiB/s wr, 965 op/s 2024-05-14T16:57:30.165 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:29 smithi069 ceph-mon[31507]: pgmap v236: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.2 MiB/s wr, 629 op/s 2024-05-14T16:57:30.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:29 smithi203 ceph-mon[35787]: pgmap v236: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.2 MiB/s wr, 629 op/s 2024-05-14T16:57:30.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:29 smithi092 ceph-mon[35763]: pgmap v236: 97 pgs: 97 active+clean; 3.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.2 MiB/s wr, 629 op/s 2024-05-14T16:57:30.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 420.0 s, 2486.8 tps, lat 40.424 ms stddev 45.613 2024-05-14T16:57:30.462 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:57:31.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:31 smithi069 ceph-mon[31507]: pgmap v237: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 8.7 MiB/s wr, 784 op/s 2024-05-14T16:57:32.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:31 smithi203 ceph-mon[35787]: pgmap v237: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 8.7 MiB/s wr, 784 op/s 2024-05-14T16:57:32.303 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:31 smithi092 ceph-mon[35763]: pgmap v237: 97 pgs: 97 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 8.7 MiB/s wr, 784 op/s 2024-05-14T16:57:33.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.994+0000 7f35182c5700 1 -- 172.21.15.69:0/3463978098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3510103590 msgr2=0x7f35101039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:33.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.994+0000 7f35182c5700 1 --2- 172.21.15.69:0/3463978098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3510103590 0x7f35101039f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f35000098f0 tx=0x7f3500009c00 comp rx=0 tx=0).stop 2024-05-14T16:57:33.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.994+0000 7f35182c5700 1 -- 172.21.15.69:0/3463978098 shutdown_connections 2024-05-14T16:57:33.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.994+0000 7f35182c5700 1 --2- 172.21.15.69:0/3463978098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3510108510 0x7f35101088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:33.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.994+0000 7f35182c5700 1 --2- 172.21.15.69:0/3463978098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3510103f30 0x7f3510110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:33.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.994+0000 7f35182c5700 1 --2- 172.21.15.69:0/3463978098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3510103590 0x7f35101039f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:33.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.994+0000 7f35182c5700 1 -- 172.21.15.69:0/3463978098 >> 172.21.15.69:0/3463978098 conn(0x7f351007a6f0 msgr2=0x7f351007ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.994+0000 7f35182c5700 1 -- 172.21.15.69:0/3463978098 shutdown_connections 2024-05-14T16:57:33.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.994+0000 7f35182c5700 1 -- 172.21.15.69:0/3463978098 wait complete. 2024-05-14T16:57:33.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f35182c5700 1 Processor -- start 2024-05-14T16:57:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f35182c5700 1 -- start start 2024-05-14T16:57:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f35182c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3510103590 0x7f351010d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:33.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f35182c5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3510103f30 0x7f351010d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:33.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f35182c5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3510108510 0x7f35101a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:33.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f35182c5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3510076de0 con 0x7f3510108510 2024-05-14T16:57:33.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f35182c5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3510076c60 con 0x7f3510103590 2024-05-14T16:57:33.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f35182c5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3510076f60 con 0x7f3510103f30 2024-05-14T16:57:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f3516862700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3510108510 0x7f35101a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:33.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f3516061700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3510103590 0x7f351010d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:33.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f3515860700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3510103f30 0x7f351010d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:33.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f3516862700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3510108510 0x7f35101a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38002/0 (socket says 172.21.15.69:38002) 2024-05-14T16:57:33.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.995+0000 7f3516862700 1 -- 172.21.15.69:0/557862924 learned_addr learned my addr 172.21.15.69:0/557862924 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:33.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.996+0000 7f3515860700 1 -- 172.21.15.69:0/557862924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3510103590 msgr2=0x7f351010d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:33.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.996+0000 7f3515860700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3510103590 0x7f351010d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:33.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.996+0000 7f3515860700 1 -- 172.21.15.69:0/557862924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3510108510 msgr2=0x7f35101a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:33.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.996+0000 7f3515860700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3510108510 0x7f35101a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:33.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.996+0000 7f3515860700 1 -- 172.21.15.69:0/557862924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35101a51c0 con 0x7f3510103f30 2024-05-14T16:57:33.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.996+0000 7f3515860700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3510103f30 0x7f351010d5d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3500009230 tx=0x7f3500009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:33.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.997+0000 7f35077fe700 1 -- 172.21.15.69:0/557862924 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35000052b0 con 0x7f3510103f30 2024-05-14T16:57:33.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.997+0000 7f35182c5700 1 -- 172.21.15.69:0/557862924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35101a5450 con 0x7f3510103f30 2024-05-14T16:57:33.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.997+0000 7f35077fe700 1 -- 172.21.15.69:0/557862924 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350000ba10 con 0x7f3510103f30 2024-05-14T16:57:33.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.997+0000 7f35077fe700 1 -- 172.21.15.69:0/557862924 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500003b20 con 0x7f3510103f30 2024-05-14T16:57:33.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:32.997+0000 7f35182c5700 1 -- 172.21.15.69:0/557862924 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35101acd30 con 0x7f3510103f30 2024-05-14T16:57:33.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.001+0000 7f35077fe700 1 -- 172.21.15.69:0/557862924 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3500003cc0 con 0x7f3510103f30 2024-05-14T16:57:33.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.001+0000 7f35077fe700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34fc0b2af0 0x7f34fc0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.002+0000 7f35077fe700 1 -- 172.21.15.69:0/557862924 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f35000e37b0 con 0x7f3510103f30 2024-05-14T16:57:33.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.002+0000 7f3516061700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34fc0b2af0 0x7f34fc0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:34.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.002+0000 7f3516061700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34fc0b2af0 0x7f34fc0b4fb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3510104770 tx=0x7f350c009310 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:34.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.003+0000 7f3522b5bb80 1 Processor -- start 2024-05-14T16:57:34.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.003+0000 7f3522b5bb80 1 -- start start 2024-05-14T16:57:34.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.003+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba64b0 0x55610bba6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:34.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.003+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba6dd0 0x55610bba1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:34.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba2350 0x55610bba27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:34.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55610bafff00 con 0x55610bba64b0 2024-05-14T16:57:34.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55610baefaa0 con 0x55610bba2350 2024-05-14T16:57:34.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55610baf3ae0 con 0x55610bba6dd0 2024-05-14T16:57:34.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3516061700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba64b0 0x55610bba6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:34.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3516061700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba64b0 0x55610bba6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38008/0 (socket says 172.21.15.69:38008) 2024-05-14T16:57:34.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3516061700 1 -- 172.21.15.69:0/566848306 learned_addr learned my addr 172.21.15.69:0/566848306 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:34.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3515860700 1 --2- 172.21.15.69:0/566848306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba6dd0 0x55610bba1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3516862700 1 --2- 172.21.15.69:0/566848306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba2350 0x55610bba27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:34.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3516061700 1 -- 172.21.15.69:0/566848306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba6dd0 msgr2=0x55610bba1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:34.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3516061700 1 --2- 172.21.15.69:0/566848306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba6dd0 0x55610bba1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3516061700 1 -- 172.21.15.69:0/566848306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba2350 msgr2=0x55610bba27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:34.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3516061700 1 --2- 172.21.15.69:0/566848306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba2350 0x55610bba27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3516061700 1 -- 172.21.15.69:0/566848306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55610ba99140 con 0x55610bba64b0 2024-05-14T16:57:34.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3516061700 1 --2- 172.21.15.69:0/566848306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba64b0 0x55610bba6890 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f350c004570 tx=0x7f350c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742c4577bc5a3c19 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:34.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3504d70700 1 -- 172.21.15.69:0/566848306 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c025070 con 0x55610bba64b0 2024-05-14T16:57:34.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3504d70700 1 -- 172.21.15.69:0/566848306 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350c016070 con 0x55610bba64b0 2024-05-14T16:57:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.004+0000 7f3504d70700 1 -- 172.21.15.69:0/566848306 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c012820 con 0x55610bba64b0 2024-05-14T16:57:34.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.006+0000 7f3522b5bb80 1 -- 172.21.15.69:0/566848306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba64b0 msgr2=0x55610bba6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:34.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.006+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/566848306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba64b0 0x55610bba6890 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f350c004570 tx=0x7f350c014000 comp rx=0 tx=0).stop 2024-05-14T16:57:34.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.006+0000 7f3522b5bb80 1 -- 172.21.15.69:0/566848306 shutdown_connections 2024-05-14T16:57:34.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.006+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/566848306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba64b0 0x55610bba6890 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.006+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/566848306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba2350 0x55610bba27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.006+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/566848306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba6dd0 0x55610bba1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.006+0000 7f3522b5bb80 1 -- 172.21.15.69:0/566848306 >> 172.21.15.69:0/566848306 conn(0x55610bacef40 msgr2=0x55610bb9c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:34.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.006+0000 7f3522b5bb80 1 -- 172.21.15.69:0/566848306 shutdown_connections 2024-05-14T16:57:34.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.006+0000 7f3522b5bb80 1 -- 172.21.15.69:0/566848306 wait complete. 2024-05-14T16:57:34.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3522b5bb80 1 -- start start 2024-05-14T16:57:34.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3522b5bb80 1 Processor -- start 2024-05-14T16:57:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba2350 0x55610bbadbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:34.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 0x55610bbaaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:34.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba6dd0 0x55610bbab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:34.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55610b9f6130 con 0x55610bba6dd0 2024-05-14T16:57:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55610ba57f30 con 0x55610bba64b0 2024-05-14T16:57:34.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55610bbae910 con 0x55610bba2350 2024-05-14T16:57:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3522b5bb80 10 client.0 _ll_register_callbacks cb 0x55610ba8aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:34.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3516862700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba6dd0 0x55610bbab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:34.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3516862700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba6dd0 0x55610bbab4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38020/0 (socket says 172.21.15.69:38020) 2024-05-14T16:57:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3516862700 1 -- 172.21.15.69:0/145963610 learned_addr learned my addr 172.21.15.69:0/145963610 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:34.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3515860700 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 0x55610bbaaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:34.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.007+0000 7f3516061700 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba2350 0x55610bbadbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:34.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f3516862700 1 -- 172.21.15.69:0/145963610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba2350 msgr2=0x55610bbadbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:34.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f3516862700 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba2350 0x55610bbadbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f3516862700 1 -- 172.21.15.69:0/145963610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 msgr2=0x55610bbaaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:34.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f3516862700 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 0x55610bbaaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f3516862700 1 -- 172.21.15.69:0/145963610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55610ba3fa60 con 0x55610bba6dd0 2024-05-14T16:57:34.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f3515860700 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 0x55610bbaaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:34.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f3516862700 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba6dd0 0x55610bbab4c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3508007b80 tx=0x7f3508007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa18de45e4f4e85 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:34.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f34f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:57:34.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f34f9ffb700 1 -- 172.21.15.69:0/145963610 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350801a070 con 0x55610bba6dd0 2024-05-14T16:57:34.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f34f9ffb700 1 -- 172.21.15.69:0/145963610 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f3508004db0 con 0x55610bba6dd0 2024-05-14T16:57:34.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f3522b5bb80 1 -- 172.21.15.69:0/145963610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f350c016070 con 0x55610bba6dd0 2024-05-14T16:57:34.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f34f9ffb700 1 -- 172.21.15.69:0/145963610 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3508025050 con 0x55610bba6dd0 2024-05-14T16:57:34.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f3522b5bb80 10 client.16230 fetch_fsmap learned FSMap version 40 2024-05-14T16:57:34.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.008+0000 7f3522b5bb80 1 -- 172.21.15.69:0/145963610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55610baa98f0 con 0x55610bba6dd0 2024-05-14T16:57:34.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.009+0000 7f34f9ffb700 1 -- 172.21.15.69:0/145963610 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350c016070 con 0x55610bba6dd0 2024-05-14T16:57:34.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.009+0000 7f34f9ffb700 1 -- 172.21.15.69:0/145963610 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f3508016c30 con 0x55610bba6dd0 2024-05-14T16:57:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.009+0000 7f3522b5bb80 10 client.16230 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T16:57:34.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.009+0000 7f3522b5bb80 10 client.16230 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:57:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.009+0000 7f3522b5bb80 20 client.16230 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:34.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.009+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55610bbbb520 0x55610bbbd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:34.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.009+0000 7f3522b5bb80 4 client.16230 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:34.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.009+0000 7f3522b5bb80 1 -- 172.21.15.69:0/145963610 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55610bb01e50 con 0x55610bbbb520 2024-05-14T16:57:34.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.009+0000 7f3516061700 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55610bbbb520 0x55610bbbd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:34.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.010+0000 7f3516061700 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55610bbbb520 0x55610bbbd910 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed05cfca7d1d066 server_cookie=429bbdeaf3353034 in_seq=0 out_seq=0 2024-05-14T16:57:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.010+0000 7f34f9ffb700 10 client.16230 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:34.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f34f9ffb700 1 -- 172.21.15.69:0/145963610 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55610bb01e50 con 0x55610bbbb520 2024-05-14T16:57:34.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f34f9ffb700 10 client.16230 handle_command_reply: tid=0 2024-05-14T16:57:34.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 client.16230 shutdown 2024-05-14T16:57:34.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 -- 172.21.15.69:0/145963610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba6dd0 msgr2=0x55610bbab4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:34.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba6dd0 0x55610bbab4c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3508007b80 tx=0x7f3508007e90 comp rx=0 tx=0).stop 2024-05-14T16:57:34.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 -- 172.21.15.69:0/145963610 shutdown_connections 2024-05-14T16:57:34.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55610bbbb520 0x55610bbbd910 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba6dd0 0x55610bbab4c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba2350 0x55610bbadbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/145963610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 0x55610bbaaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:34.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 -- 172.21.15.69:0/145963610 >> 172.21.15.69:0/145963610 conn(0x55610bacef40 msgr2=0x55610bb97050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:34.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f34f9ffb700 0 client.16230 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:34.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 -- 172.21.15.69:0/145963610 shutdown_connections 2024-05-14T16:57:34.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.011+0000 7f3522b5bb80 1 -- 172.21.15.69:0/145963610 wait complete. 2024-05-14T16:57:34.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.012+0000 7f3522b5bb80 20 client.16230 trim_cache size 0 max 16384 2024-05-14T16:57:34.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.044+0000 7f3522b5bb80 1 Processor -- start 2024-05-14T16:57:34.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.050+0000 7f3522b5bb80 1 -- start start 2024-05-14T16:57:35.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.050+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bb97160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:35.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.050+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba64b0 0x55610bbc31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:35.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba6dd0 0x55610bbc3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:35.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35101b6c20 con 0x55610bba2350 2024-05-14T16:57:35.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f35101b4450 con 0x55610bba6dd0 2024-05-14T16:57:35.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f35101b22c0 con 0x55610bba64b0 2024-05-14T16:57:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3516061700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bb97160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:35.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3516061700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bb97160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38032/0 (socket says 172.21.15.69:38032) 2024-05-14T16:57:35.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3516061700 1 -- 172.21.15.69:0/2161972627 learned_addr learned my addr 172.21.15.69:0/2161972627 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3516061700 1 -- 172.21.15.69:0/2161972627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba64b0 msgr2=0x55610bbc31c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:57:35.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3516061700 1 --2- 172.21.15.69:0/2161972627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba64b0 0x55610bbc31c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3516061700 1 -- 172.21.15.69:0/2161972627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba6dd0 msgr2=0x55610bbc3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:57:35.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3516061700 1 --2- 172.21.15.69:0/2161972627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba6dd0 0x55610bbc3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.051+0000 7f3516061700 1 -- 172.21.15.69:0/2161972627 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55610bb976a0 con 0x55610bba2350 2024-05-14T16:57:35.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.052+0000 7f3516061700 1 --2- 172.21.15.69:0/2161972627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bb97160 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f350c011be0 tx=0x7f350c009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5683e5cc834dc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.053+0000 7f34fbfff700 1 -- 172.21.15.69:0/2161972627 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c025070 con 0x55610bba2350 2024-05-14T16:57:35.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.053+0000 7f34fbfff700 1 -- 172.21.15.69:0/2161972627 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350c032030 con 0x55610bba2350 2024-05-14T16:57:35.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.054+0000 7f3522b5bb80 1 -- 172.21.15.69:0/2161972627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 msgr2=0x55610bb97160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:35.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.054+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/2161972627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bb97160 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f350c011be0 tx=0x7f350c009980 comp rx=0 tx=0).stop 2024-05-14T16:57:35.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.057+0000 7f3522b5bb80 1 -- 172.21.15.69:0/2161972627 shutdown_connections 2024-05-14T16:57:35.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.057+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/2161972627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bb97160 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.057+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/2161972627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba6dd0 0x55610bbc3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.057+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/2161972627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba64b0 0x55610bbc31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.057+0000 7f3522b5bb80 1 -- 172.21.15.69:0/2161972627 >> 172.21.15.69:0/2161972627 conn(0x55610bacef40 msgr2=0x55610bba4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:35.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.057+0000 7f3522b5bb80 1 -- 172.21.15.69:0/2161972627 shutdown_connections 2024-05-14T16:57:35.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.058+0000 7f3522b5bb80 1 -- 172.21.15.69:0/2161972627 wait complete. 2024-05-14T16:57:35.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.059+0000 7f3522b5bb80 1 -- start start 2024-05-14T16:57:35.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.059+0000 7f3522b5bb80 1 Processor -- start 2024-05-14T16:57:35.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.059+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bc1e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:35.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.059+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 0x55610bc1e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:35.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.059+0000 7f3522b5bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba6dd0 0x55610bbb0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:35.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.059+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35101b2910 con 0x55610bba2350 2024-05-14T16:57:35.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.059+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f35101b3d80 con 0x55610bba64b0 2024-05-14T16:57:35.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.059+0000 7f3522b5bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f35101b2fe0 con 0x55610bba6dd0 2024-05-14T16:57:35.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.059+0000 7f3522b5bb80 10 client.0 _ll_register_callbacks cb 0x55610ba8aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:35.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.060+0000 7f3516061700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bc1e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:35.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.060+0000 7f3516061700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bc1e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38048/0 (socket says 172.21.15.69:38048) 2024-05-14T16:57:35.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.060+0000 7f3516061700 1 -- 172.21.15.69:0/192240269 learned_addr learned my addr 172.21.15.69:0/192240269 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:35.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.061+0000 7f3515860700 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 0x55610bc1e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:35.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.061+0000 7f3516061700 1 -- 172.21.15.69:0/192240269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba6dd0 msgr2=0x55610bbb0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:35.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.061+0000 7f3516061700 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba6dd0 0x55610bbb0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.061+0000 7f3516061700 1 -- 172.21.15.69:0/192240269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 msgr2=0x55610bc1e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.061+0000 7f3516061700 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 0x55610bc1e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.061+0000 7f3516061700 1 -- 172.21.15.69:0/192240269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55610bbb0870 con 0x55610bba2350 2024-05-14T16:57:35.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.061+0000 7f3515860700 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 0x55610bc1e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:57:35.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.063+0000 7f3516061700 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bc1e0d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f350c009930 tx=0x7f350c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaa4df16d1fddb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:35.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.064+0000 7f34f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:57:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.064+0000 7f34f9ffb700 1 -- 172.21.15.69:0/192240269 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c025040 con 0x55610bba2350 2024-05-14T16:57:35.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.064+0000 7f3522b5bb80 1 -- 172.21.15.69:0/192240269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f350c032030 con 0x55610bba2350 2024-05-14T16:57:35.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.064+0000 7f34f9ffb700 1 -- 172.21.15.69:0/192240269 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f350c005460 con 0x55610bba2350 2024-05-14T16:57:35.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.065+0000 7f34f9ffb700 1 -- 172.21.15.69:0/192240269 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f350c04a050 con 0x55610bba2350 2024-05-14T16:57:35.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.065+0000 7f3522b5bb80 10 client.16236 fetch_fsmap learned FSMap version 40 2024-05-14T16:57:35.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.065+0000 7f3522b5bb80 1 -- 172.21.15.69:0/192240269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55610bc379e0 con 0x55610bba2350 2024-05-14T16:57:35.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.065+0000 7f34f9ffb700 1 -- 172.21.15.69:0/192240269 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350c032030 con 0x55610bba2350 2024-05-14T16:57:35.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.065+0000 7f34f9ffb700 1 -- 172.21.15.69:0/192240269 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f350c02b040 con 0x55610bba2350 2024-05-14T16:57:35.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.065+0000 7f3522b5bb80 10 client.16236 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T16:57:35.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.065+0000 7f3522b5bb80 10 client.16236 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:57:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.065+0000 7f3522b5bb80 20 client.16236 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:35.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.065+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55610bc38020 0x55610bc3a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:35.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.066+0000 7f3522b5bb80 4 client.16236 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 151.3, "await_for": 45.0}] 2024-05-14T16:57:35.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.066+0000 7f3522b5bb80 1 -- 172.21.15.69:0/192240269 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 151.3, "await_for": 45.0}) -- 0x55610bafb630 con 0x55610bc38020 2024-05-14T16:57:35.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.067+0000 7f3515860700 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55610bc38020 0x55610bc3a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:35.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.072+0000 7f3515860700 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55610bc38020 0x55610bc3a460 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3330daa2ce9d19 server_cookie=4002c683ec63f4c4 in_seq=0 out_seq=0 2024-05-14T16:57:35.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.072+0000 7f34f9ffb700 10 client.16236 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:35.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.329+0000 7f34f9ffb700 1 -- 172.21.15.69:0/192240269 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55610bafb630 con 0x55610bc38020 2024-05-14T16:57:35.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.329+0000 7f34f9ffb700 10 client.16236 handle_command_reply: tid=0 2024-05-14T16:57:35.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.329+0000 7f3522b5bb80 1 client.16236 shutdown 2024-05-14T16:57:35.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 1 -- 172.21.15.69:0/192240269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 msgr2=0x55610bc1e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:35.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bc1e0d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f350c009930 tx=0x7f350c003730 comp rx=0 tx=0).stop 2024-05-14T16:57:35.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 1 -- 172.21.15.69:0/192240269 shutdown_connections 2024-05-14T16:57:35.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55610bc38020 0x55610bc3a460 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55610bba2350 0x55610bc1e0d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f34f9ffb700 0 client.16236 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:35.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55610bba6dd0 0x55610bbb0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 1 --2- 172.21.15.69:0/192240269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55610bba64b0 0x55610bc1e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 1 -- 172.21.15.69:0/192240269 >> 172.21.15.69:0/192240269 conn(0x55610bacef40 msgr2=0x55610bba57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 1 -- 172.21.15.69:0/192240269 shutdown_connections 2024-05-14T16:57:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 1 -- 172.21.15.69:0/192240269 wait complete. 2024-05-14T16:57:35.737 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:57:35.745 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 40, 2024-05-14T16:57:35.756 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:57:35.763 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 26, 2024-05-14T16:57:35.771 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:57:35.782 INFO:teuthology.orchestra.run.smithi069.stdout: "37a5760b": { 2024-05-14T16:57:35.788 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 26, 2024-05-14T16:57:35.805 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:57:35.812 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:57:35.821 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:57:35.827 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:57:35.833 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:57:35.838 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:57:35.844 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 151.3, 2024-05-14T16:57:35.849 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:57:35.856 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:57:35.861 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:57:35.879 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:57:35.885 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:57:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:57:35.915 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:57:35.921 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:57:35.928 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:57:35.934 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:57:35.964 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:57:35.971 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:57:35.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f3522b5bb80 20 client.16236 trim_cache size 0 max 16384 2024-05-14T16:57:35.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f35182c5700 1 -- 172.21.15.69:0/557862924 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34fc0b2af0 msgr2=0x7f34fc0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:35.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f35182c5700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34fc0b2af0 0x7f34fc0b4fb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3510104770 tx=0x7f350c009310 comp rx=0 tx=0).stop 2024-05-14T16:57:35.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f35182c5700 1 -- 172.21.15.69:0/557862924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3510103f30 msgr2=0x7f351010d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:36.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f35182c5700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3510103f30 0x7f351010d5d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3500009230 tx=0x7f3500009480 comp rx=0 tx=0).stop 2024-05-14T16:57:36.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f35182c5700 1 -- 172.21.15.69:0/557862924 shutdown_connections 2024-05-14T16:57:36.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f35182c5700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f34fc0b2af0 0x7f34fc0b4fb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f35182c5700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3510108510 0x7f35101a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f35182c5700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3510103590 0x7f351010d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f35182c5700 1 --2- 172.21.15.69:0/557862924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3510103f30 0x7f351010d5d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.330+0000 7f35182c5700 1 -- 172.21.15.69:0/557862924 >> 172.21.15.69:0/557862924 conn(0x7f351007a6f0 msgr2=0x7f35101070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:36.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.331+0000 7f35182c5700 1 -- 172.21.15.69:0/557862924 shutdown_connections 2024-05-14T16:57:36.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:33.331+0000 7f35182c5700 1 -- 172.21.15.69:0/557862924 wait complete. 2024-05-14T16:57:36.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:33 smithi069 ceph-mon[31507]: pgmap v238: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 8.4 MiB/s wr, 753 op/s 2024-05-14T16:57:36.117 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 425.0 s, 1740.0 tps, lat 31.130 ms stddev 31.786 2024-05-14T16:57:36.148 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '37a5760b', quiesce took 12.4 seconds. Will release after: 18.9 2024-05-14T16:57:36.155 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:33 smithi092 ceph-mon[35763]: pgmap v238: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 8.4 MiB/s wr, 753 op/s 2024-05-14T16:57:36.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:33 smithi203 ceph-mon[35787]: pgmap v238: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 8.4 MiB/s wr, 753 op/s 2024-05-14T16:57:36.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:35 smithi069 ceph-mon[31507]: pgmap v239: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-05-14T16:57:36.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:35 smithi203 ceph-mon[35787]: pgmap v239: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-05-14T16:57:36.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:35 smithi092 ceph-mon[35763]: pgmap v239: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-05-14T16:57:38.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:37 smithi069 ceph-mon[31507]: pgmap v240: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 8.8 MiB/s wr, 799 op/s 2024-05-14T16:57:38.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:37 smithi203 ceph-mon[35787]: pgmap v240: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 8.8 MiB/s wr, 799 op/s 2024-05-14T16:57:38.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:37 smithi092 ceph-mon[35763]: pgmap v240: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 8.8 MiB/s wr, 799 op/s 2024-05-14T16:57:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.361+0000 7f566cda4640 1 -- 172.21.15.69:0/125084794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668111620 msgr2=0x7f5668109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.361+0000 7f566cda4640 1 --2- 172.21.15.69:0/125084794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668111620 0x7f5668109f90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f565c009890 tx=0x7f565c02a400 comp rx=0 tx=0).stop 2024-05-14T16:57:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.366+0000 7f566cda4640 1 -- 172.21.15.69:0/125084794 shutdown_connections 2024-05-14T16:57:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.366+0000 7f566cda4640 1 --2- 172.21.15.69:0/125084794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566811c230 0x7f566811e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.366+0000 7f566cda4640 1 --2- 172.21.15.69:0/125084794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668111620 0x7f5668109f90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.366+0000 7f566cda4640 1 --2- 172.21.15.69:0/125084794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668111240 0x7f5668109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.366+0000 7f566cda4640 1 -- 172.21.15.69:0/125084794 >> 172.21.15.69:0/125084794 conn(0x7f566806db20 msgr2=0x7f566806df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.368+0000 7f566cda4640 1 -- 172.21.15.69:0/125084794 shutdown_connections 2024-05-14T16:57:39.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.369+0000 7f566cda4640 1 -- 172.21.15.69:0/125084794 wait complete. 2024-05-14T16:57:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.370+0000 7f566cda4640 1 Processor -- start 2024-05-14T16:57:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f566cda4640 1 -- start start 2024-05-14T16:57:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f566cda4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668111240 0x7f566811b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f566cda4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566811c230 0x7f5668117940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f566cda4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668117e80 0x7f5668118330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f566cda4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f56681142a0 con 0x7f5668111240 2024-05-14T16:57:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f566cda4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5668114120 con 0x7f5668117e80 2024-05-14T16:57:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f566cda4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5668114420 con 0x7f566811c230 2024-05-14T16:57:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f5665d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566811c230 0x7f5668117940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f5665d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566811c230 0x7f5668117940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:47354/0 (socket says 172.21.15.69:47354) 2024-05-14T16:57:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f5665d9b640 1 -- 172.21.15.69:0/631452006 learned_addr learned my addr 172.21.15.69:0/631452006 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.371+0000 7f5666d9d640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668117e80 0x7f5668118330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.372+0000 7f5666d9d640 1 -- 172.21.15.69:0/631452006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566811c230 msgr2=0x7f5668117940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.372+0000 7f5666d9d640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566811c230 0x7f5668117940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.372+0000 7f5666d9d640 1 -- 172.21.15.69:0/631452006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668111240 msgr2=0x7f566811b860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:57:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.372+0000 7f5666d9d640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668111240 0x7f566811b860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.372+0000 7f5666d9d640 1 -- 172.21.15.69:0/631452006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56681ac020 con 0x7f5668117e80 2024-05-14T16:57:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.372+0000 7f5666d9d640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668117e80 0x7f5668118330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5660005fd0 tx=0x7f566001d380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.373+0000 7f56577fe640 1 -- 172.21.15.69:0/631452006 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566001ddc0 con 0x7f5668117e80 2024-05-14T16:57:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.373+0000 7f566cda4640 1 -- 172.21.15.69:0/631452006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56681ac250 con 0x7f5668117e80 2024-05-14T16:57:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.374+0000 7f566cda4640 1 -- 172.21.15.69:0/631452006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56681ac770 con 0x7f5668117e80 2024-05-14T16:57:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.375+0000 7f56577fe640 1 -- 172.21.15.69:0/631452006 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5660003970 con 0x7f5668117e80 2024-05-14T16:57:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.375+0000 7f56577fe640 1 -- 172.21.15.69:0/631452006 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600044d0 con 0x7f5668117e80 2024-05-14T16:57:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.378+0000 7f566fb99740 1 Processor -- start 2024-05-14T16:57:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.378+0000 7f566fb99740 1 -- start start 2024-05-14T16:57:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.378+0000 7f566fb99740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba0ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.378+0000 7f566fb99740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba17590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.378+0000 7f566fb99740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba19ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.378+0000 7f566fb99740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56179b97fc30 con 0x56179ba0d210 2024-05-14T16:57:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.378+0000 7f566fb99740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56179b96f7d0 con 0x56179ba17ad0 2024-05-14T16:57:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.378+0000 7f566fb99740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56179b973810 con 0x56179ba0c8f0 2024-05-14T16:57:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.379+0000 7f566659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba17590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.379+0000 7f566659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba17590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51696/0 (socket says 172.21.15.69:51696) 2024-05-14T16:57:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.379+0000 7f566659c640 1 -- 172.21.15.69:0/1736375582 learned_addr learned my addr 172.21.15.69:0/1736375582 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.379+0000 7f5665d9b640 1 --2- 172.21.15.69:0/1736375582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba0ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.379+0000 7f566659c640 1 -- 172.21.15.69:0/1736375582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 msgr2=0x56179ba0ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.379+0000 7f566659c640 1 --2- 172.21.15.69:0/1736375582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba0ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.379+0000 7f566659c640 1 -- 172.21.15.69:0/1736375582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 msgr2=0x56179ba19ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.379+0000 7f566659c640 1 --2- 172.21.15.69:0/1736375582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba19ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.379+0000 7f566659c640 1 -- 172.21.15.69:0/1736375582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179b684210 con 0x56179ba0d210 2024-05-14T16:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.380+0000 7f566659c640 1 --2- 172.21.15.69:0/1736375582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba17590 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f565800ba80 tx=0x7f565800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d2f64b97069e12 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.381+0000 7f5654ff9640 1 -- 172.21.15.69:0/1736375582 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658016070 con 0x56179ba0d210 2024-05-14T16:57:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.381+0000 7f56577fe640 1 -- 172.21.15.69:0/631452006 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f566002e020 con 0x7f5668117e80 2024-05-14T16:57:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.381+0000 7f5654ff9640 1 -- 172.21.15.69:0/1736375582 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56580054d0 con 0x56179ba0d210 2024-05-14T16:57:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.381+0000 7f5654ff9640 1 -- 172.21.15.69:0/1736375582 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658004de0 con 0x56179ba0d210 2024-05-14T16:57:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.383+0000 7f56577fe640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f56500b2c20 0x7f56500b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.383+0000 7f566fb99740 1 -- 172.21.15.69:0/1736375582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 msgr2=0x56179ba17590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.383+0000 7f566fb99740 1 --2- 172.21.15.69:0/1736375582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba17590 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f565800ba80 tx=0x7f565800bf40 comp rx=0 tx=0).stop 2024-05-14T16:57:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.383+0000 7f56577fe640 1 -- 172.21.15.69:0/631452006 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f56600f2c70 con 0x7f5668117e80 2024-05-14T16:57:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.384+0000 7f566fb99740 1 -- 172.21.15.69:0/1736375582 shutdown_connections 2024-05-14T16:57:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.384+0000 7f566fb99740 1 --2- 172.21.15.69:0/1736375582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba19ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.384+0000 7f566fb99740 1 --2- 172.21.15.69:0/1736375582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba17590 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.384+0000 7f566fb99740 1 --2- 172.21.15.69:0/1736375582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba0ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.384+0000 7f566fb99740 1 -- 172.21.15.69:0/1736375582 >> 172.21.15.69:0/1736375582 conn(0x56179ba08e00 msgr2=0x56179ba091e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.384+0000 7f5665d9b640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f56500b2c20 0x7f56500b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.384+0000 7f566fb99740 1 -- 172.21.15.69:0/1736375582 shutdown_connections 2024-05-14T16:57:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.384+0000 7f566fb99740 1 -- 172.21.15.69:0/1736375582 wait complete. 2024-05-14T16:57:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.385+0000 7f566fb99740 1 -- start start 2024-05-14T16:57:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.385+0000 7f566fb99740 1 Processor -- start 2024-05-14T16:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566fb99740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0c8f0 0x56179ba173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566fb99740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba0d210 0x56179ba146c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566fb99740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba17ad0 0x56179ba14c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566fb99740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56179b655790 con 0x56179ba0c8f0 2024-05-14T16:57:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566fb99740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56179b92b890 con 0x56179ba0d210 2024-05-14T16:57:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566fb99740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56179ba1a7a0 con 0x56179ba17ad0 2024-05-14T16:57:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566fb99740 10 client.0 _ll_register_callbacks cb 0x56179b92ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f5665d9b640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f56500b2c20 0x7f56500b50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f565c0057c0 tx=0x7f565c009450 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f5665d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba0d210 0x56179ba146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f5665d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba0d210 0x56179ba146c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:51020/0 (socket says 172.21.15.69:51020) 2024-05-14T16:57:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f5665d9b640 1 -- 172.21.15.69:0/3816163899 learned_addr learned my addr 172.21.15.69:0/3816163899 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566659c640 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0c8f0 0x56179ba173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566659c640 1 -- 172.21.15.69:0/3816163899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba17ad0 msgr2=0x56179ba14c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566659c640 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba17ad0 0x56179ba14c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566659c640 1 -- 172.21.15.69:0/3816163899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba0d210 msgr2=0x56179ba146c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.388+0000 7f566659c640 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba0d210 0x56179ba146c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.389+0000 7f566659c640 1 -- 172.21.15.69:0/3816163899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179b750620 con 0x56179ba0c8f0 2024-05-14T16:57:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.389+0000 7f566659c640 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0c8f0 0x56179ba173d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5658004ac0 tx=0x7f565800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ad1f129cf219b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.389+0000 7f564dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:57:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.390+0000 7f564dffb640 1 -- 172.21.15.69:0/3816163899 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658016070 con 0x56179ba0c8f0 2024-05-14T16:57:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.390+0000 7f564dffb640 1 -- 172.21.15.69:0/3816163899 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f565800dca0 con 0x56179ba0c8f0 2024-05-14T16:57:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.390+0000 7f566fb99740 1 -- 172.21.15.69:0/3816163899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56580054d0 con 0x56179ba0c8f0 2024-05-14T16:57:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.390+0000 7f564dffb640 1 -- 172.21.15.69:0/3816163899 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f565800dea0 con 0x56179ba0c8f0 2024-05-14T16:57:40.562 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:39 smithi092 ceph-mon[35763]: pgmap v241: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.2 MiB/s wr, 444 op/s 2024-05-14T16:57:40.569 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:39 smithi203 ceph-mon[35787]: pgmap v241: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.2 MiB/s wr, 444 op/s 2024-05-14T16:57:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.391+0000 7f566fb99740 10 client.16245 fetch_fsmap learned FSMap version 40 2024-05-14T16:57:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.391+0000 7f566fb99740 1 -- 172.21.15.69:0/3816163899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56179ba261a0 con 0x56179ba0c8f0 2024-05-14T16:57:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.392+0000 7f564dffb640 1 -- 172.21.15.69:0/3816163899 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56580054d0 con 0x56179ba0c8f0 2024-05-14T16:57:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.392+0000 7f564dffb640 1 -- 172.21.15.69:0/3816163899 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f5658022020 con 0x56179ba0c8f0 2024-05-14T16:57:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.393+0000 7f566fb99740 10 client.16245 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T16:57:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.393+0000 7f566fb99740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.393+0000 7f566fb99740 20 client.16245 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.393+0000 7f566fb99740 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56179ba27600 0x56179ba299f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.393+0000 7f566fb99740 4 client.16245 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.393+0000 7f566fb99740 1 -- 172.21.15.69:0/3816163899 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56179b981b80 con 0x56179ba27600 2024-05-14T16:57:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.394+0000 7f5665d9b640 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56179ba27600 0x56179ba299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.396+0000 7f5665d9b640 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56179ba27600 0x56179ba299f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47322defc204255b server_cookie=94fa149288d34e34 in_seq=0 out_seq=0 2024-05-14T16:57:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.396+0000 7f564dffb640 10 client.16245 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.397+0000 7f564dffb640 1 -- 172.21.15.69:0/3816163899 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56179b981b80 con 0x56179ba27600 2024-05-14T16:57:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.397+0000 7f564dffb640 10 client.16245 handle_command_reply: tid=0 2024-05-14T16:57:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.397+0000 7f566fb99740 1 client.16245 shutdown 2024-05-14T16:57:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.400+0000 7f566fb99740 1 -- 172.21.15.69:0/3816163899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0c8f0 msgr2=0x56179ba173d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.400+0000 7f566fb99740 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0c8f0 0x56179ba173d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5658004ac0 tx=0x7f565800b6e0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.402+0000 7f566fb99740 1 -- 172.21.15.69:0/3816163899 shutdown_connections 2024-05-14T16:57:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.402+0000 7f566fb99740 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56179ba27600 0x56179ba299f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.402+0000 7f566fb99740 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba17ad0 0x56179ba14c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.402+0000 7f566fb99740 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba0d210 0x56179ba146c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.402+0000 7f566fb99740 1 --2- 172.21.15.69:0/3816163899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0c8f0 0x56179ba173d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.402+0000 7f566fb99740 1 -- 172.21.15.69:0/3816163899 >> 172.21.15.69:0/3816163899 conn(0x56179ba08e00 msgr2=0x56179ba08100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.403+0000 7f564dffb640 0 client.16245 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.405+0000 7f566fb99740 1 -- 172.21.15.69:0/3816163899 shutdown_connections 2024-05-14T16:57:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.406+0000 7f566fb99740 1 -- 172.21.15.69:0/3816163899 wait complete. 2024-05-14T16:57:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.406+0000 7f566fb99740 20 client.16245 trim_cache size 0 max 16384 2024-05-14T16:57:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.439+0000 7f566fb99740 1 Processor -- start 2024-05-14T16:57:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.441+0000 7f566fb99740 1 -- start start 2024-05-14T16:57:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.442+0000 7f566fb99740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba0fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.442+0000 7f566fb99740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba07f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.442+0000 7f566fb99740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba16f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.442+0000 7f566659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba0fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.442+0000 7f566659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba0fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:47374/0 (socket says 172.21.15.69:47374) 2024-05-14T16:57:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.442+0000 7f566659c640 1 -- 172.21.15.69:0/2547201807 learned_addr learned my addr 172.21.15.69:0/2547201807 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.442+0000 7f566fb99740 1 -- 172.21.15.69:0/2547201807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f56480111c0 con 0x56179ba0d210 2024-05-14T16:57:40.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.443+0000 7f566fb99740 1 -- 172.21.15.69:0/2547201807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f564800ea30 con 0x56179ba17ad0 2024-05-14T16:57:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.443+0000 7f5665d9b640 1 --2- 172.21.15.69:0/2547201807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba07f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.443+0000 7f566fb99740 1 -- 172.21.15.69:0/2547201807 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f564800c8a0 con 0x56179ba0c8f0 2024-05-14T16:57:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.443+0000 7f5665d9b640 1 -- 172.21.15.69:0/2547201807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 msgr2=0x56179ba0fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.443+0000 7f5665d9b640 1 --2- 172.21.15.69:0/2547201807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba0fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.443+0000 7f5665d9b640 1 -- 172.21.15.69:0/2547201807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 msgr2=0x56179ba16f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:57:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.443+0000 7f5665d9b640 1 --2- 172.21.15.69:0/2547201807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba16f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.443+0000 7f5665d9b640 1 -- 172.21.15.69:0/2547201807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179ba12c10 con 0x56179ba0d210 2024-05-14T16:57:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.443+0000 7f5665d9b640 1 --2- 172.21.15.69:0/2547201807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba07f80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x56179ba2a230 tx=0x7f565c043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79355bed579fda5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.445+0000 7f564ffff640 1 -- 172.21.15.69:0/2547201807 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c003cc0 con 0x56179ba0d210 2024-05-14T16:57:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.445+0000 7f564ffff640 1 -- 172.21.15.69:0/2547201807 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565c052070 con 0x56179ba0d210 2024-05-14T16:57:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.445+0000 7f564ffff640 1 -- 172.21.15.69:0/2547201807 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c030720 con 0x56179ba0d210 2024-05-14T16:57:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.447+0000 7f566fb99740 1 -- 172.21.15.69:0/2547201807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 msgr2=0x56179ba07f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.447+0000 7f566fb99740 1 --2- 172.21.15.69:0/2547201807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba07f80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x56179ba2a230 tx=0x7f565c043000 comp rx=0 tx=0).stop 2024-05-14T16:57:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.448+0000 7f566fb99740 1 -- 172.21.15.69:0/2547201807 shutdown_connections 2024-05-14T16:57:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.448+0000 7f566fb99740 1 --2- 172.21.15.69:0/2547201807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba16f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.448+0000 7f566fb99740 1 --2- 172.21.15.69:0/2547201807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba07f80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.448+0000 7f566fb99740 1 --2- 172.21.15.69:0/2547201807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba0fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.448+0000 7f566fb99740 1 -- 172.21.15.69:0/2547201807 >> 172.21.15.69:0/2547201807 conn(0x56179ba08e00 msgr2=0x56179ba07870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.449+0000 7f566fb99740 1 -- 172.21.15.69:0/2547201807 shutdown_connections 2024-05-14T16:57:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.449+0000 7f566fb99740 1 -- 172.21.15.69:0/2547201807 wait complete. 2024-05-14T16:57:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.449+0000 7f566fb99740 1 -- start start 2024-05-14T16:57:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.450+0000 7f566fb99740 1 Processor -- start 2024-05-14T16:57:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.450+0000 7f566fb99740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba1fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.450+0000 7f566fb99740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba1b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f566659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba1fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f566659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba1fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:47386/0 (socket says 172.21.15.69:47386) 2024-05-14T16:57:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f566659c640 1 -- 172.21.15.69:0/1677487456 learned_addr learned my addr 172.21.15.69:0/1677487456 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f5665d9b640 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba1b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f566fb99740 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba1b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f564800cef0 con 0x56179ba0d210 2024-05-14T16:57:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f564800e360 con 0x56179ba17ad0 2024-05-14T16:57:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f564800d5c0 con 0x56179ba0c8f0 2024-05-14T16:57:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f5666d9d640 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba1b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f566fb99740 10 client.0 _ll_register_callbacks cb 0x7f565800b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f5666d9d640 1 -- 172.21.15.69:0/1677487456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 msgr2=0x56179ba1fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.451+0000 7f5666d9d640 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba1fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.452+0000 7f5666d9d640 1 -- 172.21.15.69:0/1677487456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 msgr2=0x56179ba1b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.452+0000 7f5666d9d640 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba1b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.452+0000 7f5666d9d640 1 -- 172.21.15.69:0/1677487456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179ba1be00 con 0x56179ba17ad0 2024-05-14T16:57:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.453+0000 7f5666d9d640 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba1b710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f56600f0350 tx=0x7f5660004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863852329e59ef51 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.453+0000 7f564dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:57:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.453+0000 7f564dffb640 1 -- 172.21.15.69:0/1677487456 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566002b070 con 0x56179ba17ad0 2024-05-14T16:57:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.453+0000 7f564dffb640 1 -- 172.21.15.69:0/1677487456 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f56600c28b0 con 0x56179ba17ad0 2024-05-14T16:57:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.453+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565c052070 con 0x56179ba17ad0 2024-05-14T16:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.455+0000 7f564dffb640 1 -- 172.21.15.69:0/1677487456 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f56600f07e0 con 0x56179ba17ad0 2024-05-14T16:57:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.456+0000 7f566fb99740 10 client.25715 fetch_fsmap learned FSMap version 40 2024-05-14T16:57:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.456+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56179ba91d30 con 0x56179ba17ad0 2024-05-14T16:57:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.456+0000 7f564dffb640 1 -- 172.21.15.69:0/1677487456 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56600be290 con 0x56179ba17ad0 2024-05-14T16:57:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.457+0000 7f564dffb640 1 -- 172.21.15.69:0/1677487456 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f56600b5850 con 0x56179ba17ad0 2024-05-14T16:57:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.457+0000 7f566fb99740 10 client.25715 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T16:57:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.457+0000 7f566fb99740 10 client.25715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.457+0000 7f566fb99740 20 client.25715 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.457+0000 7f566fb99740 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56179ba92430 0x56179ba94850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.457+0000 7f566fb99740 4 client.25715 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:57:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.457+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f565c052070 con 0x56179ba92430 2024-05-14T16:57:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.457+0000 7f566659c640 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56179ba92430 0x56179ba94850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.458+0000 7f566659c640 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56179ba92430 0x56179ba94850 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c39c679795df19 server_cookie=759b914a1a529e45 in_seq=0 out_seq=0 2024-05-14T16:57:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.460+0000 7f564dffb640 10 client.25715 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.592+0000 7f564dffb640 1 -- 172.21.15.69:0/1677487456 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f565c052070 con 0x56179ba92430 2024-05-14T16:57:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.592+0000 7f564dffb640 10 client.25715 handle_command_reply: tid=0 2024-05-14T16:57:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.592+0000 7f566fb99740 1 client.25715 shutdown 2024-05-14T16:57:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.592+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 msgr2=0x56179ba1b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.592+0000 7f566fb99740 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba1b710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f56600f0350 tx=0x7f5660004730 comp rx=0 tx=0).stop 2024-05-14T16:57:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 shutdown_connections 2024-05-14T16:57:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f566fb99740 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56179ba92430 0x56179ba94850 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f566fb99740 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56179ba17ad0 0x56179ba1b710 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f564dffb640 0 client.25715 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f566fb99740 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56179ba0d210 0x56179ba1b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f566fb99740 1 --2- 172.21.15.69:0/1677487456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56179ba0c8f0 0x56179ba1fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 >> 172.21.15.69:0/1677487456 conn(0x56179ba08e00 msgr2=0x56179ba08100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 shutdown_connections 2024-05-14T16:57:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f566fb99740 1 -- 172.21.15.69:0/1677487456 wait complete. 2024-05-14T16:57:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f566fb99740 20 client.25715 trim_cache size 0 max 16384 2024-05-14T16:57:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:57:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:57:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:57:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.593+0000 7f566cda4640 1 -- 172.21.15.69:0/631452006 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f56500b2c20 msgr2=0x7f56500b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f56500b2c20 0x7f56500b50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f565c0057c0 tx=0x7f565c009450 comp rx=0 tx=0).stop 2024-05-14T16:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 -- 172.21.15.69:0/631452006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668117e80 msgr2=0x7f5668118330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668117e80 0x7f5668118330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5660005fd0 tx=0x7f566001d380 comp rx=0 tx=0).stop 2024-05-14T16:57:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 -- 172.21.15.69:0/631452006 shutdown_connections 2024-05-14T16:57:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f56500b2c20 0x7f56500b50e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5668117e80 0x7f5668118330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566811c230 0x7f5668117940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 --2- 172.21.15.69:0/631452006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668111240 0x7f566811b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 -- 172.21.15.69:0/631452006 >> 172.21.15.69:0/631452006 conn(0x7f566806db20 msgr2=0x7f566811c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 -- 172.21.15.69:0/631452006 shutdown_connections 2024-05-14T16:57:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:39.594+0000 7f566cda4640 1 -- 172.21.15.69:0/631452006 wait complete. 2024-05-14T16:57:41.753 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:39 smithi069 ceph-mon[31507]: pgmap v241: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.2 MiB/s wr, 444 op/s 2024-05-14T16:57:41.782 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 430.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:57:42.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:41 smithi069 ceph-mon[31507]: pgmap v242: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 933 op/s 2024-05-14T16:57:42.213 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:41 smithi203 ceph-mon[35787]: pgmap v242: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 933 op/s 2024-05-14T16:57:42.255 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:41 smithi092 ceph-mon[35763]: pgmap v242: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 933 op/s 2024-05-14T16:57:44.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:43 smithi069 ceph-mon[31507]: pgmap v243: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 27 MiB/s wr, 1.48k op/s 2024-05-14T16:57:44.212 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:43 smithi203 ceph-mon[35787]: pgmap v243: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 27 MiB/s wr, 1.48k op/s 2024-05-14T16:57:44.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:43 smithi092 ceph-mon[35763]: pgmap v243: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 27 MiB/s wr, 1.48k op/s 2024-05-14T16:57:45.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 435.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:57:46.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:45 smithi069 ceph-mon[31507]: pgmap v244: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.35k op/s 2024-05-14T16:57:46.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:45 smithi203 ceph-mon[35787]: pgmap v244: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.35k op/s 2024-05-14T16:57:46.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:45 smithi092 ceph-mon[35763]: pgmap v244: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.35k op/s 2024-05-14T16:57:48.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:47 smithi069 ceph-mon[31507]: pgmap v245: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 25 MiB/s wr, 1.35k op/s 2024-05-14T16:57:48.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:47 smithi203 ceph-mon[35787]: pgmap v245: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 25 MiB/s wr, 1.35k op/s 2024-05-14T16:57:48.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:47 smithi092 ceph-mon[35763]: pgmap v245: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 25 MiB/s wr, 1.35k op/s 2024-05-14T16:57:50.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:49 smithi069 ceph-mon[31507]: pgmap v246: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 1.20k op/s 2024-05-14T16:57:50.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:49 smithi203 ceph-mon[35787]: pgmap v246: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 1.20k op/s 2024-05-14T16:57:50.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:49 smithi092 ceph-mon[35763]: pgmap v246: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 1.20k op/s 2024-05-14T16:57:50.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 440.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:57:52.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:51 smithi069 ceph-mon[31507]: pgmap v247: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.20k op/s 2024-05-14T16:57:52.213 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:51 smithi203 ceph-mon[35787]: pgmap v247: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.20k op/s 2024-05-14T16:57:52.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:51 smithi092 ceph-mon[35763]: pgmap v247: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.20k op/s 2024-05-14T16:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.694+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/1110476864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe98101c30 msgr2=0x7fbe981020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.694+0000 7fbe9f1b6640 1 --2- 172.21.15.69:0/1110476864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe98101c30 0x7fbe981020b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009930 tx=0x7fbe8402a420 comp rx=0 tx=0).stop 2024-05-14T16:57:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.694+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/1110476864 shutdown_connections 2024-05-14T16:57:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.694+0000 7fbe9f1b6640 1 --2- 172.21.15.69:0/1110476864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe981025f0 0x7fbe9810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.694+0000 7fbe9f1b6640 1 --2- 172.21.15.69:0/1110476864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe98101c30 0x7fbe981020b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.694+0000 7fbe9f1b6640 1 --2- 172.21.15.69:0/1110476864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe98107a70 0x7fbe98107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.694+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/1110476864 >> 172.21.15.69:0/1110476864 conn(0x7fbe980786a0 msgr2=0x7fbe98078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.694+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/1110476864 shutdown_connections 2024-05-14T16:57:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.694+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/1110476864 wait complete. 2024-05-14T16:57:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe9f1b6640 1 Processor -- start 2024-05-14T16:57:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe9f1b6640 1 -- start start 2024-05-14T16:57:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe9f1b6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe98101c30 0x7fbe98199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe9f1b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe981025f0 0x7fbe9819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe9f1b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe98107a70 0x7fbe981a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe9f1b6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98111bf0 con 0x7fbe981025f0 2024-05-14T16:57:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe9f1b6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98111a70 con 0x7fbe98107a70 2024-05-14T16:57:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe9f1b6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98111d70 con 0x7fbe98101c30 2024-05-14T16:57:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe97fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe981025f0 0x7fbe9819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe97fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe981025f0 0x7fbe9819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33046/0 (socket says 172.21.15.69:33046) 2024-05-14T16:57:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe97fff640 1 -- 172.21.15.69:0/2221304662 learned_addr learned my addr 172.21.15.69:0/2221304662 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.695+0000 7fbe9d753640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe98107a70 0x7fbe981a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe9cf52640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe98101c30 0x7fbe98199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe97fff640 1 -- 172.21.15.69:0/2221304662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe98101c30 msgr2=0x7fbe98199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe97fff640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe98101c30 0x7fbe98199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe97fff640 1 -- 172.21.15.69:0/2221304662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe98107a70 msgr2=0x7fbe981a5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:53.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe97fff640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe98107a70 0x7fbe981a5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe97fff640 1 -- 172.21.15.69:0/2221304662 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe981a5550 con 0x7fbe981025f0 2024-05-14T16:57:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe9d753640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe98107a70 0x7fbe981a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe97fff640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe981025f0 0x7fbe9819a4d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009230 tx=0x7fbe84004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe95ffb640 1 -- 172.21.15.69:0/2221304662 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8403f030 con 0x7fbe981025f0 2024-05-14T16:57:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/2221304662 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe981a5720 con 0x7fbe981025f0 2024-05-14T16:57:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe95ffb640 1 -- 172.21.15.69:0/2221304662 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe84003aa0 con 0x7fbe981025f0 2024-05-14T16:57:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe95ffb640 1 -- 172.21.15.69:0/2221304662 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84035070 con 0x7fbe981025f0 2024-05-14T16:57:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.696+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/2221304662 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe981a5b90 con 0x7fbe981025f0 2024-05-14T16:57:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbe95ffb640 1 -- 172.21.15.69:0/2221304662 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbe84042030 con 0x7fbe981025f0 2024-05-14T16:57:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbea1faa740 1 Processor -- start 2024-05-14T16:57:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbea1faa740 1 -- start start 2024-05-14T16:57:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34b7d0 0x55ad3f34bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f35aee0 0x55ad3f35d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbea1faa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ad3f2bec30 con 0x55ad3f34c0f0 2024-05-14T16:57:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbea1faa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ad3f2ae7d0 con 0x55ad3f34b7d0 2024-05-14T16:57:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbea1faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ad3f2b2810 con 0x55ad3f35aee0 2024-05-14T16:57:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbe9cf52640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbe9cf52640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33058/0 (socket says 172.21.15.69:33058) 2024-05-14T16:57:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbe9cf52640 1 -- 172.21.15.69:0/2802253270 learned_addr learned my addr 172.21.15.69:0/2802253270 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.699+0000 7fbe9d753640 1 --2- 172.21.15.69:0/2802253270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34b7d0 0x55ad3f34bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe97fff640 1 --2- 172.21.15.69:0/2802253270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f35aee0 0x55ad3f35d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe9cf52640 1 -- 172.21.15.69:0/2802253270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f35aee0 msgr2=0x55ad3f35d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe9cf52640 1 --2- 172.21.15.69:0/2802253270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f35aee0 0x55ad3f35d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe9cf52640 1 -- 172.21.15.69:0/2802253270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34b7d0 msgr2=0x55ad3f34bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe9cf52640 1 --2- 172.21.15.69:0/2802253270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34b7d0 0x55ad3f34bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe95ffb640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe880b2a10 0x7fbe880b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe9cf52640 1 -- 172.21.15.69:0/2802253270 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3efc3210 con 0x55ad3f34c0f0 2024-05-14T16:57:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe9d753640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe880b2a10 0x7fbe880b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe9cf52640 1 --2- 172.21.15.69:0/2802253270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a9a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009ca0 tx=0x7fbe8c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7908bb48745e00da server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe95ffb640 1 -- 172.21.15.69:0/2221304662 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fbe84102cc0 con 0x7fbe981025f0 2024-05-14T16:57:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe82ffd640 1 -- 172.21.15.69:0/2802253270 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c004250 con 0x55ad3f34c0f0 2024-05-14T16:57:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe82ffd640 1 -- 172.21.15.69:0/2802253270 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe8c0043f0 con 0x55ad3f34c0f0 2024-05-14T16:57:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.700+0000 7fbe82ffd640 1 -- 172.21.15.69:0/2802253270 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c019750 con 0x55ad3f34c0f0 2024-05-14T16:57:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 -- 172.21.15.69:0/2802253270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f34c0f0 msgr2=0x55ad3f35a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbe9d753640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe880b2a10 0x7fbe880b4ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbe90005fd0 tx=0x7fbe90009600 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 --2- 172.21.15.69:0/2802253270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a9a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009ca0 tx=0x7fbe8c011810 comp rx=0 tx=0).stop 2024-05-14T16:57:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 -- 172.21.15.69:0/2802253270 shutdown_connections 2024-05-14T16:57:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 --2- 172.21.15.69:0/2802253270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f35aee0 0x55ad3f35d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 --2- 172.21.15.69:0/2802253270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a9a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 --2- 172.21.15.69:0/2802253270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34b7d0 0x55ad3f34bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 -- 172.21.15.69:0/2802253270 >> 172.21.15.69:0/2802253270 conn(0x55ad3f347e00 msgr2=0x55ad3f3481e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 -- 172.21.15.69:0/2802253270 shutdown_connections 2024-05-14T16:57:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 -- 172.21.15.69:0/2802253270 wait complete. 2024-05-14T16:57:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 -- start start 2024-05-14T16:57:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.701+0000 7fbea1faa740 1 Processor -- start 2024-05-14T16:57:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34b7d0 0x55ad3f35a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 0x55ad3f357b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f358080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbea1faa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ad3ef94790 con 0x55ad3f35aee0 2024-05-14T16:57:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbea1faa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ad3f26a890 con 0x55ad3f34c0f0 2024-05-14T16:57:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbea1faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ad3f35dbb0 con 0x55ad3f34b7d0 2024-05-14T16:57:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbea1faa740 10 client.0 _ll_register_callbacks cb 0x55ad3f26dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbe9d753640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f358080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbe9d753640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f358080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33074/0 (socket says 172.21.15.69:33074) 2024-05-14T16:57:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbe9d753640 1 -- 172.21.15.69:0/3519246593 learned_addr learned my addr 172.21.15.69:0/3519246593 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbe9cf52640 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34b7d0 0x55ad3f35a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbe97fff640 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 0x55ad3f357b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbe9d753640 1 -- 172.21.15.69:0/3519246593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34b7d0 msgr2=0x55ad3f35a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbe9d753640 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34b7d0 0x55ad3f35a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbe9d753640 1 -- 172.21.15.69:0/3519246593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 msgr2=0x55ad3f357b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbe9d753640 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 0x55ad3f357b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.702+0000 7fbe9d753640 1 -- 172.21.15.69:0/3519246593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3f08f620 con 0x55ad3f35aee0 2024-05-14T16:57:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbe9d753640 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f358080 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbe900092f0 tx=0x7fbe90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c7cfa4a1068d63 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbe77fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:57:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbe77fff640 1 -- 172.21.15.69:0/3519246593 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9002b060 con 0x55ad3f35aee0 2024-05-14T16:57:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbe77fff640 1 -- 172.21.15.69:0/3519246593 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fbe9001c670 con 0x55ad3f35aee0 2024-05-14T16:57:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbea1faa740 1 -- 172.21.15.69:0/3519246593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe8c0043f0 con 0x55ad3f35aee0 2024-05-14T16:57:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbe77fff640 1 -- 172.21.15.69:0/3519246593 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbe9001c870 con 0x55ad3f35aee0 2024-05-14T16:57:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbea1faa740 10 client.16263 fetch_fsmap learned FSMap version 40 2024-05-14T16:57:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbea1faa740 1 -- 172.21.15.69:0/3519246593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ad3f3696b0 con 0x55ad3f35aee0 2024-05-14T16:57:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbe77fff640 1 -- 172.21.15.69:0/3519246593 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe8c0043f0 con 0x55ad3f35aee0 2024-05-14T16:57:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbe77fff640 1 -- 172.21.15.69:0/3519246593 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fbe9001a040 con 0x55ad3f35aee0 2024-05-14T16:57:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbea1faa740 10 client.16263 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T16:57:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbea1faa740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbea1faa740 20 client.16263 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbea1faa740 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ad3f36abc0 0x55ad3f36cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbea1faa740 4 client.16263 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.703+0000 7fbea1faa740 1 -- 172.21.15.69:0/3519246593 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad3f2c0b80 con 0x55ad3f36abc0 2024-05-14T16:57:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.704+0000 7fbe9cf52640 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ad3f36abc0 0x55ad3f36cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.704+0000 7fbe9cf52640 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ad3f36abc0 0x55ad3f36cfd0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f380e84b5a25943c server_cookie=1d0a800bc48f86e9 in_seq=0 out_seq=0 2024-05-14T16:57:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.704+0000 7fbe77fff640 10 client.16263 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbe77fff640 1 -- 172.21.15.69:0/3519246593 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ad3f2c0b80 con 0x55ad3f36abc0 2024-05-14T16:57:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbe77fff640 10 client.16263 handle_command_reply: tid=0 2024-05-14T16:57:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 client.16263 shutdown 2024-05-14T16:57:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 -- 172.21.15.69:0/3519246593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 msgr2=0x55ad3f358080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f358080 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbe900092f0 tx=0x7fbe90004a40 comp rx=0 tx=0).stop 2024-05-14T16:57:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 -- 172.21.15.69:0/3519246593 shutdown_connections 2024-05-14T16:57:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ad3f36abc0 0x55ad3f36cfd0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f358080 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbe77fff640 0 client.16263 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 0x55ad3f357b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 --2- 172.21.15.69:0/3519246593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34b7d0 0x55ad3f35a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 -- 172.21.15.69:0/3519246593 >> 172.21.15.69:0/3519246593 conn(0x55ad3f347e00 msgr2=0x55ad3f34dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 -- 172.21.15.69:0/3519246593 shutdown_connections 2024-05-14T16:57:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 1 -- 172.21.15.69:0/3519246593 wait complete. 2024-05-14T16:57:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.705+0000 7fbea1faa740 20 client.16263 trim_cache size 0 max 16384 2024-05-14T16:57:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.720+0000 7fbea1faa740 1 Processor -- start 2024-05-14T16:57:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.720+0000 7fbea1faa740 1 -- start start 2024-05-14T16:57:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.720+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34b7d0 0x55ad3f346f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.720+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.720+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f3728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.720+0000 7fbea1faa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981aed90 con 0x55ad3f35aee0 2024-05-14T16:57:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.720+0000 7fbea1faa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981ac600 con 0x55ad3f34c0f0 2024-05-14T16:57:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.720+0000 7fbea1faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981aa470 con 0x55ad3f34b7d0 2024-05-14T16:57:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.720+0000 7fbe9d753640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f3728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.720+0000 7fbe9d753640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f3728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33086/0 (socket says 172.21.15.69:33086) 2024-05-14T16:57:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe9d753640 1 -- 172.21.15.69:0/1748764822 learned_addr learned my addr 172.21.15.69:0/1748764822 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe9d753640 1 -- 172.21.15.69:0/1748764822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34b7d0 msgr2=0x55ad3f346f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe9d753640 1 --2- 172.21.15.69:0/1748764822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34b7d0 0x55ad3f346f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe9d753640 1 -- 172.21.15.69:0/1748764822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 msgr2=0x55ad3f35a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe97fff640 1 --2- 172.21.15.69:0/1748764822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe9d753640 1 --2- 172.21.15.69:0/1748764822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe9d753640 1 -- 172.21.15.69:0/1748764822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3f347c30 con 0x55ad3f35aee0 2024-05-14T16:57:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe97fff640 1 --2- 172.21.15.69:0/1748764822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:57:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe9d753640 1 --2- 172.21.15.69:0/1748764822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f3728e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbe900092c0 tx=0x7fbe90011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ffd28784b20023 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe827fc640 1 -- 172.21.15.69:0/1748764822 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90004ab0 con 0x55ad3f35aee0 2024-05-14T16:57:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe827fc640 1 -- 172.21.15.69:0/1748764822 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe90004c50 con 0x55ad3f35aee0 2024-05-14T16:57:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.721+0000 7fbe827fc640 1 -- 172.21.15.69:0/1748764822 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90005040 con 0x55ad3f35aee0 2024-05-14T16:57:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 -- 172.21.15.69:0/1748764822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 msgr2=0x55ad3f3728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 --2- 172.21.15.69:0/1748764822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f3728e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbe900092c0 tx=0x7fbe90011f90 comp rx=0 tx=0).stop 2024-05-14T16:57:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 -- 172.21.15.69:0/1748764822 shutdown_connections 2024-05-14T16:57:54.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 --2- 172.21.15.69:0/1748764822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f3728e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 --2- 172.21.15.69:0/1748764822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 --2- 172.21.15.69:0/1748764822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34b7d0 0x55ad3f346f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 -- 172.21.15.69:0/1748764822 >> 172.21.15.69:0/1748764822 conn(0x55ad3f347e00 msgr2=0x55ad3f346870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 -- 172.21.15.69:0/1748764822 shutdown_connections 2024-05-14T16:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 -- 172.21.15.69:0/1748764822 wait complete. 2024-05-14T16:57:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 -- start start 2024-05-14T16:57:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.722+0000 7fbea1faa740 1 Processor -- start 2024-05-14T16:57:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34b7d0 0x55ad3f35ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbea1faa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f35f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbea1faa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981aaac0 con 0x55ad3f35aee0 2024-05-14T16:57:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbea1faa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981abf30 con 0x55ad3f34b7d0 2024-05-14T16:57:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbea1faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981ab190 con 0x55ad3f34c0f0 2024-05-14T16:57:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbea1faa740 10 client.0 _ll_register_callbacks cb 0x7fbe90011e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe9d753640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f35f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe9d753640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f35f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33092/0 (socket says 172.21.15.69:33092) 2024-05-14T16:57:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe9d753640 1 -- 172.21.15.69:0/2704625287 learned_addr learned my addr 172.21.15.69:0/2704625287 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe97fff640 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe9cf52640 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34b7d0 0x55ad3f35ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe9d753640 1 -- 172.21.15.69:0/2704625287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34c0f0 msgr2=0x55ad3f35efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe9d753640 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe9d753640 1 -- 172.21.15.69:0/2704625287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34b7d0 msgr2=0x55ad3f35ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe9d753640 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34b7d0 0x55ad3f35ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe9d753640 1 -- 172.21.15.69:0/2704625287 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3f3d4510 con 0x55ad3f35aee0 2024-05-14T16:57:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe9d753640 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f35f4f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbe90010e90 tx=0x7fbe90005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5aed39b7b0c6c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe77fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:57:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe77fff640 1 -- 172.21.15.69:0/2704625287 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9003e070 con 0x55ad3f35aee0 2024-05-14T16:57:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe77fff640 1 -- 172.21.15.69:0/2704625287 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fbe9001ca10 con 0x55ad3f35aee0 2024-05-14T16:57:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbea1faa740 1 -- 172.21.15.69:0/2704625287 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe90004c50 con 0x55ad3f35aee0 2024-05-14T16:57:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbe77fff640 1 -- 172.21.15.69:0/2704625287 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbe9004a050 con 0x55ad3f35aee0 2024-05-14T16:57:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbea1faa740 10 client.16269 fetch_fsmap learned FSMap version 40 2024-05-14T16:57:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.723+0000 7fbea1faa740 1 -- 172.21.15.69:0/2704625287 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ad3f3d5770 con 0x55ad3f35aee0 2024-05-14T16:57:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.724+0000 7fbe77fff640 1 -- 172.21.15.69:0/2704625287 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe90004c50 con 0x55ad3f35aee0 2024-05-14T16:57:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.724+0000 7fbe77fff640 1 -- 172.21.15.69:0/2704625287 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fbe9001a070 con 0x55ad3f35aee0 2024-05-14T16:57:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.724+0000 7fbea1faa740 10 client.16269 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T16:57:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.724+0000 7fbea1faa740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.724+0000 7fbea1faa740 20 client.16269 populate_metadata read hostname 'smithi069' 2024-05-14T16:57:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.724+0000 7fbea1faa740 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ad3f36abc0 0x55ad3f3d7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.724+0000 7fbea1faa740 4 client.16269 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:57:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.724+0000 7fbea1faa740 1 -- 172.21.15.69:0/2704625287 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad3f2ba360 con 0x55ad3f36abc0 2024-05-14T16:57:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.724+0000 7fbe9cf52640 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ad3f36abc0 0x55ad3f3d7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.725+0000 7fbe77fff640 10 client.16269 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.725+0000 7fbe9cf52640 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ad3f36abc0 0x55ad3f3d7de0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c1b658ca4e46a3 server_cookie=7acffcd8ee83e7f2 in_seq=0 out_seq=0 2024-05-14T16:57:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbe77fff640 1 -- 172.21.15.69:0/2704625287 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ad3f2ba360 con 0x55ad3f36abc0 2024-05-14T16:57:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbe77fff640 10 client.16269 handle_command_reply: tid=0 2024-05-14T16:57:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbea1faa740 1 client.16269 shutdown 2024-05-14T16:57:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbea1faa740 1 -- 172.21.15.69:0/2704625287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 msgr2=0x55ad3f35f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbea1faa740 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f35f4f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbe90010e90 tx=0x7fbe90005c60 comp rx=0 tx=0).stop 2024-05-14T16:57:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbea1faa740 1 -- 172.21.15.69:0/2704625287 shutdown_connections 2024-05-14T16:57:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbea1faa740 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ad3f36abc0 0x55ad3f3d7de0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbea1faa740 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad3f35aee0 0x55ad3f35f4f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbea1faa740 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ad3f34c0f0 0x55ad3f35efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbe77fff640 0 client.16269 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:57:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbea1faa740 1 --2- 172.21.15.69:0/2704625287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad3f34b7d0 0x55ad3f35ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.738+0000 7fbea1faa740 1 -- 172.21.15.69:0/2704625287 >> 172.21.15.69:0/2704625287 conn(0x55ad3f347e00 msgr2=0x55ad3f347100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbea1faa740 1 -- 172.21.15.69:0/2704625287 shutdown_connections 2024-05-14T16:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbea1faa740 1 -- 172.21.15.69:0/2704625287 wait complete. 2024-05-14T16:57:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbea1faa740 20 client.16269 trim_cache size 0 max 16384 2024-05-14T16:57:55.399 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:57:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:57:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:57:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:57:55.424 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:53 smithi092 ceph-mon[35763]: pgmap v248: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 712 op/s 2024-05-14T16:57:55.429 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:54 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:55.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:53 smithi203 ceph-mon[35787]: pgmap v248: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 712 op/s 2024-05-14T16:57:55.440 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:54 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/2221304662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe880b2a10 msgr2=0x7fbe880b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe880b2a10 0x7fbe880b4ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbe90005fd0 tx=0x7fbe90009600 comp rx=0 tx=0).stop 2024-05-14T16:57:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/2221304662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe981025f0 msgr2=0x7fbe9819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe981025f0 0x7fbe9819a4d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009230 tx=0x7fbe84004270 comp rx=0 tx=0).stop 2024-05-14T16:57:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/2221304662 shutdown_connections 2024-05-14T16:57:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe880b2a10 0x7fbe880b4ed0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe98107a70 0x7fbe981a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe981025f0 0x7fbe9819a4d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 --2- 172.21.15.69:0/2221304662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe98101c30 0x7fbe98199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/2221304662 >> 172.21.15.69:0/2221304662 conn(0x7fbe980786a0 msgr2=0x7fbe98100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/2221304662 shutdown_connections 2024-05-14T16:57:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:57:52.739+0000 7fbe9f1b6640 1 -- 172.21.15.69:0/2221304662 wait complete. 2024-05-14T16:57:55.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:53 smithi069 ceph-mon[31507]: pgmap v248: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 712 op/s 2024-05-14T16:57:55.555 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:54 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:55.561 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 445.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:57:55.827 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:57:56.176 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:55 smithi069 ceph-mon[31507]: pgmap v249: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T16:57:56.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:55 smithi203 ceph-mon[35787]: pgmap v249: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T16:57:56.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:55 smithi092 ceph-mon[35763]: pgmap v249: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T16:57:57.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.589+0000 7f44d2645700 1 -- 172.21.15.69:0/94173385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc10ae60 msgr2=0x7f44cc116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:57.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.589+0000 7f44d2645700 1 --2- 172.21.15.69:0/94173385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc10ae60 0x7f44cc116150 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009990 tx=0x7f44b4009ca0 comp rx=0 tx=0).stop 2024-05-14T16:57:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.589+0000 7f44d2645700 1 -- 172.21.15.69:0/94173385 shutdown_connections 2024-05-14T16:57:57.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.589+0000 7f44d2645700 1 --2- 172.21.15.69:0/94173385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc116690 0x7f44cc118a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:57.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.589+0000 7f44d2645700 1 --2- 172.21.15.69:0/94173385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44cc10a540 0x7f44cc10a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:57.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.589+0000 7f44d2645700 1 --2- 172.21.15.69:0/94173385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc10ae60 0x7f44cc116150 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:57.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.589+0000 7f44d2645700 1 -- 172.21.15.69:0/94173385 >> 172.21.15.69:0/94173385 conn(0x7f44cc06e6b0 msgr2=0x7f44cc06eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:57.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.589+0000 7f44d2645700 1 -- 172.21.15.69:0/94173385 shutdown_connections 2024-05-14T16:57:57.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.589+0000 7f44d2645700 1 -- 172.21.15.69:0/94173385 wait complete. 2024-05-14T16:57:57.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d2645700 1 Processor -- start 2024-05-14T16:57:57.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d2645700 1 -- start start 2024-05-14T16:57:57.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d2645700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc10a540 0x7f44cc1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:57.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d2645700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44cc10ae60 0x7f44cc1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:57.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d2645700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc116690 0x7f44cc1a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:57.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d2645700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10d350 con 0x7f44cc116690 2024-05-14T16:57:57.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d2645700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10d1d0 con 0x7f44cc10a540 2024-05-14T16:57:57.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d2645700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10d4d0 con 0x7f44cc10ae60 2024-05-14T16:57:57.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d0be2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc116690 0x7f44cc1a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:57.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d0be2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc116690 0x7f44cc1a8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44600/0 (socket says 172.21.15.69:44600) 2024-05-14T16:57:57.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44d0be2700 1 -- 172.21.15.69:0/2345594866 learned_addr learned my addr 172.21.15.69:0/2345594866 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:57:57.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.590+0000 7f44cbfff700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc10a540 0x7f44cc1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:57.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44c3fff700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44cc10ae60 0x7f44cc1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:57.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44d0be2700 1 -- 172.21.15.69:0/2345594866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44cc10ae60 msgr2=0x7f44cc1a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:57.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44d0be2700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44cc10ae60 0x7f44cc1a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:57.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44d0be2700 1 -- 172.21.15.69:0/2345594866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc10a540 msgr2=0x7f44cc1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:57.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44d0be2700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc10a540 0x7f44cc1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:57.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44d0be2700 1 -- 172.21.15.69:0/2345594866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44cc1a96e0 con 0x7f44cc116690 2024-05-14T16:57:57.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44d0be2700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc116690 0x7f44cc1a8fa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f44c4007ee0 tx=0x7f44c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44c9ffb700 1 -- 172.21.15.69:0/2345594866 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4004bb0 con 0x7f44cc116690 2024-05-14T16:57:57.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44d2645700 1 -- 172.21.15.69:0/2345594866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44cc1a99d0 con 0x7f44cc116690 2024-05-14T16:57:57.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44c9ffb700 1 -- 172.21.15.69:0/2345594866 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44c4004d50 con 0x7f44cc116690 2024-05-14T16:57:58.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44c9ffb700 1 -- 172.21.15.69:0/2345594866 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4005920 con 0x7f44cc116690 2024-05-14T16:57:58.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.591+0000 7f44d2645700 1 -- 172.21.15.69:0/2345594866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44cc1b1250 con 0x7f44cc116690 2024-05-14T16:57:58.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.592+0000 7f44d2645700 1 -- 172.21.15.69:0/2345594866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44cc06a230 con 0x7f44cc116690 2024-05-14T16:57:58.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.594+0000 7f44c9ffb700 1 -- 172.21.15.69:0/2345594866 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f44c401a020 con 0x7f44cc116690 2024-05-14T16:57:58.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.594+0000 7f44c9ffb700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44b80b29f0 0x7f44b80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:58.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.594+0000 7f44cbfff700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44b80b29f0 0x7f44b80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:58.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.594+0000 7f44c9ffb700 1 -- 172.21.15.69:0/2345594866 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f44c40e1c80 con 0x7f44cc116690 2024-05-14T16:57:58.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.595+0000 7f44cbfff700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44b80b29f0 0x7f44b80b4eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009a10 tx=0x7f44bc009190 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:58.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.599+0000 7f44c9ffb700 1 -- 172.21.15.69:0/2345594866 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44c4010070 con 0x7f44cc116690 2024-05-14T16:57:58.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.846+0000 7f44d2645700 1 -- 172.21.15.69:0/2345594866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44cc02e560 con 0x7f44cc116690 2024-05-14T16:57:58.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.847+0000 7f44c9ffb700 1 -- 172.21.15.69:0/2345594866 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11863 (secure 0 0 0) 0x7f44c40a9650 con 0x7f44cc116690 2024-05-14T16:57:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:57:58.105 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:57:53.712905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24446,24422,24433]},"id":1}]} 2024-05-14T16:57:58.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.849+0000 7f44d2645700 1 -- 172.21.15.69:0/2345594866 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44b80b29f0 msgr2=0x7f44b80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:58.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.849+0000 7f44d2645700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44b80b29f0 0x7f44b80b4eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009a10 tx=0x7f44bc009190 comp rx=0 tx=0).stop 2024-05-14T16:57:58.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.849+0000 7f44d2645700 1 -- 172.21.15.69:0/2345594866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc116690 msgr2=0x7f44cc1a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:58.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.849+0000 7f44d2645700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc116690 0x7f44cc1a8fa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f44c4007ee0 tx=0x7f44c400d3b0 comp rx=0 tx=0).stop 2024-05-14T16:57:58.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.850+0000 7f44d2645700 1 -- 172.21.15.69:0/2345594866 shutdown_connections 2024-05-14T16:57:58.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.850+0000 7f44d2645700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f44b80b29f0 0x7f44b80b4eb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:58.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.850+0000 7f44d2645700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc116690 0x7f44cc1a8fa0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:58.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.850+0000 7f44d2645700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc10a540 0x7f44cc1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.850+0000 7f44d2645700 1 --2- 172.21.15.69:0/2345594866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44cc10ae60 0x7f44cc1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:58.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.850+0000 7f44d2645700 1 -- 172.21.15.69:0/2345594866 >> 172.21.15.69:0/2345594866 conn(0x7f44cc06e6b0 msgr2=0x7f44cc10e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:58.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.850+0000 7f44d2645700 1 -- 172.21.15.69:0/2345594866 shutdown_connections 2024-05-14T16:57:58.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:57:57.850+0000 7f44d2645700 1 -- 172.21.15.69:0/2345594866 wait complete. 2024-05-14T16:57:58.196 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 41 2024-05-14T16:57:58.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:57 smithi069 ceph-mon[31507]: pgmap v250: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-05-14T16:57:58.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:57 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2345594866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:57:58.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:57 smithi203 ceph-mon[35787]: pgmap v250: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-05-14T16:57:58.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:57 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2345594866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:57:58.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:57 smithi092 ceph-mon[35763]: pgmap v250: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-05-14T16:57:58.256 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:57 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2345594866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:57:58.716 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id 37a5760b --release --await' 2024-05-14T16:57:58.723 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id 37a5760b --release --await 2024-05-14T16:57:58.977 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:58:00.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:57:59 smithi069 ceph-mon[31507]: pgmap v251: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-14T16:58:00.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:57:59 smithi203 ceph-mon[35787]: pgmap v251: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-14T16:58:00.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:57:59 smithi092 ceph-mon[35763]: pgmap v251: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-14T16:58:00.398 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 450.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:58:00.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.958+0000 7ff350316700 1 -- 172.21.15.69:0/2135057192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481095e0 msgr2=0x7ff3481099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:00.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.958+0000 7ff350316700 1 --2- 172.21.15.69:0/2135057192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481095e0 0x7ff3481099c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff344009990 tx=0x7ff344009ca0 comp rx=0 tx=0).stop 2024-05-14T16:58:00.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.958+0000 7ff350316700 1 -- 172.21.15.69:0/2135057192 shutdown_connections 2024-05-14T16:58:01.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.958+0000 7ff350316700 1 --2- 172.21.15.69:0/2135057192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481095e0 0x7ff3481099c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.958+0000 7ff350316700 1 --2- 172.21.15.69:0/2135057192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff348102490 0x7ff3481104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.958+0000 7ff350316700 1 --2- 172.21.15.69:0/2135057192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff348101ad0 0x7ff348101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.958+0000 7ff350316700 1 -- 172.21.15.69:0/2135057192 >> 172.21.15.69:0/2135057192 conn(0x7ff348078a40 msgr2=0x7ff3480ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:01.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.958+0000 7ff350316700 1 -- 172.21.15.69:0/2135057192 shutdown_connections 2024-05-14T16:58:01.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.958+0000 7ff350316700 1 -- 172.21.15.69:0/2135057192 wait complete. 2024-05-14T16:58:01.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff350316700 1 Processor -- start 2024-05-14T16:58:01.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff350316700 1 -- start start 2024-05-14T16:58:01.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff350316700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff348101ad0 0x7ff348102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:01.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff350316700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff348102490 0x7ff3481031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff350316700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481095e0 0x7ff348103730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:01.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff350316700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff348113810 con 0x7ff3481095e0 2024-05-14T16:58:01.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff350316700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff348113690 con 0x7ff348101ad0 2024-05-14T16:58:01.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff350316700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff348113990 con 0x7ff348102490 2024-05-14T16:58:01.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34e8b3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481095e0 0x7ff348103730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:01.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34e8b3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481095e0 0x7ff348103730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44628/0 (socket says 172.21.15.69:44628) 2024-05-14T16:58:01.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34e8b3700 1 -- 172.21.15.69:0/283259729 learned_addr learned my addr 172.21.15.69:0/283259729 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:01.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff348101ad0 0x7ff348102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:01.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff348102490 0x7ff3481031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:01.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34e8b3700 1 -- 172.21.15.69:0/283259729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff348102490 msgr2=0x7ff3481031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:01.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff348102490 0x7ff3481031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34e8b3700 1 -- 172.21.15.69:0/283259729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff348101ad0 msgr2=0x7ff348102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:01.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff348101ad0 0x7ff348102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34e8b3700 1 -- 172.21.15.69:0/283259729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3481a8230 con 0x7ff3481095e0 2024-05-14T16:58:01.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.959+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff348102490 0x7ff3481031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:58:01.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.960+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481095e0 0x7ff348103730 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff340007f10 tx=0x7ff34000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:01.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.960+0000 7ff33f7fe700 1 -- 172.21.15.69:0/283259729 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340009e20 con 0x7ff3481095e0 2024-05-14T16:58:01.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.960+0000 7ff33f7fe700 1 -- 172.21.15.69:0/283259729 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff34000dbd0 con 0x7ff3481095e0 2024-05-14T16:58:01.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.960+0000 7ff350316700 1 -- 172.21.15.69:0/283259729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3481a8520 con 0x7ff3481095e0 2024-05-14T16:58:01.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.960+0000 7ff33f7fe700 1 -- 172.21.15.69:0/283259729 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340005990 con 0x7ff3481095e0 2024-05-14T16:58:01.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.960+0000 7ff350316700 1 -- 172.21.15.69:0/283259729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3481a8a10 con 0x7ff3481095e0 2024-05-14T16:58:01.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.962+0000 7ff33f7fe700 1 -- 172.21.15.69:0/283259729 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff34001a020 con 0x7ff3481095e0 2024-05-14T16:58:01.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.963+0000 7ff33f7fe700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff3340b29f0 0x7ff3340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:01.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.963+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff3340b29f0 0x7ff3340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:01.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.963+0000 7ff33f7fe700 1 -- 172.21.15.69:0/283259729 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7ff3400e1df0 con 0x7ff3481095e0 2024-05-14T16:58:01.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.963+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff3340b29f0 0x7ff3340b4eb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b5c0 tx=0x7ff344005fd0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:01.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.963+0000 7ff35abacb80 1 Processor -- start 2024-05-14T16:58:01.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff35abacb80 1 -- start start 2024-05-14T16:58:01.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcf420 0x55f3c7bcf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:01.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bcaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:01.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bcb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:01.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff35abacb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7b28e70 con 0x55f3c7bcb2c0 2024-05-14T16:58:01.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff35abacb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7b18a10 con 0x55f3c7bcfd40 2024-05-14T16:58:01.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff35abacb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7b1ca50 con 0x55f3c7bcf420 2024-05-14T16:58:01.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34e8b3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bcb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:01.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34e8b3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bcb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44630/0 (socket says 172.21.15.69:44630) 2024-05-14T16:58:01.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34e8b3700 1 -- 172.21.15.69:0/2510852552 learned_addr learned my addr 172.21.15.69:0/2510852552 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:01.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/2510852552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bcaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:01.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/2510852552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcf420 0x55f3c7bcf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:01.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34e8b3700 1 -- 172.21.15.69:0/2510852552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcf420 msgr2=0x55f3c7bcf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:01.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/2510852552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcf420 0x55f3c7bcf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34e8b3700 1 -- 172.21.15.69:0/2510852552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcfd40 msgr2=0x55f3c7bcaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/2510852552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bcaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34e8b3700 1 -- 172.21.15.69:0/2510852552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c7ac20b0 con 0x55f3c7bcb2c0 2024-05-14T16:58:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.964+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/2510852552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcf420 0x55f3c7bcf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:58:01.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/2510852552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bcb720 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff3400a8d20 tx=0x7ff3400a8aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd1276785a1b76e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:01.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff33cd70700 1 -- 172.21.15.69:0/2510852552 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400ad3d0 con 0x55f3c7bcb2c0 2024-05-14T16:58:01.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff33cd70700 1 -- 172.21.15.69:0/2510852552 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3400b1e80 con 0x55f3c7bcb2c0 2024-05-14T16:58:01.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff33cd70700 1 -- 172.21.15.69:0/2510852552 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400a9280 con 0x55f3c7bcb2c0 2024-05-14T16:58:01.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff35abacb80 1 -- 172.21.15.69:0/2510852552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 msgr2=0x55f3c7bcb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:01.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff35abacb80 1 --2- 172.21.15.69:0/2510852552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bcb720 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff3400a8d20 tx=0x7ff3400a8aa0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff35abacb80 1 -- 172.21.15.69:0/2510852552 shutdown_connections 2024-05-14T16:58:01.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff35abacb80 1 --2- 172.21.15.69:0/2510852552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bcb720 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff35abacb80 1 --2- 172.21.15.69:0/2510852552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcf420 0x55f3c7bcf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff35abacb80 1 --2- 172.21.15.69:0/2510852552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bcaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.965+0000 7ff35abacb80 1 -- 172.21.15.69:0/2510852552 >> 172.21.15.69:0/2510852552 conn(0x55f3c7af7eb0 msgr2=0x55f3c7bc54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:01.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 1 -- 172.21.15.69:0/2510852552 shutdown_connections 2024-05-14T16:58:01.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 1 -- 172.21.15.69:0/2510852552 wait complete. 2024-05-14T16:58:01.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 1 -- start start 2024-05-14T16:58:01.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 1 Processor -- start 2024-05-14T16:58:01.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bd6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:01.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:01.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bd4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:01.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7a1f0a0 con 0x55f3c7bcf420 2024-05-14T16:58:01.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7a80ea0 con 0x55f3c7bcb2c0 2024-05-14T16:58:01.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f3c7bd7880 con 0x55f3c7bcfd40 2024-05-14T16:58:01.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff35abacb80 10 client.0 _ll_register_callbacks cb 0x55f3c7ab3a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:01.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff34d8b1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:01.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff34d8b1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44642/0 (socket says 172.21.15.69:44642) 2024-05-14T16:58:01.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.966+0000 7ff34d8b1700 1 -- 172.21.15.69:0/4256587907 learned_addr learned my addr 172.21.15.69:0/4256587907 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:01.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bd6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:01.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bd4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:01.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff34d8b1700 1 -- 172.21.15.69:0/4256587907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcfd40 msgr2=0x55f3c7bd4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:01.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bd4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff34d8b1700 1 -- 172.21.15.69:0/4256587907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcb2c0 msgr2=0x55f3c7bd6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:01.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bd6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:01.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff34d8b1700 1 -- 172.21.15.69:0/4256587907 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c7a689d0 con 0x55f3c7bcf420 2024-05-14T16:58:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd3f50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff338003bc0 tx=0x7ff338003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdb2b8e5ea5d820 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:01.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff331ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:58:01.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff331ffb700 1 -- 172.21.15.69:0/4256587907 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338011b30 con 0x55f3c7bcf420 2024-05-14T16:58:01.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff331ffb700 1 -- 172.21.15.69:0/4256587907 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7ff338011cd0 con 0x55f3c7bcf420 2024-05-14T16:58:01.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff35abacb80 1 -- 172.21.15.69:0/4256587907 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3400b1e80 con 0x55f3c7bcf420 2024-05-14T16:58:01.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff331ffb700 1 -- 172.21.15.69:0/4256587907 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff338012450 con 0x55f3c7bcf420 2024-05-14T16:58:01.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff35abacb80 10 client.16287 fetch_fsmap learned FSMap version 41 2024-05-14T16:58:01.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff35abacb80 1 -- 172.21.15.69:0/4256587907 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f3c7ad2860 con 0x55f3c7bcf420 2024-05-14T16:58:01.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.967+0000 7ff331ffb700 1 -- 172.21.15.69:0/4256587907 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3400b1e80 con 0x55f3c7bcf420 2024-05-14T16:58:01.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.968+0000 7ff331ffb700 1 -- 172.21.15.69:0/4256587907 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7ff33801b700 con 0x55f3c7bcf420 2024-05-14T16:58:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.968+0000 7ff35abacb80 10 client.16287 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T16:58:01.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.968+0000 7ff35abacb80 10 client.16287 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:58:02.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.968+0000 7ff35abacb80 20 client.16287 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:02.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.968+0000 7ff35abacb80 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f3c7be4410 0x55f3c7be6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.968+0000 7ff35abacb80 4 client.16287 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:02.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.968+0000 7ff35abacb80 1 -- 172.21.15.69:0/4256587907 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3c7b2adc0 con 0x55f3c7be4410 2024-05-14T16:58:02.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.968+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f3c7be4410 0x55f3c7be6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.968+0000 7ff331ffb700 10 client.16287 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:02.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.968+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f3c7be4410 0x55f3c7be6870 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e404ea2ced443263 server_cookie=85e588e3a9685126 in_seq=0 out_seq=0 2024-05-14T16:58:02.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff331ffb700 1 -- 172.21.15.69:0/4256587907 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f3c7b2adc0 con 0x55f3c7be4410 2024-05-14T16:58:02.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff331ffb700 10 client.16287 handle_command_reply: tid=0 2024-05-14T16:58:02.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff35abacb80 1 client.16287 shutdown 2024-05-14T16:58:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff35abacb80 1 -- 172.21.15.69:0/4256587907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcf420 msgr2=0x55f3c7bd3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:02.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff35abacb80 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd3f50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff338003bc0 tx=0x7ff338003ed0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff35abacb80 1 -- 172.21.15.69:0/4256587907 shutdown_connections 2024-05-14T16:58:02.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff35abacb80 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f3c7be4410 0x55f3c7be6870 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff35abacb80 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bd4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff35abacb80 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bd6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff35abacb80 1 --2- 172.21.15.69:0/4256587907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd3f50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff35abacb80 1 -- 172.21.15.69:0/4256587907 >> 172.21.15.69:0/4256587907 conn(0x55f3c7af7eb0 msgr2=0x55f3c7bbffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:02.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff331ffb700 0 client.16287 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:02.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.969+0000 7ff35abacb80 1 -- 172.21.15.69:0/4256587907 shutdown_connections 2024-05-14T16:58:02.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.970+0000 7ff35abacb80 1 -- 172.21.15.69:0/4256587907 wait complete. 2024-05-14T16:58:02.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.970+0000 7ff35abacb80 20 client.16287 trim_cache size 0 max 16384 2024-05-14T16:58:02.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.989+0000 7ff35abacb80 1 Processor -- start 2024-05-14T16:58:02.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.989+0000 7ff35abacb80 1 -- start start 2024-05-14T16:58:02.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.989+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7be0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.989+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 0x55f3c7bc8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.989+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bc91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.989+0000 7ff35abacb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481b2aa0 con 0x55f3c7bcfd40 2024-05-14T16:58:02.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.989+0000 7ff35abacb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481b0300 con 0x55f3c7bcf420 2024-05-14T16:58:02.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.989+0000 7ff35abacb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481ae170 con 0x55f3c7bcb2c0 2024-05-14T16:58:02.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e8b3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bc91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e8b3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bc91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44652/0 (socket says 172.21.15.69:44652) 2024-05-14T16:58:02.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e8b3700 1 -- 172.21.15.69:0/3406203916 learned_addr learned my addr 172.21.15.69:0/3406203916 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:02.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/3406203916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7be0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e8b3700 1 -- 172.21.15.69:0/3406203916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcb2c0 msgr2=0x55f3c7be0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:02.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/3406203916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 0x55f3c7bc8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/3406203916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7be0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e8b3700 1 -- 172.21.15.69:0/3406203916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 msgr2=0x55f3c7bc8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:02.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/3406203916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 0x55f3c7bc8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e8b3700 1 -- 172.21.15.69:0/3406203916 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c7bc5fe0 con 0x55f3c7bcfd40 2024-05-14T16:58:02.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/3406203916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7be0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:58:02.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/3406203916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bc91d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff3400a8fb0 tx=0x7ff3400e1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e0ebe759bdb583 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:02.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff333fff700 1 -- 172.21.15.69:0/3406203916 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400ad3d0 con 0x55f3c7bcfd40 2024-05-14T16:58:02.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff333fff700 1 -- 172.21.15.69:0/3406203916 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3400df7b0 con 0x55f3c7bcfd40 2024-05-14T16:58:02.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.990+0000 7ff333fff700 1 -- 172.21.15.69:0/3406203916 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400a9280 con 0x55f3c7bcfd40 2024-05-14T16:58:02.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 -- 172.21.15.69:0/3406203916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcfd40 msgr2=0x55f3c7bc91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:02.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 --2- 172.21.15.69:0/3406203916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bc91d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff3400a8fb0 tx=0x7ff3400e1040 comp rx=0 tx=0).stop 2024-05-14T16:58:02.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 -- 172.21.15.69:0/3406203916 shutdown_connections 2024-05-14T16:58:02.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 --2- 172.21.15.69:0/3406203916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcfd40 0x55f3c7bc91d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 --2- 172.21.15.69:0/3406203916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7be0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 --2- 172.21.15.69:0/3406203916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 0x55f3c7bc8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 -- 172.21.15.69:0/3406203916 >> 172.21.15.69:0/3406203916 conn(0x55f3c7af7eb0 msgr2=0x55f3c7bd5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:02.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 -- 172.21.15.69:0/3406203916 shutdown_connections 2024-05-14T16:58:02.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 -- 172.21.15.69:0/3406203916 wait complete. 2024-05-14T16:58:02.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 -- start start 2024-05-14T16:58:02.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 Processor -- start 2024-05-14T16:58:02.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bd7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcfd40 0x55f3c7c4ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481ae7c0 con 0x55f3c7bcb2c0 2024-05-14T16:58:02.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481afc30 con 0x55f3c7bcf420 2024-05-14T16:58:02.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.991+0000 7ff35abacb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481aee90 con 0x55f3c7bcfd40 2024-05-14T16:58:02.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff35abacb80 10 client.0 _ll_register_callbacks cb 0x55f3c7ab3a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:02.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34e0b2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bd7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34e0b2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bd7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44660/0 (socket says 172.21.15.69:44660) 2024-05-14T16:58:02.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34e0b2700 1 -- 172.21.15.69:0/786669043 learned_addr learned my addr 172.21.15.69:0/786669043 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:02.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34e8b3700 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcfd40 0x55f3c7c4ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34e0b2700 1 -- 172.21.15.69:0/786669043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcfd40 msgr2=0x55f3c7c4ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:02.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcfd40 0x55f3c7c4ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34e0b2700 1 -- 172.21.15.69:0/786669043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 msgr2=0x55f3c7bd8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:02.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34e0b2700 1 -- 172.21.15.69:0/786669043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c7bd8900 con 0x55f3c7bcb2c0 2024-05-14T16:58:02.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:58:02.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff34e0b2700 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bd7ad0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff3440098e0 tx=0x7ff344009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5ed43d66079ea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:02.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff331ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:58:02.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff331ffb700 1 -- 172.21.15.69:0/786669043 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344024950 con 0x55f3c7bcb2c0 2024-05-14T16:58:02.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff331ffb700 1 -- 172.21.15.69:0/786669043 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7ff344024af0 con 0x55f3c7bcb2c0 2024-05-14T16:58:02.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff35abacb80 1 -- 172.21.15.69:0/786669043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3400df7b0 con 0x55f3c7bcb2c0 2024-05-14T16:58:02.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff331ffb700 1 -- 172.21.15.69:0/786669043 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff344024cf0 con 0x55f3c7bcb2c0 2024-05-14T16:58:02.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff35abacb80 10 client.16293 fetch_fsmap learned FSMap version 41 2024-05-14T16:58:02.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.992+0000 7ff35abacb80 1 -- 172.21.15.69:0/786669043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f3c7c503e0 con 0x55f3c7bcb2c0 2024-05-14T16:58:02.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff331ffb700 1 -- 172.21.15.69:0/786669043 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3400df7b0 con 0x55f3c7bcb2c0 2024-05-14T16:58:02.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff331ffb700 1 -- 172.21.15.69:0/786669043 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7ff34400f460 con 0x55f3c7bcb2c0 2024-05-14T16:58:02.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff35abacb80 10 client.16293 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T16:58:02.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff35abacb80 10 client.16293 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:58:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff35abacb80 20 client.16293 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff35abacb80 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f3c7c50b50 0x55f3c7c52f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff35abacb80 4 client.16293 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "37a5760b", "release": true, "await": true}] 2024-05-14T16:58:02.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff35abacb80 1 -- 172.21.15.69:0/786669043 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "37a5760b", "release": true, "await": true}) -- 0x55f3c7b245a0 con 0x55f3c7c50b50 2024-05-14T16:58:02.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f3c7c50b50 0x55f3c7c52f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff331ffb700 10 client.16293 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:02.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:00.993+0000 7ff34d8b1700 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f3c7c50b50 0x55f3c7c52f70 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550b341950e779b4 server_cookie=19d989754a3938a2 in_seq=0 out_seq=0 2024-05-14T16:58:02.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff331ffb700 1 -- 172.21.15.69:0/786669043 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f3c7b245a0 con 0x55f3c7c50b50 2024-05-14T16:58:02.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff331ffb700 10 client.16293 handle_command_reply: tid=0 2024-05-14T16:58:02.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 client.16293 shutdown 2024-05-14T16:58:02.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 -- 172.21.15.69:0/786669043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 msgr2=0x55f3c7bd7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:02.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bd7ad0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff3440098e0 tx=0x7ff344009ca0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 -- 172.21.15.69:0/786669043 shutdown_connections 2024-05-14T16:58:02.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f3c7c50b50 0x55f3c7c52f70 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff331ffb700 0 client.16293 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c7bcb2c0 0x55f3c7bd7ad0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f3c7bcfd40 0x55f3c7c4ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 --2- 172.21.15.69:0/786669043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c7bcf420 0x55f3c7bd8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 -- 172.21.15.69:0/786669043 >> 172.21.15.69:0/786669043 conn(0x55f3c7af7eb0 msgr2=0x55f3c7bc97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:02.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 -- 172.21.15.69:0/786669043 shutdown_connections 2024-05-14T16:58:02.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 1 -- 172.21.15.69:0/786669043 wait complete. 2024-05-14T16:58:02.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.002+0000 7ff35abacb80 20 client.16293 trim_cache size 0 max 16384 2024-05-14T16:58:02.928 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:58:02.933 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 41, 2024-05-14T16:58:02.940 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:58:02.945 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 28, 2024-05-14T16:58:02.952 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:58:02.957 INFO:teuthology.orchestra.run.smithi069.stdout: "37a5760b": { 2024-05-14T16:58:02.964 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 28, 2024-05-14T16:58:02.969 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:58:02.975 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:58:02.980 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:58:02.985 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:58:03.001 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:58:03.007 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:58:03.013 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 151.3, 2024-05-14T16:58:03.044 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:58:03.052 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:58:03.057 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:58:03.062 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:58:03.069 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:58:03.074 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:58:03.079 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:58:03.085 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:58:03.091 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:58:03.097 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:58:03.103 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:58:03.110 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:58:03.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:01 smithi092 ceph-mon[35763]: pgmap v252: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-14T16:58:03.123 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:01 smithi203 ceph-mon[35787]: pgmap v252: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-14T16:58:03.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.011+0000 7ff350316700 1 -- 172.21.15.69:0/283259729 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff3340b29f0 msgr2=0x7ff3340b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:03.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.011+0000 7ff350316700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff3340b29f0 0x7ff3340b4eb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b5c0 tx=0x7ff344005fd0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.011+0000 7ff350316700 1 -- 172.21.15.69:0/283259729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481095e0 msgr2=0x7ff348103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:03.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.011+0000 7ff350316700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481095e0 0x7ff348103730 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff340007f10 tx=0x7ff34000d3b0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.012+0000 7ff350316700 1 -- 172.21.15.69:0/283259729 shutdown_connections 2024-05-14T16:58:03.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.012+0000 7ff350316700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff3340b29f0 0x7ff3340b4eb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.012+0000 7ff350316700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481095e0 0x7ff348103730 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.012+0000 7ff350316700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff348101ad0 0x7ff348102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.012+0000 7ff350316700 1 --2- 172.21.15.69:0/283259729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff348102490 0x7ff3481031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.012+0000 7ff350316700 1 -- 172.21.15.69:0/283259729 >> 172.21.15.69:0/283259729 conn(0x7ff348078a40 msgr2=0x7ff348106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:03.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.013+0000 7ff350316700 1 -- 172.21.15.69:0/283259729 shutdown_connections 2024-05-14T16:58:03.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:58:01.013+0000 7ff350316700 1 -- 172.21.15.69:0/283259729 wait complete. 2024-05-14T16:58:03.237 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:01 smithi069 ceph-mon[31507]: pgmap v252: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-14T16:58:03.242 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '37a5760b', total seconds elapsed: 39.5 2024-05-14T16:58:04.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:03 smithi069 ceph-mon[31507]: pgmap v253: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-05-14T16:58:04.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:03 smithi203 ceph-mon[35787]: pgmap v253: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-05-14T16:58:04.257 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:03 smithi092 ceph-mon[35763]: pgmap v253: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-05-14T16:58:05.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:04 smithi069 ceph-mon[31507]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T16:58:05.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:04 smithi203 ceph-mon[35787]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T16:58:05.332 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:04 smithi092 ceph-mon[35763]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T16:58:05.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 455.0 s, 1770.9 tps, lat 364.983 ms stddev 2958.565 2024-05-14T16:58:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.852+0000 7f46d9378640 1 -- 172.21.15.69:0/2113928484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46d4102340 msgr2=0x7f46d410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.852+0000 7f46d9378640 1 --2- 172.21.15.69:0/2113928484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46d4102340 0x7f46d410e800 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009cd0 tx=0x7f46cc02a8e0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.856+0000 7f46d9378640 1 -- 172.21.15.69:0/2113928484 shutdown_connections 2024-05-14T16:58:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.856+0000 7f46d9378640 1 --2- 172.21.15.69:0/2113928484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46d4102340 0x7f46d410e800 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.856+0000 7f46d9378640 1 --2- 172.21.15.69:0/2113928484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d4101980 0x7f46d4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.856+0000 7f46d9378640 1 --2- 172.21.15.69:0/2113928484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46d4107940 0x7f46d4107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.856+0000 7f46d9378640 1 -- 172.21.15.69:0/2113928484 >> 172.21.15.69:0/2113928484 conn(0x7f46d4078ad0 msgr2=0x7f46d4078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.861+0000 7f46d9378640 1 -- 172.21.15.69:0/2113928484 shutdown_connections 2024-05-14T16:58:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.862+0000 7f46d9378640 1 -- 172.21.15.69:0/2113928484 wait complete. 2024-05-14T16:58:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.863+0000 7f46d9378640 1 Processor -- start 2024-05-14T16:58:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.863+0000 7f46d9378640 1 -- start start 2024-05-14T16:58:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d9378640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46d4101980 0x7f46d419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d9378640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46d4102340 0x7f46d419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d9378640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d4107940 0x7f46d4197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d9378640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4111c00 con 0x7f46d4102340 2024-05-14T16:58:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d9378640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4111a80 con 0x7f46d4107940 2024-05-14T16:58:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d9378640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4111d80 con 0x7f46d4101980 2024-05-14T16:58:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46d4101980 0x7f46d419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46d4101980 0x7f46d419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:34996/0 (socket says 172.21.15.69:34996) 2024-05-14T16:58:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d2ffd640 1 -- 172.21.15.69:0/3818664662 learned_addr learned my addr 172.21.15.69:0/3818664662 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d37fe640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d4107940 0x7f46d4197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d2ffd640 1 -- 172.21.15.69:0/3818664662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d4107940 msgr2=0x7f46d4197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d4107940 0x7f46d4197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d2ffd640 1 -- 172.21.15.69:0/3818664662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46d4102340 msgr2=0x7f46d419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46d4102340 0x7f46d419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.864+0000 7f46d2ffd640 1 -- 172.21.15.69:0/3818664662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d41979c0 con 0x7f46d4101980 2024-05-14T16:58:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.866+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46d4101980 0x7f46d419edf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f46c400e9f0 tx=0x7f46c400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.870+0000 7f46bbfff640 1 -- 172.21.15.69:0/3818664662 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c400cd80 con 0x7f46d4101980 2024-05-14T16:58:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.870+0000 7f46d9378640 1 -- 172.21.15.69:0/3818664662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d4197cb0 con 0x7f46d4101980 2024-05-14T16:58:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.870+0000 7f46bbfff640 1 -- 172.21.15.69:0/3818664662 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c40054d0 con 0x7f46d4101980 2024-05-14T16:58:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.870+0000 7f46d9378640 1 -- 172.21.15.69:0/3818664662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d41b4a90 con 0x7f46d4101980 2024-05-14T16:58:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.870+0000 7f46bbfff640 1 -- 172.21.15.69:0/3818664662 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4010690 con 0x7f46d4101980 2024-05-14T16:58:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.875+0000 7f46dc16b740 1 Processor -- start 2024-05-14T16:58:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.875+0000 7f46dc16b740 1 -- start start 2024-05-14T16:58:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c71cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 0x556ed3c7c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c7cad0 0x556ed3c7eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46dc16b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556ed3be4c30 con 0x556ed3c72210 2024-05-14T16:58:06.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46dc16b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556ed3bd47d0 con 0x556ed3c7cad0 2024-05-14T16:58:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46dc16b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556ed3bd8810 con 0x556ed3c718f0 2024-05-14T16:58:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46d37fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c71cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46d37fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c71cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:35000/0 (socket says 172.21.15.69:35000) 2024-05-14T16:58:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46d37fe640 1 -- 172.21.15.69:0/1193932422 learned_addr learned my addr 172.21.15.69:0/1193932422 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46d27fc640 1 --2- 172.21.15.69:0/1193932422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c7cad0 0x556ed3c7eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46d37fe640 1 -- 172.21.15.69:0/1193932422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c7cad0 msgr2=0x556ed3c7eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46d37fe640 1 --2- 172.21.15.69:0/1193932422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c7cad0 0x556ed3c7eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46d37fe640 1 -- 172.21.15.69:0/1193932422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 msgr2=0x556ed3c7c590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:58:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46d37fe640 1 --2- 172.21.15.69:0/1193932422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 0x556ed3c7c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.876+0000 7f46d37fe640 1 -- 172.21.15.69:0/1193932422 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed38e9210 con 0x556ed3c718f0 2024-05-14T16:58:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.878+0000 7f46d37fe640 1 --2- 172.21.15.69:0/1193932422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c71cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46cc035000 tx=0x7f46cc009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2031e62fd7377c8d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.879+0000 7f46bbfff640 1 -- 172.21.15.69:0/3818664662 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f46c4010830 con 0x7f46d4101980 2024-05-14T16:58:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.879+0000 7f46b97fa640 1 -- 172.21.15.69:0/1193932422 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc031850 con 0x556ed3c718f0 2024-05-14T16:58:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.879+0000 7f46b97fa640 1 -- 172.21.15.69:0/1193932422 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc036040 con 0x556ed3c718f0 2024-05-14T16:58:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.879+0000 7f46b97fa640 1 -- 172.21.15.69:0/1193932422 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0044a0 con 0x556ed3c718f0 2024-05-14T16:58:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.880+0000 7f46dc16b740 1 -- 172.21.15.69:0/1193932422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 msgr2=0x556ed3c71cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.880+0000 7f46dc16b740 1 --2- 172.21.15.69:0/1193932422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c71cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46cc035000 tx=0x7f46cc009a20 comp rx=0 tx=0).stop 2024-05-14T16:58:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.883+0000 7f46bbfff640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f46bc0b2bd0 0x7f46bc0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.883+0000 7f46bbfff640 1 -- 172.21.15.69:0/3818664662 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8586+0+0 (secure 0 0 0) 0x7f46c40e2a20 con 0x7f46d4101980 2024-05-14T16:58:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.883+0000 7f46d37fe640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f46bc0b2bd0 0x7f46bc0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.884+0000 7f46dc16b740 1 -- 172.21.15.69:0/1193932422 shutdown_connections 2024-05-14T16:58:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.884+0000 7f46dc16b740 1 --2- 172.21.15.69:0/1193932422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c7cad0 0x556ed3c7eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.884+0000 7f46dc16b740 1 --2- 172.21.15.69:0/1193932422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 0x556ed3c7c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.884+0000 7f46dc16b740 1 --2- 172.21.15.69:0/1193932422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c71cd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.884+0000 7f46dc16b740 1 -- 172.21.15.69:0/1193932422 >> 172.21.15.69:0/1193932422 conn(0x556ed3c6de00 msgr2=0x556ed3c6e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.884+0000 7f46dc16b740 1 -- 172.21.15.69:0/1193932422 shutdown_connections 2024-05-14T16:58:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.884+0000 7f46d37fe640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f46bc0b2bd0 0x7f46bc0b5090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009bf0 tx=0x7f46cc009800 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.885+0000 7f46dc16b740 1 -- 172.21.15.69:0/1193932422 wait complete. 2024-05-14T16:58:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.885+0000 7f46dc16b740 1 -- start start 2024-05-14T16:58:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.885+0000 7f46dc16b740 1 Processor -- start 2024-05-14T16:58:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.885+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c718f0 0x556ed3c7c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.885+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 0x556ed3c797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.885+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c7cad0 0x556ed3c79db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.885+0000 7f46dc16b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556ed38ba790 con 0x556ed3c72210 2024-05-14T16:58:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.885+0000 7f46dc16b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556ed3b90890 con 0x556ed3c718f0 2024-05-14T16:58:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.885+0000 7f46dc16b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556ed3c7f740 con 0x556ed3c7cad0 2024-05-14T16:58:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.886+0000 7f46dc16b740 10 client.0 _ll_register_callbacks cb 0x556ed3b93d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.886+0000 7f46d27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 0x556ed3c797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.886+0000 7f46d27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 0x556ed3c797a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34388/0 (socket says 172.21.15.69:34388) 2024-05-14T16:58:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.886+0000 7f46d27fc640 1 -- 172.21.15.69:0/4142832660 learned_addr learned my addr 172.21.15.69:0/4142832660 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.886+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c718f0 0x556ed3c7c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.886+0000 7f46d37fe640 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c7cad0 0x556ed3c79db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.887+0000 7f46d27fc640 1 -- 172.21.15.69:0/4142832660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c7cad0 msgr2=0x556ed3c79db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.887+0000 7f46d27fc640 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c7cad0 0x556ed3c79db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.887+0000 7f46d27fc640 1 -- 172.21.15.69:0/4142832660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c718f0 msgr2=0x556ed3c7c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.887+0000 7f46d27fc640 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c718f0 0x556ed3c7c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.887+0000 7f46d27fc640 1 -- 172.21.15.69:0/4142832660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed39b5620 con 0x556ed3c72210 2024-05-14T16:58:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.887+0000 7f46d37fe640 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c7cad0 0x556ed3c79db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:58:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.887+0000 7f46d27fc640 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 0x556ed3c797a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f46d410ec60 tx=0x7f46c800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ad06c45c7ad50f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.887+0000 7f46b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:58:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.887+0000 7f46b6ff5640 1 -- 172.21.15.69:0/4142832660 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8015070 con 0x556ed3c72210 2024-05-14T16:58:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.887+0000 7f46b6ff5640 1 -- 172.21.15.69:0/4142832660 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8586+0+0 (secure 0 0 0) 0x7f46c8005220 con 0x556ed3c72210 2024-05-14T16:58:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.888+0000 7f46dc16b740 1 -- 172.21.15.69:0/4142832660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46cc036040 con 0x556ed3c72210 2024-05-14T16:58:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.888+0000 7f46b6ff5640 1 -- 172.21.15.69:0/4142832660 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f46c8005420 con 0x556ed3c72210 2024-05-14T16:58:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.888+0000 7f46dc16b740 10 client.16299 fetch_fsmap learned FSMap version 41 2024-05-14T16:58:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.888+0000 7f46dc16b740 1 -- 172.21.15.69:0/4142832660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556ed3c8b310 con 0x556ed3c72210 2024-05-14T16:58:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.889+0000 7f46b6ff5640 1 -- 172.21.15.69:0/4142832660 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc036040 con 0x556ed3c72210 2024-05-14T16:58:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.889+0000 7f46b6ff5640 1 -- 172.21.15.69:0/4142832660 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f46c8011860 con 0x556ed3c72210 2024-05-14T16:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.889+0000 7f46dc16b740 10 client.16299 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T16:58:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.889+0000 7f46dc16b740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.889+0000 7f46dc16b740 20 client.16299 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.889+0000 7f46dc16b740 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556ed3c8cc20 0x556ed3c8f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.889+0000 7f46dc16b740 4 client.16299 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.889+0000 7f46dc16b740 1 -- 172.21.15.69:0/4142832660 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ed3be6b80 con 0x556ed3c8cc20 2024-05-14T16:58:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.892+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556ed3c8cc20 0x556ed3c8f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.893+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556ed3c8cc20 0x556ed3c8f060 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7209975de1b956 server_cookie=e8a30c022e4a3f57 in_seq=0 out_seq=0 2024-05-14T16:58:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.893+0000 7f46b6ff5640 10 client.16299 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.893+0000 7f46b6ff5640 1 -- 172.21.15.69:0/4142832660 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ed3be6b80 con 0x556ed3c8cc20 2024-05-14T16:58:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.893+0000 7f46b6ff5640 10 client.16299 handle_command_reply: tid=0 2024-05-14T16:58:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.893+0000 7f46dc16b740 1 client.16299 shutdown 2024-05-14T16:58:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 1 -- 172.21.15.69:0/4142832660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 msgr2=0x556ed3c797a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 0x556ed3c797a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f46d410ec60 tx=0x7f46c800f730 comp rx=0 tx=0).stop 2024-05-14T16:58:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 1 -- 172.21.15.69:0/4142832660 shutdown_connections 2024-05-14T16:58:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556ed3c8cc20 0x556ed3c8f060 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c7cad0 0x556ed3c79db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c72210 0x556ed3c797a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 1 --2- 172.21.15.69:0/4142832660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c718f0 0x556ed3c7c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 1 -- 172.21.15.69:0/4142832660 >> 172.21.15.69:0/4142832660 conn(0x556ed3c6de00 msgr2=0x556ed3c6d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46b6ff5640 0 client.16299 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 1 -- 172.21.15.69:0/4142832660 shutdown_connections 2024-05-14T16:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 1 -- 172.21.15.69:0/4142832660 wait complete. 2024-05-14T16:58:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.894+0000 7f46dc16b740 20 client.16299 trim_cache size 0 max 16384 2024-05-14T16:58:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.916+0000 7f46dc16b740 1 Processor -- start 2024-05-14T16:58:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.916+0000 7f46dc16b740 1 -- start start 2024-05-14T16:58:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.916+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c88670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.916+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c72210 0x556ed3c74bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.916+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c7cad0 0x556ed3c7bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.916+0000 7f46dc16b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0011270 con 0x556ed3c7cad0 2024-05-14T16:58:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.916+0000 7f46dc16b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f46c000eb10 con 0x556ed3c72210 2024-05-14T16:58:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.916+0000 7f46dc16b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46c000c980 con 0x556ed3c718f0 2024-05-14T16:58:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c88670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c88670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:35022/0 (socket says 172.21.15.69:35022) 2024-05-14T16:58:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d2ffd640 1 -- 172.21.15.69:0/3110138468 learned_addr learned my addr 172.21.15.69:0/3110138468 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d37fe640 1 --2- 172.21.15.69:0/3110138468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c7cad0 0x556ed3c7bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d27fc640 1 --2- 172.21.15.69:0/3110138468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c72210 0x556ed3c74bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d2ffd640 1 -- 172.21.15.69:0/3110138468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c72210 msgr2=0x556ed3c74bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/3110138468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c72210 0x556ed3c74bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d2ffd640 1 -- 172.21.15.69:0/3110138468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c7cad0 msgr2=0x556ed3c7bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/3110138468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c7cad0 0x556ed3c7bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d2ffd640 1 -- 172.21.15.69:0/3110138468 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed3c72040 con 0x556ed3c718f0 2024-05-14T16:58:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/3110138468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c88670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46c40a9bf0 tx=0x7f46c40cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d60dfa3b0730833 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46b8ff9640 1 -- 172.21.15.69:0/3110138468 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40a9770 con 0x556ed3c718f0 2024-05-14T16:58:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46b8ff9640 1 -- 172.21.15.69:0/3110138468 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c40aa360 con 0x556ed3c718f0 2024-05-14T16:58:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.917+0000 7f46b8ff9640 1 -- 172.21.15.69:0/3110138468 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40046c0 con 0x556ed3c718f0 2024-05-14T16:58:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.918+0000 7f46dc16b740 1 -- 172.21.15.69:0/3110138468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 msgr2=0x556ed3c88670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.918+0000 7f46dc16b740 1 --2- 172.21.15.69:0/3110138468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c88670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46c40a9bf0 tx=0x7f46c40cc040 comp rx=0 tx=0).stop 2024-05-14T16:58:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.918+0000 7f46dc16b740 1 -- 172.21.15.69:0/3110138468 shutdown_connections 2024-05-14T16:58:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.918+0000 7f46dc16b740 1 --2- 172.21.15.69:0/3110138468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c7cad0 0x556ed3c7bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.918+0000 7f46dc16b740 1 --2- 172.21.15.69:0/3110138468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c72210 0x556ed3c74bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.918+0000 7f46dc16b740 1 --2- 172.21.15.69:0/3110138468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c718f0 0x556ed3c88670 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.918+0000 7f46dc16b740 1 -- 172.21.15.69:0/3110138468 >> 172.21.15.69:0/3110138468 conn(0x556ed3c6de00 msgr2=0x556ed3c744a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.918+0000 7f46dc16b740 1 -- 172.21.15.69:0/3110138468 shutdown_connections 2024-05-14T16:58:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.918+0000 7f46dc16b740 1 -- 172.21.15.69:0/3110138468 wait complete. 2024-05-14T16:58:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.918+0000 7f46dc16b740 1 -- start start 2024-05-14T16:58:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46dc16b740 1 Processor -- start 2024-05-14T16:58:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c718f0 0x556ed3c84be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c72210 0x556ed3c76010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46dc16b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c7cad0 0x556ed3c76550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46dc16b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46c000cfd0 con 0x556ed3c718f0 2024-05-14T16:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46dc16b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f46c000e440 con 0x556ed3c7cad0 2024-05-14T16:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46dc16b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46c000d6a0 con 0x556ed3c72210 2024-05-14T16:58:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46dc16b740 10 client.0 _ll_register_callbacks cb 0x7f46c800b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c718f0 0x556ed3c84be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c718f0 0x556ed3c84be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34418/0 (socket says 172.21.15.69:34418) 2024-05-14T16:58:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d2ffd640 1 -- 172.21.15.69:0/2658593124 learned_addr learned my addr 172.21.15.69:0/2658593124 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d37fe640 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c7cad0 0x556ed3c76550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d27fc640 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c72210 0x556ed3c76010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d2ffd640 1 -- 172.21.15.69:0/2658593124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c72210 msgr2=0x556ed3c76010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c72210 0x556ed3c76010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d2ffd640 1 -- 172.21.15.69:0/2658593124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c7cad0 msgr2=0x556ed3c76550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c7cad0 0x556ed3c76550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d2ffd640 1 -- 172.21.15.69:0/2658593124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed3c76a90 con 0x556ed3c718f0 2024-05-14T16:58:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d27fc640 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c72210 0x556ed3c76010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:58:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46d2ffd640 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c718f0 0x556ed3c84be0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f46c4018000 tx=0x7f46c40b2b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff922135a779c6d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.919+0000 7f46b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:58:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.920+0000 7f46b6ff5640 1 -- 172.21.15.69:0/2658593124 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40b6020 con 0x556ed3c718f0 2024-05-14T16:58:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.920+0000 7f46b6ff5640 1 -- 172.21.15.69:0/2658593124 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8586+0+0 (secure 0 0 0) 0x7f46c40046c0 con 0x556ed3c718f0 2024-05-14T16:58:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.920+0000 7f46dc16b740 1 -- 172.21.15.69:0/2658593124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c40aa360 con 0x556ed3c718f0 2024-05-14T16:58:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.920+0000 7f46b6ff5640 1 -- 172.21.15.69:0/2658593124 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f46c40ca050 con 0x556ed3c718f0 2024-05-14T16:58:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.920+0000 7f46dc16b740 10 client.16305 fetch_fsmap learned FSMap version 41 2024-05-14T16:58:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.920+0000 7f46dc16b740 1 -- 172.21.15.69:0/2658593124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556ed3cf70d0 con 0x556ed3c718f0 2024-05-14T16:58:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.920+0000 7f46b6ff5640 1 -- 172.21.15.69:0/2658593124 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c40aa360 con 0x556ed3c718f0 2024-05-14T16:58:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.921+0000 7f46b6ff5640 1 -- 172.21.15.69:0/2658593124 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f46c40b2630 con 0x556ed3c718f0 2024-05-14T16:58:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.922+0000 7f46dc16b740 10 client.16305 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T16:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.922+0000 7f46dc16b740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.922+0000 7f46dc16b740 20 client.16305 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.922+0000 7f46dc16b740 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556ed3cf77e0 0x556ed3cf9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.922+0000 7f46dc16b740 4 client.16305 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:58:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.922+0000 7f46dc16b740 1 -- 172.21.15.69:0/2658593124 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ed3be0360 con 0x556ed3cf77e0 2024-05-14T16:58:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.923+0000 7f46d27fc640 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556ed3cf77e0 0x556ed3cf9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.924+0000 7f46d27fc640 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556ed3cf77e0 0x556ed3cf9c00 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5735c9fd3bea7bd5 server_cookie=576ed8a07ee6202d in_seq=0 out_seq=0 2024-05-14T16:58:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.924+0000 7f46b6ff5640 10 client.16305 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46b6ff5640 1 -- 172.21.15.69:0/2658593124 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556ed3be0360 con 0x556ed3cf77e0 2024-05-14T16:58:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46b6ff5640 10 client.16305 handle_command_reply: tid=0 2024-05-14T16:58:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 client.16305 shutdown 2024-05-14T16:58:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 -- 172.21.15.69:0/2658593124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c718f0 msgr2=0x556ed3c84be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c718f0 0x556ed3c84be0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f46c4018000 tx=0x7f46c40b2b30 comp rx=0 tx=0).stop 2024-05-14T16:58:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 -- 172.21.15.69:0/2658593124 shutdown_connections 2024-05-14T16:58:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556ed3cf77e0 0x556ed3cf9c00 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed3c7cad0 0x556ed3c76550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ed3c72210 0x556ed3c76010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 --2- 172.21.15.69:0/2658593124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ed3c718f0 0x556ed3c84be0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 -- 172.21.15.69:0/2658593124 >> 172.21.15.69:0/2658593124 conn(0x556ed3c6de00 msgr2=0x556ed3c74d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46b6ff5640 0 client.16305 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 -- 172.21.15.69:0/2658593124 shutdown_connections 2024-05-14T16:58:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.940+0000 7f46dc16b740 1 -- 172.21.15.69:0/2658593124 wait complete. 2024-05-14T16:58:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46dc16b740 20 client.16305 trim_cache size 0 max 16384 2024-05-14T16:58:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:58:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:58:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:58:07.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:05 smithi092 ceph-mon[35763]: pgmap v255: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-14T16:58:07.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:05 smithi203 ceph-mon[35787]: pgmap v255: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-14T16:58:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 -- 172.21.15.69:0/3818664662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f46bc0b2bd0 msgr2=0x7f46bc0b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f46bc0b2bd0 0x7f46bc0b5090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009bf0 tx=0x7f46cc009800 comp rx=0 tx=0).stop 2024-05-14T16:58:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 -- 172.21.15.69:0/3818664662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46d4101980 msgr2=0x7f46d419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46d4101980 0x7f46d419edf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f46c400e9f0 tx=0x7f46c400edf0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 -- 172.21.15.69:0/3818664662 shutdown_connections 2024-05-14T16:58:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f46bc0b2bd0 0x7f46bc0b5090 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d4107940 0x7f46d4197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46d4102340 0x7f46d419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 --2- 172.21.15.69:0/3818664662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46d4101980 0x7f46d419edf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 -- 172.21.15.69:0/3818664662 >> 172.21.15.69:0/3818664662 conn(0x7f46d4078ad0 msgr2=0x7f46d406a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 -- 172.21.15.69:0/3818664662 shutdown_connections 2024-05-14T16:58:07.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:05.941+0000 7f46d9378640 1 -- 172.21.15.69:0/3818664662 wait complete. 2024-05-14T16:58:07.874 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:05 smithi069 ceph-mon[31507]: pgmap v255: 97 pgs: 97 active+clean; 4.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-14T16:58:08.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:07 smithi069 ceph-mon[31507]: pgmap v256: 97 pgs: 97 active+clean; 3.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.9 MiB/s wr, 335 op/s 2024-05-14T16:58:08.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:07 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:08.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:07 smithi203 ceph-mon[35787]: pgmap v256: 97 pgs: 97 active+clean; 3.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.9 MiB/s wr, 335 op/s 2024-05-14T16:58:08.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:07 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:08.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:07 smithi092 ceph-mon[35763]: pgmap v256: 97 pgs: 97 active+clean; 3.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.9 MiB/s wr, 335 op/s 2024-05-14T16:58:08.255 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:07 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:09.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:08 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:09.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:08 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:09.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:08 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:58:09.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:08 smithi069 ceph-mon[31507]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T16:58:09.289 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:08 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:09.296 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:08 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:09.351 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:08 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:58:09.357 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:08 smithi092 ceph-mon[35763]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T16:58:09.362 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:08 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:09.368 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:08 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:09.373 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:08 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:58:09.381 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:08 smithi203 ceph-mon[35787]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T16:58:10.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:09 smithi069 ceph-mon[31507]: pgmap v258: 97 pgs: 97 active+clean; 3.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-05-14T16:58:10.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:09 smithi203 ceph-mon[35787]: pgmap v258: 97 pgs: 97 active+clean; 3.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-05-14T16:58:10.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:09 smithi092 ceph-mon[35763]: pgmap v258: 97 pgs: 97 active+clean; 3.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-05-14T16:58:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 460.0 s, 2183.4 tps, lat 45.635 ms stddev 46.722 2024-05-14T16:58:12.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:11 smithi203 ceph-mon[35787]: pgmap v259: 97 pgs: 97 active+clean; 3.9 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.5 MiB/s wr, 602 op/s 2024-05-14T16:58:12.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:11 smithi092 ceph-mon[35763]: pgmap v259: 97 pgs: 97 active+clean; 3.9 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.5 MiB/s wr, 602 op/s 2024-05-14T16:58:12.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:11 smithi069 ceph-mon[31507]: pgmap v259: 97 pgs: 97 active+clean; 3.9 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.5 MiB/s wr, 602 op/s 2024-05-14T16:58:14.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:13 smithi092 ceph-mon[35763]: pgmap v260: 97 pgs: 97 active+clean; 3.9 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 6.7 MiB/s wr, 751 op/s 2024-05-14T16:58:14.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:13 smithi069 ceph-mon[31507]: pgmap v260: 97 pgs: 97 active+clean; 3.9 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 6.7 MiB/s wr, 751 op/s 2024-05-14T16:58:14.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:13 smithi203 ceph-mon[35787]: pgmap v260: 97 pgs: 97 active+clean; 3.9 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 6.7 MiB/s wr, 751 op/s 2024-05-14T16:58:15.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 465.0 s, 2826.8 tps, lat 35.516 ms stddev 36.871 2024-05-14T16:58:16.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:15 smithi092 ceph-mon[35763]: pgmap v261: 97 pgs: 97 active+clean; 3.9 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 661 op/s 2024-05-14T16:58:16.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:15 smithi069 ceph-mon[31507]: pgmap v261: 97 pgs: 97 active+clean; 3.9 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 661 op/s 2024-05-14T16:58:16.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:15 smithi203 ceph-mon[35787]: pgmap v261: 97 pgs: 97 active+clean; 3.9 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 661 op/s 2024-05-14T16:58:18.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:17 smithi092 ceph-mon[35763]: pgmap v262: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 679 op/s 2024-05-14T16:58:18.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:17 smithi069 ceph-mon[31507]: pgmap v262: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 679 op/s 2024-05-14T16:58:18.482 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:17 smithi203 ceph-mon[35787]: pgmap v262: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 679 op/s 2024-05-14T16:58:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.064+0000 7effe1e83640 1 -- 172.21.15.69:0/2855312671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effdc111240 msgr2=0x7effdc109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.064+0000 7effe1e83640 1 --2- 172.21.15.69:0/2855312671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effdc111240 0x7effdc109a50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7effcc009930 tx=0x7effcc02a510 comp rx=0 tx=0).stop 2024-05-14T16:58:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.065+0000 7effe1e83640 1 -- 172.21.15.69:0/2855312671 shutdown_connections 2024-05-14T16:58:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.065+0000 7effe1e83640 1 --2- 172.21.15.69:0/2855312671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc11c230 0x7effdc11e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.065+0000 7effe1e83640 1 --2- 172.21.15.69:0/2855312671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effdc111620 0x7effdc109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.065+0000 7effe1e83640 1 --2- 172.21.15.69:0/2855312671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effdc111240 0x7effdc109a50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.065+0000 7effe1e83640 1 -- 172.21.15.69:0/2855312671 >> 172.21.15.69:0/2855312671 conn(0x7effdc06db20 msgr2=0x7effdc06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.066+0000 7effe1e83640 1 -- 172.21.15.69:0/2855312671 shutdown_connections 2024-05-14T16:58:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.066+0000 7effe1e83640 1 -- 172.21.15.69:0/2855312671 wait complete. 2024-05-14T16:58:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.067+0000 7effe1e83640 1 Processor -- start 2024-05-14T16:58:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.067+0000 7effe1e83640 1 -- start start 2024-05-14T16:58:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.067+0000 7effe1e83640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effdc111240 0x7effdc1198d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effe1e83640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc11c230 0x7effdc112950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effe1e83640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effdc112e90 0x7effdc113340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effe1e83640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7effdc121c50 con 0x7effdc112e90 2024-05-14T16:58:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effe1e83640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7effdc121ad0 con 0x7effdc11c230 2024-05-14T16:58:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effe1e83640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7effdc121dd0 con 0x7effdc111240 2024-05-14T16:58:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effdbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effdc112e90 0x7effdc113340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effdbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effdc112e90 0x7effdc113340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35514/0 (socket says 172.21.15.69:35514) 2024-05-14T16:58:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effdbfff640 1 -- 172.21.15.69:0/391835633 learned_addr learned my addr 172.21.15.69:0/391835633 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effdb7fe640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effdc111240 0x7effdc1198d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effdbfff640 1 -- 172.21.15.69:0/391835633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effdc111240 msgr2=0x7effdc1198d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effdbfff640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effdc111240 0x7effdc1198d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effdbfff640 1 -- 172.21.15.69:0/391835633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc11c230 msgr2=0x7effdc112950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effdbfff640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc11c230 0x7effdc112950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.068+0000 7effdbfff640 1 -- 172.21.15.69:0/391835633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc1173f0 con 0x7effdc112e90 2024-05-14T16:58:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.069+0000 7effdbfff640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effdc112e90 0x7effdc113340 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7effd400b9e0 tx=0x7effd400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.069+0000 7effd8ff9640 1 -- 172.21.15.69:0/391835633 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4019070 con 0x7effdc112e90 2024-05-14T16:58:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.069+0000 7effe1e83640 1 -- 172.21.15.69:0/391835633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effdc1176e0 con 0x7effdc112e90 2024-05-14T16:58:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.069+0000 7effe1e83640 1 -- 172.21.15.69:0/391835633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effdc117c80 con 0x7effdc112e90 2024-05-14T16:58:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.069+0000 7effd8ff9640 1 -- 172.21.15.69:0/391835633 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd4014440 con 0x7effdc112e90 2024-05-14T16:58:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.069+0000 7effd8ff9640 1 -- 172.21.15.69:0/391835633 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd40125a0 con 0x7effdc112e90 2024-05-14T16:58:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.073+0000 7effe4ba3740 1 Processor -- start 2024-05-14T16:58:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.073+0000 7effd8ff9640 1 -- 172.21.15.69:0/391835633 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7effd401c070 con 0x7effdc112e90 2024-05-14T16:58:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.073+0000 7effe4ba3740 1 -- start start 2024-05-14T16:58:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.075+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 0x55cca38b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.075+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.075+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38c1ad0 0x55cca38c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.075+0000 7effe4ba3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cca3829c30 con 0x55cca38b7210 2024-05-14T16:58:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.075+0000 7effe4ba3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cca38197d0 con 0x55cca38b68f0 2024-05-14T16:58:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.075+0000 7effe4ba3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cca381d810 con 0x55cca38c1ad0 2024-05-14T16:58:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.075+0000 7effdb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.075+0000 7effdb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35520/0 (socket says 172.21.15.69:35520) 2024-05-14T16:58:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.075+0000 7effdb7fe640 1 -- 172.21.15.69:0/1687032091 learned_addr learned my addr 172.21.15.69:0/1687032091 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.075+0000 7effdaffd640 1 --2- 172.21.15.69:0/1687032091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 0x55cca38b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.076+0000 7effdaffd640 1 --2- 172.21.15.69:0/1687032091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38c1ad0 0x55cca38c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.076+0000 7effd8ff9640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7effc40b2b60 0x7effc40b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.076+0000 7effdb7fe640 1 -- 172.21.15.69:0/1687032091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38c1ad0 msgr2=0x55cca38c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.076+0000 7effdb7fe640 1 --2- 172.21.15.69:0/1687032091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38c1ad0 0x55cca38c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.076+0000 7effdb7fe640 1 -- 172.21.15.69:0/1687032091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 msgr2=0x55cca38b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.076+0000 7effdb7fe640 1 --2- 172.21.15.69:0/1687032091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 0x55cca38b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.076+0000 7effdb7fe640 1 -- 172.21.15.69:0/1687032091 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cca352e210 con 0x55cca38b7210 2024-05-14T16:58:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.076+0000 7effdbfff640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7effc40b2b60 0x7effc40b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.076+0000 7effdb7fe640 1 --2- 172.21.15.69:0/1687032091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c1590 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7effcc006010 tx=0x7effcc004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e8d8f4ba55258f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.076+0000 7effd8ff9640 1 -- 172.21.15.69:0/391835633 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7effd40e1a60 con 0x7effdc112e90 2024-05-14T16:58:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.078+0000 7effc1ffb640 1 -- 172.21.15.69:0/1687032091 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc03f070 con 0x55cca38b7210 2024-05-14T16:58:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.078+0000 7effc1ffb640 1 -- 172.21.15.69:0/1687032091 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc0046c0 con 0x55cca38b7210 2024-05-14T16:58:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.078+0000 7effc1ffb640 1 -- 172.21.15.69:0/1687032091 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc035030 con 0x55cca38b7210 2024-05-14T16:58:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effe4ba3740 1 -- 172.21.15.69:0/1687032091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 msgr2=0x55cca38c1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effdbfff640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7effc40b2b60 0x7effc40b5020 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7effd40a8d70 tx=0x7effd401ff90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effe4ba3740 1 --2- 172.21.15.69:0/1687032091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c1590 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7effcc006010 tx=0x7effcc004380 comp rx=0 tx=0).stop 2024-05-14T16:58:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effe4ba3740 1 -- 172.21.15.69:0/1687032091 shutdown_connections 2024-05-14T16:58:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effe4ba3740 1 --2- 172.21.15.69:0/1687032091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38c1ad0 0x55cca38c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effe4ba3740 1 --2- 172.21.15.69:0/1687032091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c1590 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effe4ba3740 1 --2- 172.21.15.69:0/1687032091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 0x55cca38b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effe4ba3740 1 -- 172.21.15.69:0/1687032091 >> 172.21.15.69:0/1687032091 conn(0x55cca38b2e00 msgr2=0x55cca38b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effe4ba3740 1 -- 172.21.15.69:0/1687032091 shutdown_connections 2024-05-14T16:58:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effe4ba3740 1 -- 172.21.15.69:0/1687032091 wait complete. 2024-05-14T16:58:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.079+0000 7effe4ba3740 1 -- start start 2024-05-14T16:58:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effe4ba3740 1 Processor -- start 2024-05-14T16:58:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38b68f0 0x55cca38c1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38be730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 0x55cca38bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effe4ba3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cca34ff790 con 0x55cca38c1ad0 2024-05-14T16:58:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effe4ba3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cca37d5890 con 0x55cca38b7210 2024-05-14T16:58:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effe4ba3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cca38c4740 con 0x55cca38b68f0 2024-05-14T16:58:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effe4ba3740 10 client.0 _ll_register_callbacks cb 0x55cca37d8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effdb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38be730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effdaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 0x55cca38bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:20.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effdb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38be730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:50972/0 (socket says 172.21.15.69:50972) 2024-05-14T16:58:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effdb7fe640 1 -- 172.21.15.69:0/2399517470 learned_addr learned my addr 172.21.15.69:0/2399517470 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.080+0000 7effdaffd640 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38b68f0 0x55cca38c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effdb7fe640 1 -- 172.21.15.69:0/2399517470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38b68f0 msgr2=0x55cca38c1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effdb7fe640 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38b68f0 0x55cca38c1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effdb7fe640 1 -- 172.21.15.69:0/2399517470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 msgr2=0x55cca38bec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effdb7fe640 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 0x55cca38bec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effdb7fe640 1 -- 172.21.15.69:0/2399517470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cca35fa620 con 0x55cca38b7210 2024-05-14T16:58:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effdb7fe640 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38be730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7effcc02a550 tx=0x7effcc005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28fd02a2517c562b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:58:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effbaffd640 1 -- 172.21.15.69:0/2399517470 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc04c070 con 0x55cca38b7210 2024-05-14T16:58:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effbaffd640 1 -- 172.21.15.69:0/2399517470 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7effcc004a70 con 0x55cca38b7210 2024-05-14T16:58:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effe4ba3740 1 -- 172.21.15.69:0/2399517470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effcc0046c0 con 0x55cca38b7210 2024-05-14T16:58:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effbaffd640 1 -- 172.21.15.69:0/2399517470 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7effcc03f050 con 0x55cca38b7210 2024-05-14T16:58:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effe4ba3740 10 client.25772 fetch_fsmap learned FSMap version 41 2024-05-14T16:58:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.081+0000 7effe4ba3740 1 -- 172.21.15.69:0/2399517470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cca38d0200 con 0x55cca38b7210 2024-05-14T16:58:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.082+0000 7effbaffd640 1 -- 172.21.15.69:0/2399517470 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc0046c0 con 0x55cca38b7210 2024-05-14T16:58:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.082+0000 7effbaffd640 1 -- 172.21.15.69:0/2399517470 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7effcc03b460 con 0x55cca38b7210 2024-05-14T16:58:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.082+0000 7effe4ba3740 10 client.25772 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T16:58:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.082+0000 7effe4ba3740 10 client.25772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.082+0000 7effe4ba3740 20 client.25772 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.082+0000 7effe4ba3740 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cca38d1550 0x55cca38d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.082+0000 7effe4ba3740 4 client.25772 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.082+0000 7effe4ba3740 1 -- 172.21.15.69:0/2399517470 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cca382bb80 con 0x55cca38d1550 2024-05-14T16:58:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.083+0000 7effdaffd640 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cca38d1550 0x55cca38d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.084+0000 7effdaffd640 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cca38d1550 0x55cca38d3940 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0659f45fde4a4f5 server_cookie=f2d584dc74b90876 in_seq=0 out_seq=0 2024-05-14T16:58:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.084+0000 7effbaffd640 10 client.25772 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.084+0000 7effbaffd640 1 -- 172.21.15.69:0/2399517470 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cca382bb80 con 0x55cca38d1550 2024-05-14T16:58:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.084+0000 7effbaffd640 10 client.25772 handle_command_reply: tid=0 2024-05-14T16:58:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.084+0000 7effe4ba3740 1 client.25772 shutdown 2024-05-14T16:58:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 1 -- 172.21.15.69:0/2399517470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 msgr2=0x55cca38be730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38be730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7effcc02a550 tx=0x7effcc005230 comp rx=0 tx=0).stop 2024-05-14T16:58:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 1 -- 172.21.15.69:0/2399517470 shutdown_connections 2024-05-14T16:58:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cca38d1550 0x55cca38d3940 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 0x55cca38bec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38be730 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effbaffd640 0 client.25772 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 1 --2- 172.21.15.69:0/2399517470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38b68f0 0x55cca38c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 1 -- 172.21.15.69:0/2399517470 >> 172.21.15.69:0/2399517470 conn(0x55cca38b2e00 msgr2=0x55cca38b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 1 -- 172.21.15.69:0/2399517470 shutdown_connections 2024-05-14T16:58:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 1 -- 172.21.15.69:0/2399517470 wait complete. 2024-05-14T16:58:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.085+0000 7effe4ba3740 20 client.25772 trim_cache size 0 max 16384 2024-05-14T16:58:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.103+0000 7effe4ba3740 1 Processor -- start 2024-05-14T16:58:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.103+0000 7effe4ba3740 1 -- start start 2024-05-14T16:58:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.103+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 0x55cca38cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.103+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.103+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38c1ad0 0x55cca38d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.103+0000 7effe4ba3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7effc80110d0 con 0x55cca38b7210 2024-05-14T16:58:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.103+0000 7effe4ba3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7effc800e970 con 0x55cca38b68f0 2024-05-14T16:58:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.103+0000 7effe4ba3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7effc800c7e0 con 0x55cca38c1ad0 2024-05-14T16:58:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 0x55cca38cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 0x55cca38cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:50978/0 (socket says 172.21.15.69:50978) 2024-05-14T16:58:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdaffd640 1 -- 172.21.15.69:0/3790620058 learned_addr learned my addr 172.21.15.69:0/3790620058 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdb7fe640 1 --2- 172.21.15.69:0/3790620058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdaffd640 1 --2- 172.21.15.69:0/3790620058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38c1ad0 0x55cca38d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdb7fe640 1 -- 172.21.15.69:0/3790620058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38c1ad0 msgr2=0x55cca38d8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdb7fe640 1 --2- 172.21.15.69:0/3790620058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38c1ad0 0x55cca38d8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdb7fe640 1 -- 172.21.15.69:0/3790620058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 msgr2=0x55cca38cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdb7fe640 1 --2- 172.21.15.69:0/3790620058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 0x55cca38cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdb7fe640 1 -- 172.21.15.69:0/3790620058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cca38c4740 con 0x55cca38b7210 2024-05-14T16:58:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdaffd640 1 --2- 172.21.15.69:0/3790620058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 0x55cca38cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.104+0000 7effdb7fe640 1 --2- 172.21.15.69:0/3790620058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c0f70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7effcc039770 tx=0x7effcc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b467d38148ef3b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.105+0000 7effc17fa640 1 -- 172.21.15.69:0/3790620058 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc04c070 con 0x55cca38b7210 2024-05-14T16:58:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.105+0000 7effc17fa640 1 -- 172.21.15.69:0/3790620058 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc005530 con 0x55cca38b7210 2024-05-14T16:58:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.105+0000 7effc17fa640 1 -- 172.21.15.69:0/3790620058 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc035040 con 0x55cca38b7210 2024-05-14T16:58:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.106+0000 7effe4ba3740 1 -- 172.21.15.69:0/3790620058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 msgr2=0x55cca38c0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.106+0000 7effe4ba3740 1 --2- 172.21.15.69:0/3790620058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c0f70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7effcc039770 tx=0x7effcc005fd0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.106+0000 7effe4ba3740 1 -- 172.21.15.69:0/3790620058 shutdown_connections 2024-05-14T16:58:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.106+0000 7effe4ba3740 1 --2- 172.21.15.69:0/3790620058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38c1ad0 0x55cca38d8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.106+0000 7effe4ba3740 1 --2- 172.21.15.69:0/3790620058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38b7210 0x55cca38c0f70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.106+0000 7effe4ba3740 1 --2- 172.21.15.69:0/3790620058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b68f0 0x55cca38cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.106+0000 7effe4ba3740 1 -- 172.21.15.69:0/3790620058 >> 172.21.15.69:0/3790620058 conn(0x55cca38b2e00 msgr2=0x55cca38b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.106+0000 7effe4ba3740 1 -- 172.21.15.69:0/3790620058 shutdown_connections 2024-05-14T16:58:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.106+0000 7effe4ba3740 1 -- 172.21.15.69:0/3790620058 wait complete. 2024-05-14T16:58:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.106+0000 7effe4ba3740 1 -- start start 2024-05-14T16:58:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effe4ba3740 1 Processor -- start 2024-05-14T16:58:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38b68f0 0x55cca38c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effe4ba3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 0x55cca38c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effe4ba3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7effc800ce30 con 0x55cca38c1ad0 2024-05-14T16:58:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effe4ba3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7effc800e2a0 con 0x55cca38b7210 2024-05-14T16:58:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effe4ba3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7effc800d500 con 0x55cca38b68f0 2024-05-14T16:58:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effe4ba3740 10 client.0 _ll_register_callbacks cb 0x7effcc0042d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 0x55cca38c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 0x55cca38c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35556/0 (socket says 172.21.15.69:35556) 2024-05-14T16:58:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdaffd640 1 -- 172.21.15.69:0/331486406 learned_addr learned my addr 172.21.15.69:0/331486406 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdaffd640 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38b68f0 0x55cca38c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdb7fe640 1 -- 172.21.15.69:0/331486406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38b68f0 msgr2=0x55cca38c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdb7fe640 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38b68f0 0x55cca38c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdb7fe640 1 -- 172.21.15.69:0/331486406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 msgr2=0x55cca38c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdb7fe640 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 0x55cca38c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdb7fe640 1 -- 172.21.15.69:0/331486406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cca38c5e60 con 0x55cca38b7210 2024-05-14T16:58:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effdb7fe640 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38c51e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55cca38ae6d0 tx=0x7effcc009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ade51e6e82d2e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.107+0000 7effbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:58:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effbaffd640 1 -- 172.21.15.69:0/331486406 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc061060 con 0x55cca38b7210 2024-05-14T16:58:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effbaffd640 1 -- 172.21.15.69:0/331486406 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7effcc050e40 con 0x55cca38b7210 2024-05-14T16:58:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effe4ba3740 1 -- 172.21.15.69:0/331486406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effcc005530 con 0x55cca38b7210 2024-05-14T16:58:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effbaffd640 1 -- 172.21.15.69:0/331486406 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7effcc035030 con 0x55cca38b7210 2024-05-14T16:58:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effe4ba3740 10 client.25778 fetch_fsmap learned FSMap version 41 2024-05-14T16:58:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effe4ba3740 1 -- 172.21.15.69:0/331486406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cca393ba60 con 0x55cca38b7210 2024-05-14T16:58:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effbaffd640 1 -- 172.21.15.69:0/331486406 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc005530 con 0x55cca38b7210 2024-05-14T16:58:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effbaffd640 1 -- 172.21.15.69:0/331486406 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7effcc03b460 con 0x55cca38b7210 2024-05-14T16:58:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effe4ba3740 10 client.25778 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T16:58:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effe4ba3740 10 client.25778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effe4ba3740 20 client.25778 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effe4ba3740 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cca393c110 0x55cca393e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effe4ba3740 4 client.25778 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:58:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.108+0000 7effe4ba3740 1 -- 172.21.15.69:0/331486406 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cca3825360 con 0x55cca393c110 2024-05-14T16:58:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.109+0000 7effdaffd640 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cca393c110 0x55cca393e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.109+0000 7effbaffd640 10 client.25778 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.109+0000 7effdaffd640 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cca393c110 0x55cca393e530 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f834ff30de081ee server_cookie=415014695b98eaf1 in_seq=0 out_seq=0 2024-05-14T16:58:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effbaffd640 1 -- 172.21.15.69:0/331486406 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cca3825360 con 0x55cca393c110 2024-05-14T16:58:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effbaffd640 10 client.25778 handle_command_reply: tid=0 2024-05-14T16:58:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effe4ba3740 1 client.25778 shutdown 2024-05-14T16:58:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effe4ba3740 1 -- 172.21.15.69:0/331486406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 msgr2=0x55cca38c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effe4ba3740 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38c51e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55cca38ae6d0 tx=0x7effcc009650 comp rx=0 tx=0).stop 2024-05-14T16:58:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effe4ba3740 1 -- 172.21.15.69:0/331486406 shutdown_connections 2024-05-14T16:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effe4ba3740 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cca393c110 0x55cca393e530 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effe4ba3740 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cca38c1ad0 0x55cca38c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effe4ba3740 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cca38b7210 0x55cca38c51e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effe4ba3740 1 --2- 172.21.15.69:0/331486406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cca38b68f0 0x55cca38c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effe4ba3740 1 -- 172.21.15.69:0/331486406 >> 172.21.15.69:0/331486406 conn(0x55cca38b2e00 msgr2=0x55cca38b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.123+0000 7effbaffd640 0 client.25778 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.124+0000 7effe4ba3740 1 -- 172.21.15.69:0/331486406 shutdown_connections 2024-05-14T16:58:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.124+0000 7effe4ba3740 1 -- 172.21.15.69:0/331486406 wait complete. 2024-05-14T16:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.124+0000 7effe4ba3740 20 client.25778 trim_cache size 0 max 16384 2024-05-14T16:58:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:58:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:58:21.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:58:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 -- 172.21.15.69:0/391835633 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7effc40b2b60 msgr2=0x7effc40b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7effc40b2b60 0x7effc40b5020 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7effd40a8d70 tx=0x7effd401ff90 comp rx=0 tx=0).stop 2024-05-14T16:58:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 -- 172.21.15.69:0/391835633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effdc112e90 msgr2=0x7effdc113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effdc112e90 0x7effdc113340 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7effd400b9e0 tx=0x7effd400bea0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 -- 172.21.15.69:0/391835633 shutdown_connections 2024-05-14T16:58:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7effc40b2b60 0x7effc40b5020 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effdc112e90 0x7effdc113340 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc11c230 0x7effdc112950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 --2- 172.21.15.69:0/391835633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effdc111240 0x7effdc1198d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 -- 172.21.15.69:0/391835633 >> 172.21.15.69:0/391835633 conn(0x7effdc06db20 msgr2=0x7effdc10ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 -- 172.21.15.69:0/391835633 shutdown_connections 2024-05-14T16:58:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:19.125+0000 7effe1e83640 1 -- 172.21.15.69:0/391835633 wait complete. 2024-05-14T16:58:21.950 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 470.0 s, 2842.0 tps, lat 34.955 ms stddev 38.886 2024-05-14T16:58:21.967 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:19 smithi069 ceph-mon[31507]: pgmap v263: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 675 op/s 2024-05-14T16:58:21.977 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:19 smithi203 ceph-mon[35787]: pgmap v263: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 675 op/s 2024-05-14T16:58:21.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:19 smithi092 ceph-mon[35763]: pgmap v263: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 675 op/s 2024-05-14T16:58:22.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:21 smithi092 ceph-mon[35763]: pgmap v264: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 657 op/s 2024-05-14T16:58:22.256 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:21 smithi069 ceph-mon[31507]: pgmap v264: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 657 op/s 2024-05-14T16:58:22.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:21 smithi203 ceph-mon[35787]: pgmap v264: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 657 op/s 2024-05-14T16:58:24.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:23 smithi069 ceph-mon[31507]: pgmap v265: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 735 op/s 2024-05-14T16:58:24.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:23 smithi092 ceph-mon[35763]: pgmap v265: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 735 op/s 2024-05-14T16:58:24.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:23 smithi203 ceph-mon[35787]: pgmap v265: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 735 op/s 2024-05-14T16:58:25.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 475.0 s, 2457.4 tps, lat 40.914 ms stddev 42.912 2024-05-14T16:58:26.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:25 smithi092 ceph-mon[35763]: pgmap v266: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 586 op/s 2024-05-14T16:58:26.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:25 smithi069 ceph-mon[31507]: pgmap v266: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 586 op/s 2024-05-14T16:58:26.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:25 smithi203 ceph-mon[35787]: pgmap v266: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 586 op/s 2024-05-14T16:58:28.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:27 smithi092 ceph-mon[35763]: pgmap v267: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 8.3 MiB/s wr, 873 op/s 2024-05-14T16:58:28.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:27 smithi069 ceph-mon[31507]: pgmap v267: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 8.3 MiB/s wr, 873 op/s 2024-05-14T16:58:28.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:27 smithi203 ceph-mon[35787]: pgmap v267: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 8.3 MiB/s wr, 873 op/s 2024-05-14T16:58:30.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:29 smithi092 ceph-mon[35763]: pgmap v268: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 5.5 MiB/s wr, 583 op/s 2024-05-14T16:58:30.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 480.0 s, 2725.6 tps, lat 36.562 ms stddev 39.634 2024-05-14T16:58:30.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:29 smithi069 ceph-mon[31507]: pgmap v268: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 5.5 MiB/s wr, 583 op/s 2024-05-14T16:58:30.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:29 smithi203 ceph-mon[35787]: pgmap v268: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 5.5 MiB/s wr, 583 op/s 2024-05-14T16:58:32.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:31 smithi092 ceph-mon[35763]: pgmap v269: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 5.9 MiB/s wr, 634 op/s 2024-05-14T16:58:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.252+0000 7f22a8e59640 1 -- 172.21.15.69:0/739705159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a41033f0 msgr2=0x7f22a410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.252+0000 7f22a8e59640 1 --2- 172.21.15.69:0/739705159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a41033f0 0x7f22a410f900 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f229c009cd0 tx=0x7f229c02a8e0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.253+0000 7f22a8e59640 1 -- 172.21.15.69:0/739705159 shutdown_connections 2024-05-14T16:58:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.253+0000 7f22a8e59640 1 --2- 172.21.15.69:0/739705159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a41033f0 0x7f22a410f900 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.253+0000 7f22a8e59640 1 --2- 172.21.15.69:0/739705159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a4102a30 0x7f22a4102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.253+0000 7f22a8e59640 1 --2- 172.21.15.69:0/739705159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4108a40 0x7f22a4108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.253+0000 7f22a8e59640 1 -- 172.21.15.69:0/739705159 >> 172.21.15.69:0/739705159 conn(0x7f22a40fe500 msgr2=0x7f22a4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.254+0000 7f22a8e59640 1 -- 172.21.15.69:0/739705159 shutdown_connections 2024-05-14T16:58:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.254+0000 7f22a8e59640 1 -- 172.21.15.69:0/739705159 wait complete. 2024-05-14T16:58:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a8e59640 1 Processor -- start 2024-05-14T16:58:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a8e59640 1 -- start start 2024-05-14T16:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a8e59640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102a30 0x7f22a4074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a8e59640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a41033f0 0x7f22a4075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a8e59640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4108a40 0x7f22a40757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a8e59640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4112d00 con 0x7f22a4108a40 2024-05-14T16:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a8e59640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4112b80 con 0x7f22a4102a30 2024-05-14T16:58:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a8e59640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4112e80 con 0x7f22a41033f0 2024-05-14T16:58:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4108a40 0x7f22a40757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f229bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a41033f0 0x7f22a4075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4108a40 0x7f22a40757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57864/0 (socket says 172.21.15.69:57864) 2024-05-14T16:58:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f229bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a41033f0 0x7f22a4075290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:55898/0 (socket says 172.21.15.69:55898) 2024-05-14T16:58:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a2d9d640 1 -- 172.21.15.69:0/3187537120 learned_addr learned my addr 172.21.15.69:0/3187537120 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a259c640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102a30 0x7f22a4074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.255+0000 7f22a2d9d640 1 -- 172.21.15.69:0/3187537120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a41033f0 msgr2=0x7f22a4075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f22a2d9d640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a41033f0 0x7f22a4075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f22a2d9d640 1 -- 172.21.15.69:0/3187537120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102a30 msgr2=0x7f22a4074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f22a2d9d640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102a30 0x7f22a4074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f22a2d9d640 1 -- 172.21.15.69:0/3187537120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a41a7680 con 0x7f22a4108a40 2024-05-14T16:58:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f229bfff640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a41033f0 0x7f22a4075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f22a259c640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102a30 0x7f22a4074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f22a2d9d640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4108a40 0x7f22a40757d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f229c033040 tx=0x7f229c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f229b7fe640 1 -- 172.21.15.69:0/3187537120 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c004040 con 0x7f22a4108a40 2024-05-14T16:58:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f22a8e59640 1 -- 172.21.15.69:0/3187537120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a41a7910 con 0x7f22a4108a40 2024-05-14T16:58:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f229b7fe640 1 -- 172.21.15.69:0/3187537120 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c004690 con 0x7f22a4108a40 2024-05-14T16:58:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f22a8e59640 1 -- 172.21.15.69:0/3187537120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41a7eb0 con 0x7f22a4108a40 2024-05-14T16:58:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.256+0000 7f229b7fe640 1 -- 172.21.15.69:0/3187537120 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c030d80 con 0x7f22a4108a40 2024-05-14T16:58:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f229b7fe640 1 -- 172.21.15.69:0/3187537120 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f229c03b400 con 0x7f22a4108a40 2024-05-14T16:58:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f22abb73740 1 Processor -- start 2024-05-14T16:58:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f22abb73740 1 -- start start 2024-05-14T16:58:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f22abb73740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc88f0 0x558cd1fc8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f22abb73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fc9210 0x558cd1fd3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f22abb73740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:32.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f22abb73740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558cd1f3bc30 con 0x558cd1fd3ad0 2024-05-14T16:58:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f22abb73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558cd1f2b7d0 con 0x558cd1fc9210 2024-05-14T16:58:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f22abb73740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558cd1f2f810 con 0x558cd1fc88f0 2024-05-14T16:58:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f229bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f229bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57872/0 (socket says 172.21.15.69:57872) 2024-05-14T16:58:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f22a259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fc9210 0x558cd1fd3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f229bfff640 1 -- 172.21.15.69:0/3281806896 learned_addr learned my addr 172.21.15.69:0/3281806896 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.259+0000 7f229bfff640 1 --2- 172.21.15.69:0/3281806896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc88f0 0x558cd1fc8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.260+0000 7f22a259c640 1 -- 172.21.15.69:0/3281806896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc88f0 msgr2=0x558cd1fc8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:32.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.260+0000 7f22a259c640 1 --2- 172.21.15.69:0/3281806896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc88f0 0x558cd1fc8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.260+0000 7f22a259c640 1 -- 172.21.15.69:0/3281806896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fd3ad0 msgr2=0x558cd1fd5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.260+0000 7f22a259c640 1 --2- 172.21.15.69:0/3281806896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.260+0000 7f229b7fe640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f22900b2a40 0x7f22900b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.260+0000 7f22a259c640 1 -- 172.21.15.69:0/3281806896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cd1c40210 con 0x558cd1fc9210 2024-05-14T16:58:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.260+0000 7f229bfff640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f22900b2a40 0x7f22900b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.260+0000 7f229b7fe640 1 -- 172.21.15.69:0/3187537120 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f229c02e030 con 0x7f22a4108a40 2024-05-14T16:58:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.260+0000 7f22a259c640 1 --2- 172.21.15.69:0/3281806896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fc9210 0x558cd1fd3590 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2294009fd0 tx=0x7f22940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2539dbcfde3561f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.261+0000 7f2298ff9640 1 -- 172.21.15.69:0/3281806896 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294004030 con 0x558cd1fc9210 2024-05-14T16:58:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.261+0000 7f2298ff9640 1 -- 172.21.15.69:0/3281806896 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22940054d0 con 0x558cd1fc9210 2024-05-14T16:58:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f2298ff9640 1 -- 172.21.15.69:0/3281806896 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294004ce0 con 0x558cd1fc9210 2024-05-14T16:58:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f22abb73740 1 -- 172.21.15.69:0/3281806896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fc9210 msgr2=0x558cd1fd3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f229bfff640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f22900b2a40 0x7f22900b4f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x558cd1fc6220 tx=0x7f228c008040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f22abb73740 1 --2- 172.21.15.69:0/3281806896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fc9210 0x558cd1fd3590 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2294009fd0 tx=0x7f22940118d0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f22abb73740 1 -- 172.21.15.69:0/3281806896 shutdown_connections 2024-05-14T16:58:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f22abb73740 1 --2- 172.21.15.69:0/3281806896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f22abb73740 1 --2- 172.21.15.69:0/3281806896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fc9210 0x558cd1fd3590 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f22abb73740 1 --2- 172.21.15.69:0/3281806896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc88f0 0x558cd1fc8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f22abb73740 1 -- 172.21.15.69:0/3281806896 >> 172.21.15.69:0/3281806896 conn(0x558cd1fc4e00 msgr2=0x558cd1fc51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f22abb73740 1 -- 172.21.15.69:0/3281806896 shutdown_connections 2024-05-14T16:58:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.262+0000 7f22abb73740 1 -- 172.21.15.69:0/3281806896 wait complete. 2024-05-14T16:58:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22abb73740 1 -- start start 2024-05-14T16:58:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22abb73740 1 Processor -- start 2024-05-14T16:58:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22abb73740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc88f0 0x558cd1fd3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22abb73740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc9210 0x558cd1fd06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22abb73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22abb73740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558cd1c11790 con 0x558cd1fc9210 2024-05-14T16:58:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22abb73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558cd1ee7890 con 0x558cd1fd3ad0 2024-05-14T16:58:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22abb73740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558cd1fd67a0 con 0x558cd1fc88f0 2024-05-14T16:58:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22abb73740 10 client.0 _ll_register_callbacks cb 0x558cd1eead60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22a2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22a2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:41298/0 (socket says 172.21.15.69:41298) 2024-05-14T16:58:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22a2d9d640 1 -- 172.21.15.69:0/1807508072 learned_addr learned my addr 172.21.15.69:0/1807508072 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f22a259c640 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc88f0 0x558cd1fd3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.263+0000 7f229bfff640 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc9210 0x558cd1fd06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f22a2d9d640 1 -- 172.21.15.69:0/1807508072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc88f0 msgr2=0x558cd1fd3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f22a2d9d640 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc88f0 0x558cd1fd3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f22a2d9d640 1 -- 172.21.15.69:0/1807508072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc9210 msgr2=0x558cd1fd06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f22a2d9d640 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc9210 0x558cd1fd06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f22a2d9d640 1 -- 172.21.15.69:0/1807508072 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cd1d0c620 con 0x558cd1fd3ad0 2024-05-14T16:58:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f22a2d9d640 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd0c30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f229c0ca830 tx=0x7f229c0307d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75926d3dea3c0ec4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f2289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:58:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f2289ffb640 1 -- 172.21.15.69:0/1807508072 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c0cafa0 con 0x558cd1fd3ad0 2024-05-14T16:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f2289ffb640 1 -- 172.21.15.69:0/1807508072 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f229c0c6b30 con 0x558cd1fd3ad0 2024-05-14T16:58:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f22abb73740 1 -- 172.21.15.69:0/1807508072 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22940054d0 con 0x558cd1fd3ad0 2024-05-14T16:58:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f2289ffb640 1 -- 172.21.15.69:0/1807508072 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f229c0ca8a0 con 0x558cd1fd3ad0 2024-05-14T16:58:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f22abb73740 10 client.25790 fetch_fsmap learned FSMap version 41 2024-05-14T16:58:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.264+0000 7f22abb73740 1 -- 172.21.15.69:0/1807508072 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558cd1fe2210 con 0x558cd1fd3ad0 2024-05-14T16:58:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.265+0000 7f2289ffb640 1 -- 172.21.15.69:0/1807508072 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22940054d0 con 0x558cd1fd3ad0 2024-05-14T16:58:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.265+0000 7f2289ffb640 1 -- 172.21.15.69:0/1807508072 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f229c04d020 con 0x558cd1fd3ad0 2024-05-14T16:58:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.265+0000 7f22abb73740 10 client.25790 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T16:58:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.265+0000 7f22abb73740 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.266+0000 7f22abb73740 20 client.25790 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.266+0000 7f22abb73740 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558cd1fe3720 0x558cd1fe5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.266+0000 7f22abb73740 4 client.25790 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.266+0000 7f22abb73740 1 -- 172.21.15.69:0/1807508072 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cd1f3db80 con 0x558cd1fe3720 2024-05-14T16:58:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.266+0000 7f22a259c640 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558cd1fe3720 0x558cd1fe5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.267+0000 7f22a259c640 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558cd1fe3720 0x558cd1fe5b80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdea6972d9bc2ad5 server_cookie=d46ffaac35a825c3 in_seq=0 out_seq=0 2024-05-14T16:58:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.267+0000 7f2289ffb640 10 client.25790 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.267+0000 7f2289ffb640 1 -- 172.21.15.69:0/1807508072 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558cd1f3db80 con 0x558cd1fe3720 2024-05-14T16:58:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.267+0000 7f2289ffb640 10 client.25790 handle_command_reply: tid=0 2024-05-14T16:58:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.267+0000 7f22abb73740 1 client.25790 shutdown 2024-05-14T16:58:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 1 -- 172.21.15.69:0/1807508072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 msgr2=0x558cd1fd0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd0c30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f229c0ca830 tx=0x7f229c0307d0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 1 -- 172.21.15.69:0/1807508072 shutdown_connections 2024-05-14T16:58:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558cd1fe3720 0x558cd1fe5b80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f2289ffb640 0 client.25790 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fd0c30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc9210 0x558cd1fd06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 1 --2- 172.21.15.69:0/1807508072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc88f0 0x558cd1fd3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 1 -- 172.21.15.69:0/1807508072 >> 172.21.15.69:0/1807508072 conn(0x558cd1fc4e00 msgr2=0x558cd1fc4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 1 -- 172.21.15.69:0/1807508072 shutdown_connections 2024-05-14T16:58:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 1 -- 172.21.15.69:0/1807508072 wait complete. 2024-05-14T16:58:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.268+0000 7f22abb73740 20 client.25790 trim_cache size 0 max 16384 2024-05-14T16:58:33.304 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:31 smithi203 ceph-mon[35787]: pgmap v269: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 5.9 MiB/s wr, 634 op/s 2024-05-14T16:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.291+0000 7f22abb73740 1 Processor -- start 2024-05-14T16:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.292+0000 7f22abb73740 1 -- start start 2024-05-14T16:58:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.292+0000 7f22abb73740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc88f0 0x558cd1feb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.292+0000 7f22abb73740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1feb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.292+0000 7f22abb73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fcd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.292+0000 7f22abb73740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b30c0 con 0x558cd1fc88f0 2024-05-14T16:58:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.292+0000 7f22abb73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b0960 con 0x558cd1fd3ad0 2024-05-14T16:58:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.292+0000 7f22abb73740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ae7d0 con 0x558cd1fc9210 2024-05-14T16:58:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.292+0000 7f229bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1feb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.292+0000 7f229bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1feb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:55936/0 (socket says 172.21.15.69:55936) 2024-05-14T16:58:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.292+0000 7f229bfff640 1 -- 172.21.15.69:0/3869818733 learned_addr learned my addr 172.21.15.69:0/3869818733 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.293+0000 7f22a2d9d640 1 --2- 172.21.15.69:0/3869818733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fcd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.293+0000 7f229bfff640 1 -- 172.21.15.69:0/3869818733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 msgr2=0x558cd1fcd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.293+0000 7f229bfff640 1 --2- 172.21.15.69:0/3869818733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fcd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.293+0000 7f229bfff640 1 -- 172.21.15.69:0/3869818733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc88f0 msgr2=0x558cd1feb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.293+0000 7f229bfff640 1 --2- 172.21.15.69:0/3869818733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc88f0 0x558cd1feb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.293+0000 7f229bfff640 1 -- 172.21.15.69:0/3869818733 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cd1fcc2b0 con 0x558cd1fc9210 2024-05-14T16:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.293+0000 7f229bfff640 1 --2- 172.21.15.69:0/3869818733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1feb990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f228c017bc0 tx=0x7f228c009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56ead718cbf557fb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.294+0000 7f228bfff640 1 -- 172.21.15.69:0/3869818733 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c02a070 con 0x558cd1fc9210 2024-05-14T16:58:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.294+0000 7f228bfff640 1 -- 172.21.15.69:0/3869818733 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f228c01e070 con 0x558cd1fc9210 2024-05-14T16:58:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.294+0000 7f228bfff640 1 -- 172.21.15.69:0/3869818733 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c025640 con 0x558cd1fc9210 2024-05-14T16:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 -- 172.21.15.69:0/3869818733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 msgr2=0x558cd1feb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 --2- 172.21.15.69:0/3869818733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1feb990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f228c017bc0 tx=0x7f228c009110 comp rx=0 tx=0).stop 2024-05-14T16:58:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 -- 172.21.15.69:0/3869818733 shutdown_connections 2024-05-14T16:58:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 --2- 172.21.15.69:0/3869818733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fcd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 --2- 172.21.15.69:0/3869818733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1feb990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 --2- 172.21.15.69:0/3869818733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc88f0 0x558cd1feb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 -- 172.21.15.69:0/3869818733 >> 172.21.15.69:0/3869818733 conn(0x558cd1fc4e00 msgr2=0x558cd1fde690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 -- 172.21.15.69:0/3869818733 shutdown_connections 2024-05-14T16:58:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 -- 172.21.15.69:0/3869818733 wait complete. 2024-05-14T16:58:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 -- start start 2024-05-14T16:58:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.295+0000 7f22abb73740 1 Processor -- start 2024-05-14T16:58:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f22abb73740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc88f0 0x558cd1fd96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f22abb73740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1fd9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f22abb73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fda170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f22abb73740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41aee20 con 0x558cd1fc88f0 2024-05-14T16:58:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f22abb73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b0290 con 0x558cd1fd3ad0 2024-05-14T16:58:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f22abb73740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41af4f0 con 0x558cd1fc9210 2024-05-14T16:58:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f22abb73740 10 client.0 _ll_register_callbacks cb 0x7f229c0ca6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f229bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1fd9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f229bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1fd9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:55950/0 (socket says 172.21.15.69:55950) 2024-05-14T16:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f229bfff640 1 -- 172.21.15.69:0/2830435623 learned_addr learned my addr 172.21.15.69:0/2830435623 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f22a259c640 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc88f0 0x558cd1fd96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f22a2d9d640 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fda170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f229bfff640 1 -- 172.21.15.69:0/2830435623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 msgr2=0x558cd1fda170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f229bfff640 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fda170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f229bfff640 1 -- 172.21.15.69:0/2830435623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc88f0 msgr2=0x558cd1fd96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f229bfff640 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc88f0 0x558cd1fd96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f229bfff640 1 -- 172.21.15.69:0/2830435623 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cd204d110 con 0x558cd1fc9210 2024-05-14T16:58:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f229bfff640 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1fd9c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f228c008010 tx=0x7f228c018f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=344f11b694ec3b35 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f2289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:58:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f2289ffb640 1 -- 172.21.15.69:0/2830435623 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c02a050 con 0x558cd1fc9210 2024-05-14T16:58:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f22abb73740 1 -- 172.21.15.69:0/2830435623 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f228c01e070 con 0x558cd1fc9210 2024-05-14T16:58:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.296+0000 7f2289ffb640 1 -- 172.21.15.69:0/2830435623 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f228c026a10 con 0x558cd1fc9210 2024-05-14T16:58:33.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f2289ffb640 1 -- 172.21.15.69:0/2830435623 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f228c02f450 con 0x558cd1fc9210 2024-05-14T16:58:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f22abb73740 10 client.25711 fetch_fsmap learned FSMap version 41 2024-05-14T16:58:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f22abb73740 1 -- 172.21.15.69:0/2830435623 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558cd204e420 con 0x558cd1fc9210 2024-05-14T16:58:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f2289ffb640 1 -- 172.21.15.69:0/2830435623 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f228c01e070 con 0x558cd1fc9210 2024-05-14T16:58:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f2289ffb640 1 -- 172.21.15.69:0/2830435623 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f228c023050 con 0x558cd1fc9210 2024-05-14T16:58:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f22abb73740 10 client.25711 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T16:58:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f22abb73740 10 client.25711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f22abb73740 20 client.25711 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f22abb73740 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558cd204ebe0 0x558cd2051000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f22abb73740 4 client.25711 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:58:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f22abb73740 1 -- 172.21.15.69:0/2830435623 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558cd1f37360 con 0x558cd204ebe0 2024-05-14T16:58:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.297+0000 7f22a259c640 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558cd204ebe0 0x558cd2051000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.298+0000 7f2289ffb640 10 client.25711 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.298+0000 7f22a259c640 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558cd204ebe0 0x558cd2051000 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ee62d3b57dd21 server_cookie=b4c2156d911400c1 in_seq=0 out_seq=0 2024-05-14T16:58:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f2289ffb640 1 -- 172.21.15.69:0/2830435623 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558cd1f37360 con 0x558cd204ebe0 2024-05-14T16:58:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f2289ffb640 10 client.25711 handle_command_reply: tid=0 2024-05-14T16:58:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f22abb73740 1 client.25711 shutdown 2024-05-14T16:58:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f22abb73740 1 -- 172.21.15.69:0/2830435623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 msgr2=0x558cd1fd9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f22abb73740 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1fd9c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f228c008010 tx=0x7f228c018f90 comp rx=0 tx=0).stop 2024-05-14T16:58:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f22abb73740 1 -- 172.21.15.69:0/2830435623 shutdown_connections 2024-05-14T16:58:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f22abb73740 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558cd204ebe0 0x558cd2051000 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f22abb73740 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558cd1fd3ad0 0x558cd1fda170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f22abb73740 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cd1fc9210 0x558cd1fd9c30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f22abb73740 1 --2- 172.21.15.69:0/2830435623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558cd1fc88f0 0x558cd1fd96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f22abb73740 1 -- 172.21.15.69:0/2830435623 >> 172.21.15.69:0/2830435623 conn(0x558cd1fc4e00 msgr2=0x558cd1fdef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f2289ffb640 0 client.25711 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.322+0000 7f22abb73740 1 -- 172.21.15.69:0/2830435623 shutdown_connections 2024-05-14T16:58:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.323+0000 7f22abb73740 1 -- 172.21.15.69:0/2830435623 wait complete. 2024-05-14T16:58:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.323+0000 7f22abb73740 20 client.25711 trim_cache size 0 max 16384 2024-05-14T16:58:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:58:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:58:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.323+0000 7f22a8e59640 1 -- 172.21.15.69:0/3187537120 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f22900b2a40 msgr2=0x7f22900b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.323+0000 7f22a8e59640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f22900b2a40 0x7f22900b4f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x558cd1fc6220 tx=0x7f228c008040 comp rx=0 tx=0).stop 2024-05-14T16:58:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.323+0000 7f22a8e59640 1 -- 172.21.15.69:0/3187537120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4108a40 msgr2=0x7f22a40757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.323+0000 7f22a8e59640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4108a40 0x7f22a40757d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f229c033040 tx=0x7f229c003970 comp rx=0 tx=0).stop 2024-05-14T16:58:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.324+0000 7f22a8e59640 1 -- 172.21.15.69:0/3187537120 shutdown_connections 2024-05-14T16:58:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.324+0000 7f22a8e59640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f22900b2a40 0x7f22900b4f00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.324+0000 7f22a8e59640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4108a40 0x7f22a40757d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.324+0000 7f22a8e59640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a41033f0 0x7f22a4075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.324+0000 7f22a8e59640 1 --2- 172.21.15.69:0/3187537120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102a30 0x7f22a4074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.324+0000 7f22a8e59640 1 -- 172.21.15.69:0/3187537120 >> 172.21.15.69:0/3187537120 conn(0x7f22a40fe500 msgr2=0x7f22a40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.324+0000 7f22a8e59640 1 -- 172.21.15.69:0/3187537120 shutdown_connections 2024-05-14T16:58:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:32.324+0000 7f22a8e59640 1 -- 172.21.15.69:0/3187537120 wait complete. 2024-05-14T16:58:34.024 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:31 smithi069 ceph-mon[31507]: pgmap v269: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 5.9 MiB/s wr, 634 op/s 2024-05-14T16:58:34.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:33 smithi092 ceph-mon[35763]: pgmap v270: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 7.4 MiB/s wr, 778 op/s 2024-05-14T16:58:34.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:33 smithi069 ceph-mon[31507]: pgmap v270: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 7.4 MiB/s wr, 778 op/s 2024-05-14T16:58:34.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:33 smithi203 ceph-mon[35787]: pgmap v270: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 7.4 MiB/s wr, 778 op/s 2024-05-14T16:58:35.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 485.0 s, 2126.0 tps, lat 46.795 ms stddev 50.697 2024-05-14T16:58:35.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:35 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:35.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:35 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:35.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:35 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:36.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:36 smithi069 ceph-mon[31507]: pgmap v271: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.4 MiB/s wr, 574 op/s 2024-05-14T16:58:36.465 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:36 smithi203 ceph-mon[35787]: pgmap v271: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.4 MiB/s wr, 574 op/s 2024-05-14T16:58:36.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:36 smithi092 ceph-mon[35763]: pgmap v271: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.4 MiB/s wr, 574 op/s 2024-05-14T16:58:38.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:38 smithi069 ceph-mon[31507]: pgmap v272: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 9.2 MiB/s wr, 908 op/s 2024-05-14T16:58:38.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:38 smithi203 ceph-mon[35787]: pgmap v272: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 9.2 MiB/s wr, 908 op/s 2024-05-14T16:58:38.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:38 smithi092 ceph-mon[35763]: pgmap v272: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 9.2 MiB/s wr, 908 op/s 2024-05-14T16:58:39.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:39 smithi203 ceph-mon[35787]: pgmap v273: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-05-14T16:58:40.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:39 smithi092 ceph-mon[35763]: pgmap v273: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-05-14T16:58:40.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:39 smithi069 ceph-mon[31507]: pgmap v273: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-05-14T16:58:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 490.0 s, 3135.6 tps, lat 32.234 ms stddev 34.798 2024-05-14T16:58:42.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:41 smithi069 ceph-mon[31507]: pgmap v274: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.4 MiB/s wr, 782 op/s 2024-05-14T16:58:42.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:41 smithi203 ceph-mon[35787]: pgmap v274: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.4 MiB/s wr, 782 op/s 2024-05-14T16:58:42.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:41 smithi092 ceph-mon[35763]: pgmap v274: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.4 MiB/s wr, 782 op/s 2024-05-14T16:58:44.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:43 smithi069 ceph-mon[31507]: pgmap v275: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.1 MiB/s wr, 868 op/s 2024-05-14T16:58:44.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:43 smithi203 ceph-mon[35787]: pgmap v275: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.1 MiB/s wr, 868 op/s 2024-05-14T16:58:44.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:43 smithi092 ceph-mon[35763]: pgmap v275: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.1 MiB/s wr, 868 op/s 2024-05-14T16:58:45.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 495.0 s, 2361.8 tps, lat 42.240 ms stddev 88.828 2024-05-14T16:58:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.433+0000 7fadabdc1640 1 -- 172.21.15.69:0/2476132148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada4107940 msgr2=0x7fada4107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.433+0000 7fadabdc1640 1 --2- 172.21.15.69:0/2476132148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada4107940 0x7fada4107d20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fada0009930 tx=0x7fada002a420 comp rx=0 tx=0).stop 2024-05-14T16:58:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.433+0000 7fadabdc1640 1 -- 172.21.15.69:0/2476132148 shutdown_connections 2024-05-14T16:58:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.433+0000 7fadabdc1640 1 --2- 172.21.15.69:0/2476132148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4102340 0x7fada410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.433+0000 7fadabdc1640 1 --2- 172.21.15.69:0/2476132148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada4101980 0x7fada4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.433+0000 7fadabdc1640 1 --2- 172.21.15.69:0/2476132148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada4107940 0x7fada4107d20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.433+0000 7fadabdc1640 1 -- 172.21.15.69:0/2476132148 >> 172.21.15.69:0/2476132148 conn(0x7fada4078ad0 msgr2=0x7fada4078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.435+0000 7fadabdc1640 1 -- 172.21.15.69:0/2476132148 shutdown_connections 2024-05-14T16:58:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.435+0000 7fadabdc1640 1 -- 172.21.15.69:0/2476132148 wait complete. 2024-05-14T16:58:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fadabdc1640 1 Processor -- start 2024-05-14T16:58:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fadabdc1640 1 -- start start 2024-05-14T16:58:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fadabdc1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4101980 0x7fada41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fadabdc1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada4102340 0x7fada41996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fadabdc1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada4107940 0x7fada419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fadabdc1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fada4111bd0 con 0x7fada4102340 2024-05-14T16:58:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fadabdc1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fada4111a50 con 0x7fada4101980 2024-05-14T16:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fadabdc1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fada4111d50 con 0x7fada4107940 2024-05-14T16:58:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fada935c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada4102340 0x7fada41996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fada935c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada4102340 0x7fada41996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51164/0 (socket says 172.21.15.69:51164) 2024-05-14T16:58:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fada935c640 1 -- 172.21.15.69:0/2918685450 learned_addr learned my addr 172.21.15.69:0/2918685450 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fada9b5d640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4101980 0x7fada41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.436+0000 7fadaa35e640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada4107940 0x7fada419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fada9b5d640 1 -- 172.21.15.69:0/2918685450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada4107940 msgr2=0x7fada419da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fada9b5d640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada4107940 0x7fada419da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fada9b5d640 1 -- 172.21.15.69:0/2918685450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada4102340 msgr2=0x7fada41996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fada9b5d640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada4102340 0x7fada41996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:45.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fada9b5d640 1 -- 172.21.15.69:0/2918685450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada419e190 con 0x7fada4101980 2024-05-14T16:58:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fadaa35e640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada4107940 0x7fada419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:58:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fada9b5d640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4101980 0x7fada41991a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fada0031440 tx=0x7fada0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fad9affd640 1 -- 172.21.15.69:0/2918685450 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada003f070 con 0x7fada4101980 2024-05-14T16:58:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fadabdc1640 1 -- 172.21.15.69:0/2918685450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada419e420 con 0x7fada4101980 2024-05-14T16:58:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fadabdc1640 1 -- 172.21.15.69:0/2918685450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada41a5d10 con 0x7fada4101980 2024-05-14T16:58:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fad9affd640 1 -- 172.21.15.69:0/2918685450 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada0003ca0 con 0x7fada4101980 2024-05-14T16:58:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.437+0000 7fad9affd640 1 -- 172.21.15.69:0/2918685450 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0048060 con 0x7fada4101980 2024-05-14T16:58:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.440+0000 7fadaebba740 1 Processor -- start 2024-05-14T16:58:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaebba740 1 -- start start 2024-05-14T16:58:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaebba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e671bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaebba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e6720f0 0x55d53e6809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaebba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 0x55d53e6832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaebba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d53e5e4c30 con 0x55d53e6717d0 2024-05-14T16:58:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaebba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d53e5d47d0 con 0x55d53e680ee0 2024-05-14T16:58:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaebba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d53e5d8810 con 0x55d53e6720f0 2024-05-14T16:58:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaa35e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e671bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaa35e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e671bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51180/0 (socket says 172.21.15.69:51180) 2024-05-14T16:58:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaa35e640 1 -- 172.21.15.69:0/1789021916 learned_addr learned my addr 172.21.15.69:0/1789021916 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fada935c640 1 --2- 172.21.15.69:0/1789021916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 0x55d53e6832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fad9affd640 1 -- 172.21.15.69:0/2918685450 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fada003b460 con 0x7fada4101980 2024-05-14T16:58:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaa35e640 1 -- 172.21.15.69:0/1789021916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e6720f0 msgr2=0x55d53e6809a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:58:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaa35e640 1 --2- 172.21.15.69:0/1789021916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e6720f0 0x55d53e6809a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaa35e640 1 -- 172.21.15.69:0/1789021916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 msgr2=0x55d53e6832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaa35e640 1 --2- 172.21.15.69:0/1789021916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 0x55d53e6832d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fadaa35e640 1 -- 172.21.15.69:0/1789021916 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d53e2e9210 con 0x55d53e6717d0 2024-05-14T16:58:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.441+0000 7fada935c640 1 --2- 172.21.15.69:0/1789021916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 0x55d53e6832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.442+0000 7fadaa35e640 1 --2- 172.21.15.69:0/1789021916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e671bb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0103d0 tx=0x7fad9c010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c646cbe78ad38ee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.442+0000 7fad8ffff640 1 -- 172.21.15.69:0/1789021916 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c011700 con 0x55d53e6717d0 2024-05-14T16:58:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.442+0000 7fad8ffff640 1 -- 172.21.15.69:0/1789021916 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9c011d50 con 0x55d53e6717d0 2024-05-14T16:58:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.442+0000 7fad8ffff640 1 -- 172.21.15.69:0/1789021916 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00e3c0 con 0x55d53e6717d0 2024-05-14T16:58:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.442+0000 7fad9affd640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fad900b2a10 0x7fad900b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.443+0000 7fad9affd640 1 -- 172.21.15.69:0/2918685450 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fada0102a60 con 0x7fada4101980 2024-05-14T16:58:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.443+0000 7fadaebba740 1 -- 172.21.15.69:0/1789021916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 msgr2=0x55d53e671bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.443+0000 7fadaebba740 1 --2- 172.21.15.69:0/1789021916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e671bb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0103d0 tx=0x7fad9c010890 comp rx=0 tx=0).stop 2024-05-14T16:58:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.443+0000 7fadaebba740 1 -- 172.21.15.69:0/1789021916 shutdown_connections 2024-05-14T16:58:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.443+0000 7fadaebba740 1 --2- 172.21.15.69:0/1789021916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 0x55d53e6832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.443+0000 7fadaebba740 1 --2- 172.21.15.69:0/1789021916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e6720f0 0x55d53e6809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.443+0000 7fadaebba740 1 --2- 172.21.15.69:0/1789021916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e671bb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.443+0000 7fadaebba740 1 -- 172.21.15.69:0/1789021916 >> 172.21.15.69:0/1789021916 conn(0x55d53e66de00 msgr2=0x55d53e66e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.444+0000 7fada9b5d640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fad900b2a10 0x7fad900b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.444+0000 7fadaebba740 1 -- 172.21.15.69:0/1789021916 shutdown_connections 2024-05-14T16:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.444+0000 7fadaebba740 1 -- 172.21.15.69:0/1789021916 wait complete. 2024-05-14T16:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.445+0000 7fadaebba740 1 -- start start 2024-05-14T16:58:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.445+0000 7fadaebba740 1 Processor -- start 2024-05-14T16:58:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.445+0000 7fada9b5d640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fad900b2a10 0x7fad900b4ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fada0006010 tx=0x7fada0009620 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.445+0000 7fadaebba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e6717d0 0x55d53e680890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.445+0000 7fadaebba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6720f0 0x55d53e67db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.445+0000 7fadaebba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 0x55d53e67e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fada935c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6720f0 0x55d53e67db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fada935c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6720f0 0x55d53e67db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51186/0 (socket says 172.21.15.69:51186) 2024-05-14T16:58:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fadaa35e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e6717d0 0x55d53e680890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fadaa35e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e6717d0 0x55d53e680890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:47742/0 (socket says 172.21.15.69:47742) 2024-05-14T16:58:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fada935c640 1 -- 172.21.15.69:0/2021849687 learned_addr learned my addr 172.21.15.69:0/2021849687 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d53e2ba790 con 0x55d53e6720f0 2024-05-14T16:58:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d53e590890 con 0x55d53e680ee0 2024-05-14T16:58:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d53e683bb0 con 0x55d53e6717d0 2024-05-14T16:58:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fadaa35e640 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 0x55d53e67e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fada935c640 1 -- 172.21.15.69:0/2021849687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e6717d0 msgr2=0x55d53e680890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fada935c640 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e6717d0 0x55d53e680890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fada935c640 1 -- 172.21.15.69:0/2021849687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 msgr2=0x55d53e67e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fada935c640 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 0x55d53e67e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fada935c640 1 -- 172.21.15.69:0/2021849687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d53e3b5620 con 0x55d53e6720f0 2024-05-14T16:58:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fadaebba740 10 client.0 _ll_register_callbacks cb 0x55d53e593d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.446+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9c011d50 con 0x55d53e6720f0 2024-05-14T16:58:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.447+0000 7fada935c640 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6720f0 0x55d53e67db80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fad9400bc20 tx=0x7fad940039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c49d750d6890bdc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.457+0000 7fad8d7fa640 10 client.16350 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:58:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.461+0000 7fad8d7fa640 1 -- 172.21.15.69:0/2021849687 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9401b470 con 0x55d53e6720f0 2024-05-14T16:58:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.461+0000 7fad8d7fa640 1 -- 172.21.15.69:0/2021849687 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fad9401b610 con 0x55d53e6720f0 2024-05-14T16:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.461+0000 7fad8d7fa640 1 -- 172.21.15.69:0/2021849687 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fad9401b870 con 0x55d53e6720f0 2024-05-14T16:58:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.461+0000 7fadaebba740 10 client.16350 fetch_fsmap learned FSMap version 42 2024-05-14T16:58:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.461+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55d53e68f550 con 0x55d53e6720f0 2024-05-14T16:58:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.467+0000 7fad8d7fa640 1 -- 172.21.15.69:0/2021849687 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7fad9401ba10 con 0x55d53e6720f0 2024-05-14T16:58:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.467+0000 7fadaebba740 10 client.16350 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T16:58:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.467+0000 7fadaebba740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.467+0000 7fadaebba740 20 client.16350 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.467+0000 7fadaebba740 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d53e690a70 0x55d53e692f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.467+0000 7fadaebba740 4 client.16350 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.467+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d53e5e6b80 con 0x55d53e690a70 2024-05-14T16:58:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.468+0000 7fadaa35e640 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d53e690a70 0x55d53e692f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:46.585 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:45 smithi092 ceph-mon[35763]: pgmap v276: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.8 MiB/s wr, 632 op/s 2024-05-14T16:58:46.590 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:45 smithi203 ceph-mon[35787]: pgmap v276: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.8 MiB/s wr, 632 op/s 2024-05-14T16:58:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.484+0000 7fadaa35e640 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d53e690a70 0x55d53e692f10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6403cddedbaf07d server_cookie=ba3e0dd39706479 in_seq=0 out_seq=0 2024-05-14T16:58:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.484+0000 7fad8d7fa640 10 client.16350 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.485+0000 7fad8d7fa640 1 -- 172.21.15.69:0/2021849687 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d53e5e6b80 con 0x55d53e690a70 2024-05-14T16:58:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.485+0000 7fad8d7fa640 10 client.16350 handle_command_reply: tid=0 2024-05-14T16:58:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.485+0000 7fadaebba740 1 client.16350 shutdown 2024-05-14T16:58:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.485+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6720f0 msgr2=0x55d53e67db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.485+0000 7fadaebba740 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6720f0 0x55d53e67db80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fad9400bc20 tx=0x7fad940039c0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.490+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 shutdown_connections 2024-05-14T16:58:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.490+0000 7fadaebba740 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d53e690a70 0x55d53e692f10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.490+0000 7fadaebba740 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e680ee0 0x55d53e67e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.490+0000 7fadaebba740 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6720f0 0x55d53e67db80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.490+0000 7fadaebba740 1 --2- 172.21.15.69:0/2021849687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e6717d0 0x55d53e680890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.490+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 >> 172.21.15.69:0/2021849687 conn(0x55d53e66de00 msgr2=0x55d53e673c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.494+0000 7fad8d7fa640 0 client.16350 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.497+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 shutdown_connections 2024-05-14T16:58:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.501+0000 7fadaebba740 1 -- 172.21.15.69:0/2021849687 wait complete. 2024-05-14T16:58:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.501+0000 7fadaebba740 20 client.16350 trim_cache size 0 max 16384 2024-05-14T16:58:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.526+0000 7fadaebba740 1 Processor -- start 2024-05-14T16:58:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaebba740 1 -- start start 2024-05-14T16:58:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaebba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e6803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaebba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaebba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 0x55d53e698d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaebba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad880111b0 con 0x55d53e6717d0 2024-05-14T16:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaebba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad8800ea20 con 0x55d53e6720f0 2024-05-14T16:58:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaebba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad8800c890 con 0x55d53e680ee0 2024-05-14T16:58:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaa35e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e6803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fada935c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaa35e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e6803c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51190/0 (socket says 172.21.15.69:51190) 2024-05-14T16:58:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaa35e640 1 -- 172.21.15.69:0/2761720244 learned_addr learned my addr 172.21.15.69:0/2761720244 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.527+0000 7fadaa35e640 1 --2- 172.21.15.69:0/2761720244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 0x55d53e698d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.528+0000 7fada935c640 1 -- 172.21.15.69:0/2761720244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 msgr2=0x55d53e698d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.528+0000 7fada935c640 1 --2- 172.21.15.69:0/2761720244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 0x55d53e698d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.528+0000 7fada935c640 1 -- 172.21.15.69:0/2761720244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 msgr2=0x55d53e6803c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.528+0000 7fada935c640 1 --2- 172.21.15.69:0/2761720244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e6803c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.528+0000 7fada935c640 1 -- 172.21.15.69:0/2761720244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d53e668510 con 0x55d53e6720f0 2024-05-14T16:58:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.528+0000 7fadaa35e640 1 --2- 172.21.15.69:0/2761720244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e6803c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.528+0000 7fada935c640 1 --2- 172.21.15.69:0/2761720244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6987e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fad9400bbf0 tx=0x7fad94009000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aa8c2123017fe42 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.528+0000 7fad8f7fe640 1 -- 172.21.15.69:0/2761720244 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9401b470 con 0x55d53e6720f0 2024-05-14T16:58:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.528+0000 7fad8f7fe640 1 -- 172.21.15.69:0/2761720244 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9c011d50 con 0x55d53e6720f0 2024-05-14T16:58:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.528+0000 7fad8f7fe640 1 -- 172.21.15.69:0/2761720244 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94011e20 con 0x55d53e6720f0 2024-05-14T16:58:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.529+0000 7fadaebba740 1 -- 172.21.15.69:0/2761720244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 msgr2=0x55d53e6987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.529+0000 7fadaebba740 1 --2- 172.21.15.69:0/2761720244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6987e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fad9400bbf0 tx=0x7fad94009000 comp rx=0 tx=0).stop 2024-05-14T16:58:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.529+0000 7fadaebba740 1 -- 172.21.15.69:0/2761720244 shutdown_connections 2024-05-14T16:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.529+0000 7fadaebba740 1 --2- 172.21.15.69:0/2761720244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 0x55d53e698d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.529+0000 7fadaebba740 1 --2- 172.21.15.69:0/2761720244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6987e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.529+0000 7fadaebba740 1 --2- 172.21.15.69:0/2761720244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e6803c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.529+0000 7fadaebba740 1 -- 172.21.15.69:0/2761720244 >> 172.21.15.69:0/2761720244 conn(0x55d53e66de00 msgr2=0x55d53e67dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.529+0000 7fadaebba740 1 -- 172.21.15.69:0/2761720244 shutdown_connections 2024-05-14T16:58:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.529+0000 7fadaebba740 1 -- 172.21.15.69:0/2761720244 wait complete. 2024-05-14T16:58:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.529+0000 7fadaebba740 1 -- start start 2024-05-14T16:58:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fadaebba740 1 Processor -- start 2024-05-14T16:58:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fadaebba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e675070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fadaebba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fadaebba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 0x55d53e687ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fadaebba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad8800cee0 con 0x55d53e6717d0 2024-05-14T16:58:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fadaebba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad8800e350 con 0x55d53e6720f0 2024-05-14T16:58:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fadaebba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad8800d5b0 con 0x55d53e680ee0 2024-05-14T16:58:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fadaebba740 10 client.0 _ll_register_callbacks cb 0x7fad940043a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fadaa35e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e675070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fada935c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fada935c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6755b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:46590/0 (socket says 172.21.15.69:46590) 2024-05-14T16:58:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fada935c640 1 -- 172.21.15.69:0/3201744639 learned_addr learned my addr 172.21.15.69:0/3201744639 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fadaa35e640 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 0x55d53e687ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fada935c640 1 -- 172.21.15.69:0/3201744639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 msgr2=0x55d53e687ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fada935c640 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 0x55d53e687ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fada935c640 1 -- 172.21.15.69:0/3201744639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 msgr2=0x55d53e675070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fada935c640 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e675070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.530+0000 7fada935c640 1 -- 172.21.15.69:0/3201744639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d53e6884b0 con 0x55d53e6720f0 2024-05-14T16:58:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.531+0000 7fadaa35e640 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 0x55d53e687ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:58:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.531+0000 7fada935c640 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6755b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fad94000c00 tx=0x7fad9401b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e5001be5421a31 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.531+0000 7fad8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:58:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.531+0000 7fad8d7fa640 1 -- 172.21.15.69:0/3201744639 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94011d70 con 0x55d53e6720f0 2024-05-14T16:58:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.531+0000 7fad8d7fa640 1 -- 172.21.15.69:0/3201744639 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fad94011440 con 0x55d53e6720f0 2024-05-14T16:58:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.531+0000 7fadaebba740 1 -- 172.21.15.69:0/3201744639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9c011d50 con 0x55d53e6720f0 2024-05-14T16:58:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.531+0000 7fad8d7fa640 1 -- 172.21.15.69:0/3201744639 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fad94040050 con 0x55d53e6720f0 2024-05-14T16:58:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fadaebba740 10 client.25814 fetch_fsmap learned FSMap version 42 2024-05-14T16:58:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fadaebba740 1 -- 172.21.15.69:0/3201744639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d53e6897a0 con 0x55d53e6720f0 2024-05-14T16:58:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fad8d7fa640 1 -- 172.21.15.69:0/3201744639 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9c011d50 con 0x55d53e6720f0 2024-05-14T16:58:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fad8d7fa640 1 -- 172.21.15.69:0/3201744639 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7fad94004810 con 0x55d53e6720f0 2024-05-14T16:58:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fadaebba740 10 client.25814 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T16:58:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fadaebba740 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fadaebba740 20 client.25814 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fadaebba740 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d53e6f7740 0x55d53e6f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fadaebba740 4 client.25814 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:58:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fadaebba740 1 -- 172.21.15.69:0/3201744639 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d53e5e0360 con 0x55d53e6f7740 2024-05-14T16:58:47.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.532+0000 7fadaa35e640 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d53e6f7740 0x55d53e6f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.533+0000 7fad8d7fa640 10 client.25814 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.533+0000 7fadaa35e640 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d53e6f7740 0x55d53e6f9b60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebc32f8707ca8fb server_cookie=d0208a64503cd487 in_seq=0 out_seq=0 2024-05-14T16:58:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fad8d7fa640 1 -- 172.21.15.69:0/3201744639 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d53e5e0360 con 0x55d53e6f7740 2024-05-14T16:58:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fad8d7fa640 10 client.25814 handle_command_reply: tid=0 2024-05-14T16:58:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 client.25814 shutdown 2024-05-14T16:58:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 -- 172.21.15.69:0/3201744639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 msgr2=0x55d53e6755b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6755b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fad94000c00 tx=0x7fad9401b6a0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 -- 172.21.15.69:0/3201744639 shutdown_connections 2024-05-14T16:58:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d53e6f7740 0x55d53e6f9b60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d53e680ee0 0x55d53e687ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d53e6720f0 0x55d53e6755b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 --2- 172.21.15.69:0/3201744639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53e6717d0 0x55d53e675070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 -- 172.21.15.69:0/3201744639 >> 172.21.15.69:0/3201744639 conn(0x55d53e66de00 msgr2=0x55d53e66c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fad8d7fa640 0 client.25814 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 -- 172.21.15.69:0/3201744639 shutdown_connections 2024-05-14T16:58:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 1 -- 172.21.15.69:0/3201744639 wait complete. 2024-05-14T16:58:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.548+0000 7fadaebba740 20 client.25814 trim_cache size 0 max 16384 2024-05-14T16:58:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:58:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:58:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:58:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 -- 172.21.15.69:0/2918685450 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fad900b2a10 msgr2=0x7fad900b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fad900b2a10 0x7fad900b4ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fada0006010 tx=0x7fada0009620 comp rx=0 tx=0).stop 2024-05-14T16:58:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 -- 172.21.15.69:0/2918685450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4101980 msgr2=0x7fada41991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4101980 0x7fada41991a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fada0031440 tx=0x7fada0004270 comp rx=0 tx=0).stop 2024-05-14T16:58:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 -- 172.21.15.69:0/2918685450 shutdown_connections 2024-05-14T16:58:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fad900b2a10 0x7fad900b4ed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada4107940 0x7fada419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada4102340 0x7fada41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 --2- 172.21.15.69:0/2918685450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada4101980 0x7fada41991a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 -- 172.21.15.69:0/2918685450 >> 172.21.15.69:0/2918685450 conn(0x7fada4078ad0 msgr2=0x7fada4100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 -- 172.21.15.69:0/2918685450 shutdown_connections 2024-05-14T16:58:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:45.549+0000 7fadabdc1640 1 -- 172.21.15.69:0/2918685450 wait complete. 2024-05-14T16:58:47.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:45 smithi069 ceph-mon[31507]: pgmap v276: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.8 MiB/s wr, 632 op/s 2024-05-14T16:58:48.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:47 smithi069 ceph-mon[31507]: pgmap v277: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 35 MiB/s wr, 1.93k op/s 2024-05-14T16:58:48.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:47 smithi203 ceph-mon[35787]: pgmap v277: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 35 MiB/s wr, 1.93k op/s 2024-05-14T16:58:48.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:47 smithi092 ceph-mon[35763]: pgmap v277: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 35 MiB/s wr, 1.93k op/s 2024-05-14T16:58:50.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:49 smithi069 ceph-mon[31507]: pgmap v278: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 1.59k op/s 2024-05-14T16:58:50.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:49 smithi203 ceph-mon[35787]: pgmap v278: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 1.59k op/s 2024-05-14T16:58:50.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:49 smithi092 ceph-mon[35763]: pgmap v278: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 1.59k op/s 2024-05-14T16:58:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 500.0 s, 2503.0 tps, lat 39.768 ms stddev 44.646 2024-05-14T16:58:52.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:51 smithi069 ceph-mon[31507]: pgmap v279: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 1.64k op/s 2024-05-14T16:58:52.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:51 smithi203 ceph-mon[35787]: pgmap v279: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 1.64k op/s 2024-05-14T16:58:52.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:51 smithi092 ceph-mon[35763]: pgmap v279: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 1.64k op/s 2024-05-14T16:58:54.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:53 smithi069 ceph-mon[31507]: pgmap v280: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 1.72k op/s 2024-05-14T16:58:54.213 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:53 smithi203 ceph-mon[35787]: pgmap v280: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 1.72k op/s 2024-05-14T16:58:54.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:53 smithi092 ceph-mon[35763]: pgmap v280: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 1.72k op/s 2024-05-14T16:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 505.0 s, 2971.4 tps, lat 33.834 ms stddev 37.077 2024-05-14T16:58:56.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:55 smithi069 ceph-mon[31507]: pgmap v281: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 1.58k op/s 2024-05-14T16:58:56.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:55 smithi203 ceph-mon[35787]: pgmap v281: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 1.58k op/s 2024-05-14T16:58:56.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:55 smithi092 ceph-mon[35763]: pgmap v281: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 1.58k op/s 2024-05-14T16:58:58.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:57 smithi069 ceph-mon[31507]: pgmap v282: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 1.87k op/s 2024-05-14T16:58:58.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:57 smithi203 ceph-mon[35787]: pgmap v282: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 1.87k op/s 2024-05-14T16:58:58.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:57 smithi092 ceph-mon[35763]: pgmap v282: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 1.87k op/s 2024-05-14T16:58:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.688+0000 7fe75f23f640 1 -- 172.21.15.69:0/1433050918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe758103d00 msgr2=0x7fe7581040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.688+0000 7fe75f23f640 1 --2- 172.21.15.69:0/1433050918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe758103d00 0x7fe7581040e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009900 tx=0x7fe74c02a480 comp rx=0 tx=0).stop 2024-05-14T16:58:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.688+0000 7fe75f23f640 1 -- 172.21.15.69:0/1433050918 shutdown_connections 2024-05-14T16:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.688+0000 7fe75f23f640 1 --2- 172.21.15.69:0/1433050918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe75810d040 0x7fe75810f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.688+0000 7fe75f23f640 1 --2- 172.21.15.69:0/1433050918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758104620 0x7fe75810cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.688+0000 7fe75f23f640 1 --2- 172.21.15.69:0/1433050918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe758103d00 0x7fe7581040e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.688+0000 7fe75f23f640 1 -- 172.21.15.69:0/1433050918 >> 172.21.15.69:0/1433050918 conn(0x7fe758078ad0 msgr2=0x7fe758078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.688+0000 7fe75f23f640 1 -- 172.21.15.69:0/1433050918 shutdown_connections 2024-05-14T16:58:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.689+0000 7fe75f23f640 1 -- 172.21.15.69:0/1433050918 wait complete. 2024-05-14T16:58:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.689+0000 7fe75f23f640 1 Processor -- start 2024-05-14T16:58:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe75f23f640 1 -- start start 2024-05-14T16:58:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe75f23f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758103d00 0x7fe75819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe75f23f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758104620 0x7fe75819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe75f23f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe75810d040 0x7fe75819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe75f23f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe758112cc0 con 0x7fe75810d040 2024-05-14T16:58:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe75f23f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe758112b40 con 0x7fe758103d00 2024-05-14T16:58:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe75f23f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe758112e40 con 0x7fe758104620 2024-05-14T16:58:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe757fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758104620 0x7fe75819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe757fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758104620 0x7fe75819b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:44050/0 (socket says 172.21.15.69:44050) 2024-05-14T16:58:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe757fff640 1 -- 172.21.15.69:0/2375261868 learned_addr learned my addr 172.21.15.69:0/2375261868 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.690+0000 7fe75cfdb640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758103d00 0x7fe75819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe757fff640 1 -- 172.21.15.69:0/2375261868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758103d00 msgr2=0x7fe75819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe757fff640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758103d00 0x7fe75819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe757fff640 1 -- 172.21.15.69:0/2375261868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe75810d040 msgr2=0x7fe75819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe757fff640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe75810d040 0x7fe75819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe757fff640 1 -- 172.21.15.69:0/2375261868 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7581a03c0 con 0x7fe758104620 2024-05-14T16:58:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe75cfdb640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758103d00 0x7fe75819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:58:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe757fff640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758104620 0x7fe75819b960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe74800e9f0 tx=0x7fe74800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe755ffb640 1 -- 172.21.15.69:0/2375261868 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74800cd80 con 0x7fe758104620 2024-05-14T16:58:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe75f23f640 1 -- 172.21.15.69:0/2375261868 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7581a06b0 con 0x7fe758104620 2024-05-14T16:58:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe755ffb640 1 -- 172.21.15.69:0/2375261868 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7480108e0 con 0x7fe758104620 2024-05-14T16:58:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.691+0000 7fe755ffb640 1 -- 172.21.15.69:0/2375261868 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748018980 con 0x7fe758104620 2024-05-14T16:58:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.692+0000 7fe75f23f640 1 -- 172.21.15.69:0/2375261868 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7581a7f40 con 0x7fe758104620 2024-05-14T16:58:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.694+0000 7fe755ffb640 1 -- 172.21.15.69:0/2375261868 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe748018b20 con 0x7fe758104620 2024-05-14T16:58:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe762031740 1 Processor -- start 2024-05-14T16:58:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe762031740 1 -- start start 2024-05-14T16:58:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe755ffb640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe7400b2a90 0x7fe7400b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe762031740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f948f0 0x55d2c7f94cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe762031740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7f9f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe762031740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fa1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe762031740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d2c7f07c30 con 0x55d2c7f948f0 2024-05-14T16:58:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe762031740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d2c7ef77d0 con 0x55d2c7f95210 2024-05-14T16:58:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe762031740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d2c7efb810 con 0x55d2c7f9fad0 2024-05-14T16:58:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe755ffb640 1 -- 172.21.15.69:0/2375261868 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fe7480e28c0 con 0x7fe758104620 2024-05-14T16:58:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe75cfdb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f948f0 0x55d2c7f94cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe75cfdb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f948f0 0x55d2c7f94cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35362/0 (socket says 172.21.15.69:35362) 2024-05-14T16:58:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe75cfdb640 1 -- 172.21.15.69:0/510071746 learned_addr learned my addr 172.21.15.69:0/510071746 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe757fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7f9f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.695+0000 7fe75d7dc640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe7400b2a90 0x7fe7400b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.696+0000 7fe75d7dc640 1 --2- 172.21.15.69:0/510071746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fa1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.696+0000 7fe757fff640 1 -- 172.21.15.69:0/510071746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f9fad0 msgr2=0x55d2c7fa1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.696+0000 7fe757fff640 1 --2- 172.21.15.69:0/510071746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fa1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.696+0000 7fe757fff640 1 -- 172.21.15.69:0/510071746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f948f0 msgr2=0x55d2c7f94cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.696+0000 7fe757fff640 1 --2- 172.21.15.69:0/510071746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f948f0 0x55d2c7f94cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.696+0000 7fe757fff640 1 -- 172.21.15.69:0/510071746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c7c0c210 con 0x55d2c7f95210 2024-05-14T16:58:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.696+0000 7fe75cfdb640 1 --2- 172.21.15.69:0/510071746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f948f0 0x55d2c7f94cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.696+0000 7fe75d7dc640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe7400b2a90 0x7fe7400b4f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55d2c7f92220 tx=0x7fe750009740 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.697+0000 7fe757fff640 1 --2- 172.21.15.69:0/510071746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7f9f590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe7480e0000 tx=0x7fe7480a64e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de7385d76b0910a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.697+0000 7fe73effd640 1 -- 172.21.15.69:0/510071746 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480158e0 con 0x55d2c7f95210 2024-05-14T16:58:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.697+0000 7fe73effd640 1 -- 172.21.15.69:0/510071746 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7480a9b90 con 0x55d2c7f95210 2024-05-14T16:58:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.697+0000 7fe73effd640 1 -- 172.21.15.69:0/510071746 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480b2560 con 0x55d2c7f95210 2024-05-14T16:58:59.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 -- 172.21.15.69:0/510071746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 msgr2=0x55d2c7f9f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 --2- 172.21.15.69:0/510071746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7f9f590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe7480e0000 tx=0x7fe7480a64e0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 -- 172.21.15.69:0/510071746 shutdown_connections 2024-05-14T16:58:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 --2- 172.21.15.69:0/510071746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fa1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 --2- 172.21.15.69:0/510071746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7f9f590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 --2- 172.21.15.69:0/510071746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f948f0 0x55d2c7f94cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 -- 172.21.15.69:0/510071746 >> 172.21.15.69:0/510071746 conn(0x55d2c7f90e00 msgr2=0x55d2c7f911e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 -- 172.21.15.69:0/510071746 shutdown_connections 2024-05-14T16:58:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 -- 172.21.15.69:0/510071746 wait complete. 2024-05-14T16:58:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 -- start start 2024-05-14T16:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.698+0000 7fe762031740 1 Processor -- start 2024-05-14T16:58:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe762031740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f948f0 0x55d2c7f9f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe762031740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f95210 0x55d2c7f9c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe762031740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f9fad0 0x55d2c7f9ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe762031740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d2c7bdd790 con 0x55d2c7f95210 2024-05-14T16:58:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe762031740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d2c7eb3890 con 0x55d2c7f948f0 2024-05-14T16:58:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe762031740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d2c7fa27a0 con 0x55d2c7f9fad0 2024-05-14T16:58:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe762031740 10 client.0 _ll_register_callbacks cb 0x55d2c7eb6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe757fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f95210 0x55d2c7f9c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe75cfdb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f948f0 0x55d2c7f9f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe757fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f95210 0x55d2c7f9c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35376/0 (socket says 172.21.15.69:35376) 2024-05-14T16:58:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe757fff640 1 -- 172.21.15.69:0/1384769007 learned_addr learned my addr 172.21.15.69:0/1384769007 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe75d7dc640 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f9fad0 0x55d2c7f9ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe757fff640 1 -- 172.21.15.69:0/1384769007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f9fad0 msgr2=0x55d2c7f9ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe757fff640 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f9fad0 0x55d2c7f9ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe757fff640 1 -- 172.21.15.69:0/1384769007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f948f0 msgr2=0x55d2c7f9f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe757fff640 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f948f0 0x55d2c7f9f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe757fff640 1 -- 172.21.15.69:0/1384769007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c7cd8620 con 0x55d2c7f95210 2024-05-14T16:58:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe75cfdb640 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f948f0 0x55d2c7f9f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe757fff640 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f95210 0x55d2c7f9c750 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe7480a6590 tx=0x7fe74800d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937faed3f961681f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.699+0000 7fe737fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:58:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.700+0000 7fe737fff640 1 -- 172.21.15.69:0/1384769007 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480c2070 con 0x55d2c7f95210 2024-05-14T16:58:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.700+0000 7fe737fff640 1 -- 172.21.15.69:0/1384769007 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fe748010a90 con 0x55d2c7f95210 2024-05-14T16:58:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.700+0000 7fe762031740 1 -- 172.21.15.69:0/1384769007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7480a9b90 con 0x55d2c7f95210 2024-05-14T16:58:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.700+0000 7fe737fff640 1 -- 172.21.15.69:0/1384769007 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe748010c90 con 0x55d2c7f95210 2024-05-14T16:58:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.700+0000 7fe762031740 10 client.16365 fetch_fsmap learned FSMap version 42 2024-05-14T16:58:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.700+0000 7fe762031740 1 -- 172.21.15.69:0/1384769007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2c7fae250 con 0x55d2c7f95210 2024-05-14T16:58:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.701+0000 7fe737fff640 1 -- 172.21.15.69:0/1384769007 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74800b2c0 con 0x55d2c7f95210 2024-05-14T16:58:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.701+0000 7fe737fff640 1 -- 172.21.15.69:0/1384769007 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7fe7480cc0b0 con 0x55d2c7f95210 2024-05-14T16:58:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.701+0000 7fe762031740 10 client.16365 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T16:58:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.701+0000 7fe762031740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:59.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.701+0000 7fe762031740 20 client.16365 populate_metadata read hostname 'smithi069' 2024-05-14T16:58:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.701+0000 7fe762031740 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d2c7faf980 0x55d2c7fb1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.701+0000 7fe762031740 4 client.16365 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.701+0000 7fe762031740 1 -- 172.21.15.69:0/1384769007 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe7480a9b90 con 0x55d2c7faf980 2024-05-14T16:58:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.702+0000 7fe75cfdb640 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d2c7faf980 0x55d2c7fb1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.702+0000 7fe737fff640 10 client.16365 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.702+0000 7fe75cfdb640 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d2c7faf980 0x55d2c7fb1da0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d28df0e582c261 server_cookie=668b546f707d2829 in_seq=0 out_seq=0 2024-05-14T16:58:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe737fff640 1 -- 172.21.15.69:0/1384769007 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fe7480a9b90 con 0x55d2c7faf980 2024-05-14T16:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe737fff640 10 client.16365 handle_command_reply: tid=0 2024-05-14T16:58:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 client.16365 shutdown 2024-05-14T16:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 -- 172.21.15.69:0/1384769007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f95210 msgr2=0x55d2c7f9c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f95210 0x55d2c7f9c750 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe7480a6590 tx=0x7fe74800d8d0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 -- 172.21.15.69:0/1384769007 shutdown_connections 2024-05-14T16:58:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d2c7faf980 0x55d2c7fb1da0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f9fad0 0x55d2c7f9ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe737fff640 0 client.16365 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:58:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f95210 0x55d2c7f9c750 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 --2- 172.21.15.69:0/1384769007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f948f0 0x55d2c7f9f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 -- 172.21.15.69:0/1384769007 >> 172.21.15.69:0/1384769007 conn(0x55d2c7f90e00 msgr2=0x55d2c7f90100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 -- 172.21.15.69:0/1384769007 shutdown_connections 2024-05-14T16:58:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 1 -- 172.21.15.69:0/1384769007 wait complete. 2024-05-14T16:58:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.703+0000 7fe762031740 20 client.16365 trim_cache size 0 max 16384 2024-05-14T16:58:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe762031740 1 Processor -- start 2024-05-14T16:58:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe762031740 1 -- start start 2024-05-14T16:58:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe762031740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f948f0 0x55d2c7fab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe762031740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fb7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe762031740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fb7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe762031740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe744011100 con 0x55d2c7f9fad0 2024-05-14T16:58:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe762031740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe74400e970 con 0x55d2c7f95210 2024-05-14T16:58:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe762031740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe74400c7e0 con 0x55d2c7f948f0 2024-05-14T16:58:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe75d7dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fb7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe75d7dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fb7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35388/0 (socket says 172.21.15.69:35388) 2024-05-14T16:58:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe75d7dc640 1 -- 172.21.15.69:0/532250285 learned_addr learned my addr 172.21.15.69:0/532250285 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:58:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe757fff640 1 --2- 172.21.15.69:0/532250285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fb7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe75cfdb640 1 --2- 172.21.15.69:0/532250285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f948f0 0x55d2c7fab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.724+0000 7fe75d7dc640 1 -- 172.21.15.69:0/532250285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f948f0 msgr2=0x55d2c7fab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.725+0000 7fe75d7dc640 1 --2- 172.21.15.69:0/532250285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f948f0 0x55d2c7fab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.725+0000 7fe75d7dc640 1 -- 172.21.15.69:0/532250285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 msgr2=0x55d2c7fb7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.725+0000 7fe75d7dc640 1 --2- 172.21.15.69:0/532250285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fb7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.725+0000 7fe75d7dc640 1 -- 172.21.15.69:0/532250285 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c7f919d0 con 0x55d2c7f9fad0 2024-05-14T16:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.725+0000 7fe757fff640 1 --2- 172.21.15.69:0/532250285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fb7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.725+0000 7fe75d7dc640 1 --2- 172.21.15.69:0/532250285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fb7bb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe7500098a0 tx=0x7fe75000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567d7ecc382aabee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.725+0000 7fe73e7fc640 1 -- 172.21.15.69:0/532250285 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75000ed70 con 0x55d2c7f9fad0 2024-05-14T16:58:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.725+0000 7fe73e7fc640 1 -- 172.21.15.69:0/532250285 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7500118e0 con 0x55d2c7f9fad0 2024-05-14T16:58:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.725+0000 7fe73e7fc640 1 -- 172.21.15.69:0/532250285 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7500054e0 con 0x55d2c7f9fad0 2024-05-14T16:59:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.726+0000 7fe762031740 1 -- 172.21.15.69:0/532250285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 msgr2=0x55d2c7fb7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.726+0000 7fe762031740 1 --2- 172.21.15.69:0/532250285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fb7bb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe7500098a0 tx=0x7fe75000b000 comp rx=0 tx=0).stop 2024-05-14T16:59:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.726+0000 7fe762031740 1 -- 172.21.15.69:0/532250285 shutdown_connections 2024-05-14T16:59:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.726+0000 7fe762031740 1 --2- 172.21.15.69:0/532250285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fb7bb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.726+0000 7fe762031740 1 --2- 172.21.15.69:0/532250285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fb7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.726+0000 7fe762031740 1 --2- 172.21.15.69:0/532250285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f948f0 0x55d2c7fab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.726+0000 7fe762031740 1 -- 172.21.15.69:0/532250285 >> 172.21.15.69:0/532250285 conn(0x55d2c7f90e00 msgr2=0x55d2c7f90100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.726+0000 7fe762031740 1 -- 172.21.15.69:0/532250285 shutdown_connections 2024-05-14T16:59:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.726+0000 7fe762031740 1 -- 172.21.15.69:0/532250285 wait complete. 2024-05-14T16:59:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.727+0000 7fe762031740 1 -- start start 2024-05-14T16:59:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.727+0000 7fe762031740 1 Processor -- start 2024-05-14T16:59:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe762031740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f948f0 0x55d2c7fa7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe762031740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fa80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe762031740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fa5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe762031740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe74400ce30 con 0x55d2c7f9fad0 2024-05-14T16:59:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe762031740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe74400e2a0 con 0x55d2c7f95210 2024-05-14T16:59:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe762031740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe74400d500 con 0x55d2c7f948f0 2024-05-14T16:59:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe762031740 10 client.0 _ll_register_callbacks cb 0x7fe74800d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe757fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fa80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe75d7dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fa5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe757fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fa80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:41526/0 (socket says 172.21.15.69:41526) 2024-05-14T16:59:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe757fff640 1 -- 172.21.15.69:0/3237006806 learned_addr learned my addr 172.21.15.69:0/3237006806 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe75cfdb640 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f948f0 0x55d2c7fa7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe757fff640 1 -- 172.21.15.69:0/3237006806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f948f0 msgr2=0x55d2c7fa7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe757fff640 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f948f0 0x55d2c7fa7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe757fff640 1 -- 172.21.15.69:0/3237006806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 msgr2=0x55d2c7fa5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe757fff640 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fa5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe757fff640 1 -- 172.21.15.69:0/3237006806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c7fa5910 con 0x55d2c7f95210 2024-05-14T16:59:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.728+0000 7fe75d7dc640 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fa5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.729+0000 7fe757fff640 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fa80c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7480b24c0 tx=0x7fe748009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bc44bc7f989b86 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.729+0000 7fe737fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:59:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.729+0000 7fe737fff640 1 -- 172.21.15.69:0/3237006806 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480c2070 con 0x55d2c7f95210 2024-05-14T16:59:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.729+0000 7fe737fff640 1 -- 172.21.15.69:0/3237006806 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fe7480af2e0 con 0x55d2c7f95210 2024-05-14T16:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.729+0000 7fe762031740 1 -- 172.21.15.69:0/3237006806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7500118e0 con 0x55d2c7f95210 2024-05-14T16:59:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.729+0000 7fe737fff640 1 -- 172.21.15.69:0/3237006806 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe7480a9b90 con 0x55d2c7f95210 2024-05-14T16:59:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.729+0000 7fe762031740 10 client.25829 fetch_fsmap learned FSMap version 42 2024-05-14T16:59:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.729+0000 7fe762031740 1 -- 172.21.15.69:0/3237006806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2c8022660 con 0x55d2c7f95210 2024-05-14T16:59:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.730+0000 7fe737fff640 1 -- 172.21.15.69:0/3237006806 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7480b29f0 con 0x55d2c7f95210 2024-05-14T16:59:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.730+0000 7fe737fff640 1 -- 172.21.15.69:0/3237006806 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7fe7480a6740 con 0x55d2c7f95210 2024-05-14T16:59:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.730+0000 7fe762031740 10 client.25829 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T16:59:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.730+0000 7fe762031740 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.730+0000 7fe762031740 20 client.25829 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.730+0000 7fe762031740 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d2c7faf980 0x55d2c8024bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.730+0000 7fe762031740 4 client.25829 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:59:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.730+0000 7fe762031740 1 -- 172.21.15.69:0/3237006806 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe7500118e0 con 0x55d2c7faf980 2024-05-14T16:59:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.730+0000 7fe75cfdb640 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d2c7faf980 0x55d2c8024bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.731+0000 7fe75cfdb640 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d2c7faf980 0x55d2c8024bb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5356bf6d7b09fbf server_cookie=b3d0d19e55a378f4 in_seq=0 out_seq=0 2024-05-14T16:59:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.731+0000 7fe737fff640 10 client.25829 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.742+0000 7fe737fff640 1 -- 172.21.15.69:0/3237006806 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fe7500118e0 con 0x55d2c7faf980 2024-05-14T16:59:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.742+0000 7fe737fff640 10 client.25829 handle_command_reply: tid=0 2024-05-14T16:59:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.742+0000 7fe762031740 1 client.25829 shutdown 2024-05-14T16:59:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.742+0000 7fe762031740 1 -- 172.21.15.69:0/3237006806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 msgr2=0x55d2c7fa80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.742+0000 7fe762031740 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fa80c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7480b24c0 tx=0x7fe748009310 comp rx=0 tx=0).stop 2024-05-14T16:59:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.743+0000 7fe762031740 1 -- 172.21.15.69:0/3237006806 shutdown_connections 2024-05-14T16:59:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.743+0000 7fe762031740 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d2c7faf980 0x55d2c8024bb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.743+0000 7fe762031740 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2c7f9fad0 0x55d2c7fa5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.743+0000 7fe737fff640 0 client.25829 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.743+0000 7fe762031740 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d2c7f95210 0x55d2c7fa80c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.743+0000 7fe762031740 1 --2- 172.21.15.69:0/3237006806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c7f948f0 0x55d2c7fa7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.743+0000 7fe762031740 1 -- 172.21.15.69:0/3237006806 >> 172.21.15.69:0/3237006806 conn(0x55d2c7f90e00 msgr2=0x55d2c7faa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.744+0000 7fe762031740 1 -- 172.21.15.69:0/3237006806 shutdown_connections 2024-05-14T16:59:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.744+0000 7fe762031740 1 -- 172.21.15.69:0/3237006806 wait complete. 2024-05-14T16:59:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.744+0000 7fe762031740 20 client.25829 trim_cache size 0 max 16384 2024-05-14T16:59:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:59:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:59:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:59:00.572 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:58:59 smithi092 ceph-mon[35763]: pgmap v283: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.6 MiB/s wr, 579 op/s 2024-05-14T16:59:00.578 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:58:59 smithi203 ceph-mon[35787]: pgmap v283: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.6 MiB/s wr, 579 op/s 2024-05-14T16:59:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 -- 172.21.15.69:0/2375261868 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe7400b2a90 msgr2=0x7fe7400b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe7400b2a90 0x7fe7400b4f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55d2c7f92220 tx=0x7fe750009740 comp rx=0 tx=0).stop 2024-05-14T16:59:00.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 -- 172.21.15.69:0/2375261868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758104620 msgr2=0x7fe75819b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758104620 0x7fe75819b960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe74800e9f0 tx=0x7fe74800eeb0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 -- 172.21.15.69:0/2375261868 shutdown_connections 2024-05-14T16:59:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe7400b2a90 0x7fe7400b4f50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe75810d040 0x7fe75819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758104620 0x7fe75819b960 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 --2- 172.21.15.69:0/2375261868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758103d00 0x7fe75819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 -- 172.21.15.69:0/2375261868 >> 172.21.15.69:0/2375261868 conn(0x7fe758078ad0 msgr2=0x7fe758104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 -- 172.21.15.69:0/2375261868 shutdown_connections 2024-05-14T16:59:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:58:58.745+0000 7fe75f23f640 1 -- 172.21.15.69:0/2375261868 wait complete. 2024-05-14T16:59:00.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:58:59 smithi069 ceph-mon[31507]: pgmap v283: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.6 MiB/s wr, 579 op/s 2024-05-14T16:59:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 510.0 s, 2844.6 tps, lat 35.050 ms stddev 35.615 2024-05-14T16:59:01.377 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.4 2024-05-14T16:59:01.394 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:02.185 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:01 smithi069 ceph-mon[31507]: pgmap v284: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-05-14T16:59:02.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:01 smithi203 ceph-mon[35787]: pgmap v284: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-05-14T16:59:02.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:01 smithi092 ceph-mon[35763]: pgmap v284: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-05-14T16:59:04.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:03 smithi069 ceph-mon[31507]: pgmap v285: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 824 op/s 2024-05-14T16:59:04.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:03 smithi203 ceph-mon[35787]: pgmap v285: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 824 op/s 2024-05-14T16:59:04.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:03 smithi092 ceph-mon[35763]: pgmap v285: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 824 op/s 2024-05-14T16:59:04.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.364+0000 7f9211edc700 1 -- 172.21.15.69:0/1840942161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f920c10f2d0 msgr2=0x7f920c116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:04.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.364+0000 7f9211edc700 1 --2- 172.21.15.69:0/1840942161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f920c10f2d0 0x7f920c116ea0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f920400b0a0 tx=0x7f920400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:59:04.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.364+0000 7f9211edc700 1 -- 172.21.15.69:0/1840942161 shutdown_connections 2024-05-14T16:59:04.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.364+0000 7f9211edc700 1 --2- 172.21.15.69:0/1840942161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f920c10f2d0 0x7f920c116ea0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:04.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.364+0000 7f9211edc700 1 --2- 172.21.15.69:0/1840942161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c10a7a0 0x7f920c10ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:04.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.364+0000 7f9211edc700 1 --2- 172.21.15.69:0/1840942161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f920c10b150 0x7f920c10ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:04.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.364+0000 7f9211edc700 1 -- 172.21.15.69:0/1840942161 >> 172.21.15.69:0/1840942161 conn(0x7f920c06e6b0 msgr2=0x7f920c06eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:04.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.364+0000 7f9211edc700 1 -- 172.21.15.69:0/1840942161 shutdown_connections 2024-05-14T16:59:04.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.365+0000 7f9211edc700 1 -- 172.21.15.69:0/1840942161 wait complete. 2024-05-14T16:59:04.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.366+0000 7f9211edc700 1 Processor -- start 2024-05-14T16:59:04.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.366+0000 7f9211edc700 1 -- start start 2024-05-14T16:59:04.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.366+0000 7f9211edc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f920c10a7a0 0x7f920c1a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:04.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.366+0000 7f9211edc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f920c10b150 0x7f920c1a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:04.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.366+0000 7f9211edc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c10f2d0 0x7f920c1a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:04.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.366+0000 7f9211edc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f920c11a5c0 con 0x7f920c10b150 2024-05-14T16:59:04.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.366+0000 7f9211edc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f920c11a440 con 0x7f920c10f2d0 2024-05-14T16:59:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.366+0000 7f9211edc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f920c11a740 con 0x7f920c10a7a0 2024-05-14T16:59:04.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.367+0000 7f920b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f920c10a7a0 0x7f920c1a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:04.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.367+0000 7f920b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f920c10a7a0 0x7f920c1a2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:44114/0 (socket says 172.21.15.69:44114) 2024-05-14T16:59:04.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.367+0000 7f920b7fe700 1 -- 172.21.15.69:0/271337184 learned_addr learned my addr 172.21.15.69:0/271337184 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:04.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.367+0000 7f9202ffd700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f920c10b150 0x7f920c1a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:04.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.367+0000 7f920b7fe700 1 -- 172.21.15.69:0/271337184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c10f2d0 msgr2=0x7f920c1a6d30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T16:59:04.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.367+0000 7f920b7fe700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c10f2d0 0x7f920c1a6d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:04.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.367+0000 7f920b7fe700 1 -- 172.21.15.69:0/271337184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f920c10b150 msgr2=0x7f920c1a29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:04.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.367+0000 7f920b7fe700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f920c10b150 0x7f920c1a29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.367+0000 7f920b7fe700 1 -- 172.21.15.69:0/271337184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c1a7470 con 0x7f920c10a7a0 2024-05-14T16:59:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.368+0000 7f920b7fe700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f920c10a7a0 0x7f920c1a2460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00b550 tx=0x7f91fc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.368+0000 7f92097fa700 1 -- 172.21.15.69:0/271337184 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc015020 con 0x7f920c10a7a0 2024-05-14T16:59:04.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.369+0000 7f92097fa700 1 -- 172.21.15.69:0/271337184 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91fc011ce0 con 0x7f920c10a7a0 2024-05-14T16:59:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.369+0000 7f9211edc700 1 -- 172.21.15.69:0/271337184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920c1a7760 con 0x7f920c10a7a0 2024-05-14T16:59:04.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.369+0000 7f9211edc700 1 -- 172.21.15.69:0/271337184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920c1aefe0 con 0x7f920c10a7a0 2024-05-14T16:59:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.369+0000 7f92097fa700 1 -- 172.21.15.69:0/271337184 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc010d20 con 0x7f920c10a7a0 2024-05-14T16:59:04.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.371+0000 7f9211edc700 1 -- 172.21.15.69:0/271337184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ec005330 con 0x7f920c10a7a0 2024-05-14T16:59:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.372+0000 7f92097fa700 1 -- 172.21.15.69:0/271337184 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f91fc018020 con 0x7f920c10a7a0 2024-05-14T16:59:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.374+0000 7f92097fa700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f91f80b2b40 0x7f91f80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:04.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.374+0000 7f9202ffd700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f91f80b2b40 0x7f91f80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:04.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.374+0000 7f92097fa700 1 -- 172.21.15.69:0/271337184 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f91fc0e1b90 con 0x7f920c10a7a0 2024-05-14T16:59:04.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.375+0000 7f9202ffd700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f91f80b2b40 0x7f91f80b5000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91f4005950 tx=0x7f91f40058e0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:04.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.379+0000 7f92097fa700 1 -- 172.21.15.69:0/271337184 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91fc01f020 con 0x7f920c10a7a0 2024-05-14T16:59:04.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.807+0000 7f9211edc700 1 -- 172.21.15.69:0/271337184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91ec0058f0 con 0x7f920c10a7a0 2024-05-14T16:59:04.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.808+0000 7f92097fa700 1 -- 172.21.15.69:0/271337184 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11863 (secure 0 0 0) 0x7f91fc0a9670 con 0x7f920c10a7a0 2024-05-14T16:59:04.911 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:59:04.919 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:59:03.618664+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:59:04.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 -- 172.21.15.69:0/271337184 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f91f80b2b40 msgr2=0x7f91f80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:04.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f91f80b2b40 0x7f91f80b5000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91f4005950 tx=0x7f91f40058e0 comp rx=0 tx=0).stop 2024-05-14T16:59:04.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 -- 172.21.15.69:0/271337184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f920c10a7a0 msgr2=0x7f920c1a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:04.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f920c10a7a0 0x7f920c1a2460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00b550 tx=0x7f91fc00b910 comp rx=0 tx=0).stop 2024-05-14T16:59:04.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 -- 172.21.15.69:0/271337184 shutdown_connections 2024-05-14T16:59:04.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c10f2d0 0x7f920c1a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:05.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f920c10a7a0 0x7f920c1a2460 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:05.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f91f80b2b40 0x7f91f80b5000 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:05.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 --2- 172.21.15.69:0/271337184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f920c10b150 0x7f920c1a29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:05.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 -- 172.21.15.69:0/271337184 >> 172.21.15.69:0/271337184 conn(0x7f920c06e6b0 msgr2=0x7f920c0731b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:05.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 -- 172.21.15.69:0/271337184 shutdown_connections 2024-05-14T16:59:05.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:04.812+0000 7f9211edc700 1 -- 172.21.15.69:0/271337184 wait complete. 2024-05-14T16:59:05.076 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 43 2024-05-14T16:59:05.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:04 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:59:05.214 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:04 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/271337184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:05.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:04 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:59:05.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:04 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/271337184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:05.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:04 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:59:05.296 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:04 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/271337184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:05.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 515.0 s, 1579.6 tps, lat 63.520 ms stddev 131.042 2024-05-14T16:59:06.676 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:05 smithi092 ceph-mon[35763]: pgmap v286: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 588 op/s 2024-05-14T16:59:06.907 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:05 smithi203 ceph-mon[35787]: pgmap v286: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 588 op/s 2024-05-14T16:59:06.917 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0' 2024-05-14T16:59:06.922 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0 2024-05-14T16:59:06.928 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:05 smithi069 ceph-mon[31507]: pgmap v286: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 588 op/s 2024-05-14T16:59:07.300 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:08.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:07 smithi069 ceph-mon[31507]: pgmap v287: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 30 MiB/s wr, 2.07k op/s 2024-05-14T16:59:08.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:07 smithi203 ceph-mon[35787]: pgmap v287: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 30 MiB/s wr, 2.07k op/s 2024-05-14T16:59:08.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:07 smithi092 ceph-mon[35763]: pgmap v287: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 30 MiB/s wr, 2.07k op/s 2024-05-14T16:59:09.138 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:08 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:09.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:08 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:09.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:08 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:10.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.101+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/55229776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cf0101b30 msgr2=0x7f0cf0101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:10.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.101+0000 7f0cf5a4c700 1 --2- 172.21.15.69:0/55229776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cf0101b30 0x7f0cf0101fb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009990 tx=0x7f0ce4009ca0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.126 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:09 smithi069 ceph-mon[31507]: pgmap v288: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 1.78k op/s 2024-05-14T16:59:10.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:09 smithi069 ceph-mon[31507]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T16:59:10.136 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:09 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:10.143 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:09 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:10.268 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:09 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:59:10.277 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:09 smithi092 ceph-mon[35763]: pgmap v288: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 1.78k op/s 2024-05-14T16:59:10.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:09 smithi092 ceph-mon[35763]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T16:59:10.288 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:09 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:10.294 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:09 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:10.300 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:09 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:59:10.357 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:09 smithi203 ceph-mon[35787]: pgmap v288: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 1.78k op/s 2024-05-14T16:59:10.363 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:09 smithi203 ceph-mon[35787]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T16:59:10.370 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:09 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:10.375 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:09 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:10.383 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:09 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T16:59:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.104+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/55229776 shutdown_connections 2024-05-14T16:59:10.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.104+0000 7f0cf5a4c700 1 --2- 172.21.15.69:0/55229776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cf01024f0 0x7f0cf01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.105+0000 7f0cf5a4c700 1 --2- 172.21.15.69:0/55229776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf01095b0 0x7f0cf0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.105+0000 7f0cf5a4c700 1 --2- 172.21.15.69:0/55229776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cf0101b30 0x7f0cf0101fb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.105+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/55229776 >> 172.21.15.69:0/55229776 conn(0x7f0cf0078ae0 msgr2=0x7f0cf00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:10.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.107+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/55229776 shutdown_connections 2024-05-14T16:59:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.110+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/55229776 wait complete. 2024-05-14T16:59:10.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.110+0000 7f0cf5a4c700 1 Processor -- start 2024-05-14T16:59:10.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.114+0000 7f0cf5a4c700 1 -- start start 2024-05-14T16:59:10.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.114+0000 7f0cf5a4c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf0101b30 0x7f0cf010f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:10.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.114+0000 7f0cf5a4c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cf01024f0 0x7f0cf010fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:10.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.114+0000 7f0cf5a4c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cf01095b0 0x7f0cf01a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:10.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0cf5a4c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0113720 con 0x7f0cf01095b0 2024-05-14T16:59:10.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0cf5a4c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf01135a0 con 0x7f0cf01024f0 2024-05-14T16:59:10.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0cf5a4c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf01138a0 con 0x7f0cf0101b30 2024-05-14T16:59:10.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0ceeffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf0101b30 0x7f0cf010f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:10.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0ceeffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf0101b30 0x7f0cf010f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:35788/0 (socket says 172.21.15.69:35788) 2024-05-14T16:59:10.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0ceeffd700 1 -- 172.21.15.69:0/250994563 learned_addr learned my addr 172.21.15.69:0/250994563 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:10.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cf01095b0 0x7f0cf01a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:10.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0ceeffd700 1 -- 172.21.15.69:0/250994563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cf01024f0 msgr2=0x7f0cf010fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:10.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cf01024f0 0x7f0cf010fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0ceeffd700 1 -- 172.21.15.69:0/250994563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cf01095b0 msgr2=0x7f0cf01a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:10.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cf01095b0 0x7f0cf01a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0ceeffd700 1 -- 172.21.15.69:0/250994563 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf01a50b0 con 0x7f0cf0101b30 2024-05-14T16:59:10.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.115+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf0101b30 0x7f0cf010f600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000e9c0 tx=0x7f0ce000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:10.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.116+0000 7f0cf4a4a700 1 -- 172.21.15.69:0/250994563 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000cbd0 con 0x7f0cf0101b30 2024-05-14T16:59:10.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.116+0000 7f0cf4a4a700 1 -- 172.21.15.69:0/250994563 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce000cd70 con 0x7f0cf0101b30 2024-05-14T16:59:10.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.116+0000 7f0cf4a4a700 1 -- 172.21.15.69:0/250994563 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0010430 con 0x7f0cf0101b30 2024-05-14T16:59:10.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.117+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/250994563 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf01a53a0 con 0x7f0cf0101b30 2024-05-14T16:59:10.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.118+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/250994563 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf01acc30 con 0x7f0cf0101b30 2024-05-14T16:59:10.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.124+0000 7f0d002e2b80 1 Processor -- start 2024-05-14T16:59:10.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.125+0000 7f0d002e2b80 1 -- start start 2024-05-14T16:59:10.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.125+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bd4b0 0x5567f55bd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:10.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.125+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:10.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.125+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 0x5567f55b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.125+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5567f5516f00 con 0x5567f55b9350 2024-05-14T16:59:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.125+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5567f5506aa0 con 0x5567f55bddd0 2024-05-14T16:59:10.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.125+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5567f550aae0 con 0x5567f55bd4b0 2024-05-14T16:59:10.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.127+0000 7f0cef7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bd4b0 0x5567f55bd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:10.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.127+0000 7f0cef7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bd4b0 0x5567f55bd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:35802/0 (socket says 172.21.15.69:35802) 2024-05-14T16:59:10.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.127+0000 7f0cef7fe700 1 -- 172.21.15.69:0/309170347 learned_addr learned my addr 172.21.15.69:0/309170347 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:10.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.130+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/309170347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:10.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.130+0000 7f0cf4a4a700 1 -- 172.21.15.69:0/250994563 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0ce0004750 con 0x7f0cf0101b30 2024-05-14T16:59:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.131+0000 7f0cef7fe700 1 -- 172.21.15.69:0/309170347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 msgr2=0x5567f55b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:10.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.134+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/309170347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 0x5567f55b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:10.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.135+0000 7f0cf4a4a700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0cd80b2c60 0x7f0cd80b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:10.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.135+0000 7f0cf4a4a700 1 -- 172.21.15.69:0/250994563 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f0ce00e29a0 con 0x7f0cf0101b30 2024-05-14T16:59:10.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.137+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/309170347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.137+0000 7f0cef7fe700 1 -- 172.21.15.69:0/309170347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 msgr2=0x5567f55b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.137+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/309170347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 0x5567f55b97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.137+0000 7f0cef7fe700 1 -- 172.21.15.69:0/309170347 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f54b0140 con 0x5567f55bd4b0 2024-05-14T16:59:10.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.137+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/309170347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 0x5567f55b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:10.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.137+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/309170347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bd4b0 0x5567f55bd890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009430 tx=0x7f0ce8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1723992be7da0b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:10.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.141+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0cd80b2c60 0x7f0cd80b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:10.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.141+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0cd80b2c60 0x7f0cd80b5120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8019000 tx=0x7f0ce80144a0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:10.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.144+0000 7f0cd5ffb700 1 -- 172.21.15.69:0/309170347 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8012570 con 0x5567f55bd4b0 2024-05-14T16:59:10.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.144+0000 7f0cd5ffb700 1 -- 172.21.15.69:0/309170347 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce8012bc0 con 0x5567f55bd4b0 2024-05-14T16:59:10.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.144+0000 7f0cd5ffb700 1 -- 172.21.15.69:0/309170347 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8015ce0 con 0x5567f55bd4b0 2024-05-14T16:59:10.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.146+0000 7f0d002e2b80 1 -- 172.21.15.69:0/309170347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bd4b0 msgr2=0x5567f55bd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:10.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.146+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/309170347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bd4b0 0x5567f55bd890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009430 tx=0x7f0ce8009920 comp rx=0 tx=0).stop 2024-05-14T16:59:10.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.148+0000 7f0d002e2b80 1 -- 172.21.15.69:0/309170347 shutdown_connections 2024-05-14T16:59:10.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.148+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/309170347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 0x5567f55b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.148+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/309170347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bd4b0 0x5567f55bd890 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.148+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/309170347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.148+0000 7f0d002e2b80 1 -- 172.21.15.69:0/309170347 >> 172.21.15.69:0/309170347 conn(0x5567f54e5f40 msgr2=0x5567f55b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:11.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.148+0000 7f0d002e2b80 1 -- 172.21.15.69:0/309170347 shutdown_connections 2024-05-14T16:59:11.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.148+0000 7f0d002e2b80 1 -- 172.21.15.69:0/309170347 wait complete. 2024-05-14T16:59:11.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.149+0000 7f0d002e2b80 1 -- start start 2024-05-14T16:59:11.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.149+0000 7f0d002e2b80 1 Processor -- start 2024-05-14T16:59:11.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.149+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 0x5567f55c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:11.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.149+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bd4b0 0x5567f55c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:11.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.149+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bddd0 0x5567f55c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:11.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.149+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5567f540d130 con 0x5567f55b9350 2024-05-14T16:59:11.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.149+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5567f546ef30 con 0x5567f55bd4b0 2024-05-14T16:59:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.149+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5567f55d6920 con 0x5567f55bddd0 2024-05-14T16:59:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.150+0000 7f0d002e2b80 10 client.0 _ll_register_callbacks cb 0x5567f54a1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:11.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.150+0000 7f0cef7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bddd0 0x5567f55c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:11.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.150+0000 7f0cef7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bddd0 0x5567f55c2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:35804/0 (socket says 172.21.15.69:35804) 2024-05-14T16:59:11.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.150+0000 7f0cef7fe700 1 -- 172.21.15.69:0/3989245788 learned_addr learned my addr 172.21.15.69:0/3989245788 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:11.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.150+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bd4b0 0x5567f55c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.151+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 0x5567f55c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:11.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.151+0000 7f0cef7fe700 1 -- 172.21.15.69:0/3989245788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bd4b0 msgr2=0x5567f55c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:11.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.151+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bd4b0 0x5567f55c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.151+0000 7f0cef7fe700 1 -- 172.21.15.69:0/3989245788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 msgr2=0x5567f55c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:11.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.151+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 0x5567f55c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.151+0000 7f0cef7fe700 1 -- 172.21.15.69:0/3989245788 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567f5456a60 con 0x5567f55bddd0 2024-05-14T16:59:11.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.151+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bd4b0 0x5567f55c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:11.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.151+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 0x5567f55c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:11.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.151+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bddd0 0x5567f55c2460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80090b0 tx=0x7f0ce8015f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aac1136628c8c53 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.151+0000 7f0cceffd700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:59:11.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.152+0000 7f0cceffd700 1 -- 172.21.15.69:0/3989245788 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8032030 con 0x5567f55bddd0 2024-05-14T16:59:11.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.152+0000 7f0cceffd700 1 -- 172.21.15.69:0/3989245788 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f0ce801b030 con 0x5567f55bddd0 2024-05-14T16:59:11.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.152+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3989245788 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce8012bc0 con 0x5567f55bddd0 2024-05-14T16:59:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.152+0000 7f0cceffd700 1 -- 172.21.15.69:0/3989245788 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0ce803c050 con 0x5567f55bddd0 2024-05-14T16:59:11.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.152+0000 7f0d002e2b80 10 client.25762 fetch_fsmap learned FSMap version 43 2024-05-14T16:59:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.152+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3989245788 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5567f54c08f0 con 0x5567f55bddd0 2024-05-14T16:59:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.153+0000 7f0cceffd700 1 -- 172.21.15.69:0/3989245788 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce8012bc0 con 0x5567f55bddd0 2024-05-14T16:59:11.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.153+0000 7f0cceffd700 1 -- 172.21.15.69:0/3989245788 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f0ce80158e0 con 0x5567f55bddd0 2024-05-14T16:59:11.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.153+0000 7f0d002e2b80 10 client.25762 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T16:59:11.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.153+0000 7f0d002e2b80 10 client.25762 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:59:11.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.154+0000 7f0d002e2b80 20 client.25762 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:11.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.154+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5567f55e3320 0x5567f55e5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:11.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.154+0000 7f0d002e2b80 4 client.25762 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:11.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.154+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3989245788 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567f5518e50 con 0x5567f55e3320 2024-05-14T16:59:11.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.154+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5567f55e3320 0x5567f55e5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:11.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.155+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5567f55e3320 0x5567f55e5710 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9a2a9196590c012 server_cookie=4be305541b5adab8 in_seq=0 out_seq=0 2024-05-14T16:59:11.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.155+0000 7f0cceffd700 10 client.25762 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:11.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.156+0000 7f0cceffd700 1 -- 172.21.15.69:0/3989245788 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5567f5518e50 con 0x5567f55e3320 2024-05-14T16:59:11.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.156+0000 7f0cceffd700 10 client.25762 handle_command_reply: tid=0 2024-05-14T16:59:11.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.156+0000 7f0d002e2b80 1 client.25762 shutdown 2024-05-14T16:59:11.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.156+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3989245788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bddd0 msgr2=0x5567f55c2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:11.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.156+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bddd0 0x5567f55c2460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80090b0 tx=0x7f0ce8015f40 comp rx=0 tx=0).stop 2024-05-14T16:59:11.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.157+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3989245788 shutdown_connections 2024-05-14T16:59:11.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.157+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5567f55e3320 0x5567f55e5710 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.157+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55b9350 0x5567f55c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.157+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55bddd0 0x5567f55c2460 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.157+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/3989245788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bd4b0 0x5567f55c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.157+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3989245788 >> 172.21.15.69:0/3989245788 conn(0x5567f54e5f40 msgr2=0x5567f55ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.157+0000 7f0cceffd700 0 client.25762 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:11.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.157+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3989245788 shutdown_connections 2024-05-14T16:59:11.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.157+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3989245788 wait complete. 2024-05-14T16:59:11.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.157+0000 7f0d002e2b80 20 client.25762 trim_cache size 0 max 16384 2024-05-14T16:59:11.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0d002e2b80 1 Processor -- start 2024-05-14T16:59:11.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0d002e2b80 1 -- start start 2024-05-14T16:59:11.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 0x5567f55df830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:11.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:11.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55ae160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:11.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf01ae5b0 con 0x5567f55bd4b0 2024-05-14T16:59:11.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0074710 con 0x5567f55bddd0 2024-05-14T16:59:11.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0072580 con 0x5567f55b9350 2024-05-14T16:59:11.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0cee7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:11.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0cee7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55b85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49082/0 (socket says 172.21.15.69:49082) 2024-05-14T16:59:11.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0cee7fc700 1 -- 172.21.15.69:0/3533170395 learned_addr learned my addr 172.21.15.69:0/3533170395 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:11.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/3533170395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:11.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/3533170395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 0x5567f55df830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:11.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0cee7fc700 1 -- 172.21.15.69:0/3533170395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 msgr2=0x5567f55df830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:11.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/3533170395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 0x5567f55df830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0cee7fc700 1 -- 172.21.15.69:0/3533170395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 msgr2=0x5567f55ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:11.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.180+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/3533170395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.181+0000 7f0cee7fc700 1 -- 172.21.15.69:0/3533170395 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f55ae6a0 con 0x5567f55bd4b0 2024-05-14T16:59:11.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.181+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/3533170395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:11.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.181+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/3533170395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55b85b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009990 tx=0x7f0ce40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3853d14b2c56ab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:11.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.181+0000 7f0cd57fa700 1 -- 172.21.15.69:0/3533170395 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400b920 con 0x5567f55bd4b0 2024-05-14T16:59:11.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.181+0000 7f0cd57fa700 1 -- 172.21.15.69:0/3533170395 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce4027070 con 0x5567f55bd4b0 2024-05-14T16:59:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.181+0000 7f0cd57fa700 1 -- 172.21.15.69:0/3533170395 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4003d20 con 0x5567f55bd4b0 2024-05-14T16:59:11.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.182+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3533170395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 msgr2=0x5567f55b85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:11.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.182+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/3533170395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55b85b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009990 tx=0x7f0ce40096a0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.182+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3533170395 shutdown_connections 2024-05-14T16:59:11.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.182+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/3533170395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.182+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/3533170395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 0x5567f55df830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.182+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/3533170395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55b85b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.182+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3533170395 >> 172.21.15.69:0/3533170395 conn(0x5567f54e5f40 msgr2=0x5567f55c2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:11.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.182+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3533170395 shutdown_connections 2024-05-14T16:59:11.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.182+0000 7f0d002e2b80 1 -- 172.21.15.69:0/3533170395 wait complete. 2024-05-14T16:59:11.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.182+0000 7f0d002e2b80 1 -- start start 2024-05-14T16:59:11.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.183+0000 7f0d002e2b80 1 Processor -- start 2024-05-14T16:59:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.183+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 0x5567f55dbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:11.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.183+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55dc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:11.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.183+0000 7f0d002e2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55d8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:11.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.183+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0072bd0 con 0x5567f55bd4b0 2024-05-14T16:59:11.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.183+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0074040 con 0x5567f55bddd0 2024-05-14T16:59:11.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.183+0000 7f0d002e2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf00732a0 con 0x5567f55b9350 2024-05-14T16:59:11.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0d002e2b80 10 client.0 _ll_register_callbacks cb 0x5567f54a1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:11.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0ceeffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 0x5567f55dbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0ceeffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 0x5567f55dbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:35824/0 (socket says 172.21.15.69:35824) 2024-05-14T16:59:11.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0ceeffd700 1 -- 172.21.15.69:0/2995075180 learned_addr learned my addr 172.21.15.69:0/2995075180 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:11.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55d8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:11.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55dc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:11.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0ceeffd700 1 -- 172.21.15.69:0/2995075180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 msgr2=0x5567f55d8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:11.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55d8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0ceeffd700 1 -- 172.21.15.69:0/2995075180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 msgr2=0x5567f55dc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55dc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0ceeffd700 1 -- 172.21.15.69:0/2995075180 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567f55d9240 con 0x5567f55b9350 2024-05-14T16:59:11.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.184+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55dc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:11.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.185+0000 7f0cef7fe700 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55d8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:11.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.185+0000 7f0ceeffd700 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 0x5567f55dbbb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009fd0 tx=0x7f0ce00e4e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9f84b7db06d5d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:12.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.185+0000 7f0cceffd700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:59:12.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.185+0000 7f0cceffd700 1 -- 172.21.15.69:0/2995075180 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0004e00 con 0x5567f55b9350 2024-05-14T16:59:12.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.185+0000 7f0cceffd700 1 -- 172.21.15.69:0/2995075180 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f0ce0016330 con 0x5567f55b9350 2024-05-14T16:59:12.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.185+0000 7f0d002e2b80 1 -- 172.21.15.69:0/2995075180 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce4027070 con 0x5567f55b9350 2024-05-14T16:59:12.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.186+0000 7f0cceffd700 1 -- 172.21.15.69:0/2995075180 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0ce00b2380 con 0x5567f55b9350 2024-05-14T16:59:12.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.186+0000 7f0d002e2b80 10 client.25768 fetch_fsmap learned FSMap version 43 2024-05-14T16:59:12.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.186+0000 7f0d002e2b80 1 -- 172.21.15.69:0/2995075180 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5567f564e360 con 0x5567f55b9350 2024-05-14T16:59:12.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.186+0000 7f0cceffd700 1 -- 172.21.15.69:0/2995075180 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce4027070 con 0x5567f55b9350 2024-05-14T16:59:12.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.186+0000 7f0cceffd700 1 -- 172.21.15.69:0/2995075180 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f0ce00a9e40 con 0x5567f55b9350 2024-05-14T16:59:12.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.187+0000 7f0d002e2b80 10 client.25768 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T16:59:12.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.187+0000 7f0d002e2b80 10 client.25768 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:59:12.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.187+0000 7f0d002e2b80 20 client.25768 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:12.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.187+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5567f564eab0 0x5567f5650f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.187+0000 7f0d002e2b80 4 client.25768 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}] 2024-05-14T16:59:12.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.187+0000 7f0d002e2b80 1 -- 172.21.15.69:0/2995075180 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}) -- 0x5567f5512630 con 0x5567f564eab0 2024-05-14T16:59:12.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.187+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5567f564eab0 0x5567f5650f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:12.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.187+0000 7f0cee7fc700 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5567f564eab0 0x5567f5650f10 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da6ddd655c34059 server_cookie=b293e413cc78ee26 in_seq=0 out_seq=0 2024-05-14T16:59:12.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.187+0000 7f0cceffd700 10 client.25768 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:12.160 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 520.0 s, 1983.2 tps, lat 48.279 ms stddev 79.613 2024-05-14T16:59:12.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.395+0000 7f0cceffd700 1 -- 172.21.15.69:0/2995075180 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5567f5512630 con 0x5567f564eab0 2024-05-14T16:59:12.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.395+0000 7f0cceffd700 10 client.25768 handle_command_reply: tid=0 2024-05-14T16:59:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.395+0000 7f0d002e2b80 1 client.25768 shutdown 2024-05-14T16:59:12.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 1 -- 172.21.15.69:0/2995075180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 msgr2=0x5567f55dbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:12.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 0x5567f55dbbb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009fd0 tx=0x7f0ce00e4e30 comp rx=0 tx=0).stop 2024-05-14T16:59:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 1 -- 172.21.15.69:0/2995075180 shutdown_connections 2024-05-14T16:59:12.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5567f564eab0 0x5567f5650f10 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0cceffd700 0 client.25768 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:12.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567f55bddd0 0x5567f55d8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5567f55b9350 0x5567f55dbbb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 1 --2- 172.21.15.69:0/2995075180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567f55bd4b0 0x5567f55dc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 1 -- 172.21.15.69:0/2995075180 >> 172.21.15.69:0/2995075180 conn(0x5567f54e5f40 msgr2=0x5567f55bc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 1 -- 172.21.15.69:0/2995075180 shutdown_connections 2024-05-14T16:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 1 -- 172.21.15.69:0/2995075180 wait complete. 2024-05-14T16:59:12.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.396+0000 7f0d002e2b80 20 client.25768 trim_cache size 0 max 16384 2024-05-14T16:59:12.294 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:59:12.301 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 43, 2024-05-14T16:59:12.308 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:59:12.314 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 30, 2024-05-14T16:59:12.321 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:59:12.327 INFO:teuthology.orchestra.run.smithi069.stdout: "e68e8fc8": { 2024-05-14T16:59:12.364 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 30, 2024-05-14T16:59:12.370 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:59:12.376 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:59:12.382 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:59:12.388 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:59:12.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:59:12.418 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:59:12.424 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 140.4, 2024-05-14T16:59:12.431 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:59:12.436 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:59:12.444 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:59:12.489 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:59:12.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T16:59:12.500 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:59:12.508 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:59:12.513 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:59:12.518 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:59:12.542 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:59:12.548 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:59:12.556 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:59:12.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/250994563 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0cd80b2c60 msgr2=0x7f0cd80b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:12.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0cd80b2c60 0x7f0cd80b5120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8019000 tx=0x7f0ce80144a0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/250994563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf0101b30 msgr2=0x7f0cf010f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:12.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf0101b30 0x7f0cf010f600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000e9c0 tx=0x7f0ce000ed80 comp rx=0 tx=0).stop 2024-05-14T16:59:12.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/250994563 shutdown_connections 2024-05-14T16:59:12.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0cd80b2c60 0x7f0cd80b5120 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cf01095b0 0x7f0cf01a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf0101b30 0x7f0cf010f600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 --2- 172.21.15.69:0/250994563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cf01024f0 0x7f0cf010fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/250994563 >> 172.21.15.69:0/250994563 conn(0x7f0cf0078ae0 msgr2=0x7f0cf00ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/250994563 shutdown_connections 2024-05-14T16:59:12.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:10.397+0000 7f0cf5a4c700 1 -- 172.21.15.69:0/250994563 wait complete. 2024-05-14T16:59:12.674 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:11 smithi092 ceph-mon[35763]: pgmap v290: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 34 MiB/s wr, 2.09k op/s 2024-05-14T16:59:12.680 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:11 smithi203 ceph-mon[35787]: pgmap v290: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 34 MiB/s wr, 2.09k op/s 2024-05-14T16:59:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.850+0000 7fe88a859640 1 -- 172.21.15.69:0/3221378419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884102a60 msgr2=0x7fe884102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.850+0000 7fe88a859640 1 --2- 172.21.15.69:0/3221378419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884102a60 0x7fe884102ee0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe878009930 tx=0x7fe87802a420 comp rx=0 tx=0).stop 2024-05-14T16:59:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.850+0000 7fe88a859640 1 -- 172.21.15.69:0/3221378419 shutdown_connections 2024-05-14T16:59:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.850+0000 7fe88a859640 1 --2- 172.21.15.69:0/3221378419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe884103420 0x7fe88410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.850+0000 7fe88a859640 1 --2- 172.21.15.69:0/3221378419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884102a60 0x7fe884102ee0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.850+0000 7fe88a859640 1 --2- 172.21.15.69:0/3221378419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe884108a70 0x7fe884108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.850+0000 7fe88a859640 1 -- 172.21.15.69:0/3221378419 >> 172.21.15.69:0/3221378419 conn(0x7fe884079ee0 msgr2=0x7fe88407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.850+0000 7fe88a859640 1 -- 172.21.15.69:0/3221378419 shutdown_connections 2024-05-14T16:59:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.850+0000 7fe88a859640 1 -- 172.21.15.69:0/3221378419 wait complete. 2024-05-14T16:59:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.851+0000 7fe88a859640 1 Processor -- start 2024-05-14T16:59:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.851+0000 7fe88a859640 1 -- start start 2024-05-14T16:59:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.851+0000 7fe88a859640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe884102a60 0x7fe884199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.851+0000 7fe88a859640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103420 0x7fe884199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.851+0000 7fe88a859640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe884108a70 0x7fe88419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe88a859640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe884112d30 con 0x7fe884103420 2024-05-14T16:59:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe88a859640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe884112bb0 con 0x7fe884108a70 2024-05-14T16:59:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe88a859640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe884112eb0 con 0x7fe884102a60 2024-05-14T16:59:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103420 0x7fe884199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103420 0x7fe884199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49104/0 (socket says 172.21.15.69:49104) 2024-05-14T16:59:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe883fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe884102a60 0x7fe884199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe888df6640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe884108a70 0x7fe88419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8837fe640 1 -- 172.21.15.69:0/2433867518 learned_addr learned my addr 172.21.15.69:0/2433867518 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8837fe640 1 -- 172.21.15.69:0/2433867518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe884102a60 msgr2=0x7fe884199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8837fe640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe884102a60 0x7fe884199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8837fe640 1 -- 172.21.15.69:0/2433867518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe884108a70 msgr2=0x7fe88419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8837fe640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe884108a70 0x7fe88419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8837fe640 1 -- 172.21.15.69:0/2433867518 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88419e200 con 0x7fe884103420 2024-05-14T16:59:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8837fe640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103420 0x7fe884199750 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe878004770 tx=0x7fe8780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8817fa640 1 -- 172.21.15.69:0/2433867518 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87803f070 con 0x7fe884103420 2024-05-14T16:59:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe88a859640 1 -- 172.21.15.69:0/2433867518 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88419e490 con 0x7fe884103420 2024-05-14T16:59:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8817fa640 1 -- 172.21.15.69:0/2433867518 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe87803b460 con 0x7fe884103420 2024-05-14T16:59:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.852+0000 7fe8817fa640 1 -- 172.21.15.69:0/2433867518 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878035070 con 0x7fe884103420 2024-05-14T16:59:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.853+0000 7fe88a859640 1 -- 172.21.15.69:0/2433867518 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8841a5d80 con 0x7fe884103420 2024-05-14T16:59:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.855+0000 7fe8817fa640 1 -- 172.21.15.69:0/2433867518 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe87803b8d0 con 0x7fe884103420 2024-05-14T16:59:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.855+0000 7fe88d574740 1 Processor -- start 2024-05-14T16:59:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.855+0000 7fe88d574740 1 -- start start 2024-05-14T16:59:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.855+0000 7fe88d574740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffbacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.855+0000 7fe88d574740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 0x56255ffc5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.855+0000 7fe88d574740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.855+0000 7fe88d574740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56255ff2dc30 con 0x56255ffc5ad0 2024-05-14T16:59:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.855+0000 7fe88d574740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56255ff1d7d0 con 0x56255ffbb210 2024-05-14T16:59:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.855+0000 7fe88d574740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56255ff21810 con 0x56255ffba8f0 2024-05-14T16:59:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.855+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49110/0 (socket says 172.21.15.69:49110) 2024-05-14T16:59:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe8837fe640 1 -- 172.21.15.69:0/3031906787 learned_addr learned my addr 172.21.15.69:0/3031906787 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe883fff640 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 0x56255ffc5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe888df6640 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffbacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe8817fa640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe86c0b2a10 0x7fe86c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe888df6640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe86c0b2a10 0x7fe86c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe8837fe640 1 -- 172.21.15.69:0/3031906787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 msgr2=0x56255ffbacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe8837fe640 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffbacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe8837fe640 1 -- 172.21.15.69:0/3031906787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 msgr2=0x56255ffc5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe8837fe640 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 0x56255ffc5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe8817fa640 1 -- 172.21.15.69:0/2433867518 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7fe878102920 con 0x7fe884103420 2024-05-14T16:59:13.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe8837fe640 1 -- 172.21.15.69:0/3031906787 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255fc32210 con 0x56255ffc5ad0 2024-05-14T16:59:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe888df6640 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffbacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe883fff640 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 0x56255ffc5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe8837fe640 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc7ec0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe8780c9970 tx=0x7fe87802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc17619ff5ba7a16 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe888df6640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe86c0b2a10 0x7fe86c0b4ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe87c005fd0 tx=0x7fe87c004ab0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe86a7fc640 1 -- 172.21.15.69:0/3031906787 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878048020 con 0x56255ffc5ad0 2024-05-14T16:59:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe86a7fc640 1 -- 172.21.15.69:0/3031906787 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8780c9780 con 0x56255ffc5ad0 2024-05-14T16:59:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.856+0000 7fe86a7fc640 1 -- 172.21.15.69:0/3031906787 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780d2a20 con 0x56255ffc5ad0 2024-05-14T16:59:13.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.857+0000 7fe88d574740 1 -- 172.21.15.69:0/3031906787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 msgr2=0x56255ffc7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.857+0000 7fe88d574740 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc7ec0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe8780c9970 tx=0x7fe87802a420 comp rx=0 tx=0).stop 2024-05-14T16:59:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.857+0000 7fe88d574740 1 -- 172.21.15.69:0/3031906787 shutdown_connections 2024-05-14T16:59:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.857+0000 7fe88d574740 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc7ec0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.857+0000 7fe88d574740 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 0x56255ffc5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.857+0000 7fe88d574740 1 --2- 172.21.15.69:0/3031906787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffbacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.857+0000 7fe88d574740 1 -- 172.21.15.69:0/3031906787 >> 172.21.15.69:0/3031906787 conn(0x56255ffb6e00 msgr2=0x56255ffb71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.857+0000 7fe88d574740 1 -- 172.21.15.69:0/3031906787 shutdown_connections 2024-05-14T16:59:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.857+0000 7fe88d574740 1 -- 172.21.15.69:0/3031906787 wait complete. 2024-05-14T16:59:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe88d574740 1 -- start start 2024-05-14T16:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe88d574740 1 Processor -- start 2024-05-14T16:59:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe88d574740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffc5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe88d574740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 0x56255ffc2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe88d574740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe88d574740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56255fc03790 con 0x56255ffc5ad0 2024-05-14T16:59:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe88d574740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56255fed9890 con 0x56255ffbb210 2024-05-14T16:59:13.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe88d574740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56255ffc84c0 con 0x56255ffba8f0 2024-05-14T16:59:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe88d574740 10 client.0 _ll_register_callbacks cb 0x56255fedcd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe888df6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe888df6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49120/0 (socket says 172.21.15.69:49120) 2024-05-14T16:59:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe888df6640 1 -- 172.21.15.69:0/3012692931 learned_addr learned my addr 172.21.15.69:0/3012692931 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.858+0000 7fe8837fe640 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 0x56255ffc2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe883fff640 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffc5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe888df6640 1 -- 172.21.15.69:0/3012692931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 msgr2=0x56255ffc5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe888df6640 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffc5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe888df6640 1 -- 172.21.15.69:0/3012692931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 msgr2=0x56255ffc2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe888df6640 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 0x56255ffc2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe888df6640 1 -- 172.21.15.69:0/3012692931 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56255fcfe620 con 0x56255ffc5ad0 2024-05-14T16:59:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe883fff640 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffc5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe8837fe640 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 0x56255ffc2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe888df6640 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc2c70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009f70 tx=0x7fe87c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace5bc3589ba8681 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe8637fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:59:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe8637fe640 1 -- 172.21.15.69:0/3012692931 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c024070 con 0x56255ffc5ad0 2024-05-14T16:59:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe8637fe640 1 -- 172.21.15.69:0/3012692931 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7fe87c012a30 con 0x56255ffc5ad0 2024-05-14T16:59:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe88d574740 1 -- 172.21.15.69:0/3012692931 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8780c9780 con 0x56255ffc5ad0 2024-05-14T16:59:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe8637fe640 1 -- 172.21.15.69:0/3012692931 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe87c030050 con 0x56255ffc5ad0 2024-05-14T16:59:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe88d574740 10 client.16401 fetch_fsmap learned FSMap version 43 2024-05-14T16:59:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe88d574740 1 -- 172.21.15.69:0/3012692931 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56255ffd3f90 con 0x56255ffc5ad0 2024-05-14T16:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe8637fe640 1 -- 172.21.15.69:0/3012692931 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8780c9780 con 0x56255ffc5ad0 2024-05-14T16:59:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe8637fe640 1 -- 172.21.15.69:0/3012692931 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7fe87c01c070 con 0x56255ffc5ad0 2024-05-14T16:59:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe88d574740 10 client.16401 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T16:59:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.859+0000 7fe88d574740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.860+0000 7fe88d574740 20 client.16401 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.860+0000 7fe88d574740 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56255ffd54c0 0x56255ffd78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.860+0000 7fe88d574740 4 client.16401 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.860+0000 7fe88d574740 1 -- 172.21.15.69:0/3012692931 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56255ff2fb80 con 0x56255ffd54c0 2024-05-14T16:59:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.860+0000 7fe883fff640 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56255ffd54c0 0x56255ffd78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.860+0000 7fe883fff640 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56255ffd54c0 0x56255ffd78b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47be27a10b6ab96 server_cookie=90b3a3fbc0d9bad1 in_seq=0 out_seq=0 2024-05-14T16:59:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.860+0000 7fe8637fe640 10 client.16401 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe8637fe640 1 -- 172.21.15.69:0/3012692931 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56255ff2fb80 con 0x56255ffd54c0 2024-05-14T16:59:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe8637fe640 10 client.16401 handle_command_reply: tid=0 2024-05-14T16:59:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 client.16401 shutdown 2024-05-14T16:59:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 -- 172.21.15.69:0/3012692931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 msgr2=0x56255ffc2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc2c70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009f70 tx=0x7fe87c004a40 comp rx=0 tx=0).stop 2024-05-14T16:59:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 -- 172.21.15.69:0/3012692931 shutdown_connections 2024-05-14T16:59:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56255ffd54c0 0x56255ffd78b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffc5ad0 0x56255ffc2c70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffbb210 0x56255ffc2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe8637fe640 0 client.16401 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 --2- 172.21.15.69:0/3012692931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffc5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 -- 172.21.15.69:0/3012692931 >> 172.21.15.69:0/3012692931 conn(0x56255ffb6e00 msgr2=0x56255ffb6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 -- 172.21.15.69:0/3012692931 shutdown_connections 2024-05-14T16:59:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.861+0000 7fe88d574740 1 -- 172.21.15.69:0/3012692931 wait complete. 2024-05-14T16:59:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.862+0000 7fe88d574740 20 client.16401 trim_cache size 0 max 16384 2024-05-14T16:59:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe88d574740 1 Processor -- start 2024-05-14T16:59:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe88d574740 1 -- start start 2024-05-14T16:59:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe88d574740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffdd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe88d574740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffbb210 0x56255ffdd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe88d574740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 0x56255ffbf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe88d574740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841b0fe0 con 0x56255ffbb210 2024-05-14T16:59:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe88d574740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841ae880 con 0x56255ffc5ad0 2024-05-14T16:59:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe88d574740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841ac6f0 con 0x56255ffba8f0 2024-05-14T16:59:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffbb210 0x56255ffdd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffbb210 0x56255ffdd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49132/0 (socket says 172.21.15.69:49132) 2024-05-14T16:59:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe8837fe640 1 -- 172.21.15.69:0/1055048986 learned_addr learned my addr 172.21.15.69:0/1055048986 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe888df6640 1 --2- 172.21.15.69:0/1055048986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 0x56255ffbf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe883fff640 1 --2- 172.21.15.69:0/1055048986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffdd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe8837fe640 1 -- 172.21.15.69:0/1055048986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 msgr2=0x56255ffdd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe8837fe640 1 --2- 172.21.15.69:0/1055048986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffdd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe8837fe640 1 -- 172.21.15.69:0/1055048986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 msgr2=0x56255ffbf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe8837fe640 1 --2- 172.21.15.69:0/1055048986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 0x56255ffbf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe8837fe640 1 -- 172.21.15.69:0/1055048986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255ffd7f10 con 0x56255ffbb210 2024-05-14T16:59:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.877+0000 7fe883fff640 1 --2- 172.21.15.69:0/1055048986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffdd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe8837fe640 1 --2- 172.21.15.69:0/1055048986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffbb210 0x56255ffdd6e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe8780c6ee0 tx=0x7fe8780329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800634362e850879 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe869ffb640 1 -- 172.21.15.69:0/1055048986 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878048070 con 0x56255ffbb210 2024-05-14T16:59:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe869ffb640 1 -- 172.21.15.69:0/1055048986 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8780c9780 con 0x56255ffbb210 2024-05-14T16:59:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe869ffb640 1 -- 172.21.15.69:0/1055048986 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780cdf70 con 0x56255ffbb210 2024-05-14T16:59:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe88d574740 1 -- 172.21.15.69:0/1055048986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffbb210 msgr2=0x56255ffdd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe88d574740 1 --2- 172.21.15.69:0/1055048986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffbb210 0x56255ffdd6e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe8780c6ee0 tx=0x7fe8780329f0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe88d574740 1 -- 172.21.15.69:0/1055048986 shutdown_connections 2024-05-14T16:59:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe88d574740 1 --2- 172.21.15.69:0/1055048986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 0x56255ffbf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe88d574740 1 --2- 172.21.15.69:0/1055048986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffbb210 0x56255ffdd6e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe88d574740 1 --2- 172.21.15.69:0/1055048986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffba8f0 0x56255ffdd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe88d574740 1 -- 172.21.15.69:0/1055048986 >> 172.21.15.69:0/1055048986 conn(0x56255ffb6e00 msgr2=0x56255ffd0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe88d574740 1 -- 172.21.15.69:0/1055048986 shutdown_connections 2024-05-14T16:59:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.878+0000 7fe88d574740 1 -- 172.21.15.69:0/1055048986 wait complete. 2024-05-14T16:59:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe88d574740 1 -- start start 2024-05-14T16:59:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe88d574740 1 Processor -- start 2024-05-14T16:59:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe88d574740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffba8f0 0x56255ffcb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe88d574740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffbb210 0x56255ffcb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe88d574740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 0x56255ffcbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe88d574740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841acd40 con 0x56255ffba8f0 2024-05-14T16:59:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe88d574740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841ae1b0 con 0x56255ffc5ad0 2024-05-14T16:59:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe88d574740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841ad410 con 0x56255ffbb210 2024-05-14T16:59:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe88d574740 10 client.0 _ll_register_callbacks cb 0x7fe87c003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe883fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffba8f0 0x56255ffcb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe883fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffba8f0 0x56255ffcb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49134/0 (socket says 172.21.15.69:49134) 2024-05-14T16:59:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe883fff640 1 -- 172.21.15.69:0/901081116 learned_addr learned my addr 172.21.15.69:0/901081116 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe888df6640 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 0x56255ffcbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.879+0000 7fe8837fe640 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffbb210 0x56255ffcb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe883fff640 1 -- 172.21.15.69:0/901081116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffbb210 msgr2=0x56255ffcb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe883fff640 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffbb210 0x56255ffcb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe883fff640 1 -- 172.21.15.69:0/901081116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 msgr2=0x56255ffcbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe883fff640 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 0x56255ffcbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe883fff640 1 -- 172.21.15.69:0/901081116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256003ede0 con 0x56255ffba8f0 2024-05-14T16:59:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe8837fe640 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffbb210 0x56255ffcb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe888df6640 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 0x56255ffcbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe883fff640 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffba8f0 0x56255ffcb3b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe87400b6b0 tx=0x7fe87400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe21023fd9b53d4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe8637fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:59:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe8637fe640 1 -- 172.21.15.69:0/901081116 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874021030 con 0x56255ffba8f0 2024-05-14T16:59:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe8637fe640 1 -- 172.21.15.69:0/901081116 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7fe874005020 con 0x56255ffba8f0 2024-05-14T16:59:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe88d574740 1 -- 172.21.15.69:0/901081116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8780c9780 con 0x56255ffba8f0 2024-05-14T16:59:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe8637fe640 1 -- 172.21.15.69:0/901081116 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe87402b050 con 0x56255ffba8f0 2024-05-14T16:59:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe88d574740 10 client.16407 fetch_fsmap learned FSMap version 43 2024-05-14T16:59:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe88d574740 1 -- 172.21.15.69:0/901081116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5625600400a0 con 0x56255ffba8f0 2024-05-14T16:59:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe8637fe640 1 -- 172.21.15.69:0/901081116 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8780c9780 con 0x56255ffba8f0 2024-05-14T16:59:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe8637fe640 1 -- 172.21.15.69:0/901081116 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7fe8740047a0 con 0x56255ffba8f0 2024-05-14T16:59:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe88d574740 10 client.16407 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T16:59:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe88d574740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe88d574740 20 client.16407 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:14.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe88d574740 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5625600408a0 0x562560042cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.880+0000 7fe88d574740 4 client.16407 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:59:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.881+0000 7fe88d574740 1 -- 172.21.15.69:0/901081116 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56255ff29360 con 0x5625600408a0 2024-05-14T16:59:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.881+0000 7fe8837fe640 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5625600408a0 0x562560042cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.881+0000 7fe8837fe640 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5625600408a0 0x562560042cc0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da594f0cd147f138 server_cookie=53708ccc3696b4cd in_seq=0 out_seq=0 2024-05-14T16:59:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.881+0000 7fe8637fe640 10 client.16407 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe8637fe640 1 -- 172.21.15.69:0/901081116 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56255ff29360 con 0x5625600408a0 2024-05-14T16:59:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe8637fe640 10 client.16407 handle_command_reply: tid=0 2024-05-14T16:59:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe88d574740 1 client.16407 shutdown 2024-05-14T16:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe88d574740 1 -- 172.21.15.69:0/901081116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffba8f0 msgr2=0x56255ffcb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe88d574740 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffba8f0 0x56255ffcb3b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe87400b6b0 tx=0x7fe87400be30 comp rx=0 tx=0).stop 2024-05-14T16:59:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe88d574740 1 -- 172.21.15.69:0/901081116 shutdown_connections 2024-05-14T16:59:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe88d574740 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5625600408a0 0x562560042cc0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe88d574740 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56255ffc5ad0 0x56255ffcbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe8637fe640 0 client.16407 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:14.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe88d574740 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56255ffbb210 0x56255ffcb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe88d574740 1 --2- 172.21.15.69:0/901081116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56255ffba8f0 0x56255ffcb3b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe88d574740 1 -- 172.21.15.69:0/901081116 >> 172.21.15.69:0/901081116 conn(0x56255ffb6e00 msgr2=0x56255ffd0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.899+0000 7fe88d574740 1 -- 172.21.15.69:0/901081116 shutdown_connections 2024-05-14T16:59:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88d574740 1 -- 172.21.15.69:0/901081116 wait complete. 2024-05-14T16:59:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88d574740 20 client.16407 trim_cache size 0 max 16384 2024-05-14T16:59:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:59:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:59:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:59:14.616 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:13 smithi092 ceph-mon[35763]: pgmap v291: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 37 MiB/s wr, 2.08k op/s 2024-05-14T16:59:14.621 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:13 smithi092 ceph-mon[35763]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T16:59:14.628 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:13 smithi203 ceph-mon[35787]: pgmap v291: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 37 MiB/s wr, 2.08k op/s 2024-05-14T16:59:14.634 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:13 smithi203 ceph-mon[35787]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T16:59:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 -- 172.21.15.69:0/2433867518 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe86c0b2a10 msgr2=0x7fe86c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe86c0b2a10 0x7fe86c0b4ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe87c005fd0 tx=0x7fe87c004ab0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 -- 172.21.15.69:0/2433867518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103420 msgr2=0x7fe884199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103420 0x7fe884199750 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe878004770 tx=0x7fe8780042f0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 -- 172.21.15.69:0/2433867518 shutdown_connections 2024-05-14T16:59:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe86c0b2a10 0x7fe86c0b4ed0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe884108a70 0x7fe88419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103420 0x7fe884199750 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 --2- 172.21.15.69:0/2433867518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe884102a60 0x7fe884199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 -- 172.21.15.69:0/2433867518 >> 172.21.15.69:0/2433867518 conn(0x7fe884079ee0 msgr2=0x7fe8841003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 -- 172.21.15.69:0/2433867518 shutdown_connections 2024-05-14T16:59:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:11.900+0000 7fe88a859640 1 -- 172.21.15.69:0/2433867518 wait complete. 2024-05-14T16:59:14.744 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:11 smithi069 ceph-mon[31507]: pgmap v290: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 34 MiB/s wr, 2.09k op/s 2024-05-14T16:59:14.751 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:13 smithi069 ceph-mon[31507]: pgmap v291: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 37 MiB/s wr, 2.08k op/s 2024-05-14T16:59:14.756 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:13 smithi069 ceph-mon[31507]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T16:59:14.762 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e68e8fc8', quiesce took 13.4 seconds. Will release after: 6.999999999999998 2024-05-14T16:59:15.399 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 525.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:59:16.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:15 smithi069 ceph-mon[31507]: pgmap v293: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 381 op/s 2024-05-14T16:59:16.211 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:15 smithi203 ceph-mon[35787]: pgmap v293: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 381 op/s 2024-05-14T16:59:16.252 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:15 smithi092 ceph-mon[35763]: pgmap v293: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 381 op/s 2024-05-14T16:59:18.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:17 smithi069 ceph-mon[31507]: pgmap v294: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2024-05-14T16:59:18.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:17 smithi203 ceph-mon[35787]: pgmap v294: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2024-05-14T16:59:18.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:17 smithi092 ceph-mon[35763]: pgmap v294: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2024-05-14T16:59:20.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:19 smithi069 ceph-mon[31507]: pgmap v295: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 1.13k op/s 2024-05-14T16:59:20.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:19 smithi203 ceph-mon[35787]: pgmap v295: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 1.13k op/s 2024-05-14T16:59:20.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:19 smithi092 ceph-mon[35763]: pgmap v295: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 1.13k op/s 2024-05-14T16:59:20.396 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 530.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:59:21.768 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:22.059 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:22.181 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:21 smithi069 ceph-mon[31507]: pgmap v296: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-05-14T16:59:22.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:21 smithi203 ceph-mon[35787]: pgmap v296: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-05-14T16:59:22.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:21 smithi092 ceph-mon[35763]: pgmap v296: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-05-14T16:59:23.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.678+0000 7f847a93f700 1 -- 172.21.15.69:0/2311973907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f847406bbe0 msgr2=0x7f84741103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:23.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.678+0000 7f847a93f700 1 --2- 172.21.15.69:0/2311973907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f847406bbe0 0x7f84741103d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f846c00b0a0 tx=0x7f846c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.678+0000 7f847a93f700 1 -- 172.21.15.69:0/2311973907 shutdown_connections 2024-05-14T16:59:23.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.678+0000 7f847a93f700 1 --2- 172.21.15.69:0/2311973907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f847406bbe0 0x7f84741103d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.678+0000 7f847a93f700 1 --2- 172.21.15.69:0/2311973907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84741095b0 0x7f8474109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.678+0000 7f847a93f700 1 --2- 172.21.15.69:0/2311973907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f847406b220 0x7f847406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.678+0000 7f847a93f700 1 -- 172.21.15.69:0/2311973907 >> 172.21.15.69:0/2311973907 conn(0x7f84740793f0 msgr2=0x7f8474079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:23.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.678+0000 7f847a93f700 1 -- 172.21.15.69:0/2311973907 shutdown_connections 2024-05-14T16:59:23.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.678+0000 7f847a93f700 1 -- 172.21.15.69:0/2311973907 wait complete. 2024-05-14T16:59:23.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.679+0000 7f847a93f700 1 Processor -- start 2024-05-14T16:59:23.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.679+0000 7f847a93f700 1 -- start start 2024-05-14T16:59:23.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.679+0000 7f847a93f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f847406b220 0x7f8474199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:23.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.679+0000 7f847a93f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f847406bbe0 0x7f847419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.679+0000 7f847a93f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84741095b0 0x7f847419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.679+0000 7f847a93f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84741137c0 con 0x7f847406bbe0 2024-05-14T16:59:23.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.679+0000 7f847a93f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8474113640 con 0x7f84741095b0 2024-05-14T16:59:23.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.679+0000 7f847a93f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8474113940 con 0x7f847406b220 2024-05-14T16:59:23.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84737fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f847406bbe0 0x7f847419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:23.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84737fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f847406bbe0 0x7f847419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37876/0 (socket says 172.21.15.69:37876) 2024-05-14T16:59:23.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84737fe700 1 -- 172.21.15.69:0/3309207520 learned_addr learned my addr 172.21.15.69:0/3309207520 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:23.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f8478edc700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84741095b0 0x7f847419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:23.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f8473fff700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f847406b220 0x7f8474199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:23.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84737fe700 1 -- 172.21.15.69:0/3309207520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f847406b220 msgr2=0x7f8474199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:23.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84737fe700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f847406b220 0x7f8474199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:24.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84737fe700 1 -- 172.21.15.69:0/3309207520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84741095b0 msgr2=0x7f847419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:24.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84737fe700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84741095b0 0x7f847419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:24.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84737fe700 1 -- 172.21.15.69:0/3309207520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847419ed20 con 0x7f847406bbe0 2024-05-14T16:59:24.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f8478edc700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84741095b0 0x7f847419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:24.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84737fe700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f847406bbe0 0x7f847419a250 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f846800b530 tx=0x7f846800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:24.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84717fa700 1 -- 172.21.15.69:0/3309207520 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468015020 con 0x7f847406bbe0 2024-05-14T16:59:24.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84717fa700 1 -- 172.21.15.69:0/3309207520 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8468011ce0 con 0x7f847406bbe0 2024-05-14T16:59:24.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f847a93f700 1 -- 172.21.15.69:0/3309207520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847419f010 con 0x7f847406bbe0 2024-05-14T16:59:24.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f84717fa700 1 -- 172.21.15.69:0/3309207520 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468010d20 con 0x7f847406bbe0 2024-05-14T16:59:24.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.680+0000 7f847a93f700 1 -- 172.21.15.69:0/3309207520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84741a6890 con 0x7f847406bbe0 2024-05-14T16:59:24.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.681+0000 7f847a93f700 1 -- 172.21.15.69:0/3309207520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f847410ba90 con 0x7f847406bbe0 2024-05-14T16:59:24.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.683+0000 7f84717fa700 1 -- 172.21.15.69:0/3309207520 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8468018020 con 0x7f847406bbe0 2024-05-14T16:59:24.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.684+0000 7f84717fa700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f845c0b29f0 0x7f845c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:24.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.684+0000 7f8473fff700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f845c0b29f0 0x7f845c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:24.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.684+0000 7f84717fa700 1 -- 172.21.15.69:0/3309207520 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f84680e2330 con 0x7f847406bbe0 2024-05-14T16:59:24.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.684+0000 7f8473fff700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f845c0b29f0 0x7f845c0b4eb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8464006fd0 tx=0x7f8464008040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:24.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.687+0000 7f84717fa700 1 -- 172.21.15.69:0/3309207520 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f846801f020 con 0x7f847406bbe0 2024-05-14T16:59:24.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.949+0000 7f847a93f700 1 -- 172.21.15.69:0/3309207520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f847406a230 con 0x7f847406bbe0 2024-05-14T16:59:24.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.950+0000 7f84717fa700 1 -- 172.21.15.69:0/3309207520 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11863 (secure 0 0 0) 0x7f84680a9db0 con 0x7f847406bbe0 2024-05-14T16:59:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T16:59:24.226 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T16:59:03.618664+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T16:59:24.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 -- 172.21.15.69:0/3309207520 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f845c0b29f0 msgr2=0x7f845c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f845c0b29f0 0x7f845c0b4eb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8464006fd0 tx=0x7f8464008040 comp rx=0 tx=0).stop 2024-05-14T16:59:24.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 -- 172.21.15.69:0/3309207520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f847406bbe0 msgr2=0x7f847419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:24.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f847406bbe0 0x7f847419a250 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f846800b530 tx=0x7f846800b8f0 comp rx=0 tx=0).stop 2024-05-14T16:59:24.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 -- 172.21.15.69:0/3309207520 shutdown_connections 2024-05-14T16:59:24.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84741095b0 0x7f847419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:24.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f847406b220 0x7f8474199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:24.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f845c0b29f0 0x7f845c0b4eb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:24.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 --2- 172.21.15.69:0/3309207520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f847406bbe0 0x7f847419a250 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:24.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 -- 172.21.15.69:0/3309207520 >> 172.21.15.69:0/3309207520 conn(0x7f84740793f0 msgr2=0x7f8474101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:24.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 -- 172.21.15.69:0/3309207520 shutdown_connections 2024-05-14T16:59:24.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:23.952+0000 7f847a93f700 1 -- 172.21.15.69:0/3309207520 wait complete. 2024-05-14T16:59:24.344 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 43 2024-05-14T16:59:24.350 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:23 smithi069 ceph-mon[31507]: pgmap v297: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 814 op/s 2024-05-14T16:59:24.357 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:23 smithi092 ceph-mon[35763]: pgmap v297: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 814 op/s 2024-05-14T16:59:24.364 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:23 smithi203 ceph-mon[35787]: pgmap v297: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 814 op/s 2024-05-14T16:59:24.860 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id e68e8fc8 --release --await' 2024-05-14T16:59:24.867 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id e68e8fc8 --release --await 2024-05-14T16:59:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.995+0000 7f0a86493640 1 -- 172.21.15.69:0/1838711536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a80102340 msgr2=0x7f0a8010e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.995+0000 7f0a86493640 1 --2- 172.21.15.69:0/1838711536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a80102340 0x7f0a8010e800 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009cd0 tx=0x7f0a7802a8e0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.995+0000 7f0a86493640 1 -- 172.21.15.69:0/1838711536 shutdown_connections 2024-05-14T16:59:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.995+0000 7f0a86493640 1 --2- 172.21.15.69:0/1838711536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a80102340 0x7f0a8010e800 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.995+0000 7f0a86493640 1 --2- 172.21.15.69:0/1838711536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a80101980 0x7f0a80101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.995+0000 7f0a86493640 1 --2- 172.21.15.69:0/1838711536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a80107940 0x7f0a80107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.995+0000 7f0a86493640 1 -- 172.21.15.69:0/1838711536 >> 172.21.15.69:0/1838711536 conn(0x7f0a80078ad0 msgr2=0x7f0a80078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.995+0000 7f0a86493640 1 -- 172.21.15.69:0/1838711536 shutdown_connections 2024-05-14T16:59:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.995+0000 7f0a86493640 1 -- 172.21.15.69:0/1838711536 wait complete. 2024-05-14T16:59:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.996+0000 7f0a86493640 1 Processor -- start 2024-05-14T16:59:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.996+0000 7f0a86493640 1 -- start start 2024-05-14T16:59:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.996+0000 7f0a86493640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a80101980 0x7f0a801991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.996+0000 7f0a86493640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a80102340 0x7f0a80199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.996+0000 7f0a86493640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a80107940 0x7f0a8019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.996+0000 7f0a86493640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a80111c00 con 0x7f0a80107940 2024-05-14T16:59:25.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.996+0000 7f0a86493640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a80111a80 con 0x7f0a80101980 2024-05-14T16:59:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.996+0000 7f0a86493640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0a80111d80 con 0x7f0a80102340 2024-05-14T16:59:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.996+0000 7f0a84a30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a80107940 0x7f0a8019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.996+0000 7f0a84a30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a80107940 0x7f0a8019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39528/0 (socket says 172.21.15.69:39528) 2024-05-14T16:59:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a84a30640 1 -- 172.21.15.69:0/630860220 learned_addr learned my addr 172.21.15.69:0/630860220 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a80101980 0x7f0a801991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a80102340 0x7f0a80199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a84a30640 1 -- 172.21.15.69:0/630860220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a80102340 msgr2=0x7f0a80199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a84a30640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a80102340 0x7f0a80199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a84a30640 1 -- 172.21.15.69:0/630860220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a80101980 msgr2=0x7f0a801991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a84a30640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a80101980 0x7f0a801991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a84a30640 1 -- 172.21.15.69:0/630860220 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8019e1e0 con 0x7f0a80107940 2024-05-14T16:59:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a84a30640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a80107940 0x7f0a8019dab0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005cc0 tx=0x7f0a78003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a7d7fa640 1 -- 172.21.15.69:0/630860220 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78004450 con 0x7f0a80107940 2024-05-14T16:59:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a7d7fa640 1 -- 172.21.15.69:0/630860220 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a78031c90 con 0x7f0a80107940 2024-05-14T16:59:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a7d7fa640 1 -- 172.21.15.69:0/630860220 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7803b400 con 0x7f0a80107940 2024-05-14T16:59:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a86493640 1 -- 172.21.15.69:0/630860220 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8019e470 con 0x7f0a80107940 2024-05-14T16:59:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:24.997+0000 7f0a86493640 1 -- 172.21.15.69:0/630860220 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a801a5d60 con 0x7f0a80107940 2024-05-14T16:59:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7d7fa640 1 -- 172.21.15.69:0/630860220 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0a78004680 con 0x7f0a80107940 2024-05-14T16:59:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a891b1740 1 Processor -- start 2024-05-14T16:59:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a891b1740 1 -- start start 2024-05-14T16:59:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a235acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2365590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 0x5609a2367ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a891b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5609a22cdc30 con 0x5609a235b210 2024-05-14T16:59:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a891b1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5609a22bd7d0 con 0x5609a235a8f0 2024-05-14T16:59:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a891b1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5609a22c1810 con 0x5609a2365ad0 2024-05-14T16:59:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2365590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2365590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39544/0 (socket says 172.21.15.69:39544) 2024-05-14T16:59:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7ffff640 1 -- 172.21.15.69:0/4111505266 learned_addr learned my addr 172.21.15.69:0/4111505266 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/4111505266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a235acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/4111505266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 0x5609a2367ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7ffff640 1 -- 172.21.15.69:0/4111505266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 msgr2=0x5609a2367ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/4111505266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 0x5609a2367ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7ffff640 1 -- 172.21.15.69:0/4111505266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 msgr2=0x5609a235acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/4111505266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a235acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7d7fa640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a680b2a90 0x7f0a680b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7ffff640 1 -- 172.21.15.69:0/4111505266 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a1fd2210 con 0x5609a235b210 2024-05-14T16:59:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.000+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/4111505266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a235acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.001+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/4111505266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2365590 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000bc90 tx=0x7f0a700107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d3a98c72f5b875 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.001+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a680b2a90 0x7f0a680b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.001+0000 7f0a7d7fa640 1 -- 172.21.15.69:0/630860220 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f0a7802e030 con 0x7f0a80107940 2024-05-14T16:59:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.001+0000 7f0a667fc640 1 -- 172.21.15.69:0/4111505266 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7001c480 con 0x5609a235b210 2024-05-14T16:59:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.001+0000 7f0a667fc640 1 -- 172.21.15.69:0/4111505266 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a7001cad0 con 0x5609a235b210 2024-05-14T16:59:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.001+0000 7f0a667fc640 1 -- 172.21.15.69:0/4111505266 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70011ae0 con 0x5609a235b210 2024-05-14T16:59:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.001+0000 7f0a891b1740 1 -- 172.21.15.69:0/4111505266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 msgr2=0x5609a2365590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.001+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a680b2a90 0x7f0a680b4f50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5609a2358220 tx=0x7f0a74004730 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.001+0000 7f0a891b1740 1 --2- 172.21.15.69:0/4111505266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2365590 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000bc90 tx=0x7f0a700107a0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 -- 172.21.15.69:0/4111505266 shutdown_connections 2024-05-14T16:59:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 --2- 172.21.15.69:0/4111505266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 0x5609a2367ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 --2- 172.21.15.69:0/4111505266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2365590 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 --2- 172.21.15.69:0/4111505266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a235acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 -- 172.21.15.69:0/4111505266 >> 172.21.15.69:0/4111505266 conn(0x5609a2356e00 msgr2=0x5609a23571e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 -- 172.21.15.69:0/4111505266 shutdown_connections 2024-05-14T16:59:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 -- 172.21.15.69:0/4111505266 wait complete. 2024-05-14T16:59:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 -- start start 2024-05-14T16:59:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 Processor -- start 2024-05-14T16:59:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a23653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2362690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 0x5609a2362bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5609a1fa3790 con 0x5609a235b210 2024-05-14T16:59:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5609a2279890 con 0x5609a235a8f0 2024-05-14T16:59:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5609a23687a0 con 0x5609a2365ad0 2024-05-14T16:59:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.002+0000 7f0a891b1740 10 client.0 _ll_register_callbacks cb 0x5609a227cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2362690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2362690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39554/0 (socket says 172.21.15.69:39554) 2024-05-14T16:59:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7f7fe640 1 -- 172.21.15.69:0/3380363153 learned_addr learned my addr 172.21.15.69:0/3380363153 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a84a30640 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 0x5609a2362bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a23653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7f7fe640 1 -- 172.21.15.69:0/3380363153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 msgr2=0x5609a2362bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 0x5609a2362bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7f7fe640 1 -- 172.21.15.69:0/3380363153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 msgr2=0x5609a23653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a23653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7f7fe640 1 -- 172.21.15.69:0/3380363153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a209e620 con 0x5609a235b210 2024-05-14T16:59:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a23653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a84a30640 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 0x5609a2362bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2362690 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0a74011c20 tx=0x7f0a74005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3093b8c1a52cb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/3380363153 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74011e60 con 0x5609a235b210 2024-05-14T16:59:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/3380363153 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f0a74027600 con 0x5609a235b210 2024-05-14T16:59:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a891b1740 1 -- 172.21.15.69:0/3380363153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a7001cad0 con 0x5609a235b210 2024-05-14T16:59:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/3380363153 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0a7400fe90 con 0x5609a235b210 2024-05-14T16:59:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a891b1740 10 client.16425 fetch_fsmap learned FSMap version 43 2024-05-14T16:59:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.003+0000 7f0a891b1740 1 -- 172.21.15.69:0/3380363153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5609a2374180 con 0x5609a235b210 2024-05-14T16:59:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.004+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/3380363153 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a7001cad0 con 0x5609a235b210 2024-05-14T16:59:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.004+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/3380363153 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f0a7401f880 con 0x5609a235b210 2024-05-14T16:59:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.004+0000 7f0a891b1740 10 client.16425 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T16:59:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.004+0000 7f0a891b1740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.004+0000 7f0a891b1740 20 client.16425 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.004+0000 7f0a891b1740 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5609a2375700 0x5609a2377af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.004+0000 7f0a891b1740 4 client.16425 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.004+0000 7f0a891b1740 1 -- 172.21.15.69:0/3380363153 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a22cfb80 con 0x5609a2375700 2024-05-14T16:59:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.004+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5609a2375700 0x5609a2377af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.005+0000 7f0a5f7fe640 10 client.16425 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.005+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5609a2375700 0x5609a2377af0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3af93334913d6a77 server_cookie=68a0b5a8a18c in_seq=0 out_seq=0 2024-05-14T16:59:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.005+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/3380363153 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609a22cfb80 con 0x5609a2375700 2024-05-14T16:59:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.005+0000 7f0a5f7fe640 10 client.16425 handle_command_reply: tid=0 2024-05-14T16:59:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.005+0000 7f0a891b1740 1 client.16425 shutdown 2024-05-14T16:59:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 1 -- 172.21.15.69:0/3380363153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 msgr2=0x5609a2362690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2362690 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0a74011c20 tx=0x7f0a74005850 comp rx=0 tx=0).stop 2024-05-14T16:59:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 1 -- 172.21.15.69:0/3380363153 shutdown_connections 2024-05-14T16:59:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5609a2375700 0x5609a2377af0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a2365ad0 0x5609a2362bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235b210 0x5609a2362690 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 1 --2- 172.21.15.69:0/3380363153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a23653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a5f7fe640 0 client.16425 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 1 -- 172.21.15.69:0/3380363153 >> 172.21.15.69:0/3380363153 conn(0x5609a2356e00 msgr2=0x5609a2356100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 1 -- 172.21.15.69:0/3380363153 shutdown_connections 2024-05-14T16:59:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 1 -- 172.21.15.69:0/3380363153 wait complete. 2024-05-14T16:59:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.006+0000 7f0a891b1740 20 client.16425 trim_cache size 0 max 16384 2024-05-14T16:59:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a891b1740 1 Processor -- start 2024-05-14T16:59:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a891b1740 1 -- start start 2024-05-14T16:59:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a23716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a235b210 0x5609a237d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a2365ad0 0x5609a237d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a891b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a801b0f30 con 0x5609a2365ad0 2024-05-14T16:59:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a891b1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a801ae7d0 con 0x5609a235a8f0 2024-05-14T16:59:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a891b1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0a801ac640 con 0x5609a235b210 2024-05-14T16:59:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a84a30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a2365ad0 0x5609a237d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a84a30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a2365ad0 0x5609a237d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39562/0 (socket says 172.21.15.69:39562) 2024-05-14T16:59:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a84a30640 1 -- 172.21.15.69:0/2725367732 learned_addr learned my addr 172.21.15.69:0/2725367732 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/2725367732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a23716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/2725367732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a235b210 0x5609a237d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a84a30640 1 -- 172.21.15.69:0/2725367732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a235b210 msgr2=0x5609a237d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a84a30640 1 --2- 172.21.15.69:0/2725367732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a235b210 0x5609a237d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a84a30640 1 -- 172.21.15.69:0/2725367732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 msgr2=0x5609a23716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.021+0000 7f0a84a30640 1 --2- 172.21.15.69:0/2725367732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a23716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.022+0000 7f0a84a30640 1 -- 172.21.15.69:0/2725367732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a2361640 con 0x5609a2365ad0 2024-05-14T16:59:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.022+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/2725367732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a23716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.022+0000 7f0a84a30640 1 --2- 172.21.15.69:0/2725367732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a2365ad0 0x5609a237d940 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0a780c7c50 tx=0x7f0a780ca3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75cea1d0cab4ca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.022+0000 7f0a65ffb640 1 -- 172.21.15.69:0/2725367732 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a780ca4f0 con 0x5609a2365ad0 2024-05-14T16:59:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.022+0000 7f0a65ffb640 1 -- 172.21.15.69:0/2725367732 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a78036040 con 0x5609a2365ad0 2024-05-14T16:59:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.022+0000 7f0a65ffb640 1 -- 172.21.15.69:0/2725367732 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a780ced90 con 0x5609a2365ad0 2024-05-14T16:59:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 -- 172.21.15.69:0/2725367732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a2365ad0 msgr2=0x5609a237d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 --2- 172.21.15.69:0/2725367732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a2365ad0 0x5609a237d940 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0a780c7c50 tx=0x7f0a780ca3f0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 -- 172.21.15.69:0/2725367732 shutdown_connections 2024-05-14T16:59:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 --2- 172.21.15.69:0/2725367732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a2365ad0 0x5609a237d940 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 --2- 172.21.15.69:0/2725367732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a235b210 0x5609a237d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 --2- 172.21.15.69:0/2725367732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a235a8f0 0x5609a23716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 -- 172.21.15.69:0/2725367732 >> 172.21.15.69:0/2725367732 conn(0x5609a2356e00 msgr2=0x5609a2356100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 -- 172.21.15.69:0/2725367732 shutdown_connections 2024-05-14T16:59:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 -- 172.21.15.69:0/2725367732 wait complete. 2024-05-14T16:59:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 -- start start 2024-05-14T16:59:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 Processor -- start 2024-05-14T16:59:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235a8f0 0x5609a236b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a235b210 0x5609a236bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a2365ad0 0x5609a236c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:26.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a801acc90 con 0x5609a235a8f0 2024-05-14T16:59:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a801ae100 con 0x5609a2365ad0 2024-05-14T16:59:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0a801ad360 con 0x5609a235b210 2024-05-14T16:59:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.023+0000 7f0a891b1740 10 client.0 _ll_register_callbacks cb 0x7f0a740118d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235a8f0 0x5609a236b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235a8f0 0x5609a236b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39568/0 (socket says 172.21.15.69:39568) 2024-05-14T16:59:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a7ffff640 1 -- 172.21.15.69:0/2085187303 learned_addr learned my addr 172.21.15.69:0/2085187303 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a235b210 0x5609a236bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a84a30640 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a2365ad0 0x5609a236c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a7ffff640 1 -- 172.21.15.69:0/2085187303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a235b210 msgr2=0x5609a236bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a235b210 0x5609a236bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a7ffff640 1 -- 172.21.15.69:0/2085187303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a2365ad0 msgr2=0x5609a236c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a2365ad0 0x5609a236c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a7ffff640 1 -- 172.21.15.69:0/2085187303 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a2377e60 con 0x5609a235a8f0 2024-05-14T16:59:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a7ffff640 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235a8f0 0x5609a236b660 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000bc90 tx=0x7f0a7000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3514fa801440a2bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:59:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/2085187303 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70004220 con 0x5609a235a8f0 2024-05-14T16:59:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a891b1740 1 -- 172.21.15.69:0/2085187303 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a78036040 con 0x5609a235a8f0 2024-05-14T16:59:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/2085187303 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f0a700043c0 con 0x5609a235a8f0 2024-05-14T16:59:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/2085187303 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0a70005040 con 0x5609a235a8f0 2024-05-14T16:59:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a891b1740 10 client.16431 fetch_fsmap learned FSMap version 43 2024-05-14T16:59:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.024+0000 7f0a891b1740 1 -- 172.21.15.69:0/2085187303 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5609a23e0160 con 0x5609a235a8f0 2024-05-14T16:59:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/2085187303 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a78036040 con 0x5609a235a8f0 2024-05-14T16:59:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/2085187303 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f0a70011cb0 con 0x5609a235a8f0 2024-05-14T16:59:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a891b1740 10 client.16431 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T16:59:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a891b1740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a891b1740 20 client.16431 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a891b1740 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5609a23e0650 0x5609a23e2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a891b1740 4 client.16431 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:59:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a891b1740 1 -- 172.21.15.69:0/2085187303 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609a22c9360 con 0x5609a23e0650 2024-05-14T16:59:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5609a23e0650 0x5609a23e2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a5f7fe640 10 client.16431 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.025+0000 7f0a7f7fe640 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5609a23e0650 0x5609a23e2a70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66138a6c817d39d0 server_cookie=b1bc2b82aaab1eb9 in_seq=0 out_seq=0 2024-05-14T16:59:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a5f7fe640 1 -- 172.21.15.69:0/2085187303 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5609a22c9360 con 0x5609a23e0650 2024-05-14T16:59:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a5f7fe640 10 client.16431 handle_command_reply: tid=0 2024-05-14T16:59:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 client.16431 shutdown 2024-05-14T16:59:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 -- 172.21.15.69:0/2085187303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235a8f0 msgr2=0x5609a236b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235a8f0 0x5609a236b660 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000bc90 tx=0x7f0a7000b5b0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 -- 172.21.15.69:0/2085187303 shutdown_connections 2024-05-14T16:59:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5609a23e0650 0x5609a23e2a70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609a2365ad0 0x5609a236c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5609a235b210 0x5609a236bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a5f7fe640 0 client.16431 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 --2- 172.21.15.69:0/2085187303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609a235a8f0 0x5609a236b660 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 -- 172.21.15.69:0/2085187303 >> 172.21.15.69:0/2085187303 conn(0x5609a2356e00 msgr2=0x5609a2370690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 -- 172.21.15.69:0/2085187303 shutdown_connections 2024-05-14T16:59:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 1 -- 172.21.15.69:0/2085187303 wait complete. 2024-05-14T16:59:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.038+0000 7f0a891b1740 20 client.16431 trim_cache size 0 max 16384 2024-05-14T16:59:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:59:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:59:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:59:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 -- 172.21.15.69:0/630860220 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a680b2a90 msgr2=0x7f0a680b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a680b2a90 0x7f0a680b4f50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5609a2358220 tx=0x7f0a74004730 comp rx=0 tx=0).stop 2024-05-14T16:59:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 -- 172.21.15.69:0/630860220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a80107940 msgr2=0x7f0a8019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a80107940 0x7f0a8019dab0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005cc0 tx=0x7f0a78003c20 comp rx=0 tx=0).stop 2024-05-14T16:59:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 -- 172.21.15.69:0/630860220 shutdown_connections 2024-05-14T16:59:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a680b2a90 0x7f0a680b4f50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a80107940 0x7f0a8019dab0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a80102340 0x7f0a80199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 --2- 172.21.15.69:0/630860220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a80101980 0x7f0a801991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 -- 172.21.15.69:0/630860220 >> 172.21.15.69:0/630860220 conn(0x7f0a80078ad0 msgr2=0x7f0a8006a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 -- 172.21.15.69:0/630860220 shutdown_connections 2024-05-14T16:59:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:25.039+0000 7f0a86493640 1 -- 172.21.15.69:0/630860220 wait complete. 2024-05-14T16:59:27.463 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:27.475 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:24 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3309207520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:27.500 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 535.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:59:27.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:25 smithi069 ceph-mon[31507]: pgmap v298: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 736 op/s 2024-05-14T16:59:27.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:24 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3309207520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:27.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:25 smithi092 ceph-mon[35763]: pgmap v298: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 736 op/s 2024-05-14T16:59:27.543 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:24 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3309207520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:27.551 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:25 smithi203 ceph-mon[35787]: pgmap v298: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 736 op/s 2024-05-14T16:59:27.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.816+0000 7f528a951700 1 -- 172.21.15.69:0/108452753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52841024d0 msgr2=0x7f5284102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:27.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.816+0000 7f528a951700 1 --2- 172.21.15.69:0/108452753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52841024d0 0x7f5284102950 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f526c009990 tx=0x7f526c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.816+0000 7f528a951700 1 -- 172.21.15.69:0/108452753 shutdown_connections 2024-05-14T16:59:27.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.816+0000 7f528a951700 1 --2- 172.21.15.69:0/108452753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5284102e90 0x7f528410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.816+0000 7f528a951700 1 --2- 172.21.15.69:0/108452753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52841084d0 0x7f52841088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.816+0000 7f528a951700 1 --2- 172.21.15.69:0/108452753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52841024d0 0x7f5284102950 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.816+0000 7f528a951700 1 -- 172.21.15.69:0/108452753 >> 172.21.15.69:0/108452753 conn(0x7f5284078490 msgr2=0x7f52840788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.816+0000 7f528a951700 1 -- 172.21.15.69:0/108452753 shutdown_connections 2024-05-14T16:59:27.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.816+0000 7f528a951700 1 -- 172.21.15.69:0/108452753 wait complete. 2024-05-14T16:59:27.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.816+0000 7f528a951700 1 Processor -- start 2024-05-14T16:59:27.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f528a951700 1 -- start start 2024-05-14T16:59:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f528a951700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52841024d0 0x7f528419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:27.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f528a951700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5284102e90 0x7f528419afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:27.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f528a951700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52841084d0 0x7f52841a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:27.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f528a951700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5284112610 con 0x7f52841024d0 2024-05-14T16:59:27.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f528a951700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5284112490 con 0x7f5284102e90 2024-05-14T16:59:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f528a951700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5284112790 con 0x7f52841084d0 2024-05-14T16:59:27.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f5283fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52841024d0 0x7f528419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:27.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f5283fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52841024d0 0x7f528419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39582/0 (socket says 172.21.15.69:39582) 2024-05-14T16:59:27.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f5283fff700 1 -- 172.21.15.69:0/1803610157 learned_addr learned my addr 172.21.15.69:0/1803610157 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:27.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f5288eee700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52841084d0 0x7f52841a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:27.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f527bfff700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5284102e90 0x7f528419afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:27.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f5283fff700 1 -- 172.21.15.69:0/1803610157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52841084d0 msgr2=0x7f52841a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:27.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f5283fff700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52841084d0 0x7f52841a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f5283fff700 1 -- 172.21.15.69:0/1803610157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5284102e90 msgr2=0x7f528419afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:27.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f5283fff700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5284102e90 0x7f528419afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:27.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f5283fff700 1 -- 172.21.15.69:0/1803610157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52841a6030 con 0x7f52841024d0 2024-05-14T16:59:27.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f527bfff700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5284102e90 0x7f528419afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:27.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.817+0000 7f5283fff700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52841024d0 0x7f528419aa70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f527400e9a0 tx=0x7f527400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.818+0000 7f5281ffb700 1 -- 172.21.15.69:0/1803610157 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527400cbd0 con 0x7f52841024d0 2024-05-14T16:59:27.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.818+0000 7f528a951700 1 -- 172.21.15.69:0/1803610157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52841a6200 con 0x7f52841024d0 2024-05-14T16:59:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.818+0000 7f5281ffb700 1 -- 172.21.15.69:0/1803610157 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527400cd70 con 0x7f52841024d0 2024-05-14T16:59:27.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.818+0000 7f5281ffb700 1 -- 172.21.15.69:0/1803610157 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274018680 con 0x7f52841024d0 2024-05-14T16:59:27.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.818+0000 7f528a951700 1 -- 172.21.15.69:0/1803610157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52841a6710 con 0x7f52841024d0 2024-05-14T16:59:27.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.820+0000 7f5281ffb700 1 -- 172.21.15.69:0/1803610157 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5274010430 con 0x7f52841024d0 2024-05-14T16:59:27.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.821+0000 7f5281ffb700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52640b29f0 0x7f52640b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:27.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.821+0000 7f527bfff700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52640b29f0 0x7f52640b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:27.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.821+0000 7f5281ffb700 1 -- 172.21.15.69:0/1803610157 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f52740e2810 con 0x7f52841024d0 2024-05-14T16:59:27.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.821+0000 7f527bfff700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52640b29f0 0x7f52640b4eb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f526c00b2e0 tx=0x7f526c01a040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:27.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.821+0000 7f52951e7b80 1 Processor -- start 2024-05-14T16:59:28.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.821+0000 7f52951e7b80 1 -- start start 2024-05-14T16:59:28.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1f210 0x556473a1f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1fb30 0x556473a198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:28.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a19ee0 0x556473a1a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:28.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f52951e7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556473973d90 con 0x556473a19ee0 2024-05-14T16:59:28.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f52951e7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556473963930 con 0x556473a1fb30 2024-05-14T16:59:28.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f52951e7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556473967970 con 0x556473a1f210 2024-05-14T16:59:28.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5288eee700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a19ee0 0x556473a1a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:28.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5288eee700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a19ee0 0x556473a1a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39592/0 (socket says 172.21.15.69:39592) 2024-05-14T16:59:28.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5288eee700 1 -- 172.21.15.69:0/2176702887 learned_addr learned my addr 172.21.15.69:0/2176702887 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:28.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f527bfff700 1 --2- 172.21.15.69:0/2176702887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1fb30 0x556473a198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:28.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5283fff700 1 --2- 172.21.15.69:0/2176702887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1f210 0x556473a1f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:28.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5288eee700 1 -- 172.21.15.69:0/2176702887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1f210 msgr2=0x556473a1f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:28.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5288eee700 1 --2- 172.21.15.69:0/2176702887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1f210 0x556473a1f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5288eee700 1 -- 172.21.15.69:0/2176702887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1fb30 msgr2=0x556473a198c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:28.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5288eee700 1 --2- 172.21.15.69:0/2176702887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1fb30 0x556473a198c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5288eee700 1 -- 172.21.15.69:0/2176702887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564738b39d0 con 0x556473a19ee0 2024-05-14T16:59:28.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5283fff700 1 --2- 172.21.15.69:0/2176702887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1f210 0x556473a1f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:28.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f5288eee700 1 --2- 172.21.15.69:0/2176702887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a19ee0 0x556473a1a340 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f527c0098f0 tx=0x7f527c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b4152d16e78251 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:28.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f527a7fc700 1 -- 172.21.15.69:0/2176702887 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c004bb0 con 0x556473a19ee0 2024-05-14T16:59:28.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.822+0000 7f527a7fc700 1 -- 172.21.15.69:0/2176702887 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527c004d50 con 0x556473a19ee0 2024-05-14T16:59:28.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.823+0000 7f527a7fc700 1 -- 172.21.15.69:0/2176702887 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c007da0 con 0x556473a19ee0 2024-05-14T16:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.823+0000 7f52951e7b80 1 -- 172.21.15.69:0/2176702887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a19ee0 msgr2=0x556473a1a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:28.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.823+0000 7f52951e7b80 1 --2- 172.21.15.69:0/2176702887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a19ee0 0x556473a1a340 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f527c0098f0 tx=0x7f527c009c00 comp rx=0 tx=0).stop 2024-05-14T16:59:28.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.823+0000 7f52951e7b80 1 -- 172.21.15.69:0/2176702887 shutdown_connections 2024-05-14T16:59:28.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.823+0000 7f52951e7b80 1 --2- 172.21.15.69:0/2176702887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a19ee0 0x556473a1a340 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.823+0000 7f52951e7b80 1 --2- 172.21.15.69:0/2176702887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1f210 0x556473a1f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.823+0000 7f52951e7b80 1 --2- 172.21.15.69:0/2176702887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1fb30 0x556473a198c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.823+0000 7f52951e7b80 1 -- 172.21.15.69:0/2176702887 >> 172.21.15.69:0/2176702887 conn(0x556473940350 msgr2=0x556473a0fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:28.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.823+0000 7f52951e7b80 1 -- 172.21.15.69:0/2176702887 shutdown_connections 2024-05-14T16:59:28.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.823+0000 7f52951e7b80 1 -- 172.21.15.69:0/2176702887 wait complete. 2024-05-14T16:59:28.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f52951e7b80 1 -- start start 2024-05-14T16:59:28.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f52951e7b80 1 Processor -- start 2024-05-14T16:59:28.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 0x556473a25820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:28.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 0x556473a22bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:28.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a23100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:28.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f52951e7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55647386a0a0 con 0x556473a1fb30 2024-05-14T16:59:28.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f52951e7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5564739058c0 con 0x556473a1f210 2024-05-14T16:59:28.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f52951e7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556473a26590 con 0x556473a19ee0 2024-05-14T16:59:28.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f52951e7b80 10 client.0 _ll_register_callbacks cb 0x5564738fa680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:28.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f5288eee700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a23100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:28.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f5288eee700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a23100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39596/0 (socket says 172.21.15.69:39596) 2024-05-14T16:59:28.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f5288eee700 1 -- 172.21.15.69:0/4205207917 learned_addr learned my addr 172.21.15.69:0/4205207917 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:28.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f527bfff700 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 0x556473a22bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:28.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.824+0000 7f5283fff700 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 0x556473a25820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:28.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f5288eee700 1 -- 172.21.15.69:0/4205207917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 msgr2=0x556473a25820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:28.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f5288eee700 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 0x556473a25820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f5288eee700 1 -- 172.21.15.69:0/4205207917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 msgr2=0x556473a22bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:28.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f5288eee700 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 0x556473a22bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f5288eee700 1 -- 172.21.15.69:0/4205207917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556473902790 con 0x556473a1fb30 2024-05-14T16:59:28.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f5283fff700 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 0x556473a25820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:28.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f527bfff700 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 0x556473a22bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:28.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f5288eee700 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a23100 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f527c007980 tx=0x7f527c007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96b2bbff060b0e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:28.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52737fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:59:28.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52737fe700 1 -- 172.21.15.69:0/4205207917 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c022070 con 0x556473a1fb30 2024-05-14T16:59:28.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52737fe700 1 -- 172.21.15.69:0/4205207917 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f527c0046e0 con 0x556473a1fb30 2024-05-14T16:59:28.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52951e7b80 1 -- 172.21.15.69:0/4205207917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f527c004d50 con 0x556473a1fb30 2024-05-14T16:59:28.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52737fe700 1 -- 172.21.15.69:0/4205207917 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f527c02e050 con 0x556473a1fb30 2024-05-14T16:59:28.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52951e7b80 10 client.16443 fetch_fsmap learned FSMap version 43 2024-05-14T16:59:28.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52951e7b80 1 -- 172.21.15.69:0/4205207917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556473a30c90 con 0x556473a1fb30 2024-05-14T16:59:28.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52737fe700 1 -- 172.21.15.69:0/4205207917 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527c004d50 con 0x556473a1fb30 2024-05-14T16:59:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52737fe700 1 -- 172.21.15.69:0/4205207917 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f527c01e410 con 0x556473a1fb30 2024-05-14T16:59:28.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52951e7b80 10 client.16443 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T16:59:28.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52951e7b80 10 client.16443 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:59:28.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.825+0000 7f52951e7b80 20 client.16443 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:28.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.826+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556473a33160 0x556473a35550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:28.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.826+0000 7f52951e7b80 4 client.16443 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:28.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.826+0000 7f52951e7b80 1 -- 172.21.15.69:0/4205207917 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564739e1460 con 0x556473a33160 2024-05-14T16:59:28.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.826+0000 7f5283fff700 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556473a33160 0x556473a35550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.826+0000 7f52737fe700 10 client.16443 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:28.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.826+0000 7f5283fff700 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556473a33160 0x556473a35550 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df063ca7a8daf5d2 server_cookie=6930eead72497acc in_seq=0 out_seq=0 2024-05-14T16:59:28.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52737fe700 1 -- 172.21.15.69:0/4205207917 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564739e1460 con 0x556473a33160 2024-05-14T16:59:28.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52737fe700 10 client.16443 handle_command_reply: tid=0 2024-05-14T16:59:28.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 client.16443 shutdown 2024-05-14T16:59:28.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 -- 172.21.15.69:0/4205207917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 msgr2=0x556473a23100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:28.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a23100 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f527c007980 tx=0x7f527c007bb0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 -- 172.21.15.69:0/4205207917 shutdown_connections 2024-05-14T16:59:28.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556473a33160 0x556473a35550 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a23100 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 0x556473a25820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52737fe700 0 client.16443 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:28.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4205207917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 0x556473a22bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:28.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 -- 172.21.15.69:0/4205207917 >> 172.21.15.69:0/4205207917 conn(0x556473940350 msgr2=0x556473a0a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:28.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 -- 172.21.15.69:0/4205207917 shutdown_connections 2024-05-14T16:59:28.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 1 -- 172.21.15.69:0/4205207917 wait complete. 2024-05-14T16:59:28.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.827+0000 7f52951e7b80 20 client.16443 trim_cache size 0 max 16384 2024-05-14T16:59:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f52951e7b80 1 Processor -- start 2024-05-14T16:59:28.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f52951e7b80 1 -- start start 2024-05-14T16:59:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 0x556473a2f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:28.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 0x556473a0a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a1e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:28.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f52951e7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52841ae5e0 con 0x556473a1fb30 2024-05-14T16:59:28.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f52951e7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f52841abe40 con 0x556473a1f210 2024-05-14T16:59:28.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f52951e7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f52841a9cb0 con 0x556473a19ee0 2024-05-14T16:59:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f5288eee700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a1e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:28.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f5288eee700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a1e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39612/0 (socket says 172.21.15.69:39612) 2024-05-14T16:59:28.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f5288eee700 1 -- 172.21.15.69:0/4293825390 learned_addr learned my addr 172.21.15.69:0/4293825390 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:28.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f5283fff700 1 --2- 172.21.15.69:0/4293825390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 0x556473a2f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:28.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f527bfff700 1 --2- 172.21.15.69:0/4293825390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 0x556473a0a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:28.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f5288eee700 1 -- 172.21.15.69:0/4293825390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 msgr2=0x556473a2f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:28.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f5288eee700 1 --2- 172.21.15.69:0/4293825390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 0x556473a2f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f5288eee700 1 -- 172.21.15.69:0/4293825390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 msgr2=0x556473a0a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:29.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.847+0000 7f5288eee700 1 --2- 172.21.15.69:0/4293825390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 0x556473a0a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.848+0000 7f5288eee700 1 -- 172.21.15.69:0/4293825390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556473a19c00 con 0x556473a1fb30 2024-05-14T16:59:29.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.848+0000 7f527bfff700 1 --2- 172.21.15.69:0/4293825390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 0x556473a0a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:29.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.848+0000 7f5288eee700 1 --2- 172.21.15.69:0/4293825390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a1e2f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f527c003c20 tx=0x7f527c0073e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3078a1c7bf8e83 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:29.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.848+0000 7f5279ffb700 1 -- 172.21.15.69:0/4293825390 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c022050 con 0x556473a1fb30 2024-05-14T16:59:29.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.848+0000 7f5279ffb700 1 -- 172.21.15.69:0/4293825390 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527c01e410 con 0x556473a1fb30 2024-05-14T16:59:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.848+0000 7f5279ffb700 1 -- 172.21.15.69:0/4293825390 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c015b90 con 0x556473a1fb30 2024-05-14T16:59:29.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 -- 172.21.15.69:0/4293825390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 msgr2=0x556473a1e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:29.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4293825390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a1e2f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f527c003c20 tx=0x7f527c0073e0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 -- 172.21.15.69:0/4293825390 shutdown_connections 2024-05-14T16:59:29.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4293825390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1fb30 0x556473a1e2f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4293825390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a19ee0 0x556473a2f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4293825390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a1f210 0x556473a0a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 -- 172.21.15.69:0/4293825390 >> 172.21.15.69:0/4293825390 conn(0x556473940350 msgr2=0x556473a1dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:29.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 -- 172.21.15.69:0/4293825390 shutdown_connections 2024-05-14T16:59:29.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 -- 172.21.15.69:0/4293825390 wait complete. 2024-05-14T16:59:29.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 -- start start 2024-05-14T16:59:29.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.849+0000 7f52951e7b80 1 Processor -- start 2024-05-14T16:59:29.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a19ee0 0x556473a28b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1f210 0x556473a29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f52951e7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1fb30 0x556473a9d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:29.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f52951e7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52841aa300 con 0x556473a1f210 2024-05-14T16:59:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f52951e7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f52841ab770 con 0x556473a19ee0 2024-05-14T16:59:29.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f52951e7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f52841aa9d0 con 0x556473a1fb30 2024-05-14T16:59:29.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f52951e7b80 10 client.0 _ll_register_callbacks cb 0x5564738fa680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:29.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f527bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1f210 0x556473a29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:29.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f5283fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a19ee0 0x556473a28b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:29.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f5283fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a19ee0 0x556473a28b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:48738/0 (socket says 172.21.15.69:48738) 2024-05-14T16:59:29.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f5283fff700 1 -- 172.21.15.69:0/4232533397 learned_addr learned my addr 172.21.15.69:0/4232533397 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:29.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f527bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1f210 0x556473a29080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39628/0 (socket says 172.21.15.69:39628) 2024-05-14T16:59:29.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f5288eee700 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1fb30 0x556473a9d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:29.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f5283fff700 1 -- 172.21.15.69:0/4232533397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1fb30 msgr2=0x556473a9d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:29.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f5283fff700 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1fb30 0x556473a9d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f5283fff700 1 -- 172.21.15.69:0/4232533397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1f210 msgr2=0x556473a29080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:29.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f5283fff700 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1f210 0x556473a29080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f5283fff700 1 -- 172.21.15.69:0/4232533397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556473a9dd50 con 0x556473a19ee0 2024-05-14T16:59:29.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f5283fff700 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a19ee0 0x556473a28b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f52740a5b10 tx=0x7f5274010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92080e5455abc35 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:29.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f52737fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:59:29.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.850+0000 7f52737fe700 1 -- 172.21.15.69:0/4232533397 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52740c3060 con 0x556473a19ee0 2024-05-14T16:59:29.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52737fe700 1 -- 172.21.15.69:0/4232533397 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f52740a60d0 con 0x556473a19ee0 2024-05-14T16:59:29.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52951e7b80 1 -- 172.21.15.69:0/4232533397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f527c01e410 con 0x556473a19ee0 2024-05-14T16:59:29.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52737fe700 1 -- 172.21.15.69:0/4232533397 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f527408e030 con 0x556473a19ee0 2024-05-14T16:59:29.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52951e7b80 10 client.25895 fetch_fsmap learned FSMap version 43 2024-05-14T16:59:29.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52951e7b80 1 -- 172.21.15.69:0/4232533397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556473a15510 con 0x556473a19ee0 2024-05-14T16:59:29.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52737fe700 1 -- 172.21.15.69:0/4232533397 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527c01e410 con 0x556473a19ee0 2024-05-14T16:59:29.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52737fe700 1 -- 172.21.15.69:0/4232533397 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f5274010720 con 0x556473a19ee0 2024-05-14T16:59:29.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52951e7b80 10 client.25895 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T16:59:29.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52951e7b80 10 client.25895 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T16:59:29.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52951e7b80 20 client.25895 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:29.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556473a33160 0x556473a17bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:29.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52951e7b80 4 client.25895 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e68e8fc8", "release": true, "await": true}] 2024-05-14T16:59:29.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f52951e7b80 1 -- 172.21.15.69:0/4232533397 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e68e8fc8", "release": true, "await": true}) -- 0x556473975ce0 con 0x556473a33160 2024-05-14T16:59:29.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.851+0000 7f527bfff700 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556473a33160 0x556473a17bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:29.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.852+0000 7f52737fe700 10 client.25895 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:29.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.852+0000 7f527bfff700 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556473a33160 0x556473a17bc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8433eb474869aaac server_cookie=fa24382e573bdba9 in_seq=0 out_seq=0 2024-05-14T16:59:29.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52737fe700 1 -- 172.21.15.69:0/4232533397 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556473975ce0 con 0x556473a33160 2024-05-14T16:59:29.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52737fe700 10 client.25895 handle_command_reply: tid=0 2024-05-14T16:59:29.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 client.25895 shutdown 2024-05-14T16:59:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 -- 172.21.15.69:0/4232533397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a19ee0 msgr2=0x556473a28b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:29.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a19ee0 0x556473a28b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f52740a5b10 tx=0x7f5274010cb0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 -- 172.21.15.69:0/4232533397 shutdown_connections 2024-05-14T16:59:29.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556473a33160 0x556473a17bc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556473a1fb30 0x556473a9d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52737fe700 0 client.25895 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:29.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556473a19ee0 0x556473a28b40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 --2- 172.21.15.69:0/4232533397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556473a1f210 0x556473a29080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 -- 172.21.15.69:0/4232533397 >> 172.21.15.69:0/4232533397 conn(0x556473940350 msgr2=0x556473a1e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 -- 172.21.15.69:0/4232533397 shutdown_connections 2024-05-14T16:59:29.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 1 -- 172.21.15.69:0/4232533397 wait complete. 2024-05-14T16:59:29.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.862+0000 7f52951e7b80 20 client.25895 trim_cache size 0 max 16384 2024-05-14T16:59:29.619 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T16:59:29.624 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 43, 2024-05-14T16:59:29.630 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T16:59:29.656 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 32, 2024-05-14T16:59:29.663 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T16:59:29.668 INFO:teuthology.orchestra.run.smithi069.stdout: "e68e8fc8": { 2024-05-14T16:59:29.675 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 32, 2024-05-14T16:59:29.680 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T16:59:29.686 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:59:29.692 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:59:29.697 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:59:29.703 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T16:59:29.708 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T16:59:29.713 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 140.4, 2024-05-14T16:59:29.718 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T16:59:29.748 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T16:59:29.754 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T16:59:29.759 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T16:59:29.766 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T16:59:29.773 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T16:59:29.787 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:59:29.792 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:59:29.797 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:59:29.804 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:59:29.809 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T16:59:29.815 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T16:59:29.821 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:27 smithi092 ceph-mon[35763]: pgmap v299: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.4 MiB/s wr, 684 op/s 2024-05-14T16:59:29.828 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:27 smithi203 ceph-mon[35787]: pgmap v299: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.4 MiB/s wr, 684 op/s 2024-05-14T16:59:29.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 -- 172.21.15.69:0/1803610157 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52640b29f0 msgr2=0x7f52640b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:29.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52640b29f0 0x7f52640b4eb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f526c00b2e0 tx=0x7f526c01a040 comp rx=0 tx=0).stop 2024-05-14T16:59:29.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 -- 172.21.15.69:0/1803610157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52841024d0 msgr2=0x7f528419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:29.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52841024d0 0x7f528419aa70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f527400e9a0 tx=0x7f527400ecb0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 -- 172.21.15.69:0/1803610157 shutdown_connections 2024-05-14T16:59:29.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f52640b29f0 0x7f52640b4eb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52841024d0 0x7f528419aa70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52841084d0 0x7f52841a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 --2- 172.21.15.69:0/1803610157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5284102e90 0x7f528419afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:29.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 -- 172.21.15.69:0/1803610157 >> 172.21.15.69:0/1803610157 conn(0x7f5284078490 msgr2=0x7f5284100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:29.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 -- 172.21.15.69:0/1803610157 shutdown_connections 2024-05-14T16:59:29.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T16:59:26.863+0000 7f528a951700 1 -- 172.21.15.69:0/1803610157 wait complete. 2024-05-14T16:59:29.935 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:27 smithi069 ceph-mon[31507]: pgmap v299: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.4 MiB/s wr, 684 op/s 2024-05-14T16:59:29.942 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e68e8fc8', total seconds elapsed: 28.6 2024-05-14T16:59:30.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:29 smithi069 ceph-mon[31507]: pgmap v300: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-05-14T16:59:30.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:29 smithi203 ceph-mon[35787]: pgmap v300: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-05-14T16:59:30.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:29 smithi092 ceph-mon[35763]: pgmap v300: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-05-14T16:59:30.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 540.0 s, 2229.5 tps, lat 181.379 ms stddev 1583.170 2024-05-14T16:59:32.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:31 smithi203 ceph-mon[35787]: pgmap v301: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.6 MiB/s wr, 96 op/s 2024-05-14T16:59:32.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:31 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:59:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:31 smithi069 ceph-mon[31507]: pgmap v301: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.6 MiB/s wr, 96 op/s 2024-05-14T16:59:32.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:31 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:59:32.607 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:31 smithi092 ceph-mon[35763]: pgmap v301: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.6 MiB/s wr, 96 op/s 2024-05-14T16:59:32.613 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:31 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:59:34.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:33 smithi203 ceph-mon[35787]: pgmap v302: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.8 MiB/s wr, 253 op/s 2024-05-14T16:59:34.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:33 smithi092 ceph-mon[35763]: pgmap v302: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.8 MiB/s wr, 253 op/s 2024-05-14T16:59:34.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:33 smithi069 ceph-mon[31507]: pgmap v302: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.8 MiB/s wr, 253 op/s 2024-05-14T16:59:35.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 545.0 s, 2639.0 tps, lat 37.748 ms stddev 44.607 2024-05-14T16:59:36.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:35 smithi092 ceph-mon[35763]: pgmap v303: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.8 MiB/s wr, 250 op/s 2024-05-14T16:59:36.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:35 smithi069 ceph-mon[31507]: pgmap v303: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.8 MiB/s wr, 250 op/s 2024-05-14T16:59:36.509 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:35 smithi203 ceph-mon[35787]: pgmap v303: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.8 MiB/s wr, 250 op/s 2024-05-14T16:59:37.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:36 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:59:37.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:36 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:59:37.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:36 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:59:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.169+0000 7f9e0bc93640 1 -- 172.21.15.69:0/3762639343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e04107960 msgr2=0x7f9e04107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.169+0000 7f9e0bc93640 1 --2- 172.21.15.69:0/3762639343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e04107960 0x7f9e04107d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009900 tx=0x7f9e0002a3f0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.169+0000 7f9e0bc93640 1 -- 172.21.15.69:0/3762639343 shutdown_connections 2024-05-14T16:59:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.169+0000 7f9e0bc93640 1 --2- 172.21.15.69:0/3762639343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e04102310 0x7f9e0410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.169+0000 7f9e0bc93640 1 --2- 172.21.15.69:0/3762639343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04101950 0x7f9e04101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.169+0000 7f9e0bc93640 1 --2- 172.21.15.69:0/3762639343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e04107960 0x7f9e04107d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.169+0000 7f9e0bc93640 1 -- 172.21.15.69:0/3762639343 >> 172.21.15.69:0/3762639343 conn(0x7f9e04078a70 msgr2=0x7f9e04078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.169+0000 7f9e0bc93640 1 -- 172.21.15.69:0/3762639343 shutdown_connections 2024-05-14T16:59:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.169+0000 7f9e0bc93640 1 -- 172.21.15.69:0/3762639343 wait complete. 2024-05-14T16:59:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.170+0000 7f9e0bc93640 1 Processor -- start 2024-05-14T16:59:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.170+0000 7f9e0bc93640 1 -- start start 2024-05-14T16:59:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e0bc93640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04101950 0x7f9e04197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e0bc93640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e04102310 0x7f9e041980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e0bc93640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e04107960 0x7f9e04198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e0bc93640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e04111c70 con 0x7f9e04101950 2024-05-14T16:59:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e09a2f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04101950 0x7f9e04197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e0bc93640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9e04111af0 con 0x7f9e04107960 2024-05-14T16:59:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e0bc93640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e04111df0 con 0x7f9e04102310 2024-05-14T16:59:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e09a2f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04101950 0x7f9e04197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39294/0 (socket says 172.21.15.69:39294) 2024-05-14T16:59:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e09a2f640 1 -- 172.21.15.69:0/1058169636 learned_addr learned my addr 172.21.15.69:0/1058169636 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e0922e640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e04102310 0x7f9e041980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e0a230640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e04107960 0x7f9e04198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e09a2f640 1 -- 172.21.15.69:0/1058169636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e04102310 msgr2=0x7f9e041980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e09a2f640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e04102310 0x7f9e041980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e09a2f640 1 -- 172.21.15.69:0/1058169636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e04107960 msgr2=0x7f9e04198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e09a2f640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e04107960 0x7f9e04198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.171+0000 7f9e09a2f640 1 -- 172.21.15.69:0/1058169636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e041a7460 con 0x7f9e04101950 2024-05-14T16:59:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.172+0000 7f9e09a2f640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04101950 0x7f9e04197bb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009850 tx=0x7f9e00030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.172+0000 7f9dfaffd640 1 -- 172.21.15.69:0/1058169636 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0003f070 con 0x7f9e04101950 2024-05-14T16:59:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.173+0000 7f9dfaffd640 1 -- 172.21.15.69:0/1058169636 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e00004460 con 0x7f9e04101950 2024-05-14T16:59:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.173+0000 7f9dfaffd640 1 -- 172.21.15.69:0/1058169636 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00035030 con 0x7f9e04101950 2024-05-14T16:59:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.173+0000 7f9e0bc93640 1 -- 172.21.15.69:0/1058169636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e041a76f0 con 0x7f9e04101950 2024-05-14T16:59:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.173+0000 7f9e0bc93640 1 -- 172.21.15.69:0/1058169636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e041a7b90 con 0x7f9e04101950 2024-05-14T16:59:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.177+0000 7f9dfaffd640 1 -- 172.21.15.69:0/1058169636 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9e00042070 con 0x7f9e04101950 2024-05-14T16:59:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.177+0000 7f9e0e9b5740 1 Processor -- start 2024-05-14T16:59:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0e9b5740 1 -- start start 2024-05-14T16:59:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496100 0x5577aa4964e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 0x5577aa4a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa4a12e0 0x5577aa4a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5577aa400c30 con 0x5577aa496a20 2024-05-14T16:59:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5577aa3f07d0 con 0x5577aa496100 2024-05-14T16:59:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5577aa3f4810 con 0x5577aa4a12e0 2024-05-14T16:59:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0a230640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496100 0x5577aa4964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0a230640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496100 0x5577aa4964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:53074/0 (socket says 172.21.15.69:53074) 2024-05-14T16:59:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0a230640 1 -- 172.21.15.69:0/1592944784 learned_addr learned my addr 172.21.15.69:0/1592944784 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e09a2f640 1 --2- 172.21.15.69:0/1592944784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 0x5577aa4a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0a230640 1 -- 172.21.15.69:0/1592944784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa4a12e0 msgr2=0x5577aa4a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0a230640 1 --2- 172.21.15.69:0/1592944784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa4a12e0 0x5577aa4a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0a230640 1 -- 172.21.15.69:0/1592944784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 msgr2=0x5577aa4a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0a230640 1 --2- 172.21.15.69:0/1592944784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 0x5577aa4a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.178+0000 7f9e0a230640 1 -- 172.21.15.69:0/1592944784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577aa105210 con 0x5577aa496100 2024-05-14T16:59:38.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.179+0000 7f9e0a230640 1 --2- 172.21.15.69:0/1592944784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496100 0x5577aa4964e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009f50 tx=0x7f9dfc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7094620479401e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.179+0000 7f9deffff640 1 -- 172.21.15.69:0/1592944784 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc004d20 con 0x5577aa496100 2024-05-14T16:59:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.179+0000 7f9deffff640 1 -- 172.21.15.69:0/1592944784 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dfc004050 con 0x5577aa496100 2024-05-14T16:59:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.179+0000 7f9deffff640 1 -- 172.21.15.69:0/1592944784 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc005280 con 0x5577aa496100 2024-05-14T16:59:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.179+0000 7f9dfaffd640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9df00b2bd0 0x7f9df00b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.179+0000 7f9e09a2f640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9df00b2bd0 0x7f9df00b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.180+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1592944784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496100 msgr2=0x5577aa4964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.180+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/1592944784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496100 0x5577aa4964e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009f50 tx=0x7f9dfc0118d0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.180+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1592944784 shutdown_connections 2024-05-14T16:59:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.180+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/1592944784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa4a12e0 0x5577aa4a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.180+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/1592944784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 0x5577aa4a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.180+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/1592944784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496100 0x5577aa4964e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.180+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1592944784 >> 172.21.15.69:0/1592944784 conn(0x5577aa492610 msgr2=0x5577aa4929f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.180+0000 7f9dfaffd640 1 -- 172.21.15.69:0/1058169636 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f9e00102fa0 con 0x7f9e04101950 2024-05-14T16:59:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.181+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1592944784 shutdown_connections 2024-05-14T16:59:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.181+0000 7f9e09a2f640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9df00b2bd0 0x7f9df00b5090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5577aa4976a0 tx=0x7f9e00108040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.181+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1592944784 wait complete. 2024-05-14T16:59:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.181+0000 7f9e0e9b5740 1 -- start start 2024-05-14T16:59:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.181+0000 7f9e0e9b5740 1 Processor -- start 2024-05-14T16:59:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 0x5577aa49e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496a20 0x5577aa49edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa4a12e0 0x5577aa490150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5577aa0d6790 con 0x5577aa496100 2024-05-14T16:59:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5577aa3ac890 con 0x5577aa496a20 2024-05-14T16:59:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5577aa4a3fb0 con 0x5577aa4a12e0 2024-05-14T16:59:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0922e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496a20 0x5577aa49edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0922e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496a20 0x5577aa49edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:53078/0 (socket says 172.21.15.69:53078) 2024-05-14T16:59:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0922e640 1 -- 172.21.15.69:0/2570890961 learned_addr learned my addr 172.21.15.69:0/2570890961 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0e9b5740 10 client.0 _ll_register_callbacks cb 0x5577aa3afd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0a230640 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 0x5577aa49e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.182+0000 7f9e0a230640 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa4a12e0 0x5577aa490150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9e0922e640 1 -- 172.21.15.69:0/2570890961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa4a12e0 msgr2=0x5577aa490150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9e0922e640 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa4a12e0 0x5577aa490150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9e0922e640 1 -- 172.21.15.69:0/2570890961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 msgr2=0x5577aa49e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9e0922e640 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 0x5577aa49e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9e0922e640 1 -- 172.21.15.69:0/2570890961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577aa1d1620 con 0x5577aa496a20 2024-05-14T16:59:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9e0a230640 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 0x5577aa49e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9e0922e640 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496a20 0x5577aa49edc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9df40116c0 tx=0x7f9df4011b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=116caf35271317d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9ded7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T16:59:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9ded7fa640 1 -- 172.21.15.69:0/2570890961 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df401e060 con 0x5577aa496a20 2024-05-14T16:59:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9ded7fa640 1 -- 172.21.15.69:0/2570890961 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f9df400fd70 con 0x5577aa496a20 2024-05-14T16:59:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/2570890961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dfc004050 con 0x5577aa496a20 2024-05-14T16:59:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9ded7fa640 1 -- 172.21.15.69:0/2570890961 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9df4019430 con 0x5577aa496a20 2024-05-14T16:59:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9e0e9b5740 10 client.25907 fetch_fsmap learned FSMap version 45 2024-05-14T16:59:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/2570890961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5577aa491ce0 con 0x5577aa496a20 2024-05-14T16:59:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.183+0000 7f9ded7fa640 1 -- 172.21.15.69:0/2570890961 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df4019a80 con 0x5577aa496a20 2024-05-14T16:59:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.184+0000 7f9ded7fa640 1 -- 172.21.15.69:0/2570890961 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 45) ==== 8342+0+0 (secure 0 0 0) 0x7f9df4021760 con 0x5577aa496a20 2024-05-14T16:59:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.184+0000 7f9e0e9b5740 10 client.25907 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T16:59:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.184+0000 7f9e0e9b5740 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.184+0000 7f9e0e9b5740 20 client.25907 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.184+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5577aa4b9650 0x5577aa4bba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.184+0000 7f9e0e9b5740 4 client.25907 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.184+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/2570890961 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9dfc004050 con 0x5577aa4b9650 2024-05-14T16:59:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.184+0000 7f9e0a230640 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5577aa4b9650 0x5577aa4bba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.185+0000 7f9ded7fa640 10 client.25907 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.185+0000 7f9e0a230640 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5577aa4b9650 0x5577aa4bba40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4e201a53067bf server_cookie=48a4afcb459102ca in_seq=0 out_seq=0 2024-05-14T16:59:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9ded7fa640 1 -- 172.21.15.69:0/2570890961 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f9dfc004050 con 0x5577aa4b9650 2024-05-14T16:59:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9ded7fa640 10 client.25907 handle_command_reply: tid=0 2024-05-14T16:59:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9e0e9b5740 1 client.25907 shutdown 2024-05-14T16:59:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/2570890961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496a20 msgr2=0x5577aa49edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496a20 0x5577aa49edc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9df40116c0 tx=0x7f9df4011b80 comp rx=0 tx=0).stop 2024-05-14T16:59:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/2570890961 shutdown_connections 2024-05-14T16:59:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5577aa4b9650 0x5577aa4bba40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa4a12e0 0x5577aa490150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa496a20 0x5577aa49edc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/2570890961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 0x5577aa49e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/2570890961 >> 172.21.15.69:0/2570890961 conn(0x5577aa492610 msgr2=0x5577aa497b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.186+0000 7f9ded7fa640 0 client.25907 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.187+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/2570890961 shutdown_connections 2024-05-14T16:59:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.187+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/2570890961 wait complete. 2024-05-14T16:59:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.187+0000 7f9e0e9b5740 20 client.25907 trim_cache size 0 max 16384 2024-05-14T16:59:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.208+0000 7f9e0e9b5740 1 Processor -- start 2024-05-14T16:59:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.208+0000 7f9e0e9b5740 1 -- start start 2024-05-14T16:59:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.208+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 0x5577aa49d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.208+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa496a20 0x5577aa498210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.208+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 0x5577aa498750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.208+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041aeac0 con 0x5577aa496100 2024-05-14T16:59:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.208+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041ac360 con 0x5577aa4a12e0 2024-05-14T16:59:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.208+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041aa210 con 0x5577aa496a20 2024-05-14T16:59:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.209+0000 7f9e0a230640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 0x5577aa49d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.209+0000 7f9e0a230640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 0x5577aa49d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39322/0 (socket says 172.21.15.69:39322) 2024-05-14T16:59:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.209+0000 7f9e0a230640 1 -- 172.21.15.69:0/3394363709 learned_addr learned my addr 172.21.15.69:0/3394363709 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.209+0000 7f9e0a230640 1 --2- 172.21.15.69:0/3394363709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 0x5577aa498750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.209+0000 7f9e0a230640 1 -- 172.21.15.69:0/3394363709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa496a20 msgr2=0x5577aa498210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:59:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.209+0000 7f9e0a230640 1 --2- 172.21.15.69:0/3394363709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa496a20 0x5577aa498210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.209+0000 7f9e0a230640 1 -- 172.21.15.69:0/3394363709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 msgr2=0x5577aa49d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.209+0000 7f9e0a230640 1 --2- 172.21.15.69:0/3394363709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 0x5577aa49d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.209+0000 7f9e0a230640 1 -- 172.21.15.69:0/3394363709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577aa492440 con 0x5577aa4a12e0 2024-05-14T16:59:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.209+0000 7f9e0a230640 1 --2- 172.21.15.69:0/3394363709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 0x5577aa498750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009560 tx=0x7f9dfc004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd2ed3e2e6106d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.210+0000 7f9def7fe640 1 -- 172.21.15.69:0/3394363709 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc004ca0 con 0x5577aa4a12e0 2024-05-14T16:59:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.210+0000 7f9def7fe640 1 -- 172.21.15.69:0/3394363709 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dfc004e40 con 0x5577aa4a12e0 2024-05-14T16:59:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.210+0000 7f9def7fe640 1 -- 172.21.15.69:0/3394363709 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc005770 con 0x5577aa4a12e0 2024-05-14T16:59:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/3394363709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 msgr2=0x5577aa498750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/3394363709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 0x5577aa498750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009560 tx=0x7f9dfc004710 comp rx=0 tx=0).stop 2024-05-14T16:59:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/3394363709 shutdown_connections 2024-05-14T16:59:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/3394363709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 0x5577aa498750 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/3394363709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa496a20 0x5577aa498210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/3394363709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496100 0x5577aa49d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/3394363709 >> 172.21.15.69:0/3394363709 conn(0x5577aa492610 msgr2=0x5577aa497b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/3394363709 shutdown_connections 2024-05-14T16:59:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/3394363709 wait complete. 2024-05-14T16:59:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 -- start start 2024-05-14T16:59:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.211+0000 7f9e0e9b5740 1 Processor -- start 2024-05-14T16:59:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.212+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa496100 0x5577aa48dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.212+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 0x5577aa48b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.212+0000 7f9e0e9b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 0x5577aa48b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.212+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041aa860 con 0x5577aa496a20 2024-05-14T16:59:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.212+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041abc90 con 0x5577aa4a12e0 2024-05-14T16:59:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.212+0000 7f9e0e9b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041aaef0 con 0x5577aa496100 2024-05-14T16:59:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.212+0000 7f9e0e9b5740 10 client.0 _ll_register_callbacks cb 0x7f9df4009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.213+0000 7f9e0922e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 0x5577aa48b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.213+0000 7f9e0922e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 0x5577aa48b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39338/0 (socket says 172.21.15.69:39338) 2024-05-14T16:59:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.213+0000 7f9e0922e640 1 -- 172.21.15.69:0/1448294040 learned_addr learned my addr 172.21.15.69:0/1448294040 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.213+0000 7f9e0a230640 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa496100 0x5577aa48dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9e0a230640 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 0x5577aa48b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9e0922e640 1 -- 172.21.15.69:0/1448294040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa496100 msgr2=0x5577aa48dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9e0922e640 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa496100 0x5577aa48dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9e0922e640 1 -- 172.21.15.69:0/1448294040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 msgr2=0x5577aa48b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9e0922e640 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 0x5577aa48b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9e0922e640 1 -- 172.21.15.69:0/1448294040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577aa4973c0 con 0x5577aa496a20 2024-05-14T16:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9e0a230640 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 0x5577aa48b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9e0922e640 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 0x5577aa48b120 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9df4021d70 tx=0x7f9df4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632e25fd48c9201c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9ded7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:59:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9ded7fa640 1 -- 172.21.15.69:0/1448294040 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df40054c0 con 0x5577aa496a20 2024-05-14T16:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9ded7fa640 1 -- 172.21.15.69:0/1448294040 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f9df40213a0 con 0x5577aa496a20 2024-05-14T16:59:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.214+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1448294040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5577aa402b80 con 0x5577aa496a20 2024-05-14T16:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9ded7fa640 1 -- 172.21.15.69:0/1448294040 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9df401e030 con 0x5577aa496a20 2024-05-14T16:59:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9e0e9b5740 10 client.16467 fetch_fsmap learned FSMap version 45 2024-05-14T16:59:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1448294040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5577aa523c70 con 0x5577aa496a20 2024-05-14T16:59:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9ded7fa640 1 -- 172.21.15.69:0/1448294040 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5577aa402b80 con 0x5577aa496a20 2024-05-14T16:59:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9ded7fa640 1 -- 172.21.15.69:0/1448294040 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 45) ==== 8342+0+0 (secure 0 0 0) 0x7f9df4003af0 con 0x5577aa496a20 2024-05-14T16:59:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9e0e9b5740 10 client.16467 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T16:59:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9e0e9b5740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9e0e9b5740 20 client.16467 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5577aa5244a0 0x5577aa5268c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9e0e9b5740 4 client.16467 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:59:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.215+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1448294040 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577aa3fc360 con 0x5577aa5244a0 2024-05-14T16:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.216+0000 7f9e0a230640 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5577aa5244a0 0x5577aa5268c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.216+0000 7f9e0a230640 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5577aa5244a0 0x5577aa5268c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b49bb773e3fcbb1 server_cookie=3c25822914739d7e in_seq=0 out_seq=0 2024-05-14T16:59:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.216+0000 7f9ded7fa640 10 client.16467 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:39.927 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:37 smithi069 ceph-mon[31507]: pgmap v304: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 16 MiB/s wr, 678 op/s 2024-05-14T16:59:39.933 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:37 smithi203 ceph-mon[35787]: pgmap v304: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 16 MiB/s wr, 678 op/s 2024-05-14T16:59:39.943 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:37 smithi092 ceph-mon[35763]: pgmap v304: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 16 MiB/s wr, 678 op/s 2024-05-14T16:59:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9ded7fa640 1 -- 172.21.15.69:0/1448294040 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5577aa3fc360 con 0x5577aa5244a0 2024-05-14T16:59:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9ded7fa640 10 client.16467 handle_command_reply: tid=0 2024-05-14T16:59:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 client.16467 shutdown 2024-05-14T16:59:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1448294040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 msgr2=0x5577aa48b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 0x5577aa48b120 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9df4021d70 tx=0x7f9df4005f10 comp rx=0 tx=0).stop 2024-05-14T16:59:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1448294040 shutdown_connections 2024-05-14T16:59:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5577aa5244a0 0x5577aa5268c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577aa4a12e0 0x5577aa48b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aa496a20 0x5577aa48b120 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 --2- 172.21.15.69:0/1448294040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577aa496100 0x5577aa48dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9ded7fa640 0 client.16467 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1448294040 >> 172.21.15.69:0/1448294040 conn(0x5577aa492610 msgr2=0x5577aa498390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1448294040 shutdown_connections 2024-05-14T16:59:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 1 -- 172.21.15.69:0/1448294040 wait complete. 2024-05-14T16:59:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.638+0000 7f9e0e9b5740 20 client.16467 trim_cache size 0 max 16384 2024-05-14T16:59:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:59:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:59:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:59:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 -- 172.21.15.69:0/1058169636 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9df00b2bd0 msgr2=0x7f9df00b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9df00b2bd0 0x7f9df00b5090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5577aa4976a0 tx=0x7f9e00108040 comp rx=0 tx=0).stop 2024-05-14T16:59:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 -- 172.21.15.69:0/1058169636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04101950 msgr2=0x7f9e04197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04101950 0x7f9e04197bb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009850 tx=0x7f9e00030650 comp rx=0 tx=0).stop 2024-05-14T16:59:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 -- 172.21.15.69:0/1058169636 shutdown_connections 2024-05-14T16:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9df00b2bd0 0x7f9df00b5090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e04107960 0x7f9e04198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e04102310 0x7f9e041980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 --2- 172.21.15.69:0/1058169636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04101950 0x7f9e04197bb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 -- 172.21.15.69:0/1058169636 >> 172.21.15.69:0/1058169636 conn(0x7f9e04078a70 msgr2=0x7f9e041042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 -- 172.21.15.69:0/1058169636 shutdown_connections 2024-05-14T16:59:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:38.639+0000 7f9e0bc93640 1 -- 172.21.15.69:0/1058169636 wait complete. 2024-05-14T16:59:40.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:39 smithi092 ceph-mon[35763]: pgmap v305: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 MiB/s wr, 672 op/s 2024-05-14T16:59:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 550.0 s, 2528.0 tps, lat 39.845 ms stddev 44.501 2024-05-14T16:59:40.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:39 smithi069 ceph-mon[31507]: pgmap v305: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 MiB/s wr, 672 op/s 2024-05-14T16:59:40.466 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:39 smithi203 ceph-mon[35787]: pgmap v305: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 MiB/s wr, 672 op/s 2024-05-14T16:59:42.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:41 smithi092 ceph-mon[35763]: pgmap v306: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 17 MiB/s wr, 741 op/s 2024-05-14T16:59:42.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:41 smithi069 ceph-mon[31507]: pgmap v306: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 17 MiB/s wr, 741 op/s 2024-05-14T16:59:42.463 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:41 smithi203 ceph-mon[35787]: pgmap v306: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 17 MiB/s wr, 741 op/s 2024-05-14T16:59:44.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:43 smithi092 ceph-mon[35763]: pgmap v307: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 19 MiB/s wr, 874 op/s 2024-05-14T16:59:44.266 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:43 smithi203 ceph-mon[35787]: pgmap v307: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 19 MiB/s wr, 874 op/s 2024-05-14T16:59:44.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:43 smithi069 ceph-mon[31507]: pgmap v307: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 19 MiB/s wr, 874 op/s 2024-05-14T16:59:45.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 555.0 s, 3045.4 tps, lat 32.513 ms stddev 39.253 2024-05-14T16:59:46.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:45 smithi092 ceph-mon[35763]: pgmap v308: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 14 MiB/s wr, 715 op/s 2024-05-14T16:59:46.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:45 smithi069 ceph-mon[31507]: pgmap v308: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 14 MiB/s wr, 715 op/s 2024-05-14T16:59:46.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:45 smithi203 ceph-mon[35787]: pgmap v308: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 14 MiB/s wr, 715 op/s 2024-05-14T16:59:48.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:47 smithi092 ceph-mon[35763]: pgmap v309: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2024-05-14T16:59:48.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:47 smithi069 ceph-mon[31507]: pgmap v309: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2024-05-14T16:59:48.469 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:47 smithi203 ceph-mon[35787]: pgmap v309: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2024-05-14T16:59:50.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:49 smithi092 ceph-mon[35763]: pgmap v310: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.38k op/s 2024-05-14T16:59:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 560.0 s, 2013.4 tps, lat 50.199 ms stddev 106.031 2024-05-14T16:59:50.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:49 smithi069 ceph-mon[31507]: pgmap v310: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.38k op/s 2024-05-14T16:59:50.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:49 smithi203 ceph-mon[35787]: pgmap v310: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.38k op/s 2024-05-14T16:59:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.760+0000 7f45d62d1640 1 -- 172.21.15.69:0/4122803821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0101ce0 msgr2=0x7f45d010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.760+0000 7f45d62d1640 1 --2- 172.21.15.69:0/4122803821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0101ce0 0x7f45d010f950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009be0 tx=0x7f45c802a6f0 comp rx=0 tx=0).stop 2024-05-14T16:59:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.760+0000 7f45d62d1640 1 -- 172.21.15.69:0/4122803821 shutdown_connections 2024-05-14T16:59:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.760+0000 7f45d62d1640 1 --2- 172.21.15.69:0/4122803821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0101ce0 0x7f45d010f950 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.760+0000 7f45d62d1640 1 --2- 172.21.15.69:0/4122803821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0101320 0x7f45d01017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.760+0000 7f45d62d1640 1 --2- 172.21.15.69:0/4122803821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d0107aa0 0x7f45d0107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.760+0000 7f45d62d1640 1 -- 172.21.15.69:0/4122803821 >> 172.21.15.69:0/4122803821 conn(0x7f45d0076cd0 msgr2=0x7f45d00770e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.760+0000 7f45d62d1640 1 -- 172.21.15.69:0/4122803821 shutdown_connections 2024-05-14T16:59:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.760+0000 7f45d62d1640 1 -- 172.21.15.69:0/4122803821 wait complete. 2024-05-14T16:59:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.761+0000 7f45d62d1640 1 Processor -- start 2024-05-14T16:59:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.761+0000 7f45d62d1640 1 -- start start 2024-05-14T16:59:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.761+0000 7f45d62d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d0101320 0x7f45d0199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.761+0000 7f45d62d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0101ce0 0x7f45d01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.761+0000 7f45d62d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0107aa0 0x7f45d019d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45d62d1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0112d80 con 0x7f45d0101ce0 2024-05-14T16:59:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45d62d1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0112c00 con 0x7f45d0101320 2024-05-14T16:59:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45d62d1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0112f00 con 0x7f45d0107aa0 2024-05-14T16:59:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0101ce0 0x7f45d01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0101ce0 0x7f45d01996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60862/0 (socket says 172.21.15.69:60862) 2024-05-14T16:59:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45cffff640 1 -- 172.21.15.69:0/385557645 learned_addr learned my addr 172.21.15.69:0/385557645 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45d486e640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0107aa0 0x7f45d019d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45c7fff640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d0101320 0x7f45d0199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45d486e640 1 -- 172.21.15.69:0/385557645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d0101320 msgr2=0x7f45d0199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45d486e640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d0101320 0x7f45d0199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45d486e640 1 -- 172.21.15.69:0/385557645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0101ce0 msgr2=0x7f45d01996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45d486e640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0101ce0 0x7f45d01996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45d486e640 1 -- 172.21.15.69:0/385557645 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d019e120 con 0x7f45d0107aa0 2024-05-14T16:59:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.762+0000 7f45c7fff640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d0101320 0x7f45d0199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.763+0000 7f45d486e640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0107aa0 0x7f45d019d9f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f45c802ac10 tx=0x7f45c80316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.763+0000 7f45cdffb640 1 -- 172.21.15.69:0/385557645 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8041020 con 0x7f45d0107aa0 2024-05-14T16:59:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.763+0000 7f45cdffb640 1 -- 172.21.15.69:0/385557645 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c80318e0 con 0x7f45d0107aa0 2024-05-14T16:59:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.763+0000 7f45cdffb640 1 -- 172.21.15.69:0/385557645 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8036040 con 0x7f45d0107aa0 2024-05-14T16:59:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.763+0000 7f45d62d1640 1 -- 172.21.15.69:0/385557645 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d019e3b0 con 0x7f45d0107aa0 2024-05-14T16:59:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.763+0000 7f45d62d1640 1 -- 172.21.15.69:0/385557645 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d019e820 con 0x7f45d0107aa0 2024-05-14T16:59:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.766+0000 7f45cdffb640 1 -- 172.21.15.69:0/385557645 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f45c8044020 con 0x7f45d0107aa0 2024-05-14T16:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.766+0000 7f45d8fea740 1 Processor -- start 2024-05-14T16:59:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.766+0000 7f45d8fea740 1 -- start start 2024-05-14T16:59:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.768+0000 7f45cdffb640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f45bc0b2a90 0x7f45bc0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.768+0000 7f45cdffb640 1 -- 172.21.15.69:0/385557645 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f45c81036c0 con 0x7f45d0107aa0 2024-05-14T16:59:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.769+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.769+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 0x55e1421c4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.769+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1421c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.769+0000 7f45c7fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.769+0000 7f45cffff640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f45bc0b2a90 0x7f45bc0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.769+0000 7f45c7fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421c09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:50072/0 (socket says 172.21.15.69:50072) 2024-05-14T16:59:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.769+0000 7f45c7fff640 1 -- 172.21.15.69:0/3961816413 learned_addr learned my addr 172.21.15.69:0/3961816413 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.769+0000 7f45d8fea740 1 -- 172.21.15.69:0/3961816413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e142129c30 con 0x55e1421c4d20 2024-05-14T16:59:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.769+0000 7f45d8fea740 1 -- 172.21.15.69:0/3961816413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e1421197d0 con 0x55e1421c0fc0 2024-05-14T16:59:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.769+0000 7f45d8fea740 1 -- 172.21.15.69:0/3961816413 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e14211d810 con 0x55e1421c0610 2024-05-14T16:59:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.770+0000 7f45cffff640 1 --2- 172.21.15.69:0/3961816413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 0x55e1421c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.770+0000 7f45d486e640 1 --2- 172.21.15.69:0/3961816413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1421c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.770+0000 7f45cffff640 1 -- 172.21.15.69:0/3961816413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 msgr2=0x55e1421c09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.770+0000 7f45cffff640 1 --2- 172.21.15.69:0/3961816413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421c09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.770+0000 7f45cffff640 1 -- 172.21.15.69:0/3961816413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 msgr2=0x55e1421c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.770+0000 7f45cffff640 1 --2- 172.21.15.69:0/3961816413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1421c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.770+0000 7f45cffff640 1 -- 172.21.15.69:0/3961816413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e141e2e210 con 0x55e1421c0fc0 2024-05-14T16:59:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.770+0000 7f45cffff640 1 --2- 172.21.15.69:0/3961816413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 0x55e1421c4700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f45b8005fd0 tx=0x7f45b801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce923510076deb01 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:53.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.770+0000 7f45cffff640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f45bc0b2a90 0x7f45bc0b4f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45d0100e30 tx=0x7f45b800bc00 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.771+0000 7f45c67fc640 1 -- 172.21.15.69:0/3961816413 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8012110 con 0x55e1421c0fc0 2024-05-14T16:59:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.771+0000 7f45c67fc640 1 -- 172.21.15.69:0/3961816413 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b80122b0 con 0x55e1421c0fc0 2024-05-14T16:59:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.771+0000 7f45c67fc640 1 -- 172.21.15.69:0/3961816413 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8004c80 con 0x55e1421c0fc0 2024-05-14T16:59:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.772+0000 7f45d8fea740 1 -- 172.21.15.69:0/3961816413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 msgr2=0x55e1421c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.772+0000 7f45d8fea740 1 --2- 172.21.15.69:0/3961816413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 0x55e1421c4700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f45b8005fd0 tx=0x7f45b801c000 comp rx=0 tx=0).stop 2024-05-14T16:59:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.772+0000 7f45d8fea740 1 -- 172.21.15.69:0/3961816413 shutdown_connections 2024-05-14T16:59:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.772+0000 7f45d8fea740 1 --2- 172.21.15.69:0/3961816413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1421c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.772+0000 7f45d8fea740 1 --2- 172.21.15.69:0/3961816413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 0x55e1421c4700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.772+0000 7f45d8fea740 1 --2- 172.21.15.69:0/3961816413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.772+0000 7f45d8fea740 1 -- 172.21.15.69:0/3961816413 >> 172.21.15.69:0/3961816413 conn(0x55e1421b28b0 msgr2=0x55e1421b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.772+0000 7f45d8fea740 1 -- 172.21.15.69:0/3961816413 shutdown_connections 2024-05-14T16:59:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.772+0000 7f45d8fea740 1 -- 172.21.15.69:0/3961816413 wait complete. 2024-05-14T16:59:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.772+0000 7f45d8fea740 1 -- start start 2024-05-14T16:59:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45d8fea740 1 Processor -- start 2024-05-14T16:59:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 0x55e1421c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1421c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45d8fea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e141dff790 con 0x55e1421c4d20 2024-05-14T16:59:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45d8fea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e1420d5890 con 0x55e1421c0fc0 2024-05-14T16:59:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45d8fea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e1421cceb0 con 0x55e1421c0610 2024-05-14T16:59:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45c7fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45d486e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1421c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45c7fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:50074/0 (socket says 172.21.15.69:50074) 2024-05-14T16:59:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45c7fff640 1 -- 172.21.15.69:0/285079910 learned_addr learned my addr 172.21.15.69:0/285079910 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45cffff640 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 0x55e1421c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45d8fea740 10 client.0 _ll_register_callbacks cb 0x55e1420d8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45c7fff640 1 -- 172.21.15.69:0/285079910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 msgr2=0x55e1421c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45c7fff640 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 0x55e1421c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45c7fff640 1 -- 172.21.15.69:0/285079910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 msgr2=0x55e1421c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45c7fff640 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1421c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.773+0000 7f45c7fff640 1 -- 172.21.15.69:0/285079910 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e141efa620 con 0x55e1421c0610 2024-05-14T16:59:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45c7fff640 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421cc330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f45c0012480 tx=0x7f45c0012940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1caf26b2eb6f27d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T16:59:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45af7fe640 1 -- 172.21.15.69:0/285079910 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0004460 con 0x55e1421c0610 2024-05-14T16:59:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45af7fe640 1 -- 172.21.15.69:0/285079910 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f45c00035f0 con 0x55e1421c0610 2024-05-14T16:59:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45d8fea740 1 -- 172.21.15.69:0/285079910 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b80122b0 con 0x55e1421c0610 2024-05-14T16:59:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45af7fe640 1 -- 172.21.15.69:0/285079910 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f45c00037f0 con 0x55e1421c0610 2024-05-14T16:59:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45d8fea740 10 client.25834 fetch_fsmap learned FSMap version 45 2024-05-14T16:59:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45d8fea740 1 -- 172.21.15.69:0/285079910 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e1421d8990 con 0x55e1421c0610 2024-05-14T16:59:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45af7fe640 1 -- 172.21.15.69:0/285079910 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b80122b0 con 0x55e1421c0610 2024-05-14T16:59:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45af7fe640 1 -- 172.21.15.69:0/285079910 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 45) ==== 8342+0+0 (secure 0 0 0) 0x7f45c0019070 con 0x55e1421c0610 2024-05-14T16:59:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45d8fea740 10 client.25834 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T16:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.774+0000 7f45d8fea740 10 client.25834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.775+0000 7f45d8fea740 20 client.25834 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.775+0000 7f45d8fea740 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e1421d9e70 0x55e1421dc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.775+0000 7f45d8fea740 4 client.25834 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.775+0000 7f45d8fea740 1 -- 172.21.15.69:0/285079910 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e14212bb80 con 0x55e1421d9e70 2024-05-14T16:59:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.775+0000 7f45cffff640 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e1421d9e70 0x55e1421dc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.775+0000 7f45af7fe640 10 client.25834 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.775+0000 7f45cffff640 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e1421d9e70 0x55e1421dc2a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2665f07f44ed1d server_cookie=9913d47a65788379 in_seq=0 out_seq=0 2024-05-14T16:59:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45af7fe640 1 -- 172.21.15.69:0/285079910 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e14212bb80 con 0x55e1421d9e70 2024-05-14T16:59:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45af7fe640 10 client.25834 handle_command_reply: tid=0 2024-05-14T16:59:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45d8fea740 1 client.25834 shutdown 2024-05-14T16:59:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45d8fea740 1 -- 172.21.15.69:0/285079910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 msgr2=0x55e1421cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45d8fea740 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421cc330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f45c0012480 tx=0x7f45c0012940 comp rx=0 tx=0).stop 2024-05-14T16:59:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45d8fea740 1 -- 172.21.15.69:0/285079910 shutdown_connections 2024-05-14T16:59:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45d8fea740 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e1421d9e70 0x55e1421dc2a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45d8fea740 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1421c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45d8fea740 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0fc0 0x55e1421c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45d8fea740 1 --2- 172.21.15.69:0/285079910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0610 0x55e1421cc330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45d8fea740 1 -- 172.21.15.69:0/285079910 >> 172.21.15.69:0/285079910 conn(0x55e1421b28b0 msgr2=0x55e1421bfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.778+0000 7f45af7fe640 0 client.25834 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.779+0000 7f45d8fea740 1 -- 172.21.15.69:0/285079910 shutdown_connections 2024-05-14T16:59:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.779+0000 7f45d8fea740 1 -- 172.21.15.69:0/285079910 wait complete. 2024-05-14T16:59:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.779+0000 7f45d8fea740 20 client.25834 trim_cache size 0 max 16384 2024-05-14T16:59:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.797+0000 7f45d8fea740 1 Processor -- start 2024-05-14T16:59:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.797+0000 7f45d8fea740 1 -- start start 2024-05-14T16:59:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.797+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c0610 0x55e1421c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.797+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 0x55e1421d5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.797+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c4d20 0x55e1421b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.797+0000 7f45d8fea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01aeca0 con 0x55e1421c0610 2024-05-14T16:59:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.798+0000 7f45d8fea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01ac540 con 0x55e1421c4d20 2024-05-14T16:59:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.798+0000 7f45d8fea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01aa3b0 con 0x55e1421c0fc0 2024-05-14T16:59:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.798+0000 7f45d486e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c4d20 0x55e1421b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.798+0000 7f45d486e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c4d20 0x55e1421b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:55152/0 (socket says 172.21.15.69:55152) 2024-05-14T16:59:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.798+0000 7f45d486e640 1 -- 172.21.15.69:0/3582875816 learned_addr learned my addr 172.21.15.69:0/3582875816 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.799+0000 7f45c7fff640 1 --2- 172.21.15.69:0/3582875816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c0610 0x55e1421c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.799+0000 7f45cffff640 1 --2- 172.21.15.69:0/3582875816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 0x55e1421d5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.799+0000 7f45d486e640 1 -- 172.21.15.69:0/3582875816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 msgr2=0x55e1421d5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.799+0000 7f45d486e640 1 --2- 172.21.15.69:0/3582875816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 0x55e1421d5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.799+0000 7f45d486e640 1 -- 172.21.15.69:0/3582875816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c0610 msgr2=0x55e1421c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:53.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.799+0000 7f45d486e640 1 --2- 172.21.15.69:0/3582875816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c0610 0x55e1421c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.799+0000 7f45d486e640 1 -- 172.21.15.69:0/3582875816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1421ae830 con 0x55e1421c4d20 2024-05-14T16:59:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.799+0000 7f45c7fff640 1 --2- 172.21.15.69:0/3582875816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c0610 0x55e1421c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.799+0000 7f45d486e640 1 --2- 172.21.15.69:0/3582875816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c4d20 0x55e1421b1870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f45c8006010 tx=0x7f45c80ca100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deccee92455562e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.800+0000 7f45c5ffb640 1 -- 172.21.15.69:0/3582875816 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80ca4a0 con 0x55e1421c4d20 2024-05-14T16:59:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.800+0000 7f45c5ffb640 1 -- 172.21.15.69:0/3582875816 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c80cb090 con 0x55e1421c4d20 2024-05-14T16:59:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.800+0000 7f45c5ffb640 1 -- 172.21.15.69:0/3582875816 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80c6db0 con 0x55e1421c4d20 2024-05-14T16:59:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 -- 172.21.15.69:0/3582875816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c4d20 msgr2=0x55e1421b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 --2- 172.21.15.69:0/3582875816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c4d20 0x55e1421b1870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f45c8006010 tx=0x7f45c80ca100 comp rx=0 tx=0).stop 2024-05-14T16:59:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 -- 172.21.15.69:0/3582875816 shutdown_connections 2024-05-14T16:59:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 --2- 172.21.15.69:0/3582875816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c4d20 0x55e1421b1870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 --2- 172.21.15.69:0/3582875816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 0x55e1421d5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 --2- 172.21.15.69:0/3582875816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c0610 0x55e1421c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 -- 172.21.15.69:0/3582875816 >> 172.21.15.69:0/3582875816 conn(0x55e1421b28b0 msgr2=0x55e1421c3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 -- 172.21.15.69:0/3582875816 shutdown_connections 2024-05-14T16:59:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 -- 172.21.15.69:0/3582875816 wait complete. 2024-05-14T16:59:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 -- start start 2024-05-14T16:59:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.801+0000 7f45d8fea740 1 Processor -- start 2024-05-14T16:59:54.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0610 0x55e1421e1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 0x55e1421e1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d8fea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1422437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d8fea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01aaa00 con 0x55e1421c4d20 2024-05-14T16:59:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d8fea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01abe70 con 0x55e1421c0610 2024-05-14T16:59:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d8fea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01ab0d0 con 0x55e1421c0fc0 2024-05-14T16:59:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d8fea740 10 client.0 _ll_register_callbacks cb 0x7f45c000bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d486e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1422437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d486e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1422437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60888/0 (socket says 172.21.15.69:60888) 2024-05-14T16:59:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45cffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 0x55e1421e1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d486e640 1 -- 172.21.15.69:0/2290627656 learned_addr learned my addr 172.21.15.69:0/2290627656 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T16:59:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45c7fff640 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0610 0x55e1421e1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d486e640 1 -- 172.21.15.69:0/2290627656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 msgr2=0x55e1421e1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d486e640 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 0x55e1421e1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d486e640 1 -- 172.21.15.69:0/2290627656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0610 msgr2=0x55e1421e1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d486e640 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0610 0x55e1421e1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45d486e640 1 -- 172.21.15.69:0/2290627656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1421e2590 con 0x55e1421c4d20 2024-05-14T16:59:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.802+0000 7f45cffff640 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 0x55e1421e1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.803+0000 7f45d486e640 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1422437b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f45c80ca690 tx=0x7f45c80d3570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7dea201537d2fe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.803+0000 7f45af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T16:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.803+0000 7f45af7fe640 1 -- 172.21.15.69:0/2290627656 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80ca4a0 con 0x55e1421c4d20 2024-05-14T16:59:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.803+0000 7f45af7fe640 1 -- 172.21.15.69:0/2290627656 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f45c8004d50 con 0x55e1421c4d20 2024-05-14T16:59:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.803+0000 7f45d8fea740 1 -- 172.21.15.69:0/2290627656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c80cb090 con 0x55e1421c4d20 2024-05-14T16:59:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.803+0000 7f45af7fe640 1 -- 172.21.15.69:0/2290627656 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f45c80ca990 con 0x55e1421c4d20 2024-05-14T16:59:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.803+0000 7f45d8fea740 10 client.16482 fetch_fsmap learned FSMap version 45 2024-05-14T16:59:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.803+0000 7f45d8fea740 1 -- 172.21.15.69:0/2290627656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e142244e00 con 0x55e1421c4d20 2024-05-14T16:59:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.803+0000 7f45af7fe640 1 -- 172.21.15.69:0/2290627656 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c80c6d90 con 0x55e1421c4d20 2024-05-14T16:59:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.803+0000 7f45af7fe640 1 -- 172.21.15.69:0/2290627656 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 45) ==== 8342+0+0 (secure 0 0 0) 0x7f45c80c9660 con 0x55e1421c4d20 2024-05-14T16:59:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.804+0000 7f45d8fea740 10 client.16482 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T16:59:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.804+0000 7f45d8fea740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.804+0000 7f45d8fea740 20 client.16482 populate_metadata read hostname 'smithi069' 2024-05-14T16:59:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.804+0000 7f45d8fea740 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e1422453f0 0x55e142247810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.804+0000 7f45d8fea740 4 client.16482 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:59:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.804+0000 7f45d8fea740 1 -- 172.21.15.69:0/2290627656 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f45c80cb090 con 0x55e1422453f0 2024-05-14T16:59:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.804+0000 7f45c7fff640 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e1422453f0 0x55e142247810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.804+0000 7f45af7fe640 10 client.16482 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.804+0000 7f45c7fff640 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e1422453f0 0x55e142247810 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e047f0e92516968b server_cookie=ee504f35fbc34e32 in_seq=0 out_seq=0 2024-05-14T16:59:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45af7fe640 1 -- 172.21.15.69:0/2290627656 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f45c80cb090 con 0x55e1422453f0 2024-05-14T16:59:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45af7fe640 10 client.16482 handle_command_reply: tid=0 2024-05-14T16:59:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45d8fea740 1 client.16482 shutdown 2024-05-14T16:59:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45d8fea740 1 -- 172.21.15.69:0/2290627656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 msgr2=0x55e1422437b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45d8fea740 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1422437b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f45c80ca690 tx=0x7f45c80d3570 comp rx=0 tx=0).stop 2024-05-14T16:59:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45d8fea740 1 -- 172.21.15.69:0/2290627656 shutdown_connections 2024-05-14T16:59:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45d8fea740 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e1422453f0 0x55e142247810 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45d8fea740 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1421c4d20 0x55e1422437b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45af7fe640 0 client.16482 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T16:59:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45d8fea740 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e1421c0fc0 0x55e1421e1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45d8fea740 1 --2- 172.21.15.69:0/2290627656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1421c0610 0x55e1421e1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.818+0000 7f45d8fea740 1 -- 172.21.15.69:0/2290627656 >> 172.21.15.69:0/2290627656 conn(0x55e1421b28b0 msgr2=0x55e1421cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.819+0000 7f45d8fea740 1 -- 172.21.15.69:0/2290627656 shutdown_connections 2024-05-14T16:59:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.819+0000 7f45d8fea740 1 -- 172.21.15.69:0/2290627656 wait complete. 2024-05-14T16:59:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.819+0000 7f45d8fea740 20 client.16482 trim_cache size 0 max 16384 2024-05-14T16:59:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T16:59:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T16:59:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T16:59:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 -- 172.21.15.69:0/385557645 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f45bc0b2a90 msgr2=0x7f45bc0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f45bc0b2a90 0x7f45bc0b4f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45d0100e30 tx=0x7f45b800bc00 comp rx=0 tx=0).stop 2024-05-14T16:59:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 -- 172.21.15.69:0/385557645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0107aa0 msgr2=0x7f45d019d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0107aa0 0x7f45d019d9f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f45c802ac10 tx=0x7f45c80316f0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 -- 172.21.15.69:0/385557645 shutdown_connections 2024-05-14T16:59:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f45bc0b2a90 0x7f45bc0b4f50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0107aa0 0x7f45d019d9f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0101ce0 0x7f45d01996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 --2- 172.21.15.69:0/385557645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d0101320 0x7f45d0199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 -- 172.21.15.69:0/385557645 >> 172.21.15.69:0/385557645 conn(0x7f45d0076cd0 msgr2=0x7f45d00fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 -- 172.21.15.69:0/385557645 shutdown_connections 2024-05-14T16:59:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T16:59:51.820+0000 7f45d62d1640 1 -- 172.21.15.69:0/385557645 wait complete. 2024-05-14T16:59:54.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:51 smithi069 ceph-mon[31507]: pgmap v311: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-05-14T16:59:54.617 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:54 smithi069 ceph-mon[31507]: pgmap v312: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 34 MiB/s wr, 1.96k op/s 2024-05-14T16:59:54.624 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:51 smithi203 ceph-mon[35787]: pgmap v311: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-05-14T16:59:54.630 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:54 smithi203 ceph-mon[35787]: pgmap v312: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 34 MiB/s wr, 1.96k op/s 2024-05-14T16:59:54.635 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:51 smithi092 ceph-mon[35763]: pgmap v311: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-05-14T16:59:54.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:54 smithi092 ceph-mon[35763]: pgmap v312: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 34 MiB/s wr, 1.96k op/s 2024-05-14T16:59:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 565.0 s, 2644.2 tps, lat 37.359 ms stddev 41.623 2024-05-14T16:59:56.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:56 smithi069 ceph-mon[31507]: pgmap v313: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2024-05-14T16:59:56.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:56 smithi203 ceph-mon[35787]: pgmap v313: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2024-05-14T16:59:56.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:56 smithi092 ceph-mon[35763]: pgmap v313: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2024-05-14T16:59:58.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 16:59:58 smithi069 ceph-mon[31507]: pgmap v314: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 34 MiB/s wr, 2.02k op/s 2024-05-14T16:59:58.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 16:59:58 smithi203 ceph-mon[35787]: pgmap v314: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 34 MiB/s wr, 2.02k op/s 2024-05-14T16:59:58.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 16:59:58 smithi092 ceph-mon[35763]: pgmap v314: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 34 MiB/s wr, 2.02k op/s 2024-05-14T17:00:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 570.0 s, 2635.8 tps, lat 38.291 ms stddev 43.572 2024-05-14T17:00:00.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:00 smithi069 ceph-mon[31507]: pgmap v315: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 927 op/s 2024-05-14T17:00:00.487 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:00 smithi069 ceph-mon[31507]: overall HEALTH_OK 2024-05-14T17:00:00.566 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:00 smithi092 ceph-mon[35763]: pgmap v315: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 927 op/s 2024-05-14T17:00:00.573 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:00 smithi092 ceph-mon[35763]: overall HEALTH_OK 2024-05-14T17:00:00.579 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:00 smithi203 ceph-mon[35787]: pgmap v315: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 927 op/s 2024-05-14T17:00:00.585 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:00 smithi203 ceph-mon[35787]: overall HEALTH_OK 2024-05-14T17:00:02.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:02 smithi069 ceph-mon[31507]: pgmap v316: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 930 op/s 2024-05-14T17:00:02.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:02 smithi203 ceph-mon[35787]: pgmap v316: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 930 op/s 2024-05-14T17:00:02.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:02 smithi092 ceph-mon[35763]: pgmap v316: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 930 op/s 2024-05-14T17:00:04.279 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:04 smithi069 ceph-mon[31507]: pgmap v317: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 20 MiB/s wr, 1.20k op/s 2024-05-14T17:00:04.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:04 smithi203 ceph-mon[35787]: pgmap v317: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 20 MiB/s wr, 1.20k op/s 2024-05-14T17:00:04.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:04 smithi092 ceph-mon[35763]: pgmap v317: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 20 MiB/s wr, 1.20k op/s 2024-05-14T17:00:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.943+0000 7f3ab5151640 1 -- 172.21.15.69:0/475115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102a30 msgr2=0x7f3ab0102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.943+0000 7f3ab5151640 1 --2- 172.21.15.69:0/475115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102a30 0x7f3ab0102eb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009930 tx=0x7f3a9802a420 comp rx=0 tx=0).stop 2024-05-14T17:00:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.943+0000 7f3ab5151640 1 -- 172.21.15.69:0/475115 shutdown_connections 2024-05-14T17:00:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.943+0000 7f3ab5151640 1 --2- 172.21.15.69:0/475115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab01033f0 0x7f3ab010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.943+0000 7f3ab5151640 1 --2- 172.21.15.69:0/475115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102a30 0x7f3ab0102eb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.943+0000 7f3ab5151640 1 --2- 172.21.15.69:0/475115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0108a40 0x7f3ab0108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.943+0000 7f3ab5151640 1 -- 172.21.15.69:0/475115 >> 172.21.15.69:0/475115 conn(0x7f3ab00fe500 msgr2=0x7f3ab0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.943+0000 7f3ab5151640 1 -- 172.21.15.69:0/475115 shutdown_connections 2024-05-14T17:00:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.943+0000 7f3ab5151640 1 -- 172.21.15.69:0/475115 wait complete. 2024-05-14T17:00:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.944+0000 7f3ab5151640 1 Processor -- start 2024-05-14T17:00:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3ab5151640 1 -- start start 2024-05-14T17:00:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3ab5151640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102a30 0x7f3ab0199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3ab5151640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab01033f0 0x7f3ab0199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3ab5151640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab0108a40 0x7f3ab019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3ab5151640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab0112d50 con 0x7f3ab0102a30 2024-05-14T17:00:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3ab5151640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab0112bd0 con 0x7f3ab01033f0 2024-05-14T17:00:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3ab5151640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab0112ed0 con 0x7f3ab0108a40 2024-05-14T17:00:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab01033f0 0x7f3ab0199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab01033f0 0x7f3ab0199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:52498/0 (socket says 172.21.15.69:52498) 2024-05-14T17:00:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3aa7fff640 1 -- 172.21.15.69:0/3483662565 learned_addr learned my addr 172.21.15.69:0/3483662565 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3aaf59e640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab0108a40 0x7f3ab019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.945+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102a30 0x7f3ab0199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.946+0000 7f3aa7fff640 1 -- 172.21.15.69:0/3483662565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab0108a40 msgr2=0x7f3ab019d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.946+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab0108a40 0x7f3ab019d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.946+0000 7f3aa7fff640 1 -- 172.21.15.69:0/3483662565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102a30 msgr2=0x7f3ab0199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.946+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102a30 0x7f3ab0199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.946+0000 7f3aa7fff640 1 -- 172.21.15.69:0/3483662565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab019e000 con 0x7f3ab01033f0 2024-05-14T17:00:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.946+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab01033f0 0x7f3ab0199550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3a9802a940 tx=0x7f3a98005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.946+0000 7f3aacd99640 1 -- 172.21.15.69:0/3483662565 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98035030 con 0x7f3ab01033f0 2024-05-14T17:00:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.946+0000 7f3aacd99640 1 -- 172.21.15.69:0/3483662565 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a98041070 con 0x7f3ab01033f0 2024-05-14T17:00:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.946+0000 7f3ab5151640 1 -- 172.21.15.69:0/3483662565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab019e290 con 0x7f3ab01033f0 2024-05-14T17:00:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.946+0000 7f3aacd99640 1 -- 172.21.15.69:0/3483662565 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98030d20 con 0x7f3ab01033f0 2024-05-14T17:00:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.947+0000 7f3ab5151640 1 -- 172.21.15.69:0/3483662565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab0074c80 con 0x7f3ab01033f0 2024-05-14T17:00:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.951+0000 7f3ab7e6c740 1 Processor -- start 2024-05-14T17:00:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.951+0000 7f3aacd99640 1 -- 172.21.15.69:0/3483662565 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3a980035f0 con 0x7f3ab01033f0 2024-05-14T17:00:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.951+0000 7f3ab7e6c740 1 -- start start 2024-05-14T17:00:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea6e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea78da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea7b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ea6e9d8c30 con 0x55ea6ea6ea20 2024-05-14T17:00:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ea6e9c87d0 con 0x55ea6ea792e0 2024-05-14T17:00:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea6e9cc810 con 0x55ea6ea6e100 2024-05-14T17:00:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea7b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3aaf59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea6e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3aaed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea78da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3aaf59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea6e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:55014/0 (socket says 172.21.15.69:55014) 2024-05-14T17:00:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3aaf59e640 1 -- 172.21.15.69:0/2657764729 learned_addr learned my addr 172.21.15.69:0/2657764729 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3aa7fff640 1 -- 172.21.15.69:0/2657764729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 msgr2=0x55ea6ea6e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/2657764729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea6e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3aa7fff640 1 -- 172.21.15.69:0/2657764729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea6ea20 msgr2=0x55ea6ea78da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/2657764729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea78da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.952+0000 7f3aa7fff640 1 -- 172.21.15.69:0/2657764729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea6e6dd210 con 0x55ea6ea792e0 2024-05-14T17:00:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.953+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/2657764729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea7b6d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3a981007d0 tx=0x7f3a9810b1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1635aeb763ecbffa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.953+0000 7f3aa57fa640 1 -- 172.21.15.69:0/2657764729 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9810bdc0 con 0x55ea6ea792e0 2024-05-14T17:00:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.953+0000 7f3aa57fa640 1 -- 172.21.15.69:0/2657764729 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a98110070 con 0x55ea6ea792e0 2024-05-14T17:00:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.953+0000 7f3aa57fa640 1 -- 172.21.15.69:0/2657764729 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98103740 con 0x55ea6ea792e0 2024-05-14T17:00:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.954+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2657764729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 msgr2=0x55ea6ea7b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.954+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2657764729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea7b6d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3a981007d0 tx=0x7f3a9810b1b0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.954+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2657764729 shutdown_connections 2024-05-14T17:00:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.954+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2657764729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea7b6d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.954+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2657764729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea78da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.954+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2657764729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea6e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.954+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2657764729 >> 172.21.15.69:0/2657764729 conn(0x55ea6ea6a610 msgr2=0x55ea6ea6a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.954+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2657764729 shutdown_connections 2024-05-14T17:00:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.954+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2657764729 wait complete. 2024-05-14T17:00:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3aacd99640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3a9c0b29a0 0x7f3a9c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3ab7e6c740 1 -- start start 2024-05-14T17:00:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3a9c0b29a0 0x7f3a9c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3aacd99640 1 -- 172.21.15.69:0/3483662565 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f3a98102fe0 con 0x7f3ab01033f0 2024-05-14T17:00:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3ab7e6c740 1 Processor -- start 2024-05-14T17:00:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3a9c0b29a0 0x7f3a9c0b4e60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55ea6ea6f6a0 tx=0x7f3aa0005f30 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6e100 0x55ea6ea78c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea75f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ea6e6ae790 con 0x55ea6ea792e0 2024-05-14T17:00:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ea6e984890 con 0x55ea6ea6e100 2024-05-14T17:00:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea6ea7bcd0 con 0x55ea6ea6ea20 2024-05-14T17:00:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3ab7e6c740 10 client.0 _ll_register_callbacks cb 0x55ea6e987d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3aaf59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3aaf59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea764d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39450/0 (socket says 172.21.15.69:39450) 2024-05-14T17:00:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3aaf59e640 1 -- 172.21.15.69:0/87102670 learned_addr learned my addr 172.21.15.69:0/87102670 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6e100 0x55ea6ea78c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.955+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea75f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3aaf59e640 1 -- 172.21.15.69:0/87102670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6ea20 msgr2=0x55ea6ea75f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3aaf59e640 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea75f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3aaf59e640 1 -- 172.21.15.69:0/87102670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6e100 msgr2=0x55ea6ea78c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3aaf59e640 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6e100 0x55ea6ea78c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3aaf59e640 1 -- 172.21.15.69:0/87102670 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea6e7a9620 con 0x55ea6ea792e0 2024-05-14T17:00:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea75f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6e100 0x55ea6ea78c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3aaf59e640 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea764d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8011d10 tx=0x7f3aa800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c5e067e3098407 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3a967fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:00:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3a967fc640 1 -- 172.21.15.69:0/87102670 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800fe20 con 0x55ea6ea792e0 2024-05-14T17:00:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3a967fc640 1 -- 172.21.15.69:0/87102670 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f3aa8017050 con 0x55ea6ea792e0 2024-05-14T17:00:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.956+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/87102670 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a98110070 con 0x55ea6ea792e0 2024-05-14T17:00:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3a967fc640 1 -- 172.21.15.69:0/87102670 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3aa8027050 con 0x55ea6ea792e0 2024-05-14T17:00:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3ab7e6c740 10 client.16494 fetch_fsmap learned FSMap version 45 2024-05-14T17:00:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/87102670 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ea6ea877b0 con 0x55ea6ea792e0 2024-05-14T17:00:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3a967fc640 1 -- 172.21.15.69:0/87102670 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a98110070 con 0x55ea6ea792e0 2024-05-14T17:00:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3a967fc640 1 -- 172.21.15.69:0/87102670 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 45) ==== 8342+0+0 (secure 0 0 0) 0x7f3aa8022030 con 0x55ea6ea792e0 2024-05-14T17:00:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3ab7e6c740 10 client.16494 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:00:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3ab7e6c740 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3ab7e6c740 20 client.16494 populate_metadata read hostname 'smithi069' 2024-05-14T17:00:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ea6ea88ce0 0x55ea6ea8b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3ab7e6c740 4 client.16494 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/87102670 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea6e9dab80 con 0x55ea6ea88ce0 2024-05-14T17:00:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.957+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ea6ea88ce0 0x55ea6ea8b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.958+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ea6ea88ce0 0x55ea6ea8b0d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e017585eb91f85a server_cookie=9ee9d77998de25d1 in_seq=0 out_seq=0 2024-05-14T17:00:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.958+0000 7f3a967fc640 10 client.16494 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3a967fc640 1 -- 172.21.15.69:0/87102670 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea6e9dab80 con 0x55ea6ea88ce0 2024-05-14T17:00:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3a967fc640 10 client.16494 handle_command_reply: tid=0 2024-05-14T17:00:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 client.16494 shutdown 2024-05-14T17:00:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/87102670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 msgr2=0x55ea6ea764d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea764d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8011d10 tx=0x7f3aa800f790 comp rx=0 tx=0).stop 2024-05-14T17:00:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/87102670 shutdown_connections 2024-05-14T17:00:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ea6ea88ce0 0x55ea6ea8b0d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3a967fc640 0 client.16494 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea764d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea75f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/87102670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6e100 0x55ea6ea78c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/87102670 >> 172.21.15.69:0/87102670 conn(0x55ea6ea6a610 msgr2=0x55ea6ea6fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/87102670 shutdown_connections 2024-05-14T17:00:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.959+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/87102670 wait complete. 2024-05-14T17:00:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.960+0000 7f3ab7e6c740 20 client.16494 trim_cache size 0 max 16384 2024-05-14T17:00:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.983+0000 7f3ab7e6c740 1 Processor -- start 2024-05-14T17:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3ab7e6c740 1 -- start start 2024-05-14T17:00:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea61100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea70210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea78790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01aebd0 con 0x55ea6ea6ea20 2024-05-14T17:00:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01ac470 con 0x55ea6ea792e0 2024-05-14T17:00:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01aa300 con 0x55ea6ea6e100 2024-05-14T17:00:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3aaf59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea78790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3aaf59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea78790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:52522/0 (socket says 172.21.15.69:52522) 2024-05-14T17:00:06.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3aaf59e640 1 -- 172.21.15.69:0/2302572120 learned_addr learned my addr 172.21.15.69:0/2302572120 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.984+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/2302572120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea61100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.985+0000 7f3aaf59e640 1 -- 172.21.15.69:0/2302572120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 msgr2=0x55ea6ea61100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.985+0000 7f3aaf59e640 1 --2- 172.21.15.69:0/2302572120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea61100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.985+0000 7f3aaf59e640 1 -- 172.21.15.69:0/2302572120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea6ea20 msgr2=0x55ea6ea70210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.985+0000 7f3aaf59e640 1 --2- 172.21.15.69:0/2302572120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea70210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.985+0000 7f3aaf59e640 1 -- 172.21.15.69:0/2302572120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea6ea6e810 con 0x55ea6ea792e0 2024-05-14T17:00:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.985+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/2302572120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea61100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.985+0000 7f3aaf59e640 1 --2- 172.21.15.69:0/2302572120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea78790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800ffb0 tx=0x7f3aa8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d073a7ce70509380 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.985+0000 7f3aa4ff9640 1 -- 172.21.15.69:0/2302572120 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa801a3f0 con 0x55ea6ea792e0 2024-05-14T17:00:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.985+0000 7f3aa4ff9640 1 -- 172.21.15.69:0/2302572120 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa801a970 con 0x55ea6ea792e0 2024-05-14T17:00:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.985+0000 7f3aa4ff9640 1 -- 172.21.15.69:0/2302572120 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800e940 con 0x55ea6ea792e0 2024-05-14T17:00:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.986+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2302572120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 msgr2=0x55ea6ea78790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.986+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2302572120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea78790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800ffb0 tx=0x7f3aa8009c20 comp rx=0 tx=0).stop 2024-05-14T17:00:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.986+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2302572120 shutdown_connections 2024-05-14T17:00:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.986+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2302572120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea792e0 0x55ea6ea78790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.987+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2302572120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea70210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.987+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2302572120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea61100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.987+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2302572120 >> 172.21.15.69:0/2302572120 conn(0x55ea6ea6a610 msgr2=0x55ea6ea6fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.987+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2302572120 shutdown_connections 2024-05-14T17:00:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.987+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2302572120 wait complete. 2024-05-14T17:00:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3ab7e6c740 1 -- start start 2024-05-14T17:00:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3ab7e6c740 1 Processor -- start 2024-05-14T17:00:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea81420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:06.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea72a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3ab7e6c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea72f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01aa950 con 0x55ea6ea792e0 2024-05-14T17:00:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01abda0 con 0x55ea6ea6ea20 2024-05-14T17:00:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3ab7e6c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab01ab000 con 0x55ea6ea6e100 2024-05-14T17:00:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3ab7e6c740 10 client.0 _ll_register_callbacks cb 0x7f3aa8011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3aaf59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea72f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea72a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea72a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:52534/0 (socket says 172.21.15.69:52534) 2024-05-14T17:00:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3aaf59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea72f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39478/0 (socket says 172.21.15.69:39478) 2024-05-14T17:00:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3aa7fff640 1 -- 172.21.15.69:0/2787953827 learned_addr learned my addr 172.21.15.69:0/2787953827 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.988+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea81420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3aa7fff640 1 -- 172.21.15.69:0/2787953827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 msgr2=0x55ea6ea81420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea81420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3aa7fff640 1 -- 172.21.15.69:0/2787953827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 msgr2=0x55ea6ea72f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea72f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3aa7fff640 1 -- 172.21.15.69:0/2787953827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea6ea73760 con 0x55ea6ea6ea20 2024-05-14T17:00:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3aa7fff640 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea72a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3a98100830 tx=0x7f3a9803ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62dec5a5938a8000 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3a967fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:00:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3a967fc640 1 -- 172.21.15.69:0/2787953827 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a980ca620 con 0x55ea6ea6ea20 2024-05-14T17:00:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3a967fc640 1 -- 172.21.15.69:0/2787953827 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f3a980ca9f0 con 0x55ea6ea6ea20 2024-05-14T17:00:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2787953827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa801a970 con 0x55ea6ea6ea20 2024-05-14T17:00:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3a967fc640 1 -- 172.21.15.69:0/2787953827 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3a980cded0 con 0x55ea6ea6ea20 2024-05-14T17:00:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3ab7e6c740 10 client.25946 fetch_fsmap learned FSMap version 45 2024-05-14T17:00:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2787953827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ea6eaf34f0 con 0x55ea6ea6ea20 2024-05-14T17:00:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3a967fc640 1 -- 172.21.15.69:0/2787953827 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa801a970 con 0x55ea6ea6ea20 2024-05-14T17:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3a967fc640 1 -- 172.21.15.69:0/2787953827 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 45) ==== 8342+0+0 (secure 0 0 0) 0x7f3a9804e030 con 0x55ea6ea6ea20 2024-05-14T17:00:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3ab7e6c740 10 client.25946 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:00:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3ab7e6c740 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3ab7e6c740 20 client.25946 populate_metadata read hostname 'smithi069' 2024-05-14T17:00:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ea6ea88ce0 0x55ea6eaf5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3ab7e6c740 4 client.25946 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:00:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.989+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2787953827 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea6e9d4360 con 0x55ea6ea88ce0 2024-05-14T17:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.990+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ea6ea88ce0 0x55ea6eaf5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.990+0000 7f3a967fc640 10 client.25946 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:04.990+0000 7f3aaed9d640 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ea6ea88ce0 0x55ea6eaf5ba0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0cb24064e4f4c8 server_cookie=a3c9e5e529035864 in_seq=0 out_seq=0 2024-05-14T17:00:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.040+0000 7f3a967fc640 1 -- 172.21.15.69:0/2787953827 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ea6e9d4360 con 0x55ea6ea88ce0 2024-05-14T17:00:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.040+0000 7f3a967fc640 10 client.25946 handle_command_reply: tid=0 2024-05-14T17:00:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.040+0000 7f3ab7e6c740 1 client.25946 shutdown 2024-05-14T17:00:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.043+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2787953827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6ea20 msgr2=0x55ea6ea72a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.043+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea72a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3a98100830 tx=0x7f3a9803ded0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.043+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2787953827 shutdown_connections 2024-05-14T17:00:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.043+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ea6ea88ce0 0x55ea6eaf5ba0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.043+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ea6ea792e0 0x55ea6ea72f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.043+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ea6ea6ea20 0x55ea6ea72a10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.043+0000 7f3ab7e6c740 1 --2- 172.21.15.69:0/2787953827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea6ea6e100 0x55ea6ea81420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.043+0000 7f3a967fc640 0 client.25946 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.043+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2787953827 >> 172.21.15.69:0/2787953827 conn(0x55ea6ea6a610 msgr2=0x55ea6ea70390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.044+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2787953827 shutdown_connections 2024-05-14T17:00:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.044+0000 7f3ab7e6c740 1 -- 172.21.15.69:0/2787953827 wait complete. 2024-05-14T17:00:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.044+0000 7f3ab7e6c740 20 client.25946 trim_cache size 0 max 16384 2024-05-14T17:00:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:00:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:00:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:00:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.048+0000 7f3ab5151640 1 -- 172.21.15.69:0/3483662565 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3a9c0b29a0 msgr2=0x7f3a9c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.048+0000 7f3ab5151640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3a9c0b29a0 0x7f3a9c0b4e60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55ea6ea6f6a0 tx=0x7f3aa0005f30 comp rx=0 tx=0).stop 2024-05-14T17:00:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.048+0000 7f3ab5151640 1 -- 172.21.15.69:0/3483662565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab01033f0 msgr2=0x7f3ab0199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.048+0000 7f3ab5151640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab01033f0 0x7f3ab0199550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3a9802a940 tx=0x7f3a98005dd0 comp rx=0 tx=0).stop 2024-05-14T17:00:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.048+0000 7f3ab5151640 1 -- 172.21.15.69:0/3483662565 shutdown_connections 2024-05-14T17:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.048+0000 7f3ab5151640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3a9c0b29a0 0x7f3a9c0b4e60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.049+0000 7f3ab5151640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab0108a40 0x7f3ab019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.049+0000 7f3ab5151640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab01033f0 0x7f3ab0199550 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.049+0000 7f3ab5151640 1 --2- 172.21.15.69:0/3483662565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102a30 0x7f3ab0199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.049+0000 7f3ab5151640 1 -- 172.21.15.69:0/3483662565 >> 172.21.15.69:0/3483662565 conn(0x7f3ab00fe500 msgr2=0x7f3ab00fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:07.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.049+0000 7f3ab5151640 1 -- 172.21.15.69:0/3483662565 shutdown_connections 2024-05-14T17:00:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:05.049+0000 7f3ab5151640 1 -- 172.21.15.69:0/3483662565 wait complete. 2024-05-14T17:00:07.125 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 575.0 s, 1766.0 tps, lat 56.599 ms stddev 115.528 2024-05-14T17:00:07.187 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:06 smithi069 ceph-mon[31507]: pgmap v318: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.2 MiB/s wr, 569 op/s 2024-05-14T17:00:07.194 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:06 smithi092 ceph-mon[35763]: pgmap v318: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.2 MiB/s wr, 569 op/s 2024-05-14T17:00:07.200 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:06 smithi203 ceph-mon[35787]: pgmap v318: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.2 MiB/s wr, 569 op/s 2024-05-14T17:00:07.834 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.9 2024-05-14T17:00:07.841 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:00:08.190 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:08.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:08 smithi069 ceph-mon[31507]: pgmap v319: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 26 MiB/s wr, 2.04k op/s 2024-05-14T17:00:08.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:08 smithi203 ceph-mon[35787]: pgmap v319: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 26 MiB/s wr, 2.04k op/s 2024-05-14T17:00:08.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:08 smithi092 ceph-mon[35763]: pgmap v319: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 26 MiB/s wr, 2.04k op/s 2024-05-14T17:00:10.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.246+0000 7f4b15713700 1 -- 172.21.15.69:0/3760298517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b10102440 msgr2=0x7f4b101028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:10.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.246+0000 7f4b15713700 1 --2- 172.21.15.69:0/3760298517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b10102440 0x7f4b101028c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4b040098f0 tx=0x7f4b04009c00 comp rx=0 tx=0).stop 2024-05-14T17:00:10.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.246+0000 7f4b15713700 1 -- 172.21.15.69:0/3760298517 shutdown_connections 2024-05-14T17:00:10.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.246+0000 7f4b15713700 1 --2- 172.21.15.69:0/3760298517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10108520 0x7f4b10108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:10.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.246+0000 7f4b15713700 1 --2- 172.21.15.69:0/3760298517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b10102e00 0x7f4b1010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:10.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.246+0000 7f4b15713700 1 --2- 172.21.15.69:0/3760298517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b10102440 0x7f4b101028c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:10.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.246+0000 7f4b15713700 1 -- 172.21.15.69:0/3760298517 >> 172.21.15.69:0/3760298517 conn(0x7f4b100793f0 msgr2=0x7f4b10079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:10.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.246+0000 7f4b15713700 1 -- 172.21.15.69:0/3760298517 shutdown_connections 2024-05-14T17:00:10.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.246+0000 7f4b15713700 1 -- 172.21.15.69:0/3760298517 wait complete. 2024-05-14T17:00:10.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.247+0000 7f4b15713700 1 Processor -- start 2024-05-14T17:00:10.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.247+0000 7f4b15713700 1 -- start start 2024-05-14T17:00:10.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.247+0000 7f4b15713700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10102440 0x7f4b10116ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:10.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.247+0000 7f4b15713700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b10102e00 0x7f4b10112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:10.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b15713700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b10108520 0x7f4b101132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b15713700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10111530 con 0x7f4b10102440 2024-05-14T17:00:10.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b15713700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101113b0 con 0x7f4b10108520 2024-05-14T17:00:10.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b15713700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101116b0 con 0x7f4b10102e00 2024-05-14T17:00:10.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b0effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10102440 0x7f4b10116ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:10.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b0effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10102440 0x7f4b10116ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39484/0 (socket says 172.21.15.69:39484) 2024-05-14T17:00:10.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b0effd700 1 -- 172.21.15.69:0/2620949389 learned_addr learned my addr 172.21.15.69:0/2620949389 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b0f7fe700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b10108520 0x7f4b101132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:10.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b0e7fc700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b10102e00 0x7f4b10112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:10.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b0effd700 1 -- 172.21.15.69:0/2620949389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b10102e00 msgr2=0x7f4b10112d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:10.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b0effd700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b10102e00 0x7f4b10112d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:10.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b0effd700 1 -- 172.21.15.69:0/2620949389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b10108520 msgr2=0x7f4b101132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:10.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b0effd700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b10108520 0x7f4b101132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:10.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.248+0000 7f4b0effd700 1 -- 172.21.15.69:0/2620949389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b101ac690 con 0x7f4b10102440 2024-05-14T17:00:10.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.249+0000 7f4b0effd700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10102440 0x7f4b10116ac0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000e9c0 tx=0x7f4b0000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:10.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.250+0000 7f4af7fff700 1 -- 172.21.15.69:0/2620949389 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000cbd0 con 0x7f4b10102440 2024-05-14T17:00:10.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.250+0000 7f4af7fff700 1 -- 172.21.15.69:0/2620949389 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b0000cd70 con 0x7f4b10102440 2024-05-14T17:00:10.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.250+0000 7f4af7fff700 1 -- 172.21.15.69:0/2620949389 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00005580 con 0x7f4b10102440 2024-05-14T17:00:10.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.251+0000 7f4b15713700 1 -- 172.21.15.69:0/2620949389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b101ac860 con 0x7f4b10102440 2024-05-14T17:00:10.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.251+0000 7f4b15713700 1 -- 172.21.15.69:0/2620949389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b101accc0 con 0x7f4b10102440 2024-05-14T17:00:10.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.252+0000 7f4af5ffb700 1 -- 172.21.15.69:0/2620949389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af0005330 con 0x7f4b10102440 2024-05-14T17:00:10.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.256+0000 7f4af7fff700 1 -- 172.21.15.69:0/2620949389 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4b00004750 con 0x7f4b10102440 2024-05-14T17:00:10.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.257+0000 7f4af7fff700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4af80b2b40 0x7f4af80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:10.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.257+0000 7f4af7fff700 1 -- 172.21.15.69:0/2620949389 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f4b000e2900 con 0x7f4b10102440 2024-05-14T17:00:10.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.258+0000 7f4b0e7fc700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4af80b2b40 0x7f4af80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:10.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.258+0000 7f4b0e7fc700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4af80b2b40 0x7f4af80b5000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b10114370 tx=0x7f4b04019040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.260+0000 7f4af7fff700 1 -- 172.21.15.69:0/2620949389 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b00054df0 con 0x7f4b10102440 2024-05-14T17:00:10.704 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:10 smithi092 ceph-mon[35763]: pgmap v320: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2024-05-14T17:00:10.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:10 smithi092 ceph-mon[35763]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T17:00:10.716 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:10 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:10.725 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:10 smithi203 ceph-mon[35787]: pgmap v320: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2024-05-14T17:00:10.730 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:10 smithi203 ceph-mon[35787]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T17:00:10.736 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:10 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:10.741 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 580.0 s, 2733.6 tps, lat 36.352 ms stddev 49.247 2024-05-14T17:00:10.752 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:10 smithi069 ceph-mon[31507]: pgmap v320: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2024-05-14T17:00:10.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:10 smithi069 ceph-mon[31507]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T17:00:10.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:10 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:10.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.737+0000 7f4af5ffb700 1 -- 172.21.15.69:0/2620949389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4af0005620 con 0x7f4b10102440 2024-05-14T17:00:10.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.740+0000 7f4af7fff700 1 -- 172.21.15.69:0/2620949389 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11861 (secure 0 0 0) 0x7f4b000aa320 con 0x7f4b10102440 2024-05-14T17:00:10.822 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:00:10.843 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:00:08.714890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T17:00:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.744+0000 7f4af5ffb700 1 -- 172.21.15.69:0/2620949389 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4af80b2b40 msgr2=0x7f4af80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.744+0000 7f4af5ffb700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4af80b2b40 0x7f4af80b5000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b10114370 tx=0x7f4b04019040 comp rx=0 tx=0).stop 2024-05-14T17:00:10.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.744+0000 7f4af5ffb700 1 -- 172.21.15.69:0/2620949389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10102440 msgr2=0x7f4b10116ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:10.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.744+0000 7f4af5ffb700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10102440 0x7f4b10116ac0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000e9c0 tx=0x7f4b0000ecd0 comp rx=0 tx=0).stop 2024-05-14T17:00:10.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.748+0000 7f4af5ffb700 1 -- 172.21.15.69:0/2620949389 shutdown_connections 2024-05-14T17:00:10.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.748+0000 7f4af5ffb700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4af80b2b40 0x7f4af80b5000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:10.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.748+0000 7f4af5ffb700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b10102440 0x7f4b10116ac0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:10.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.748+0000 7f4af5ffb700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b10108520 0x7f4b101132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:10.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.748+0000 7f4af5ffb700 1 --2- 172.21.15.69:0/2620949389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b10102e00 0x7f4b10112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:10.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.748+0000 7f4af5ffb700 1 -- 172.21.15.69:0/2620949389 >> 172.21.15.69:0/2620949389 conn(0x7f4b100793f0 msgr2=0x7f4b10100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:10.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.750+0000 7f4af5ffb700 1 -- 172.21.15.69:0/2620949389 shutdown_connections 2024-05-14T17:00:10.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:10.751+0000 7f4af5ffb700 1 -- 172.21.15.69:0/2620949389 wait complete. 2024-05-14T17:00:10.937 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 46 2024-05-14T17:00:11.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:11 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:00:11.469 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:11 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:11.486 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:11 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:11.527 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:11 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2620949389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:11.533 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:11 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:00:11.539 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:11 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:11.546 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:11 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:11.551 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:11 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2620949389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:11.557 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:11 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:00:11.563 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:11 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:11.568 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:11 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:11.575 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:11 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2620949389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:12.028 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 137.9 --await-for 45.0' 2024-05-14T17:00:12.034 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 137.9 --await-for 45.0 2024-05-14T17:00:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:12.458 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:12 smithi069 ceph-mon[31507]: pgmap v322: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 26 MiB/s wr, 2.10k op/s 2024-05-14T17:00:12.466 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:12 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:12.473 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:12 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:12.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:12 smithi203 ceph-mon[35787]: pgmap v322: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 26 MiB/s wr, 2.10k op/s 2024-05-14T17:00:12.542 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:12 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:12.626 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:12 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:12.633 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:12 smithi092 ceph-mon[35763]: pgmap v322: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 26 MiB/s wr, 2.10k op/s 2024-05-14T17:00:12.638 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:12 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:12.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:12 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:13.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:13 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:13.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:13 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:13.467 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:13 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.474 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:13 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.480 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:13 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.485 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:13 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:13 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:13.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:13 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:13.517 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:13 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.523 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:13 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.528 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:13 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.606 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:13 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.638 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:13 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:13.718 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:13 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:13.724 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:13 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.732 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:13 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.746 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:13 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:13.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:13 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:14.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:14 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-05-14T17:00:14.465 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:14 smithi069 ceph-mon[31507]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-05-14T17:00:14.474 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:14 smithi069 ceph-mon[31507]: pgmap v323: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 26 MiB/s wr, 2.11k op/s 2024-05-14T17:00:14.485 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:14 smithi069 ceph-mon[31507]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T17:00:14.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:14 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-05-14T17:00:14.572 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:14 smithi203 ceph-mon[35787]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-05-14T17:00:14.618 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:14 smithi203 ceph-mon[35787]: pgmap v323: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 26 MiB/s wr, 2.11k op/s 2024-05-14T17:00:14.629 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:14 smithi203 ceph-mon[35787]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T17:00:14.640 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:14 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-05-14T17:00:14.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:14 smithi092 ceph-mon[35763]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-05-14T17:00:14.664 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:14 smithi092 ceph-mon[35763]: pgmap v323: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 26 MiB/s wr, 2.11k op/s 2024-05-14T17:00:14.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:14 smithi092 ceph-mon[35763]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T17:00:15.290 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:15 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:15.396 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 585.0 s, 1564.5 tps, lat 63.071 ms stddev 75.229 2024-05-14T17:00:15.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:15 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:15.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:15 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:15.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.648+0000 7fa960082700 1 -- 172.21.15.69:0/2541253069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa958106ef0 msgr2=0x7fa95810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.648+0000 7fa960082700 1 --2- 172.21.15.69:0/2541253069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa958106ef0 0x7fa95810eac0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa95001c1d0 tx=0x7fa95001c4e0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.648+0000 7fa960082700 1 -- 172.21.15.69:0/2541253069 shutdown_connections 2024-05-14T17:00:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.648+0000 7fa960082700 1 --2- 172.21.15.69:0/2541253069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa958101e10 0x7fa9581021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.648+0000 7fa960082700 1 --2- 172.21.15.69:0/2541253069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa958106ef0 0x7fa95810eac0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.648+0000 7fa960082700 1 --2- 172.21.15.69:0/2541253069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9581027c0 0x7fa958106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.648+0000 7fa960082700 1 -- 172.21.15.69:0/2541253069 >> 172.21.15.69:0/2541253069 conn(0x7fa958078ae0 msgr2=0x7fa9580ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:15.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.649+0000 7fa960082700 1 -- 172.21.15.69:0/2541253069 shutdown_connections 2024-05-14T17:00:15.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.649+0000 7fa960082700 1 -- 172.21.15.69:0/2541253069 wait complete. 2024-05-14T17:00:15.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.649+0000 7fa960082700 1 Processor -- start 2024-05-14T17:00:15.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa960082700 1 -- start start 2024-05-14T17:00:15.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa960082700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa958101e10 0x7fa95810eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:15.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa960082700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9581027c0 0x7fa958109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:15.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa960082700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958106ef0 0x7fa95810a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:15.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa960082700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa95806eeb0 con 0x7fa958101e10 2024-05-14T17:00:15.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa960082700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa95806ed30 con 0x7fa958106ef0 2024-05-14T17:00:15.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa960082700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa95806f030 con 0x7fa9581027c0 2024-05-14T17:00:15.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa95e61f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958106ef0 0x7fa95810a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa95e61f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958106ef0 0x7fa95810a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:54822/0 (socket says 172.21.15.69:54822) 2024-05-14T17:00:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa95e61f700 1 -- 172.21.15.69:0/3680885854 learned_addr learned my addr 172.21.15.69:0/3680885854 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa95d61d700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9581027c0 0x7fa958109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.650+0000 7fa95de1e700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa958101e10 0x7fa95810eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.651+0000 7fa95e61f700 1 -- 172.21.15.69:0/3680885854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9581027c0 msgr2=0x7fa958109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:15.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.651+0000 7fa95e61f700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9581027c0 0x7fa958109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.651+0000 7fa95e61f700 1 -- 172.21.15.69:0/3680885854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa958101e10 msgr2=0x7fa95810eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:15.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.651+0000 7fa95e61f700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa958101e10 0x7fa95810eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.651+0000 7fa95e61f700 1 -- 172.21.15.69:0/3680885854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95819b3b0 con 0x7fa958106ef0 2024-05-14T17:00:15.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.651+0000 7fa95d61d700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9581027c0 0x7fa958109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:15.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.651+0000 7fa95e61f700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958106ef0 0x7fa95810a2c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa950018800 tx=0x7fa950018830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:16.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.653+0000 7fa94effd700 1 -- 172.21.15.69:0/3680885854 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950018d70 con 0x7fa958106ef0 2024-05-14T17:00:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.653+0000 7fa94effd700 1 -- 172.21.15.69:0/3680885854 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa95001a6b0 con 0x7fa958106ef0 2024-05-14T17:00:16.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.653+0000 7fa94effd700 1 -- 172.21.15.69:0/3680885854 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950005400 con 0x7fa958106ef0 2024-05-14T17:00:16.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.653+0000 7fa960082700 1 -- 172.21.15.69:0/3680885854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95819b640 con 0x7fa958106ef0 2024-05-14T17:00:16.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.653+0000 7fa960082700 1 -- 172.21.15.69:0/3680885854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95819ba40 con 0x7fa958106ef0 2024-05-14T17:00:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.657+0000 7fa94effd700 1 -- 172.21.15.69:0/3680885854 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa950022460 con 0x7fa958106ef0 2024-05-14T17:00:16.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.658+0000 7fa96a918b80 1 Processor -- start 2024-05-14T17:00:16.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.658+0000 7fa96a918b80 1 -- start start 2024-05-14T17:00:16.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.659+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75a9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.659+0000 7fa94effd700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9440b6ff0 0x7fa9440b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.659+0000 7fa94effd700 1 -- 172.21.15.69:0/3680885854 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fa9500f48b0 con 0x7fa958106ef0 2024-05-14T17:00:16.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.659+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a9750 0x5637b75a3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa95de1e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa95de1e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75a9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45614/0 (socket says 172.21.15.69:45614) 2024-05-14T17:00:16.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa96a918b80 1 -- 172.21.15.69:0/4060238397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5637b7501e20 con 0x5637b75a8e30 2024-05-14T17:00:16.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa95de1e700 1 -- 172.21.15.69:0/4060238397 learned_addr learned my addr 172.21.15.69:0/4060238397 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:16.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa96a918b80 1 -- 172.21.15.69:0/4060238397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5637b74f19c0 con 0x5637b75a9750 2024-05-14T17:00:16.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa96a918b80 1 -- 172.21.15.69:0/4060238397 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5637b74f5a00 con 0x5637b75a3b60 2024-05-14T17:00:16.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa95e61f700 1 --2- 172.21.15.69:0/4060238397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa95d61d700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9440b6ff0 0x7fa9440b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa95e61f700 1 -- 172.21.15.69:0/4060238397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a9750 msgr2=0x5637b75a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:16.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa95e61f700 1 --2- 172.21.15.69:0/4060238397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a9750 0x5637b75a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa95e61f700 1 -- 172.21.15.69:0/4060238397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 msgr2=0x5637b75a9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:16.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa95e61f700 1 --2- 172.21.15.69:0/4060238397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75a9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.660+0000 7fa95e61f700 1 -- 172.21.15.69:0/4060238397 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b7441a60 con 0x5637b75a3b60 2024-05-14T17:00:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.661+0000 7fa95e61f700 1 --2- 172.21.15.69:0/4060238397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75a3fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa9500f2080 tx=0x7fa9500bb490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78ac0f696da13505 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:16.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.661+0000 7fa95d61d700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9440b6ff0 0x7fa9440b94b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x5637b75a9b50 tx=0x7fa94800a390 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:16.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.661+0000 7fa943fff700 1 -- 172.21.15.69:0/4060238397 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9500c43b0 con 0x5637b75a3b60 2024-05-14T17:00:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.661+0000 7fa943fff700 1 -- 172.21.15.69:0/4060238397 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9500bb7b0 con 0x5637b75a3b60 2024-05-14T17:00:16.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.661+0000 7fa943fff700 1 -- 172.21.15.69:0/4060238397 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95001acf0 con 0x5637b75a3b60 2024-05-14T17:00:16.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.662+0000 7fa96a918b80 1 -- 172.21.15.69:0/4060238397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 msgr2=0x5637b75a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.662+0000 7fa96a918b80 1 --2- 172.21.15.69:0/4060238397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75a3fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa9500f2080 tx=0x7fa9500bb490 comp rx=0 tx=0).stop 2024-05-14T17:00:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.662+0000 7fa96a918b80 1 -- 172.21.15.69:0/4060238397 shutdown_connections 2024-05-14T17:00:16.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.662+0000 7fa96a918b80 1 --2- 172.21.15.69:0/4060238397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.663+0000 7fa96a918b80 1 --2- 172.21.15.69:0/4060238397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75a3fe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.663+0000 7fa96a918b80 1 --2- 172.21.15.69:0/4060238397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a9750 0x5637b75a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.663+0000 7fa96a918b80 1 -- 172.21.15.69:0/4060238397 >> 172.21.15.69:0/4060238397 conn(0x5637b74ce3e0 msgr2=0x5637b759dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:16.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.663+0000 7fa96a918b80 1 -- 172.21.15.69:0/4060238397 shutdown_connections 2024-05-14T17:00:16.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.663+0000 7fa96a918b80 1 -- 172.21.15.69:0/4060238397 wait complete. 2024-05-14T17:00:16.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.663+0000 7fa96a918b80 1 -- start start 2024-05-14T17:00:16.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.663+0000 7fa96a918b80 1 Processor -- start 2024-05-14T17:00:16.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a3b60 0x5637b75af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a9750 0x5637b75ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa96a918b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5637b73f8130 con 0x5637b75a8e30 2024-05-14T17:00:16.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa96a918b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5637b7493950 con 0x5637b75a3b60 2024-05-14T17:00:16.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa96a918b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5637b74f5a00 con 0x5637b75a9750 2024-05-14T17:00:16.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa96a918b80 10 client.0 _ll_register_callbacks cb 0x5637b7488710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:16.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa95d61d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa95d61d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45628/0 (socket says 172.21.15.69:45628) 2024-05-14T17:00:16.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa95d61d700 1 -- 172.21.15.69:0/1312751462 learned_addr learned my addr 172.21.15.69:0/1312751462 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa95de1e700 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a3b60 0x5637b75af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.664+0000 7fa95e61f700 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a9750 0x5637b75ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.665+0000 7fa95de1e700 1 -- 172.21.15.69:0/1312751462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a9750 msgr2=0x5637b75ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.665+0000 7fa95de1e700 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a9750 0x5637b75ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.665+0000 7fa95de1e700 1 -- 172.21.15.69:0/1312751462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 msgr2=0x5637b75ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:16.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.665+0000 7fa95de1e700 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.665+0000 7fa95de1e700 1 -- 172.21.15.69:0/1312751462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b7490820 con 0x5637b75a3b60 2024-05-14T17:00:16.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.665+0000 7fa95de1e700 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a3b60 0x5637b75af540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa9540126d0 tx=0x7fa954012a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0383dea77f44b5b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:16.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.666+0000 7fa9417fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:00:16.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.666+0000 7fa9417fa700 1 -- 172.21.15.69:0/1312751462 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95400f540 con 0x5637b75a3b60 2024-05-14T17:00:16.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.666+0000 7fa9417fa700 1 -- 172.21.15.69:0/1312751462 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fa95400f6e0 con 0x5637b75a3b60 2024-05-14T17:00:16.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.666+0000 7fa96a918b80 1 -- 172.21.15.69:0/1312751462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9500bb7b0 con 0x5637b75a3b60 2024-05-14T17:00:16.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.666+0000 7fa9417fa700 1 -- 172.21.15.69:0/1312751462 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa95400f8e0 con 0x5637b75a3b60 2024-05-14T17:00:16.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.666+0000 7fa96a918b80 10 client.25958 fetch_fsmap learned FSMap version 47 2024-05-14T17:00:16.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.666+0000 7fa96a918b80 1 -- 172.21.15.69:0/1312751462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5637b75ba370 con 0x5637b75a3b60 2024-05-14T17:00:16.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.666+0000 7fa9417fa700 1 -- 172.21.15.69:0/1312751462 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9500bb7b0 con 0x5637b75a3b60 2024-05-14T17:00:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.667+0000 7fa9417fa700 1 -- 172.21.15.69:0/1312751462 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fa954005020 con 0x5637b75a3b60 2024-05-14T17:00:16.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.667+0000 7fa96a918b80 10 client.25958 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:00:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.667+0000 7fa96a918b80 10 client.25958 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:00:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.667+0000 7fa96a918b80 20 client.25958 populate_metadata read hostname 'smithi069' 2024-05-14T17:00:16.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.667+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5637b75bc9b0 0x5637b75bee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.667+0000 7fa96a918b80 4 client.25958 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:16.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.667+0000 7fa96a918b80 1 -- 172.21.15.69:0/1312751462 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637b756f4f0 con 0x5637b75bc9b0 2024-05-14T17:00:16.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.667+0000 7fa95d61d700 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5637b75bc9b0 0x5637b75bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.668+0000 7fa9417fa700 10 client.25958 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:17.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.668+0000 7fa95d61d700 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5637b75bc9b0 0x5637b75bee00 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f61c54afa39c0b4 server_cookie=31ede2c329ce0be1 in_seq=0 out_seq=0 2024-05-14T17:00:17.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.668+0000 7fa9417fa700 1 -- 172.21.15.69:0/1312751462 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5637b756f4f0 con 0x5637b75bc9b0 2024-05-14T17:00:17.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.668+0000 7fa9417fa700 10 client.25958 handle_command_reply: tid=0 2024-05-14T17:00:17.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.669+0000 7fa96a918b80 1 client.25958 shutdown 2024-05-14T17:00:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.671+0000 7fa96a918b80 1 -- 172.21.15.69:0/1312751462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a3b60 msgr2=0x5637b75af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:17.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.671+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a3b60 0x5637b75af540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa9540126d0 tx=0x7fa954012a90 comp rx=0 tx=0).stop 2024-05-14T17:00:17.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.671+0000 7fa96a918b80 1 -- 172.21.15.69:0/1312751462 shutdown_connections 2024-05-14T17:00:17.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.671+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5637b75bc9b0 0x5637b75bee00 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.671+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a9750 0x5637b75ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.671+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a3b60 0x5637b75af540 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.671+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1312751462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.671+0000 7fa96a918b80 1 -- 172.21.15.69:0/1312751462 >> 172.21.15.69:0/1312751462 conn(0x5637b74ce3e0 msgr2=0x5637b7598860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:17.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.671+0000 7fa9417fa700 0 client.25958 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:17.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.672+0000 7fa96a918b80 1 -- 172.21.15.69:0/1312751462 shutdown_connections 2024-05-14T17:00:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.672+0000 7fa96a918b80 1 -- 172.21.15.69:0/1312751462 wait complete. 2024-05-14T17:00:17.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.672+0000 7fa96a918b80 20 client.25958 trim_cache size 0 max 16384 2024-05-14T17:00:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa96a918b80 1 Processor -- start 2024-05-14T17:00:17.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa96a918b80 1 -- start start 2024-05-14T17:00:17.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a8e30 0x5637b7598970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:17.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a9750 0x5637b75aa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:17.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa96a918b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581a5c90 con 0x5637b75a9750 2024-05-14T17:00:17.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa96a918b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581a3490 con 0x5637b75a8e30 2024-05-14T17:00:17.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa96a918b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581a1300 con 0x5637b75a3b60 2024-05-14T17:00:17.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95e61f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a9750 0x5637b75aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95e61f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a9750 0x5637b75aa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45634/0 (socket says 172.21.15.69:45634) 2024-05-14T17:00:17.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95e61f700 1 -- 172.21.15.69:0/3545292777 learned_addr learned my addr 172.21.15.69:0/3545292777 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:17.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95de1e700 1 --2- 172.21.15.69:0/3545292777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95d61d700 1 --2- 172.21.15.69:0/3545292777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a8e30 0x5637b7598970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95de1e700 1 -- 172.21.15.69:0/3545292777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a8e30 msgr2=0x5637b7598970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:17.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95de1e700 1 --2- 172.21.15.69:0/3545292777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a8e30 0x5637b7598970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95de1e700 1 -- 172.21.15.69:0/3545292777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a9750 msgr2=0x5637b75aa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:17.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95de1e700 1 --2- 172.21.15.69:0/3545292777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a9750 0x5637b75aa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95de1e700 1 -- 172.21.15.69:0/3545292777 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b75bf480 con 0x5637b75a3b60 2024-05-14T17:00:17.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.698+0000 7fa95e61f700 1 --2- 172.21.15.69:0/3545292777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a9750 0x5637b75aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:17.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.699+0000 7fa95de1e700 1 --2- 172.21.15.69:0/3545292777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75b8ca0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa9540126d0 tx=0x7fa954012ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63e2ca2ded05685a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:17.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.699+0000 7fa9437fe700 1 -- 172.21.15.69:0/3545292777 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95400fa80 con 0x5637b75a3b60 2024-05-14T17:00:17.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.699+0000 7fa9437fe700 1 -- 172.21.15.69:0/3545292777 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa95400fc20 con 0x5637b75a3b60 2024-05-14T17:00:17.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.699+0000 7fa9437fe700 1 -- 172.21.15.69:0/3545292777 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95401c420 con 0x5637b75a3b60 2024-05-14T17:00:17.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 -- 172.21.15.69:0/3545292777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 msgr2=0x5637b75b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:17.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 --2- 172.21.15.69:0/3545292777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75b8ca0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa9540126d0 tx=0x7fa954012ed0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 -- 172.21.15.69:0/3545292777 shutdown_connections 2024-05-14T17:00:17.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 --2- 172.21.15.69:0/3545292777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a9750 0x5637b75aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 --2- 172.21.15.69:0/3545292777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75b8ca0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 --2- 172.21.15.69:0/3545292777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a8e30 0x5637b7598970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 -- 172.21.15.69:0/3545292777 >> 172.21.15.69:0/3545292777 conn(0x5637b74ce3e0 msgr2=0x5637b75a9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:17.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 -- 172.21.15.69:0/3545292777 shutdown_connections 2024-05-14T17:00:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 -- 172.21.15.69:0/3545292777 wait complete. 2024-05-14T17:00:17.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 -- start start 2024-05-14T17:00:17.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 Processor -- start 2024-05-14T17:00:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.700+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:17.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa96a918b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a9750 0x5637b75b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa96a918b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581a1950 con 0x5637b75a8e30 2024-05-14T17:00:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa96a918b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581a2dc0 con 0x5637b75a9750 2024-05-14T17:00:17.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa96a918b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581a2020 con 0x5637b75a3b60 2024-05-14T17:00:17.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa96a918b80 10 client.0 _ll_register_callbacks cb 0x5637b7488710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:17.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95d61d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95d61d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45648/0 (socket says 172.21.15.69:45648) 2024-05-14T17:00:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95d61d700 1 -- 172.21.15.69:0/1573337643 learned_addr learned my addr 172.21.15.69:0/1573337643 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95de1e700 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95e61f700 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a9750 0x5637b75b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95d61d700 1 -- 172.21.15.69:0/1573337643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 msgr2=0x5637b75b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:17.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95d61d700 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95d61d700 1 -- 172.21.15.69:0/1573337643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a9750 msgr2=0x5637b75b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:17.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95d61d700 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a9750 0x5637b75b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95d61d700 1 -- 172.21.15.69:0/1573337643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b75b29f0 con 0x5637b75a8e30 2024-05-14T17:00:17.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa95d61d700 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75b5740 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x5637b75adeb0 tx=0x7fa948005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d347a26cbb3483 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:17.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa9417fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:00:17.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa9417fa700 1 -- 172.21.15.69:0/1573337643 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948025450 con 0x5637b75a8e30 2024-05-14T17:00:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa9417fa700 1 -- 172.21.15.69:0/1573337643 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fa9480255f0 con 0x5637b75a8e30 2024-05-14T17:00:17.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.701+0000 7fa96a918b80 1 -- 172.21.15.69:0/1573337643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa95400fc20 con 0x5637b75a8e30 2024-05-14T17:00:17.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa9417fa700 1 -- 172.21.15.69:0/1573337643 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa9480257f0 con 0x5637b75a8e30 2024-05-14T17:00:17.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa96a918b80 10 client.16521 fetch_fsmap learned FSMap version 47 2024-05-14T17:00:17.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa96a918b80 1 -- 172.21.15.69:0/1573337643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5637b7627f10 con 0x5637b75a8e30 2024-05-14T17:00:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa9417fa700 1 -- 172.21.15.69:0/1573337643 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa95400fc20 con 0x5637b75a8e30 2024-05-14T17:00:18.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa9417fa700 1 -- 172.21.15.69:0/1573337643 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fa948025ce0 con 0x5637b75a8e30 2024-05-14T17:00:18.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa96a918b80 10 client.16521 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:00:18.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa96a918b80 10 client.16521 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:00:18.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa96a918b80 20 client.16521 populate_metadata read hostname 'smithi069' 2024-05-14T17:00:19.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5637b7628740 0x5637b762ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:19.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa96a918b80 4 client.16521 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.9, "await_for": 45.0}] 2024-05-14T17:00:19.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa96a918b80 1 -- 172.21.15.69:0/1573337643 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.9, "await_for": 45.0}) -- 0x5637b7503d70 con 0x5637b7628740 2024-05-14T17:00:19.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.702+0000 7fa95de1e700 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5637b7628740 0x5637b762ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:19.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.703+0000 7fa9417fa700 10 client.16521 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:19.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:15.703+0000 7fa95de1e700 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5637b7628740 0x5637b762ab60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4df66fbda4a23b0 server_cookie=fa05089e58e16be9 in_seq=0 out_seq=0 2024-05-14T17:00:19.562 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:16 smithi069 ceph-mon[31507]: pgmap v325: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 428 op/s 2024-05-14T17:00:19.570 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:16 smithi092 ceph-mon[35763]: pgmap v325: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 428 op/s 2024-05-14T17:00:19.579 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: pgmap v326: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.6 MiB/s wr, 806 op/s 2024-05-14T17:00:19.585 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:19.590 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:19.636 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:19.723 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:19.733 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:19.743 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:19.756 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:19.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:19.816 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:19.901 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:19.914 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:19.926 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:19.940 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:00:19.953 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:00:19.966 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:19.975 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:18 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-14T17:00:19.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:18 smithi092 ceph-mon[35763]: Unable to set osd_memory_target on smithi069 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-14T17:00:19.995 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:18 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-14T17:00:20.006 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:18 smithi092 ceph-mon[35763]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-14T17:00:20.032 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:16 smithi203 ceph-mon[35787]: pgmap v325: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 428 op/s 2024-05-14T17:00:20.082 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: pgmap v326: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.6 MiB/s wr, 806 op/s 2024-05-14T17:00:20.091 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:20.097 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:20.108 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:20.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:20.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:20.198 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:20.207 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:20.215 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:20.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:20.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:20.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:20.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:20.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:00:20.269 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:00:20.274 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:20.312 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:18 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-14T17:00:20.373 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:18 smithi203 ceph-mon[35787]: Unable to set osd_memory_target on smithi069 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-14T17:00:20.381 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:18 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-14T17:00:20.388 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:18 smithi203 ceph-mon[35787]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-14T17:00:20.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:19 smithi092 ceph-mon[35763]: pgmap v327: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.9 MiB/s wr, 661 op/s 2024-05-14T17:00:20.427 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:19 smithi203 ceph-mon[35787]: pgmap v327: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.9 MiB/s wr, 661 op/s 2024-05-14T17:00:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 -- 172.21.15.69:0/3662998006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd388101ca0 msgr2=0x7fd388102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 --2- 172.21.15.69:0/3662998006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd388101ca0 0x7fd388102120 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009900 tx=0x7fd37c02a410 comp rx=0 tx=0).stop 2024-05-14T17:00:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 -- 172.21.15.69:0/3662998006 shutdown_connections 2024-05-14T17:00:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 --2- 172.21.15.69:0/3662998006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd388102660 0x7fd38810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 --2- 172.21.15.69:0/3662998006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd388101ca0 0x7fd388102120 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 --2- 172.21.15.69:0/3662998006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3881068f0 0x7fd388106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 -- 172.21.15.69:0/3662998006 >> 172.21.15.69:0/3662998006 conn(0x7fd388077750 msgr2=0x7fd388077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 -- 172.21.15.69:0/3662998006 shutdown_connections 2024-05-14T17:00:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 -- 172.21.15.69:0/3662998006 wait complete. 2024-05-14T17:00:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 Processor -- start 2024-05-14T17:00:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.152+0000 7fd38e846640 1 -- start start 2024-05-14T17:00:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd38e846640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd388101ca0 0x7fd38819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd38e846640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd388102660 0x7fd38819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd38e846640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3881068f0 0x7fd38819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd38e846640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd388114ef0 con 0x7fd388102660 2024-05-14T17:00:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd38e846640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd388114d70 con 0x7fd388101ca0 2024-05-14T17:00:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd38e846640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd388115070 con 0x7fd3881068f0 2024-05-14T17:00:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd388102660 0x7fd38819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd388102660 0x7fd38819b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45666/0 (socket says 172.21.15.69:45666) 2024-05-14T17:00:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3877fe640 1 -- 172.21.15.69:0/3660606531 learned_addr learned my addr 172.21.15.69:0/3660606531 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd38cde3640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3881068f0 0x7fd38819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd387fff640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd388101ca0 0x7fd38819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3877fe640 1 -- 172.21.15.69:0/3660606531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3881068f0 msgr2=0x7fd38819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3877fe640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3881068f0 0x7fd38819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3877fe640 1 -- 172.21.15.69:0/3660606531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd388101ca0 msgr2=0x7fd38819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3877fe640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd388101ca0 0x7fd38819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3877fe640 1 -- 172.21.15.69:0/3660606531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3881a0320 con 0x7fd388102660 2024-05-14T17:00:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3877fe640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd388102660 0x7fd38819b870 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd37c005940 tx=0x7fd37c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3857fa640 1 -- 172.21.15.69:0/3660606531 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c03f030 con 0x7fd388102660 2024-05-14T17:00:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd3857fa640 1 -- 172.21.15.69:0/3660606531 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd37c003cb0 con 0x7fd388102660 2024-05-14T17:00:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.153+0000 7fd38e846640 1 -- 172.21.15.69:0/3660606531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3881a05b0 con 0x7fd388102660 2024-05-14T17:00:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.154+0000 7fd3857fa640 1 -- 172.21.15.69:0/3660606531 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c035070 con 0x7fd388102660 2024-05-14T17:00:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.154+0000 7fd38e846640 1 -- 172.21.15.69:0/3660606531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881a7ea0 con 0x7fd388102660 2024-05-14T17:00:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.156+0000 7fd391633740 1 Processor -- start 2024-05-14T17:00:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.156+0000 7fd3857fa640 1 -- 172.21.15.69:0/3660606531 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd37c042030 con 0x7fd388102660 2024-05-14T17:00:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.156+0000 7fd391633740 1 -- start start 2024-05-14T17:00:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.156+0000 7fd391633740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f3dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.156+0000 7fd391633740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f48590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.156+0000 7fd391633740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f4aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.156+0000 7fd391633740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563ec9eb0c30 con 0x563ec9f3e210 2024-05-14T17:00:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.156+0000 7fd391633740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563ec9ea07d0 con 0x563ec9f48ad0 2024-05-14T17:00:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.156+0000 7fd391633740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563ec9ea4810 con 0x563ec9f3d8f0 2024-05-14T17:00:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd387fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f48590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd387fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f48590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45674/0 (socket says 172.21.15.69:45674) 2024-05-14T17:00:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd387fff640 1 -- 172.21.15.69:0/65905844 learned_addr learned my addr 172.21.15.69:0/65905844 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd3877fe640 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f4aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd38cde3640 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f3dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd387fff640 1 -- 172.21.15.69:0/65905844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 msgr2=0x563ec9f3dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd387fff640 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f3dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd387fff640 1 -- 172.21.15.69:0/65905844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 msgr2=0x563ec9f4aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd387fff640 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f4aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd387fff640 1 -- 172.21.15.69:0/65905844 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec9bb5210 con 0x563ec9f3e210 2024-05-14T17:00:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd38cde3640 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f3dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd3877fe640 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f4aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd3857fa640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd3700b2a00 0x7fd3700b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd38cde3640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd3700b2a00 0x7fd3700b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd387fff640 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f48590 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd37800bc90 tx=0x7fd3780107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6709e9afdfb26 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd3857fa640 1 -- 172.21.15.69:0/3660606531 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fd37c102e50 con 0x7fd388102660 2024-05-14T17:00:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd36e7fc640 1 -- 172.21.15.69:0/65905844 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37801c480 con 0x563ec9f3e210 2024-05-14T17:00:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd36e7fc640 1 -- 172.21.15.69:0/65905844 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd37801cad0 con 0x563ec9f3e210 2024-05-14T17:00:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.157+0000 7fd36e7fc640 1 -- 172.21.15.69:0/65905844 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378011ae0 con 0x563ec9f3e210 2024-05-14T17:00:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd391633740 1 -- 172.21.15.69:0/65905844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 msgr2=0x563ec9f48590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd38cde3640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd3700b2a00 0x7fd3700b4ec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd380005fd0 tx=0x7fd3800096a0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd391633740 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f48590 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd37800bc90 tx=0x7fd3780107a0 comp rx=0 tx=0).stop 2024-05-14T17:00:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd391633740 1 -- 172.21.15.69:0/65905844 shutdown_connections 2024-05-14T17:00:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd391633740 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f4aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd391633740 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f48590 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd391633740 1 --2- 172.21.15.69:0/65905844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f3dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd391633740 1 -- 172.21.15.69:0/65905844 >> 172.21.15.69:0/65905844 conn(0x563ec9f39e00 msgr2=0x563ec9f3a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd391633740 1 -- 172.21.15.69:0/65905844 shutdown_connections 2024-05-14T17:00:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd391633740 1 -- 172.21.15.69:0/65905844 wait complete. 2024-05-14T17:00:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.158+0000 7fd391633740 1 -- start start 2024-05-14T17:00:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd391633740 1 Processor -- start 2024-05-14T17:00:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd391633740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f48450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd391633740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f3e210 0x563ec9f45740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd391633740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f48ad0 0x563ec9f45cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd391633740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563ec9b86790 con 0x563ec9f48ad0 2024-05-14T17:00:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd391633740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563ec9e5c890 con 0x563ec9f3e210 2024-05-14T17:00:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd391633740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563ec9f4b7a0 con 0x563ec9f3d8f0 2024-05-14T17:00:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd391633740 10 client.0 _ll_register_callbacks cb 0x563ec9e5fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd38cde3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f48ad0 0x563ec9f45cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd38cde3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f48ad0 0x563ec9f45cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45682/0 (socket says 172.21.15.69:45682) 2024-05-14T17:00:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd38cde3640 1 -- 172.21.15.69:0/2102268289 learned_addr learned my addr 172.21.15.69:0/2102268289 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd3877fe640 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f3e210 0x563ec9f45740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd387fff640 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f48450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd38cde3640 1 -- 172.21.15.69:0/2102268289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 msgr2=0x563ec9f48450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd38cde3640 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f48450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd38cde3640 1 -- 172.21.15.69:0/2102268289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f3e210 msgr2=0x563ec9f45740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd38cde3640 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f3e210 0x563ec9f45740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.159+0000 7fd38cde3640 1 -- 172.21.15.69:0/2102268289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec9c81620 con 0x563ec9f48ad0 2024-05-14T17:00:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd38cde3640 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f48ad0 0x563ec9f45cc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd3800114c0 tx=0x7fd380004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8befb67e1daf9581 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd3677fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:00:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd3677fe640 1 -- 172.21.15.69:0/2102268289 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380011e00 con 0x563ec9f48ad0 2024-05-14T17:00:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd3677fe640 1 -- 172.21.15.69:0/2102268289 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fd380029420 con 0x563ec9f48ad0 2024-05-14T17:00:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd391633740 1 -- 172.21.15.69:0/2102268289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd37801cad0 con 0x563ec9f48ad0 2024-05-14T17:00:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd3677fe640 1 -- 172.21.15.69:0/2102268289 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd380029620 con 0x563ec9f48ad0 2024-05-14T17:00:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd391633740 10 client.16533 fetch_fsmap learned FSMap version 47 2024-05-14T17:00:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd391633740 1 -- 172.21.15.69:0/2102268289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x563ec9f57240 con 0x563ec9f48ad0 2024-05-14T17:00:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd3677fe640 1 -- 172.21.15.69:0/2102268289 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd37801cad0 con 0x563ec9f48ad0 2024-05-14T17:00:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd3677fe640 1 -- 172.21.15.69:0/2102268289 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fd38001f7a0 con 0x563ec9f48ad0 2024-05-14T17:00:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd391633740 10 client.16533 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:00:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd391633740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd391633740 20 client.16533 populate_metadata read hostname 'smithi069' 2024-05-14T17:00:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd391633740 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563ec9f587c0 0x563ec9f5abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd391633740 4 client.16533 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.160+0000 7fd391633740 1 -- 172.21.15.69:0/2102268289 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ec9eb2b80 con 0x563ec9f587c0 2024-05-14T17:00:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.161+0000 7fd387fff640 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563ec9f587c0 0x563ec9f5abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.161+0000 7fd3677fe640 10 client.16533 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.161+0000 7fd387fff640 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563ec9f587c0 0x563ec9f5abb0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a074c23ba127497a server_cookie=794cb89a52a630d4 in_seq=0 out_seq=0 2024-05-14T17:00:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd3677fe640 1 -- 172.21.15.69:0/2102268289 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ec9eb2b80 con 0x563ec9f587c0 2024-05-14T17:00:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd3677fe640 10 client.16533 handle_command_reply: tid=0 2024-05-14T17:00:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 client.16533 shutdown 2024-05-14T17:00:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 -- 172.21.15.69:0/2102268289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f48ad0 msgr2=0x563ec9f45cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f48ad0 0x563ec9f45cc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd3800114c0 tx=0x7fd380004ab0 comp rx=0 tx=0).stop 2024-05-14T17:00:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 -- 172.21.15.69:0/2102268289 shutdown_connections 2024-05-14T17:00:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563ec9f587c0 0x563ec9f5abb0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f48ad0 0x563ec9f45cc0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f3e210 0x563ec9f45740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd3677fe640 0 client.16533 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 --2- 172.21.15.69:0/2102268289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f48450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 -- 172.21.15.69:0/2102268289 >> 172.21.15.69:0/2102268289 conn(0x563ec9f39e00 msgr2=0x563ec9f39100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 -- 172.21.15.69:0/2102268289 shutdown_connections 2024-05-14T17:00:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 1 -- 172.21.15.69:0/2102268289 wait complete. 2024-05-14T17:00:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.162+0000 7fd391633740 20 client.16533 trim_cache size 0 max 16384 2024-05-14T17:00:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd391633740 1 Processor -- start 2024-05-14T17:00:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd391633740 1 -- start start 2024-05-14T17:00:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd391633740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd391633740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd391633740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd391633740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b3190 con 0x563ec9f3e210 2024-05-14T17:00:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd391633740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b09d0 con 0x563ec9f48ad0 2024-05-14T17:00:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd391633740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881ae840 con 0x563ec9f3d8f0 2024-05-14T17:00:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45698/0 (socket says 172.21.15.69:45698) 2024-05-14T17:00:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd3877fe640 1 -- 172.21.15.69:0/1156608248 learned_addr learned my addr 172.21.15.69:0/1156608248 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd38cde3640 1 --2- 172.21.15.69:0/1156608248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd3877fe640 1 -- 172.21.15.69:0/1156608248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 msgr2=0x563ec9f604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd3877fe640 1 --2- 172.21.15.69:0/1156608248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd3877fe640 1 -- 172.21.15.69:0/1156608248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 msgr2=0x563ec9f429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd3877fe640 1 --2- 172.21.15.69:0/1156608248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd3877fe640 1 -- 172.21.15.69:0/1156608248 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec9f5af20 con 0x563ec9f3e210 2024-05-14T17:00:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd38cde3640 1 --2- 172.21.15.69:0/1156608248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd3877fe640 1 --2- 172.21.15.69:0/1156608248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f609e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0c6250 tx=0x7fd37c0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc697488ed4e802 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.177+0000 7fd36dffb640 1 -- 172.21.15.69:0/1156608248 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c049030 con 0x563ec9f3e210 2024-05-14T17:00:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd36dffb640 1 -- 172.21.15.69:0/1156608248 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd37c0c9ce0 con 0x563ec9f3e210 2024-05-14T17:00:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd36dffb640 1 -- 172.21.15.69:0/1156608248 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0ce4d0 con 0x563ec9f3e210 2024-05-14T17:00:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd391633740 1 -- 172.21.15.69:0/1156608248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 msgr2=0x563ec9f609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd391633740 1 --2- 172.21.15.69:0/1156608248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f609e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0c6250 tx=0x7fd37c0045a0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd391633740 1 -- 172.21.15.69:0/1156608248 shutdown_connections 2024-05-14T17:00:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd391633740 1 --2- 172.21.15.69:0/1156608248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd391633740 1 --2- 172.21.15.69:0/1156608248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f609e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd391633740 1 --2- 172.21.15.69:0/1156608248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd391633740 1 -- 172.21.15.69:0/1156608248 >> 172.21.15.69:0/1156608248 conn(0x563ec9f39e00 msgr2=0x563ec9f53660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd391633740 1 -- 172.21.15.69:0/1156608248 shutdown_connections 2024-05-14T17:00:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd391633740 1 -- 172.21.15.69:0/1156608248 wait complete. 2024-05-14T17:00:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.178+0000 7fd391633740 1 -- start start 2024-05-14T17:00:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd391633740 1 Processor -- start 2024-05-14T17:00:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd391633740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f4e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd391633740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f4ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd391633740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f4f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd391633740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881aee90 con 0x563ec9f3e210 2024-05-14T17:00:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd391633740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b0300 con 0x563ec9f48ad0 2024-05-14T17:00:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd391633740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881af560 con 0x563ec9f3d8f0 2024-05-14T17:00:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd391633740 10 client.0 _ll_register_callbacks cb 0x7fd380009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f4ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f4ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45706/0 (socket says 172.21.15.69:45706) 2024-05-14T17:00:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3877fe640 1 -- 172.21.15.69:0/699475020 learned_addr learned my addr 172.21.15.69:0/699475020 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd38cde3640 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f4f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3877fe640 1 -- 172.21.15.69:0/699475020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 msgr2=0x563ec9f4e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3877fe640 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f4e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3877fe640 1 -- 172.21.15.69:0/699475020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 msgr2=0x563ec9f4f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3877fe640 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f4f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3877fe640 1 -- 172.21.15.69:0/699475020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec9fc2300 con 0x563ec9f3e210 2024-05-14T17:00:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3877fe640 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f4ed90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd37c100640 tx=0x7fd37c100ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c97c533ee01251 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3677fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:00:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3677fe640 1 -- 172.21.15.69:0/699475020 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0e4060 con 0x563ec9f3e210 2024-05-14T17:00:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd3677fe640 1 -- 172.21.15.69:0/699475020 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fd37c0ce170 con 0x563ec9f3e210 2024-05-14T17:00:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.179+0000 7fd391633740 1 -- 172.21.15.69:0/699475020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd37c0c9ce0 con 0x563ec9f3e210 2024-05-14T17:00:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd3677fe640 1 -- 172.21.15.69:0/699475020 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd37c049050 con 0x563ec9f3e210 2024-05-14T17:00:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd391633740 10 client.16539 fetch_fsmap learned FSMap version 47 2024-05-14T17:00:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd391633740 1 -- 172.21.15.69:0/699475020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x563ec9fc3580 con 0x563ec9f3e210 2024-05-14T17:00:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd3677fe640 1 -- 172.21.15.69:0/699475020 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd37c0c9ce0 con 0x563ec9f3e210 2024-05-14T17:00:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd3677fe640 1 -- 172.21.15.69:0/699475020 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fd37c100ce0 con 0x563ec9f3e210 2024-05-14T17:00:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd391633740 10 client.16539 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:00:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd391633740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd391633740 20 client.16539 populate_metadata read hostname 'smithi069' 2024-05-14T17:00:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd391633740 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563ec9fc3ab0 0x563ec9fc5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd391633740 4 client.16539 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:00:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd391633740 1 -- 172.21.15.69:0/699475020 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ec9eac360 con 0x563ec9fc3ab0 2024-05-14T17:00:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd387fff640 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563ec9fc3ab0 0x563ec9fc5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd387fff640 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563ec9fc3ab0 0x563ec9fc5ea0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1da317b709dd29 server_cookie=11f4930757efaabd in_seq=0 out_seq=0 2024-05-14T17:00:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.180+0000 7fd3677fe640 10 client.16539 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:30.518 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: pgmap v326: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.6 MiB/s wr, 806 op/s 2024-05-14T17:00:30.567 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:30.617 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:30.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:30.726 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:30.776 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:30.826 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:30.876 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:30.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:30.976 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:31.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:31.076 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:31.126 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:00:31.176 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:00:31.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:00:31.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:00:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.204+0000 7fd3677fe640 1 -- 172.21.15.69:0/699475020 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563ec9eac360 con 0x563ec9fc3ab0 2024-05-14T17:00:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.204+0000 7fd3677fe640 10 client.16539 handle_command_reply: tid=0 2024-05-14T17:00:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.204+0000 7fd391633740 1 client.16539 shutdown 2024-05-14T17:00:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.204+0000 7fd391633740 1 -- 172.21.15.69:0/699475020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 msgr2=0x563ec9f4ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.204+0000 7fd391633740 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f4ed90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd37c100640 tx=0x7fd37c100ac0 comp rx=0 tx=0).stop 2024-05-14T17:00:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd391633740 1 -- 172.21.15.69:0/699475020 shutdown_connections 2024-05-14T17:00:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd391633740 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x563ec9fc3ab0 0x563ec9fc5ea0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd391633740 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563ec9f48ad0 0x563ec9f4f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd391633740 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ec9f3e210 0x563ec9f4ed90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd3677fe640 0 client.16539 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd391633740 1 --2- 172.21.15.69:0/699475020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ec9f3d8f0 0x563ec9f4e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd391633740 1 -- 172.21.15.69:0/699475020 >> 172.21.15.69:0/699475020 conn(0x563ec9f39e00 msgr2=0x563ec9f53ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd391633740 1 -- 172.21.15.69:0/699475020 shutdown_connections 2024-05-14T17:00:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd391633740 1 -- 172.21.15.69:0/699475020 wait complete. 2024-05-14T17:00:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd391633740 20 client.16539 trim_cache size 0 max 16384 2024-05-14T17:00:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:00:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:00:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:00:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 -- 172.21.15.69:0/3660606531 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd3700b2a00 msgr2=0x7fd3700b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd3700b2a00 0x7fd3700b4ec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd380005fd0 tx=0x7fd3800096a0 comp rx=0 tx=0).stop 2024-05-14T17:00:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 -- 172.21.15.69:0/3660606531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd388102660 msgr2=0x7fd38819b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd388102660 0x7fd38819b870 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd37c005940 tx=0x7fd37c0042f0 comp rx=0 tx=0).stop 2024-05-14T17:00:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 -- 172.21.15.69:0/3660606531 shutdown_connections 2024-05-14T17:00:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd3700b2a00 0x7fd3700b4ec0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3881068f0 0x7fd38819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd388102660 0x7fd38819b870 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 --2- 172.21.15.69:0/3660606531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd388101ca0 0x7fd38819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 -- 172.21.15.69:0/3660606531 >> 172.21.15.69:0/3660606531 conn(0x7fd388077750 msgr2=0x7fd3880ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 -- 172.21.15.69:0/3660606531 shutdown_connections 2024-05-14T17:00:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:18.205+0000 7fd38e846640 1 -- 172.21.15.69:0/3660606531 wait complete. 2024-05-14T17:00:32.895 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:18 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-14T17:00:32.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:18 smithi069 ceph-mon[31507]: Unable to set osd_memory_target on smithi069 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-14T17:00:32.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:18 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-14T17:00:33.068 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:18 smithi069 ceph-mon[31507]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-14T17:00:33.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:19 smithi069 ceph-mon[31507]: pgmap v327: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.9 MiB/s wr, 661 op/s 2024-05-14T17:00:33.220 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 590.0 s, 165.3 tps, lat 64.755 ms stddev 69.893 2024-05-14T17:00:33.270 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:21 smithi069 ceph-mon[31507]: pgmap v328: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.8 MiB/s wr, 654 op/s 2024-05-14T17:00:33.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa9417fa700 1 -- 172.21.15.69:0/1573337643 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5637b7503d70 con 0x5637b7628740 2024-05-14T17:00:33.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa9417fa700 10 client.16521 handle_command_reply: tid=0 2024-05-14T17:00:33.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 client.16521 shutdown 2024-05-14T17:00:33.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 -- 172.21.15.69:0/1573337643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 msgr2=0x5637b75b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75b5740 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x5637b75adeb0 tx=0x7fa948005960 comp rx=0 tx=0).stop 2024-05-14T17:00:33.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 -- 172.21.15.69:0/1573337643 shutdown_connections 2024-05-14T17:00:33.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5637b7628740 0x5637b762ab60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:33.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637b75a9750 0x5637b75b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637b75a3b60 0x5637b75b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:33.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 --2- 172.21.15.69:0/1573337643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637b75a8e30 0x5637b75b5740 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:33.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa9417fa700 0 client.16521 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:33.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 -- 172.21.15.69:0/1573337643 >> 172.21.15.69:0/1573337643 conn(0x5637b74ce3e0 msgr2=0x5637b75ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:33.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 -- 172.21.15.69:0/1573337643 shutdown_connections 2024-05-14T17:00:33.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 1 -- 172.21.15.69:0/1573337643 wait complete. 2024-05-14T17:00:34.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.735+0000 7fa96a918b80 20 client.16521 trim_cache size 0 max 16384 2024-05-14T17:00:34.129 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:00:34.178 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 47, 2024-05-14T17:00:34.228 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T17:00:34.281 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 34, 2024-05-14T17:00:34.328 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T17:00:34.378 INFO:teuthology.orchestra.run.smithi069.stdout: "c837d4f6": { 2024-05-14T17:00:34.428 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 34, 2024-05-14T17:00:34.478 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T17:00:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:00:34.578 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T17:00:34.628 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:00:34.679 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:00:34.728 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T17:00:34.778 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 137.9, 2024-05-14T17:00:34.828 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T17:00:34.945 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T17:00:34.995 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T17:00:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:00:35.095 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T17:00:35.145 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:00:35.195 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:00:35.245 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:00:35.295 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:00:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:00:35.395 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:00:35.445 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:00:35.496 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:21 smithi092 ceph-mon[35763]: pgmap v328: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.8 MiB/s wr, 654 op/s 2024-05-14T17:00:35.545 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:23 smithi092 ceph-mon[35763]: pgmap v329: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 1.50k op/s 2024-05-14T17:00:35.604 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:25 smithi092 ceph-mon[35763]: pgmap v330: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 15 MiB/s wr, 1.35k op/s 2024-05-14T17:00:35.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:27 smithi092 ceph-mon[35763]: pgmap v331: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 14 MiB/s wr, 1.25k op/s 2024-05-14T17:00:35.704 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:29 smithi092 ceph-mon[35763]: pgmap v332: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1k op/s 2024-05-14T17:00:35.804 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:31 smithi092 ceph-mon[35763]: pgmap v333: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1k op/s 2024-05-14T17:00:35.886 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:33 smithi092 ceph-mon[35763]: pgmap v334: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 994 op/s 2024-05-14T17:00:35.937 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:21 smithi203 ceph-mon[35787]: pgmap v328: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.8 MiB/s wr, 654 op/s 2024-05-14T17:00:35.987 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:23 smithi203 ceph-mon[35787]: pgmap v329: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 1.50k op/s 2024-05-14T17:00:36.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:25 smithi203 ceph-mon[35787]: pgmap v330: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 15 MiB/s wr, 1.35k op/s 2024-05-14T17:00:36.087 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:27 smithi203 ceph-mon[35787]: pgmap v331: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 14 MiB/s wr, 1.25k op/s 2024-05-14T17:00:36.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:29 smithi203 ceph-mon[35787]: pgmap v332: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1k op/s 2024-05-14T17:00:36.196 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:31 smithi203 ceph-mon[35787]: pgmap v333: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1k op/s 2024-05-14T17:00:36.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:33 smithi203 ceph-mon[35787]: pgmap v334: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 994 op/s 2024-05-14T17:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 -- 172.21.15.69:0/3680885854 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9440b6ff0 msgr2=0x7fa9440b94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:36.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9440b6ff0 0x7fa9440b94b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x5637b75a9b50 tx=0x7fa94800a390 comp rx=0 tx=0).stop 2024-05-14T17:00:36.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 -- 172.21.15.69:0/3680885854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958106ef0 msgr2=0x7fa95810a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:36.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958106ef0 0x7fa95810a2c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa950018800 tx=0x7fa950018830 comp rx=0 tx=0).stop 2024-05-14T17:00:36.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 -- 172.21.15.69:0/3680885854 shutdown_connections 2024-05-14T17:00:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9440b6ff0 0x7fa9440b94b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:36.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa958101e10 0x7fa95810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:36.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958106ef0 0x7fa95810a2c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:36.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 --2- 172.21.15.69:0/3680885854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9581027c0 0x7fa958109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:36.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 -- 172.21.15.69:0/3680885854 >> 172.21.15.69:0/3680885854 conn(0x7fa958078ae0 msgr2=0x7fa958106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:36.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 -- 172.21.15.69:0/3680885854 shutdown_connections 2024-05-14T17:00:36.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:23.736+0000 7fa960082700 1 -- 172.21.15.69:0/3680885854 wait complete. 2024-05-14T17:00:36.930 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:23 smithi069 ceph-mon[31507]: pgmap v329: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 1.50k op/s 2024-05-14T17:00:36.980 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:35 smithi092 ceph-mon[35763]: pgmap v335: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-05-14T17:00:37.030 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:35 smithi203 ceph-mon[35787]: pgmap v335: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-05-14T17:00:37.081 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 595.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:00:37.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:25 smithi069 ceph-mon[31507]: pgmap v330: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 15 MiB/s wr, 1.35k op/s 2024-05-14T17:00:37.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:27 smithi069 ceph-mon[31507]: pgmap v331: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 14 MiB/s wr, 1.25k op/s 2024-05-14T17:00:37.230 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:29 smithi069 ceph-mon[31507]: pgmap v332: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1k op/s 2024-05-14T17:00:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 600.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:00:37.331 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c837d4f6', quiesce took 29.5 seconds. Will release after: -11.600000000000001 2024-05-14T17:00:37.385 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:00:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.304+0000 7f0508389640 1 -- 172.21.15.69:0/664748820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0500107940 msgr2=0x7f0500107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.304+0000 7f0508389640 1 --2- 172.21.15.69:0/664748820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0500107940 0x7f0500107d20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009930 tx=0x7f04fc02a420 comp rx=0 tx=0).stop 2024-05-14T17:00:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 -- 172.21.15.69:0/664748820 shutdown_connections 2024-05-14T17:00:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 --2- 172.21.15.69:0/664748820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0500102340 0x7f050010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 --2- 172.21.15.69:0/664748820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0500101980 0x7f0500101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 --2- 172.21.15.69:0/664748820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0500107940 0x7f0500107d20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 -- 172.21.15.69:0/664748820 >> 172.21.15.69:0/664748820 conn(0x7f0500078ad0 msgr2=0x7f0500078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 -- 172.21.15.69:0/664748820 shutdown_connections 2024-05-14T17:00:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 -- 172.21.15.69:0/664748820 wait complete. 2024-05-14T17:00:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 Processor -- start 2024-05-14T17:00:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 -- start start 2024-05-14T17:00:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0500101980 0x7f050019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.305+0000 7f0508389640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0500102340 0x7f050019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0508389640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0500107940 0x7f05001a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0508389640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0500111c00 con 0x7f0500102340 2024-05-14T17:00:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0508389640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0500111a80 con 0x7f0500107940 2024-05-14T17:00:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0508389640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0500111d80 con 0x7f0500101980 2024-05-14T17:00:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0505924640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0500102340 0x7f050019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0505924640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0500102340 0x7f050019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50056/0 (socket says 172.21.15.69:50056) 2024-05-14T17:00:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0505924640 1 -- 172.21.15.69:0/2106226882 learned_addr learned my addr 172.21.15.69:0/2106226882 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0506125640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0500101980 0x7f050019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0506926640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0500107940 0x7f05001a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0505924640 1 -- 172.21.15.69:0/2106226882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0500101980 msgr2=0x7f050019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0505924640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0500101980 0x7f050019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0505924640 1 -- 172.21.15.69:0/2106226882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0500107940 msgr2=0x7f05001a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0505924640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0500107940 0x7f05001a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0505924640 1 -- 172.21.15.69:0/2106226882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05001a5640 con 0x7f0500102340 2024-05-14T17:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0506926640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0500107940 0x7f05001a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0505924640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0500102340 0x7f050019a5c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f04f000e9f0 tx=0x7f04f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f04f77fe640 1 -- 172.21.15.69:0/2106226882 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f000cd80 con 0x7f0500102340 2024-05-14T17:00:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f04f77fe640 1 -- 172.21.15.69:0/2106226882 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f00108e0 con 0x7f0500102340 2024-05-14T17:00:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f04f77fe640 1 -- 172.21.15.69:0/2106226882 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0018980 con 0x7f0500102340 2024-05-14T17:00:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0508389640 1 -- 172.21.15.69:0/2106226882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05001a5810 con 0x7f0500102340 2024-05-14T17:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.306+0000 7f0508389640 1 -- 172.21.15.69:0/2106226882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05001a5db0 con 0x7f0500102340 2024-05-14T17:00:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.309+0000 7f050b180740 1 Processor -- start 2024-05-14T17:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.309+0000 7f04f77fe640 1 -- 172.21.15.69:0/2106226882 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f04f0010430 con 0x7f0500102340 2024-05-14T17:00:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.309+0000 7f050b180740 1 -- start start 2024-05-14T17:00:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f050b180740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f050b180740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f050b180740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768bdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f050b180740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561976823c30 con 0x5619768b08f0 2024-05-14T17:00:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f050b180740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5619768137d0 con 0x5619768b1210 2024-05-14T17:00:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f050b180740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561976817810 con 0x5619768bbad0 2024-05-14T17:00:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506926640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506926640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50070/0 (socket says 172.21.15.69:50070) 2024-05-14T17:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506926640 1 -- 172.21.15.69:0/2344898059 learned_addr learned my addr 172.21.15.69:0/2344898059 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506125640 1 --2- 172.21.15.69:0/2344898059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0505924640 1 --2- 172.21.15.69:0/2344898059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768bdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506926640 1 -- 172.21.15.69:0/2344898059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 msgr2=0x5619768bdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506926640 1 --2- 172.21.15.69:0/2344898059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768bdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506926640 1 -- 172.21.15.69:0/2344898059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 msgr2=0x5619768bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506926640 1 --2- 172.21.15.69:0/2344898059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506926640 1 -- 172.21.15.69:0/2344898059 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561976528210 con 0x5619768b08f0 2024-05-14T17:00:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f04f77fe640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04ec0b2a90 0x7f04ec0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506926640 1 --2- 172.21.15.69:0/2344898059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768b0cd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009e40 tx=0x7f04f800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97026052ddfcb76f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.310+0000 7f0506125640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04ec0b2a90 0x7f04ec0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.311+0000 7f04f4bfe640 1 -- 172.21.15.69:0/2344898059 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8004d20 con 0x5619768b08f0 2024-05-14T17:00:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.311+0000 7f04f77fe640 1 -- 172.21.15.69:0/2106226882 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f04f00e2a50 con 0x7f0500102340 2024-05-14T17:00:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.311+0000 7f04f4bfe640 1 -- 172.21.15.69:0/2344898059 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f80054d0 con 0x5619768b08f0 2024-05-14T17:00:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.311+0000 7f04f4bfe640 1 -- 172.21.15.69:0/2344898059 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8011770 con 0x5619768b08f0 2024-05-14T17:00:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.311+0000 7f050b180740 1 -- 172.21.15.69:0/2344898059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 msgr2=0x5619768b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.311+0000 7f0506125640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04ec0b2a90 0x7f04ec0b4f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f04fc000c00 tx=0x7f04fc005a40 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.311+0000 7f050b180740 1 --2- 172.21.15.69:0/2344898059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768b0cd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009e40 tx=0x7f04f800f730 comp rx=0 tx=0).stop 2024-05-14T17:00:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.311+0000 7f050b180740 1 -- 172.21.15.69:0/2344898059 shutdown_connections 2024-05-14T17:00:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.311+0000 7f050b180740 1 --2- 172.21.15.69:0/2344898059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768bdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.311+0000 7f050b180740 1 --2- 172.21.15.69:0/2344898059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 --2- 172.21.15.69:0/2344898059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768b0cd0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 -- 172.21.15.69:0/2344898059 >> 172.21.15.69:0/2344898059 conn(0x5619768ace00 msgr2=0x5619768ad1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 -- 172.21.15.69:0/2344898059 shutdown_connections 2024-05-14T17:00:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 -- 172.21.15.69:0/2344898059 wait complete. 2024-05-14T17:00:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 -- start start 2024-05-14T17:00:41.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 Processor -- start 2024-05-14T17:00:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768b1210 0x5619768b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768bbad0 0x5619768b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5619764f9790 con 0x5619768b08f0 2024-05-14T17:00:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5619767cf890 con 0x5619768bbad0 2024-05-14T17:00:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5619768be7a0 con 0x5619768b1210 2024-05-14T17:00:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f050b180740 10 client.0 _ll_register_callbacks cb 0x5619767d2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f0506125640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f0506125640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50076/0 (socket says 172.21.15.69:50076) 2024-05-14T17:00:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.312+0000 7f0506125640 1 -- 172.21.15.69:0/109786247 learned_addr learned my addr 172.21.15.69:0/109786247 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f0505924640 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768b1210 0x5619768b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f0506926640 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768bbad0 0x5619768b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f0506125640 1 -- 172.21.15.69:0/109786247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768b1210 msgr2=0x5619768b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f0506125640 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768b1210 0x5619768b87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f0506125640 1 -- 172.21.15.69:0/109786247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768bbad0 msgr2=0x5619768b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f0506125640 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768bbad0 0x5619768b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f0506125640 1 -- 172.21.15.69:0/109786247 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619765f4620 con 0x5619768b08f0 2024-05-14T17:00:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f0506926640 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768bbad0 0x5619768b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f0506125640 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bb4f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009930 tx=0x7f04fc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a863138f5fa8cfe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f04e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:00:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f04e9ffb640 1 -- 172.21.15.69:0/109786247 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc04c070 con 0x5619768b08f0 2024-05-14T17:00:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f04e9ffb640 1 -- 172.21.15.69:0/109786247 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f04fc003cc0 con 0x5619768b08f0 2024-05-14T17:00:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f050b180740 1 -- 172.21.15.69:0/109786247 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f80054d0 con 0x5619768b08f0 2024-05-14T17:00:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f04e9ffb640 1 -- 172.21.15.69:0/109786247 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f04fc058050 con 0x5619768b08f0 2024-05-14T17:00:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f050b180740 10 client.16551 fetch_fsmap learned FSMap version 47 2024-05-14T17:00:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f050b180740 1 -- 172.21.15.69:0/109786247 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5619768ca280 con 0x5619768b08f0 2024-05-14T17:00:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f04e9ffb640 1 -- 172.21.15.69:0/109786247 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f80054d0 con 0x5619768b08f0 2024-05-14T17:00:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.313+0000 7f04e9ffb640 1 -- 172.21.15.69:0/109786247 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7f04fc004990 con 0x5619768b08f0 2024-05-14T17:00:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.314+0000 7f050b180740 10 client.16551 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:00:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.314+0000 7f050b180740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.314+0000 7f050b180740 20 client.16551 populate_metadata read hostname 'smithi069' 2024-05-14T17:00:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.314+0000 7f050b180740 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5619768cb740 0x5619768cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.314+0000 7f050b180740 4 client.16551 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.314+0000 7f050b180740 1 -- 172.21.15.69:0/109786247 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561976825b80 con 0x5619768cb740 2024-05-14T17:00:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.314+0000 7f0505924640 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5619768cb740 0x5619768cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.314+0000 7f04e9ffb640 10 client.16551 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.314+0000 7f0505924640 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5619768cb740 0x5619768cdb30 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663636324968f18a server_cookie=ba390e4c06d1d22d in_seq=0 out_seq=0 2024-05-14T17:00:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f04e9ffb640 1 -- 172.21.15.69:0/109786247 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561976825b80 con 0x5619768cb740 2024-05-14T17:00:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f04e9ffb640 10 client.16551 handle_command_reply: tid=0 2024-05-14T17:00:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 client.16551 shutdown 2024-05-14T17:00:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 -- 172.21.15.69:0/109786247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 msgr2=0x5619768bb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bb4f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009930 tx=0x7f04fc009260 comp rx=0 tx=0).stop 2024-05-14T17:00:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 -- 172.21.15.69:0/109786247 shutdown_connections 2024-05-14T17:00:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5619768cb740 0x5619768cdb30 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768bbad0 0x5619768b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f04e9ffb640 0 client.16551 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768b1210 0x5619768b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 --2- 172.21.15.69:0/109786247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bb4f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 -- 172.21.15.69:0/109786247 >> 172.21.15.69:0/109786247 conn(0x5619768ace00 msgr2=0x5619768ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 -- 172.21.15.69:0/109786247 shutdown_connections 2024-05-14T17:00:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 1 -- 172.21.15.69:0/109786247 wait complete. 2024-05-14T17:00:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.315+0000 7f050b180740 20 client.16551 trim_cache size 0 max 16384 2024-05-14T17:00:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.330+0000 7f050b180740 1 Processor -- start 2024-05-14T17:00:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.330+0000 7f050b180740 1 -- start start 2024-05-14T17:00:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.330+0000 7f050b180740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.330+0000 7f050b180740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.330+0000 7f050b180740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.330+0000 7f050b180740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05001b1070 con 0x5619768b08f0 2024-05-14T17:00:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.330+0000 7f050b180740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f05001ae8b0 con 0x5619768b1210 2024-05-14T17:00:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.330+0000 7f050b180740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05001ac720 con 0x5619768bbad0 2024-05-14T17:00:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0506125640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0506125640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50082/0 (socket says 172.21.15.69:50082) 2024-05-14T17:00:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0506125640 1 -- 172.21.15.69:0/1581773333 learned_addr learned my addr 172.21.15.69:0/1581773333 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0506926640 1 --2- 172.21.15.69:0/1581773333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0505924640 1 --2- 172.21.15.69:0/1581773333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0506125640 1 -- 172.21.15.69:0/1581773333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 msgr2=0x5619768d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0506125640 1 --2- 172.21.15.69:0/1581773333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768d3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0506125640 1 -- 172.21.15.69:0/1581773333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 msgr2=0x5619768bb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0506125640 1 --2- 172.21.15.69:0/1581773333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0506125640 1 -- 172.21.15.69:0/1581773333 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619768b9160 con 0x5619768b08f0 2024-05-14T17:00:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0505924640 1 --2- 172.21.15.69:0/1581773333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f0506125640 1 --2- 172.21.15.69:0/1581773333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768c7670 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f04fc031440 tx=0x7f04fc0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86696f42ce93875 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f04ebfff640 1 -- 172.21.15.69:0/1581773333 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc04c070 con 0x5619768b08f0 2024-05-14T17:00:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f04ebfff640 1 -- 172.21.15.69:0/1581773333 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc061070 con 0x5619768b08f0 2024-05-14T17:00:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.331+0000 7f04ebfff640 1 -- 172.21.15.69:0/1581773333 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc035040 con 0x5619768b08f0 2024-05-14T17:00:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 -- 172.21.15.69:0/1581773333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 msgr2=0x5619768c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 --2- 172.21.15.69:0/1581773333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768c7670 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f04fc031440 tx=0x7f04fc0092d0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 -- 172.21.15.69:0/1581773333 shutdown_connections 2024-05-14T17:00:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 --2- 172.21.15.69:0/1581773333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 --2- 172.21.15.69:0/1581773333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 --2- 172.21.15.69:0/1581773333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768c7670 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 -- 172.21.15.69:0/1581773333 >> 172.21.15.69:0/1581773333 conn(0x5619768ace00 msgr2=0x5619768b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 -- 172.21.15.69:0/1581773333 shutdown_connections 2024-05-14T17:00:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 -- 172.21.15.69:0/1581773333 wait complete. 2024-05-14T17:00:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 -- start start 2024-05-14T17:00:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 Processor -- start 2024-05-14T17:00:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05001acd70 con 0x5619768b08f0 2024-05-14T17:00:46.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f05001ae1e0 con 0x5619768b1210 2024-05-14T17:00:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05001ad440 con 0x5619768bbad0 2024-05-14T17:00:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f050b180740 10 client.0 _ll_register_callbacks cb 0x7f04fc0318a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f0506125640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f0506125640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50098/0 (socket says 172.21.15.69:50098) 2024-05-14T17:00:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.332+0000 7f0506125640 1 -- 172.21.15.69:0/807256037 learned_addr learned my addr 172.21.15.69:0/807256037 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f0505924640 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f0506926640 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f0506125640 1 -- 172.21.15.69:0/807256037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 msgr2=0x5619768bffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f0506125640 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768bffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f0506125640 1 -- 172.21.15.69:0/807256037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 msgr2=0x5619768bfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f0506125640 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f0506125640 1 -- 172.21.15.69:0/807256037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561976934fd0 con 0x5619768b08f0 2024-05-14T17:00:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f0506125640 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bf520 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f04fc02a940 tx=0x7f04fc003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f42c42d73b0c1d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f04e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:00:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f04e9ffb640 1 -- 172.21.15.69:0/807256037 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc069070 con 0x5619768b08f0 2024-05-14T17:00:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f04e9ffb640 1 -- 172.21.15.69:0/807256037 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f04fc004ad0 con 0x5619768b08f0 2024-05-14T17:00:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f050b180740 1 -- 172.21.15.69:0/807256037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04fc061070 con 0x5619768b08f0 2024-05-14T17:00:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f04e9ffb640 1 -- 172.21.15.69:0/807256037 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f04fc075050 con 0x5619768b08f0 2024-05-14T17:00:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f050b180740 10 client.16557 fetch_fsmap learned FSMap version 47 2024-05-14T17:00:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f050b180740 1 -- 172.21.15.69:0/807256037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5619769361b0 con 0x5619768b08f0 2024-05-14T17:00:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f04e9ffb640 1 -- 172.21.15.69:0/807256037 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc061070 con 0x5619768b08f0 2024-05-14T17:00:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.333+0000 7f04e9ffb640 1 -- 172.21.15.69:0/807256037 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7f04fc0488e0 con 0x5619768b08f0 2024-05-14T17:00:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.334+0000 7f050b180740 10 client.16557 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:00:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.334+0000 7f050b180740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.334+0000 7f050b180740 20 client.16557 populate_metadata read hostname 'smithi069' 2024-05-14T17:00:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.334+0000 7f050b180740 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561976936850 0x561976938c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.334+0000 7f050b180740 4 client.16557 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:00:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.334+0000 7f050b180740 1 -- 172.21.15.69:0/807256037 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56197681f360 con 0x561976936850 2024-05-14T17:00:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.334+0000 7f0505924640 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561976936850 0x561976938c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.334+0000 7f04e9ffb640 10 client.16557 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.334+0000 7f0505924640 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561976936850 0x561976938c70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd220ca9b855c866 server_cookie=37e0e5963c7f1d9f in_seq=0 out_seq=0 2024-05-14T17:00:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f04e9ffb640 1 -- 172.21.15.69:0/807256037 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56197681f360 con 0x561976936850 2024-05-14T17:00:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f04e9ffb640 10 client.16557 handle_command_reply: tid=0 2024-05-14T17:00:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 client.16557 shutdown 2024-05-14T17:00:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 -- 172.21.15.69:0/807256037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 msgr2=0x5619768bf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bf520 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f04fc02a940 tx=0x7f04fc003cf0 comp rx=0 tx=0).stop 2024-05-14T17:00:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 -- 172.21.15.69:0/807256037 shutdown_connections 2024-05-14T17:00:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561976936850 0x561976938c70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619768bbad0 0x5619768bffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619768b1210 0x5619768bfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f04e9ffb640 0 client.16557 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 --2- 172.21.15.69:0/807256037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619768b08f0 0x5619768bf520 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 -- 172.21.15.69:0/807256037 >> 172.21.15.69:0/807256037 conn(0x5619768ace00 msgr2=0x5619768c6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 -- 172.21.15.69:0/807256037 shutdown_connections 2024-05-14T17:00:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 1 -- 172.21.15.69:0/807256037 wait complete. 2024-05-14T17:00:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.347+0000 7f050b180740 20 client.16557 trim_cache size 0 max 16384 2024-05-14T17:00:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:00:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:00:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:00:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 -- 172.21.15.69:0/2106226882 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04ec0b2a90 msgr2=0x7f04ec0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04ec0b2a90 0x7f04ec0b4f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f04fc000c00 tx=0x7f04fc005a40 comp rx=0 tx=0).stop 2024-05-14T17:00:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 -- 172.21.15.69:0/2106226882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0500102340 msgr2=0x7f050019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0500102340 0x7f050019a5c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f04f000e9f0 tx=0x7f04f000eeb0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 -- 172.21.15.69:0/2106226882 shutdown_connections 2024-05-14T17:00:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04ec0b2a90 0x7f04ec0b4f50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0500107940 0x7f05001a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0500102340 0x7f050019a5c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 --2- 172.21.15.69:0/2106226882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0500101980 0x7f050019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 -- 172.21.15.69:0/2106226882 >> 172.21.15.69:0/2106226882 conn(0x7f0500078ad0 msgr2=0x7f0500104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 -- 172.21.15.69:0/2106226882 shutdown_connections 2024-05-14T17:00:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:31.348+0000 7f0508389640 1 -- 172.21.15.69:0/2106226882 wait complete. 2024-05-14T17:00:50.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:31 smithi069 ceph-mon[31507]: pgmap v333: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1k op/s 2024-05-14T17:00:50.181 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:33 smithi069 ceph-mon[31507]: pgmap v334: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 994 op/s 2024-05-14T17:00:50.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 605.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:00:50.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:35 smithi069 ceph-mon[31507]: pgmap v335: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-05-14T17:00:50.329 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:37 smithi069 ceph-mon[31507]: pgmap v336: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-05-14T17:00:50.379 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:39 smithi069 ceph-mon[31507]: pgmap v337: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:00:50.429 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:37 smithi092 ceph-mon[35763]: pgmap v336: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-05-14T17:00:50.479 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:39 smithi092 ceph-mon[35763]: pgmap v337: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:00:50.529 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:41 smithi092 ceph-mon[35763]: pgmap v338: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:00:50.579 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:43 smithi092 ceph-mon[35763]: pgmap v339: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-05-14T17:00:50.629 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:45 smithi092 ceph-mon[35763]: pgmap v340: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-05-14T17:00:50.679 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:47 smithi092 ceph-mon[35763]: pgmap v341: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-05-14T17:00:50.729 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:49 smithi092 ceph-mon[35763]: pgmap v342: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-05-14T17:00:50.779 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:37 smithi203 ceph-mon[35787]: pgmap v336: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-05-14T17:00:50.829 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:39 smithi203 ceph-mon[35787]: pgmap v337: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:00:50.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:41 smithi203 ceph-mon[35787]: pgmap v338: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:00:50.929 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:43 smithi203 ceph-mon[35787]: pgmap v339: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-05-14T17:00:50.988 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:45 smithi203 ceph-mon[35787]: pgmap v340: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-05-14T17:00:51.089 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:47 smithi203 ceph-mon[35787]: pgmap v341: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-05-14T17:00:51.138 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:49 smithi203 ceph-mon[35787]: pgmap v342: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-05-14T17:00:51.188 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 610.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:00:51.239 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:41 smithi069 ceph-mon[31507]: pgmap v338: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:00:51.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:43 smithi069 ceph-mon[31507]: pgmap v339: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-05-14T17:00:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.445+0000 7fc8b422a640 1 -- 172.21.15.69:0/1872522027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8ac101a90 msgr2=0x7fc8ac101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.445+0000 7fc8b422a640 1 --2- 172.21.15.69:0/1872522027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8ac101a90 0x7fc8ac101e70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009930 tx=0x7fc8a802a4d0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.445+0000 7fc8b422a640 1 -- 172.21.15.69:0/1872522027 shutdown_connections 2024-05-14T17:00:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.445+0000 7fc8b422a640 1 --2- 172.21.15.69:0/1872522027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac106e90 0x7fc8ac107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.445+0000 7fc8b422a640 1 --2- 172.21.15.69:0/1872522027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8ac1023b0 0x7fc8ac106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.445+0000 7fc8b422a640 1 --2- 172.21.15.69:0/1872522027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8ac101a90 0x7fc8ac101e70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.445+0000 7fc8b422a640 1 -- 172.21.15.69:0/1872522027 >> 172.21.15.69:0/1872522027 conn(0x7fc8ac078b60 msgr2=0x7fc8ac078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.445+0000 7fc8b422a640 1 -- 172.21.15.69:0/1872522027 shutdown_connections 2024-05-14T17:00:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.446+0000 7fc8b422a640 1 -- 172.21.15.69:0/1872522027 wait complete. 2024-05-14T17:00:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.446+0000 7fc8b422a640 1 Processor -- start 2024-05-14T17:00:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.446+0000 7fc8b422a640 1 -- start start 2024-05-14T17:00:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.446+0000 7fc8b422a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8ac101a90 0x7fc8ac1a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.446+0000 7fc8b422a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac1023b0 0x7fc8ac1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.446+0000 7fc8b422a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8ac106e90 0x7fc8ac19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.446+0000 7fc8b422a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac1192e0 con 0x7fc8ac106e90 2024-05-14T17:00:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.446+0000 7fc8b422a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac119160 con 0x7fc8ac101a90 2024-05-14T17:00:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.446+0000 7fc8b422a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac119460 con 0x7fc8ac1023b0 2024-05-14T17:00:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b27c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8ac106e90 0x7fc8ac19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b27c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8ac106e90 0x7fc8ac19f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41944/0 (socket says 172.21.15.69:41944) 2024-05-14T17:00:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b27c7640 1 -- 172.21.15.69:0/2127196305 learned_addr learned my addr 172.21.15.69:0/2127196305 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac1023b0 0x7fc8ac1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8ac101a90 0x7fc8ac1a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b27c7640 1 -- 172.21.15.69:0/2127196305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac1023b0 msgr2=0x7fc8ac1a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b27c7640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac1023b0 0x7fc8ac1a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b27c7640 1 -- 172.21.15.69:0/2127196305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8ac101a90 msgr2=0x7fc8ac1a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b27c7640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8ac101a90 0x7fc8ac1a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b27c7640 1 -- 172.21.15.69:0/2127196305 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac19fda0 con 0x7fc8ac106e90 2024-05-14T17:00:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac1023b0 0x7fc8ac1a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b27c7640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8ac106e90 0x7fc8ac19f620 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400e960 tx=0x7fc8a400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8a2ffd640 1 -- 172.21.15.69:0/2127196305 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4004d20 con 0x7fc8ac106e90 2024-05-14T17:00:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8a2ffd640 1 -- 172.21.15.69:0/2127196305 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a4005020 con 0x7fc8ac106e90 2024-05-14T17:00:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b422a640 1 -- 172.21.15.69:0/2127196305 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ac1a0090 con 0x7fc8ac106e90 2024-05-14T17:00:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8a2ffd640 1 -- 172.21.15.69:0/2127196305 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a40116d0 con 0x7fc8ac106e90 2024-05-14T17:00:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.447+0000 7fc8b422a640 1 -- 172.21.15.69:0/2127196305 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ac1ac2e0 con 0x7fc8ac106e90 2024-05-14T17:00:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.450+0000 7fc8b7028740 1 Processor -- start 2024-05-14T17:00:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.450+0000 7fc8a2ffd640 1 -- 172.21.15.69:0/2127196305 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc8a40051d0 con 0x7fc8ac106e90 2024-05-14T17:00:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b7028740 1 -- start start 2024-05-14T17:00:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6357378f0 0x55c635737cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c635738210 0x55c635742590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c635744ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b7028740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c6356aac30 con 0x55c6357378f0 2024-05-14T17:00:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b7028740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c63569a7d0 con 0x55c635742ad0 2024-05-14T17:00:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b7028740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c63569e810 con 0x55c635738210 2024-05-14T17:00:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b17c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6357378f0 0x55c635737cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b17c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6357378f0 0x55c635737cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41952/0 (socket says 172.21.15.69:41952) 2024-05-14T17:00:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b17c5640 1 -- 172.21.15.69:0/394423811 learned_addr learned my addr 172.21.15.69:0/394423811 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/394423811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c635738210 0x55c635742590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/394423811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c635744ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b17c5640 1 -- 172.21.15.69:0/394423811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c635738210 msgr2=0x55c635742590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/394423811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c635738210 0x55c635742590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b17c5640 1 -- 172.21.15.69:0/394423811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 msgr2=0x55c635744ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.451+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/394423811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c635744ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.452+0000 7fc8b17c5640 1 -- 172.21.15.69:0/394423811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6353af210 con 0x55c6357378f0 2024-05-14T17:00:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.452+0000 7fc8a2ffd640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc8980b2a10 0x7fc8980b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.452+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc8980b2a10 0x7fc8980b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.452+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/394423811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6357378f0 0x55c635737cd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc89c004620 tx=0x7fc89c00b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4565067cc186e344 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.452+0000 7fc8a2ffd640 1 -- 172.21.15.69:0/2127196305 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fc8a40e2aa0 con 0x7fc8ac106e90 2024-05-14T17:00:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.452+0000 7fc897fff640 1 -- 172.21.15.69:0/394423811 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c00f3f0 con 0x55c6357378f0 2024-05-14T17:00:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.452+0000 7fc897fff640 1 -- 172.21.15.69:0/394423811 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c00fa40 con 0x55c6357378f0 2024-05-14T17:00:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.452+0000 7fc897fff640 1 -- 172.21.15.69:0/394423811 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c012920 con 0x55c6357378f0 2024-05-14T17:00:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 -- 172.21.15.69:0/394423811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6357378f0 msgr2=0x55c635737cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc8980b2a10 0x7fc8980b4ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8000c00 tx=0x7fc8a80058b0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 --2- 172.21.15.69:0/394423811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6357378f0 0x55c635737cd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc89c004620 tx=0x7fc89c00b6f0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 -- 172.21.15.69:0/394423811 shutdown_connections 2024-05-14T17:00:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 --2- 172.21.15.69:0/394423811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c635744ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 --2- 172.21.15.69:0/394423811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c635738210 0x55c635742590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 --2- 172.21.15.69:0/394423811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6357378f0 0x55c635737cd0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 -- 172.21.15.69:0/394423811 >> 172.21.15.69:0/394423811 conn(0x55c635733e00 msgr2=0x55c6357341e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 -- 172.21.15.69:0/394423811 shutdown_connections 2024-05-14T17:00:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 -- 172.21.15.69:0/394423811 wait complete. 2024-05-14T17:00:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 -- start start 2024-05-14T17:00:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.453+0000 7fc8b7028740 1 Processor -- start 2024-05-14T17:00:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c635742400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635738210 0x55c63573f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635742ad0 0x55c63573fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b7028740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c635380790 con 0x55c635742ad0 2024-05-14T17:00:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b7028740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c635656890 con 0x55c635738210 2024-05-14T17:00:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b7028740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c6357457a0 con 0x55c6357378f0 2024-05-14T17:00:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b7028740 10 client.0 _ll_register_callbacks cb 0x55c635659d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b27c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635742ad0 0x55c63573fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b27c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635742ad0 0x55c63573fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41958/0 (socket says 172.21.15.69:41958) 2024-05-14T17:00:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b27c7640 1 -- 172.21.15.69:0/4276928416 learned_addr learned my addr 172.21.15.69:0/4276928416 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635738210 0x55c63573f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c635742400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b27c7640 1 -- 172.21.15.69:0/4276928416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 msgr2=0x55c635742400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b27c7640 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c635742400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b27c7640 1 -- 172.21.15.69:0/4276928416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635738210 msgr2=0x55c63573f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b27c7640 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635738210 0x55c63573f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b27c7640 1 -- 172.21.15.69:0/4276928416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c63547b620 con 0x55c635742ad0 2024-05-14T17:00:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b27c7640 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635742ad0 0x55c63573fc30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40a9d00 tx=0x7fc8a40a9880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3843550537f9211a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8957fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:00:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8957fa640 1 -- 172.21.15.69:0/4276928416 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a401f070 con 0x55c635742ad0 2024-05-14T17:00:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8957fa640 1 -- 172.21.15.69:0/4276928416 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fc8a40b2740 con 0x55c635742ad0 2024-05-14T17:00:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.454+0000 7fc8b7028740 1 -- 172.21.15.69:0/4276928416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc89c00fa40 con 0x55c635742ad0 2024-05-14T17:00:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8957fa640 1 -- 172.21.15.69:0/4276928416 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc8a40ae120 con 0x55c635742ad0 2024-05-14T17:00:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8b7028740 10 client.16569 fetch_fsmap learned FSMap version 47 2024-05-14T17:00:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8b7028740 1 -- 172.21.15.69:0/4276928416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c6357511a0 con 0x55c635742ad0 2024-05-14T17:00:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8957fa640 1 -- 172.21.15.69:0/4276928416 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c00fa40 con 0x55c635742ad0 2024-05-14T17:00:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8957fa640 1 -- 172.21.15.69:0/4276928416 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fc8a400f040 con 0x55c635742ad0 2024-05-14T17:00:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8b7028740 10 client.16569 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:00:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8b7028740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8b7028740 20 client.16569 populate_metadata read hostname 'smithi069' 2024-05-14T17:00:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8b7028740 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c6357526b0 0x55c635754aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8b7028740 4 client.16569 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8b7028740 1 -- 172.21.15.69:0/4276928416 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6356acb80 con 0x55c6357526b0 2024-05-14T17:00:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.455+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c6357526b0 0x55c635754aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.456+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c6357526b0 0x55c635754aa0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94831cc405d79fb server_cookie=f0b238b4468c8ab5 in_seq=0 out_seq=0 2024-05-14T17:00:57.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.456+0000 7fc8957fa640 10 client.16569 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.456+0000 7fc8957fa640 1 -- 172.21.15.69:0/4276928416 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6356acb80 con 0x55c6357526b0 2024-05-14T17:00:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.456+0000 7fc8957fa640 10 client.16569 handle_command_reply: tid=0 2024-05-14T17:00:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.456+0000 7fc8b7028740 1 client.16569 shutdown 2024-05-14T17:00:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.456+0000 7fc8b7028740 1 -- 172.21.15.69:0/4276928416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635742ad0 msgr2=0x55c63573fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.456+0000 7fc8b7028740 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635742ad0 0x55c63573fc30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40a9d00 tx=0x7fc8a40a9880 comp rx=0 tx=0).stop 2024-05-14T17:00:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.457+0000 7fc8b7028740 1 -- 172.21.15.69:0/4276928416 shutdown_connections 2024-05-14T17:00:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.457+0000 7fc8b7028740 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c6357526b0 0x55c635754aa0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.457+0000 7fc8b7028740 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635742ad0 0x55c63573fc30 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.457+0000 7fc8b7028740 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635738210 0x55c63573f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.457+0000 7fc8b7028740 1 --2- 172.21.15.69:0/4276928416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c635742400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.457+0000 7fc8957fa640 0 client.16569 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:00:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.457+0000 7fc8b7028740 1 -- 172.21.15.69:0/4276928416 >> 172.21.15.69:0/4276928416 conn(0x55c635733e00 msgr2=0x55c635733100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.457+0000 7fc8b7028740 1 -- 172.21.15.69:0/4276928416 shutdown_connections 2024-05-14T17:00:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.457+0000 7fc8b7028740 1 -- 172.21.15.69:0/4276928416 wait complete. 2024-05-14T17:00:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.457+0000 7fc8b7028740 20 client.16569 trim_cache size 0 max 16384 2024-05-14T17:00:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.472+0000 7fc8b7028740 1 Processor -- start 2024-05-14T17:00:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.472+0000 7fc8b7028740 1 -- start start 2024-05-14T17:00:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.472+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c63574e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.472+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c635741f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.472+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c63575a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.472+0000 7fc8b7028740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac1b75f0 con 0x55c635738210 2024-05-14T17:00:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.472+0000 7fc8b7028740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac1b4e30 con 0x55c635742ad0 2024-05-14T17:00:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.472+0000 7fc8b7028740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac1b2ca0 con 0x55c6357378f0 2024-05-14T17:00:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b17c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c635741f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b17c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c635741f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41974/0 (socket says 172.21.15.69:41974) 2024-05-14T17:00:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b17c5640 1 -- 172.21.15.69:0/1619958193 learned_addr learned my addr 172.21.15.69:0/1619958193 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:00:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/1619958193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c63574e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b27c7640 1 --2- 172.21.15.69:0/1619958193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c63575a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b17c5640 1 -- 172.21.15.69:0/1619958193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 msgr2=0x55c63574e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/1619958193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c63574e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b17c5640 1 -- 172.21.15.69:0/1619958193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 msgr2=0x55c63575a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/1619958193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c63575a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b17c5640 1 -- 172.21.15.69:0/1619958193 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6357550d0 con 0x55c635738210 2024-05-14T17:00:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/1619958193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c635741f30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00bed0 tx=0x7fc89c005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9d9c9238b15886 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8977fe640 1 -- 172.21.15.69:0/1619958193 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c02a070 con 0x55c635738210 2024-05-14T17:00:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8977fe640 1 -- 172.21.15.69:0/1619958193 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c0110d0 con 0x55c635738210 2024-05-14T17:00:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.473+0000 7fc8977fe640 1 -- 172.21.15.69:0/1619958193 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c004680 con 0x55c635738210 2024-05-14T17:00:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 -- 172.21.15.69:0/1619958193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 msgr2=0x55c635741f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 --2- 172.21.15.69:0/1619958193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c635741f30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00bed0 tx=0x7fc89c005b20 comp rx=0 tx=0).stop 2024-05-14T17:00:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 -- 172.21.15.69:0/1619958193 shutdown_connections 2024-05-14T17:00:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 --2- 172.21.15.69:0/1619958193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c63575a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 --2- 172.21.15.69:0/1619958193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c635741f30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 --2- 172.21.15.69:0/1619958193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c63574e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 -- 172.21.15.69:0/1619958193 >> 172.21.15.69:0/1619958193 conn(0x55c635733e00 msgr2=0x55c635739b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 -- 172.21.15.69:0/1619958193 shutdown_connections 2024-05-14T17:01:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 -- 172.21.15.69:0/1619958193 wait complete. 2024-05-14T17:01:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 -- start start 2024-05-14T17:01:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.474+0000 7fc8b7028740 1 Processor -- start 2024-05-14T17:01:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c63573c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c63573ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b7028740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c63573cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b7028740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac1b32f0 con 0x55c635738210 2024-05-14T17:01:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b7028740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac1b4760 con 0x55c635742ad0 2024-05-14T17:01:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b7028740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac1b39c0 con 0x55c6357378f0 2024-05-14T17:01:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b7028740 10 client.0 _ll_register_callbacks cb 0x7fc8a40a94e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b17c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c63573ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b17c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c63573ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41990/0 (socket says 172.21.15.69:41990) 2024-05-14T17:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b17c5640 1 -- 172.21.15.69:0/2308569767 learned_addr learned my addr 172.21.15.69:0/2308569767 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b27c7640 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c63573cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c63573c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b17c5640 1 -- 172.21.15.69:0/2308569767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 msgr2=0x55c63573c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c63573c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b17c5640 1 -- 172.21.15.69:0/2308569767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 msgr2=0x55c63573cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c63573cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b17c5640 1 -- 172.21.15.69:0/2308569767 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6357bbec0 con 0x55c635738210 2024-05-14T17:01:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b17c5640 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c63573ca40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc89c011ca0 tx=0x7fc89c011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e623fea8f1d2258 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8957fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:01:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8957fa640 1 -- 172.21.15.69:0/2308569767 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c02a040 con 0x55c635738210 2024-05-14T17:01:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8957fa640 1 -- 172.21.15.69:0/2308569767 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fc89c00f3f0 con 0x55c635738210 2024-05-14T17:01:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.475+0000 7fc8b7028740 1 -- 172.21.15.69:0/2308569767 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc89c0110d0 con 0x55c635738210 2024-05-14T17:01:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8957fa640 1 -- 172.21.15.69:0/2308569767 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc89c00f6f0 con 0x55c635738210 2024-05-14T17:01:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8b7028740 10 client.16575 fetch_fsmap learned FSMap version 47 2024-05-14T17:01:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8b7028740 1 -- 172.21.15.69:0/2308569767 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c6357bd170 con 0x55c635738210 2024-05-14T17:01:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8957fa640 1 -- 172.21.15.69:0/2308569767 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c0110d0 con 0x55c635738210 2024-05-14T17:01:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8957fa640 1 -- 172.21.15.69:0/2308569767 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fc89c004810 con 0x55c635738210 2024-05-14T17:01:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8b7028740 10 client.16575 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:01:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8b7028740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8b7028740 20 client.16575 populate_metadata read hostname 'smithi069' 2024-05-14T17:01:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8b7028740 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c6357bd670 0x55c6357bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8b7028740 4 client.16575 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8b7028740 1 -- 172.21.15.69:0/2308569767 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6356a6360 con 0x55c6357bd670 2024-05-14T17:01:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.476+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c6357bd670 0x55c6357bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.477+0000 7fc8957fa640 10 client.16575 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.477+0000 7fc8b1fc6640 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c6357bd670 0x55c6357bfa90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4fc0531c7a1d79 server_cookie=88041c06eb8a0616 in_seq=0 out_seq=0 2024-05-14T17:01:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8957fa640 1 -- 172.21.15.69:0/2308569767 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c6356a6360 con 0x55c6357bd670 2024-05-14T17:01:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8957fa640 10 client.16575 handle_command_reply: tid=0 2024-05-14T17:01:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 client.16575 shutdown 2024-05-14T17:01:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 -- 172.21.15.69:0/2308569767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 msgr2=0x55c63573ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c63573ca40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc89c011ca0 tx=0x7fc89c011ef0 comp rx=0 tx=0).stop 2024-05-14T17:01:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 -- 172.21.15.69:0/2308569767 shutdown_connections 2024-05-14T17:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c6357bd670 0x55c6357bfa90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c635742ad0 0x55c63573cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c635738210 0x55c63573ca40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 --2- 172.21.15.69:0/2308569767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6357378f0 0x55c63573c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8957fa640 0 client.16575 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 -- 172.21.15.69:0/2308569767 >> 172.21.15.69:0/2308569767 conn(0x55c635733e00 msgr2=0x55c6357392a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 -- 172.21.15.69:0/2308569767 shutdown_connections 2024-05-14T17:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 1 -- 172.21.15.69:0/2308569767 wait complete. 2024-05-14T17:01:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b7028740 20 client.16575 trim_cache size 0 max 16384 2024-05-14T17:01:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:01:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:01:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:01:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 -- 172.21.15.69:0/2127196305 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc8980b2a10 msgr2=0x7fc8980b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc8980b2a10 0x7fc8980b4ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8000c00 tx=0x7fc8a80058b0 comp rx=0 tx=0).stop 2024-05-14T17:01:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 -- 172.21.15.69:0/2127196305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8ac106e90 msgr2=0x7fc8ac19f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8ac106e90 0x7fc8ac19f620 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400e960 tx=0x7fc8a400ee20 comp rx=0 tx=0).stop 2024-05-14T17:01:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 -- 172.21.15.69:0/2127196305 shutdown_connections 2024-05-14T17:01:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc8980b2a10 0x7fc8980b4ed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8ac106e90 0x7fc8ac19f620 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac1023b0 0x7fc8ac1a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 --2- 172.21.15.69:0/2127196305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8ac101a90 0x7fc8ac1a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 -- 172.21.15.69:0/2127196305 >> 172.21.15.69:0/2127196305 conn(0x7fc8ac078b60 msgr2=0x7fc8ac104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 -- 172.21.15.69:0/2127196305 shutdown_connections 2024-05-14T17:01:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:44.489+0000 7fc8b422a640 1 -- 172.21.15.69:0/2127196305 wait complete. 2024-05-14T17:01:03.538 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 615.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:03.578 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:51 smithi092 ceph-mon[35763]: pgmap v343: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-05-14T17:01:03.619 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:52 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2394436415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:01:03.661 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:53 smithi092 ceph-mon[35763]: pgmap v344: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:01:03.703 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:54 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:03.744 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:55 smithi092 ceph-mon[35763]: pgmap v345: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-05-14T17:01:03.794 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:57 smithi092 ceph-mon[35763]: pgmap v346: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-05-14T17:01:03.836 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:00:59 smithi092 ceph-mon[35763]: pgmap v347: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-05-14T17:01:03.878 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:01 smithi092 ceph-mon[35763]: pgmap v348: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-05-14T17:01:03.920 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:51 smithi203 ceph-mon[35787]: pgmap v343: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-05-14T17:01:04.011 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:52 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2394436415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:01:04.086 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:53 smithi203 ceph-mon[35787]: pgmap v344: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:01:04.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:54 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:04.186 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:55 smithi203 ceph-mon[35787]: pgmap v345: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-05-14T17:01:04.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:57 smithi203 ceph-mon[35787]: pgmap v346: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-05-14T17:01:04.269 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:00:59 smithi203 ceph-mon[35787]: pgmap v347: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-05-14T17:01:04.319 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:01 smithi203 ceph-mon[35787]: pgmap v348: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-05-14T17:01:04.362 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:45 smithi069 ceph-mon[31507]: pgmap v340: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-05-14T17:01:04.403 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:47 smithi069 ceph-mon[31507]: pgmap v341: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-05-14T17:01:04.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:49 smithi069 ceph-mon[31507]: pgmap v342: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-05-14T17:01:04.488 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 620.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:04.529 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:04.570 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:51 smithi069 ceph-mon[31507]: pgmap v343: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-05-14T17:01:04.614 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:03 smithi092 ceph-mon[35763]: pgmap v349: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 67 KiB/s wr, 29 op/s 2024-05-14T17:01:04.653 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:03 smithi203 ceph-mon[35787]: pgmap v349: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 67 KiB/s wr, 29 op/s 2024-05-14T17:01:04.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.318+0000 7fba09eb9700 1 -- 172.21.15.69:0/4134886475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba04102c40 msgr2=0x7fba041030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:04.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.318+0000 7fba09eb9700 1 --2- 172.21.15.69:0/4134886475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba04102c40 0x7fba041030c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009990 tx=0x7fb9f8009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.318+0000 7fba09eb9700 1 -- 172.21.15.69:0/4134886475 shutdown_connections 2024-05-14T17:01:04.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.318+0000 7fba09eb9700 1 --2- 172.21.15.69:0/4134886475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba04103600 0x7fba0410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.318+0000 7fba09eb9700 1 --2- 172.21.15.69:0/4134886475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba041084c0 0x7fba041088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.318+0000 7fba09eb9700 1 --2- 172.21.15.69:0/4134886475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba04102c40 0x7fba041030c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.318+0000 7fba09eb9700 1 -- 172.21.15.69:0/4134886475 >> 172.21.15.69:0/4134886475 conn(0x7fba04078ab0 msgr2=0x7fba040ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:05.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.318+0000 7fba09eb9700 1 -- 172.21.15.69:0/4134886475 shutdown_connections 2024-05-14T17:01:05.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.318+0000 7fba09eb9700 1 -- 172.21.15.69:0/4134886475 wait complete. 2024-05-14T17:01:05.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba09eb9700 1 Processor -- start 2024-05-14T17:01:05.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba09eb9700 1 -- start start 2024-05-14T17:01:05.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba09eb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba04102c40 0x7fba0419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:05.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba09eb9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba04103600 0x7fba0419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:05.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba09eb9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba041084c0 0x7fba04197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:05.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba09eb9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba041125d0 con 0x7fba041084c0 2024-05-14T17:01:05.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba09eb9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba04112450 con 0x7fba04102c40 2024-05-14T17:01:05.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba09eb9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba04112750 con 0x7fba04103600 2024-05-14T17:01:05.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba03fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba041084c0 0x7fba04197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:05.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba03fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba041084c0 0x7fba04197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56508/0 (socket says 172.21.15.69:56508) 2024-05-14T17:01:05.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.319+0000 7fba03fff700 1 -- 172.21.15.69:0/2394436415 learned_addr learned my addr 172.21.15.69:0/2394436415 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:05.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba02ffd700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba04103600 0x7fba0419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:05.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba037fe700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba04102c40 0x7fba0419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:05.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba03fff700 1 -- 172.21.15.69:0/2394436415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba04103600 msgr2=0x7fba0419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:05.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba03fff700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba04103600 0x7fba0419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:05.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba03fff700 1 -- 172.21.15.69:0/2394436415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba04102c40 msgr2=0x7fba0419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:05.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba03fff700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba04102c40 0x7fba0419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:05.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba03fff700 1 -- 172.21.15.69:0/2394436415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba04198410 con 0x7fba041084c0 2024-05-14T17:01:06.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba03fff700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba041084c0 0x7fba04197b10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc007ee0 tx=0x7fb9fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:06.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba00ff9700 1 -- 172.21.15.69:0/2394436415 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc004bb0 con 0x7fba041084c0 2024-05-14T17:01:06.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba00ff9700 1 -- 172.21.15.69:0/2394436415 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9fc004d50 con 0x7fba041084c0 2024-05-14T17:01:06.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba09eb9700 1 -- 172.21.15.69:0/2394436415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba04198700 con 0x7fba041084c0 2024-05-14T17:01:06.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba00ff9700 1 -- 172.21.15.69:0/2394436415 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc005920 con 0x7fba041084c0 2024-05-14T17:01:06.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.320+0000 7fba09eb9700 1 -- 172.21.15.69:0/2394436415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba041b54b0 con 0x7fba041084c0 2024-05-14T17:01:06.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.322+0000 7fba09eb9700 1 -- 172.21.15.69:0/2394436415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba04103c50 con 0x7fba041084c0 2024-05-14T17:01:06.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.323+0000 7fba00ff9700 1 -- 172.21.15.69:0/2394436415 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb9fc01a020 con 0x7fba041084c0 2024-05-14T17:01:06.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.324+0000 7fba00ff9700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb9ec0b29f0 0x7fb9ec0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:06.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.324+0000 7fba037fe700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb9ec0b29f0 0x7fb9ec0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:06.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.324+0000 7fba00ff9700 1 -- 172.21.15.69:0/2394436415 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fb9fc0e1ce0 con 0x7fba041084c0 2024-05-14T17:01:06.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.324+0000 7fba037fe700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb9ec0b29f0 0x7fb9ec0b4eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba0406ae90 tx=0x7fb9f4007400 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:06.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.328+0000 7fba00ff9700 1 -- 172.21.15.69:0/2394436415 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb9fc010070 con 0x7fba041084c0 2024-05-14T17:01:06.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.587+0000 7fba09eb9700 1 -- 172.21.15.69:0/2394436415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba041b5680 con 0x7fba041084c0 2024-05-14T17:01:06.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.588+0000 7fba00ff9700 1 -- 172.21.15.69:0/2394436415 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11863 (secure 0 0 0) 0x7fb9fc0a96b0 con 0x7fba041084c0 2024-05-14T17:01:06.855 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:01:06.905 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:00:13.714949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25027":{"gid":25027,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1947390593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1947390593},{"type":"v1","addr":"172.21.15.203:6837","nonce":1947390593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24446,24422,24433]},"id":1}]} 2024-05-14T17:01:06.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 -- 172.21.15.69:0/2394436415 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb9ec0b29f0 msgr2=0x7fb9ec0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb9ec0b29f0 0x7fb9ec0b4eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba0406ae90 tx=0x7fb9f4007400 comp rx=0 tx=0).stop 2024-05-14T17:01:07.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 -- 172.21.15.69:0/2394436415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba041084c0 msgr2=0x7fba04197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:07.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba041084c0 0x7fba04197b10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc007ee0 tx=0x7fb9fc00d3b0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 -- 172.21.15.69:0/2394436415 shutdown_connections 2024-05-14T17:01:07.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb9ec0b29f0 0x7fb9ec0b4eb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba041084c0 0x7fba04197b10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba04102c40 0x7fba0419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 --2- 172.21.15.69:0/2394436415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba04103600 0x7fba0419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 -- 172.21.15.69:0/2394436415 >> 172.21.15.69:0/2394436415 conn(0x7fba04078ab0 msgr2=0x7fba04101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:07.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 -- 172.21.15.69:0/2394436415 shutdown_connections 2024-05-14T17:01:07.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:00:52.590+0000 7fba09eb9700 1 -- 172.21.15.69:0/2394436415 wait complete. 2024-05-14T17:01:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 47 2024-05-14T17:01:07.656 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:52 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2394436415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:01:07.706 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:05 smithi092 ceph-mon[35763]: pgmap v350: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 62 KiB/s wr, 26 op/s 2024-05-14T17:01:07.756 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70573]: *** Caught signal (Segmentation fault) ** 2024-05-14T17:01:07.805 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[70573]: in thread 7f229df77700 thread_name:safe_timer 2024-05-14T17:01:07.856 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:04 smithi203 podman[72532]: 2024-05-14 17:01:04.823250589 +0000 UTC m=+0.021557557 container died d649402271e5a1d7485b2a2f5894f22907eda364d08f4c4f70fd0d756625e4cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0) 2024-05-14T17:01:07.906 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:05 smithi203 podman[72532]: 2024-05-14 17:01:05.522223204 +0000 UTC m=+0.720530194 container remove d649402271e5a1d7485b2a2f5894f22907eda364d08f4c4f70fd0d756625e4cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-14T17:01:07.956 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:05 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-05-14T17:01:08.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:05 smithi203 ceph-mon[35787]: pgmap v350: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 62 KiB/s wr, 26 op/s 2024-05-14T17:01:08.057 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id c837d4f6 --release --await' 2024-05-14T17:01:08.106 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id c837d4f6 --release --await 2024-05-14T17:01:08.173 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:05 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-14T17:01:08.223 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:05 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Consumed 1.811s CPU time. 2024-05-14T17:01:08.273 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:53 smithi069 ceph-mon[31507]: pgmap v344: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:01:08.322 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:54 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:08.373 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 625.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:08.423 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:55 smithi069 ceph-mon[31507]: pgmap v345: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-05-14T17:01:08.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:07 smithi092 ceph-mon[35763]: pgmap v351: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-05-14T17:01:08.523 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:07 smithi203 ceph-mon[35787]: pgmap v351: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-05-14T17:01:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.589+0000 7f6659ee1640 1 -- 172.21.15.69:0/3470120245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6654108a70 msgr2=0x7f6654108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.589+0000 7f6659ee1640 1 --2- 172.21.15.69:0/3470120245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6654108a70 0x7f6654108e50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6644009930 tx=0x7f664402a420 comp rx=0 tx=0).stop 2024-05-14T17:01:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.589+0000 7f6659ee1640 1 -- 172.21.15.69:0/3470120245 shutdown_connections 2024-05-14T17:01:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.589+0000 7f6659ee1640 1 --2- 172.21.15.69:0/3470120245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6654103420 0x7f665410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.589+0000 7f6659ee1640 1 --2- 172.21.15.69:0/3470120245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6654102a60 0x7f6654102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.589+0000 7f6659ee1640 1 --2- 172.21.15.69:0/3470120245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6654108a70 0x7f6654108e50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.590+0000 7f6659ee1640 1 -- 172.21.15.69:0/3470120245 >> 172.21.15.69:0/3470120245 conn(0x7f6654079ee0 msgr2=0x7f665407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.590+0000 7f6659ee1640 1 -- 172.21.15.69:0/3470120245 shutdown_connections 2024-05-14T17:01:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.590+0000 7f6659ee1640 1 -- 172.21.15.69:0/3470120245 wait complete. 2024-05-14T17:01:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.590+0000 7f6659ee1640 1 Processor -- start 2024-05-14T17:01:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.590+0000 7f6659ee1640 1 -- start start 2024-05-14T17:01:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f6659ee1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6654102a60 0x7f6654199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f6659ee1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6654103420 0x7f6654199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f6659ee1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6654108a70 0x7f665419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f6659ee1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6654112d00 con 0x7f6654102a60 2024-05-14T17:01:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f6659ee1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6654112b80 con 0x7f6654103420 2024-05-14T17:01:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f6659ee1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6654112e80 con 0x7f6654108a70 2024-05-14T17:01:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f66537fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6654102a60 0x7f6654199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f66537fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6654102a60 0x7f6654199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39056/0 (socket says 172.21.15.69:39056) 2024-05-14T17:01:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f66537fe640 1 -- 172.21.15.69:0/3771649921 learned_addr learned my addr 172.21.15.69:0/3771649921 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f6653fff640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6654108a70 0x7f665419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f6652ffd640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6654103420 0x7f6654199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f66537fe640 1 -- 172.21.15.69:0/3771649921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6654108a70 msgr2=0x7f665419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f66537fe640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6654108a70 0x7f665419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f66537fe640 1 -- 172.21.15.69:0/3771649921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6654103420 msgr2=0x7f6654199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f66537fe640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6654103420 0x7f6654199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f66537fe640 1 -- 172.21.15.69:0/3771649921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665419e0a0 con 0x7f6654102a60 2024-05-14T17:01:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f66537fe640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6654102a60 0x7f6654199100 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6644005940 tx=0x7f6644004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f6650ff9640 1 -- 172.21.15.69:0/3771649921 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664403f030 con 0x7f6654102a60 2024-05-14T17:01:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.591+0000 7f6650ff9640 1 -- 172.21.15.69:0/3771649921 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6644003aa0 con 0x7f6654102a60 2024-05-14T17:01:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.592+0000 7f6659ee1640 1 -- 172.21.15.69:0/3771649921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665419e330 con 0x7f6654102a60 2024-05-14T17:01:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.592+0000 7f6650ff9640 1 -- 172.21.15.69:0/3771649921 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644035070 con 0x7f6654102a60 2024-05-14T17:01:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.592+0000 7f6659ee1640 1 -- 172.21.15.69:0/3771649921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66541a5c20 con 0x7f6654102a60 2024-05-14T17:01:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.594+0000 7f6650ff9640 1 -- 172.21.15.69:0/3771649921 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6644042030 con 0x7f6654102a60 2024-05-14T17:01:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.594+0000 7f665cbff740 1 Processor -- start 2024-05-14T17:01:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.594+0000 7f665cbff740 1 -- start start 2024-05-14T17:01:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f665cbff740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6adb8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f665cbff740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adc3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f665cbff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f665cbff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558b6ad2bc30 con 0x558b6adc3ad0 2024-05-14T17:01:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f665cbff740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558b6ad1b7d0 con 0x558b6adb9210 2024-05-14T17:01:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f665cbff740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558b6ad1f810 con 0x558b6adb88f0 2024-05-14T17:01:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39066/0 (socket says 172.21.15.69:39066) 2024-05-14T17:01:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6652ffd640 1 -- 172.21.15.69:0/3461348919 learned_addr learned my addr 172.21.15.69:0/3461348919 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f66537fe640 1 --2- 172.21.15.69:0/3461348919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adc3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6653fff640 1 --2- 172.21.15.69:0/3461348919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6adb8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6650ff9640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f663c0b2a90 0x7f663c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6652ffd640 1 -- 172.21.15.69:0/3461348919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 msgr2=0x558b6adb8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6652ffd640 1 --2- 172.21.15.69:0/3461348919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6adb8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6652ffd640 1 -- 172.21.15.69:0/3461348919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 msgr2=0x558b6adc3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6652ffd640 1 --2- 172.21.15.69:0/3461348919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adc3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6652ffd640 1 -- 172.21.15.69:0/3461348919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b6aa30210 con 0x558b6adc3ad0 2024-05-14T17:01:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f66537fe640 1 --2- 172.21.15.69:0/3461348919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adc3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6653fff640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f663c0b2a90 0x7f663c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6650ff9640 1 -- 172.21.15.69:0/3771649921 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f6644102bd0 con 0x7f6654102a60 2024-05-14T17:01:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.595+0000 7f6652ffd640 1 --2- 172.21.15.69:0/3461348919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc5ec0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f664800cc40 tx=0x7f664800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a3926021302804 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f6639ffb640 1 -- 172.21.15.69:0/3461348919 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648014070 con 0x558b6adc3ad0 2024-05-14T17:01:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f6639ffb640 1 -- 172.21.15.69:0/3461348919 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66480047a0 con 0x558b6adc3ad0 2024-05-14T17:01:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f6639ffb640 1 -- 172.21.15.69:0/3461348919 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648010670 con 0x558b6adc3ad0 2024-05-14T17:01:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f665cbff740 1 -- 172.21.15.69:0/3461348919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 msgr2=0x558b6adc5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f665cbff740 1 --2- 172.21.15.69:0/3461348919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc5ec0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f664800cc40 tx=0x7f664800e720 comp rx=0 tx=0).stop 2024-05-14T17:01:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f6653fff640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f663c0b2a90 0x7f663c0b4f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f664c007e70 tx=0x7f664c004a40 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f665cbff740 1 -- 172.21.15.69:0/3461348919 shutdown_connections 2024-05-14T17:01:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f665cbff740 1 --2- 172.21.15.69:0/3461348919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc5ec0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f665cbff740 1 --2- 172.21.15.69:0/3461348919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adc3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f665cbff740 1 --2- 172.21.15.69:0/3461348919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6adb8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f665cbff740 1 -- 172.21.15.69:0/3461348919 >> 172.21.15.69:0/3461348919 conn(0x558b6adb4e00 msgr2=0x558b6adb51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f665cbff740 1 -- 172.21.15.69:0/3461348919 shutdown_connections 2024-05-14T17:01:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.596+0000 7f665cbff740 1 -- 172.21.15.69:0/3461348919 wait complete. 2024-05-14T17:01:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f665cbff740 1 -- start start 2024-05-14T17:01:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f665cbff740 1 Processor -- start 2024-05-14T17:01:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f665cbff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb88f0 0x558b6adc3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f665cbff740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adc0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f665cbff740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adc3ad0 0x558b6adc0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f665cbff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558b6aa01790 con 0x558b6adb88f0 2024-05-14T17:01:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f665cbff740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558b6acd7890 con 0x558b6adb9210 2024-05-14T17:01:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f665cbff740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558b6adc67a0 con 0x558b6adc3ad0 2024-05-14T17:01:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f665cbff740 10 client.0 _ll_register_callbacks cb 0x558b6acdad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f66537fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb88f0 0x558b6adc3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f66537fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb88f0 0x558b6adc3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39074/0 (socket says 172.21.15.69:39074) 2024-05-14T17:01:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f66537fe640 1 -- 172.21.15.69:0/725092653 learned_addr learned my addr 172.21.15.69:0/725092653 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.597+0000 7f6652ffd640 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adc0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f6653fff640 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adc3ad0 0x558b6adc0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f66537fe640 1 -- 172.21.15.69:0/725092653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adc3ad0 msgr2=0x558b6adc0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f66537fe640 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adc3ad0 0x558b6adc0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f66537fe640 1 -- 172.21.15.69:0/725092653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 msgr2=0x558b6adc0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f66537fe640 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adc0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f66537fe640 1 -- 172.21.15.69:0/725092653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b6aafc620 con 0x558b6adb88f0 2024-05-14T17:01:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f6653fff640 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adc3ad0 0x558b6adc0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f6652ffd640 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adc0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f66537fe640 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb88f0 0x558b6adc3430 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f66440c7110 tx=0x7f6644031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09e3470e2936bf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f6632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:01:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f6632ffd640 1 -- 172.21.15.69:0/725092653 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644049040 con 0x558b6adb88f0 2024-05-14T17:01:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f6632ffd640 1 -- 172.21.15.69:0/725092653 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f66440c6130 con 0x558b6adb88f0 2024-05-14T17:01:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f665cbff740 1 -- 172.21.15.69:0/725092653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66480047a0 con 0x558b6adb88f0 2024-05-14T17:01:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f6632ffd640 1 -- 172.21.15.69:0/725092653 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f66440c97f0 con 0x558b6adb88f0 2024-05-14T17:01:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f665cbff740 10 client.16593 fetch_fsmap learned FSMap version 48 2024-05-14T17:01:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f665cbff740 1 -- 172.21.15.69:0/725092653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558b6add2200 con 0x558b6adb88f0 2024-05-14T17:01:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f6632ffd640 1 -- 172.21.15.69:0/725092653 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66480047a0 con 0x558b6adb88f0 2024-05-14T17:01:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f6632ffd640 1 -- 172.21.15.69:0/725092653 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 48) ==== 8346+0+0 (secure 0 0 0) 0x7f66440e0020 con 0x558b6adb88f0 2024-05-14T17:01:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f665cbff740 10 client.16593 fetch_fsmap finished waiting for FSMap version 48 2024-05-14T17:01:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f665cbff740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f665cbff740 20 client.16593 populate_metadata read hostname 'smithi069' 2024-05-14T17:01:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f665cbff740 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558b6add3700 0x558b6add5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f665cbff740 4 client.16593 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:01:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.598+0000 7f665cbff740 1 -- 172.21.15.69:0/725092653 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b6ad2db80 con 0x558b6add3700 2024-05-14T17:01:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.599+0000 7f6652ffd640 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558b6add3700 0x558b6add5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.599+0000 7f6632ffd640 10 client.16593 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.599+0000 7f6652ffd640 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558b6add3700 0x558b6add5b10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965c07940e7d528e server_cookie=b09a9576272d5873 in_seq=0 out_seq=0 2024-05-14T17:01:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f6632ffd640 1 -- 172.21.15.69:0/725092653 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b6ad2db80 con 0x558b6add3700 2024-05-14T17:01:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f6632ffd640 10 client.16593 handle_command_reply: tid=0 2024-05-14T17:01:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 client.16593 shutdown 2024-05-14T17:01:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 -- 172.21.15.69:0/725092653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb88f0 msgr2=0x558b6adc3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb88f0 0x558b6adc3430 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f66440c7110 tx=0x7f6644031400 comp rx=0 tx=0).stop 2024-05-14T17:01:14.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 -- 172.21.15.69:0/725092653 shutdown_connections 2024-05-14T17:01:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558b6add3700 0x558b6add5b10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adc3ad0 0x558b6adc0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f6632ffd640 0 client.16593 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adc0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 --2- 172.21.15.69:0/725092653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb88f0 0x558b6adc3430 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 -- 172.21.15.69:0/725092653 >> 172.21.15.69:0/725092653 conn(0x558b6adb4e00 msgr2=0x558b6adb4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 -- 172.21.15.69:0/725092653 shutdown_connections 2024-05-14T17:01:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 1 -- 172.21.15.69:0/725092653 wait complete. 2024-05-14T17:01:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.600+0000 7f665cbff740 20 client.16593 trim_cache size 0 max 16384 2024-05-14T17:01:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.615+0000 7f665cbff740 1 Processor -- start 2024-05-14T17:01:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.615+0000 7f665cbff740 1 -- start start 2024-05-14T17:01:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.615+0000 7f665cbff740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6adcbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.615+0000 7f665cbff740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adcf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.615+0000 7f665cbff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.615+0000 7f665cbff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f66541b0f50 con 0x558b6adc3ad0 2024-05-14T17:01:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.615+0000 7f665cbff740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f66541ae7c0 con 0x558b6adb9210 2024-05-14T17:01:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.615+0000 7f665cbff740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66541ac630 con 0x558b6adb88f0 2024-05-14T17:01:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6653fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6653fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39084/0 (socket says 172.21.15.69:39084) 2024-05-14T17:01:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6653fff640 1 -- 172.21.15.69:0/338624879 learned_addr learned my addr 172.21.15.69:0/338624879 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6652ffd640 1 --2- 172.21.15.69:0/338624879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adcf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f66537fe640 1 --2- 172.21.15.69:0/338624879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6adcbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6653fff640 1 -- 172.21.15.69:0/338624879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 msgr2=0x558b6adcbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6653fff640 1 --2- 172.21.15.69:0/338624879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6adcbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6653fff640 1 -- 172.21.15.69:0/338624879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 msgr2=0x558b6adcf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6653fff640 1 --2- 172.21.15.69:0/338624879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adcf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6653fff640 1 -- 172.21.15.69:0/338624879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b6adb9040 con 0x558b6adc3ad0 2024-05-14T17:01:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6652ffd640 1 --2- 172.21.15.69:0/338624879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adcf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f6653fff640 1 --2- 172.21.15.69:0/338624879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc2f60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f664c01e530 tx=0x7f664c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc9b7be2b560829 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f66397fa640 1 -- 172.21.15.69:0/338624879 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c012920 con 0x558b6adc3ad0 2024-05-14T17:01:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f66397fa640 1 -- 172.21.15.69:0/338624879 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f664c01b420 con 0x558b6adc3ad0 2024-05-14T17:01:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.616+0000 7f66397fa640 1 -- 172.21.15.69:0/338624879 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c026640 con 0x558b6adc3ad0 2024-05-14T17:01:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 -- 172.21.15.69:0/338624879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 msgr2=0x558b6adc2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 --2- 172.21.15.69:0/338624879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc2f60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f664c01e530 tx=0x7f664c013000 comp rx=0 tx=0).stop 2024-05-14T17:01:17.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 -- 172.21.15.69:0/338624879 shutdown_connections 2024-05-14T17:01:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 --2- 172.21.15.69:0/338624879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adc3ad0 0x558b6adc2f60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 --2- 172.21.15.69:0/338624879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adb9210 0x558b6adcf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 --2- 172.21.15.69:0/338624879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6adcbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 -- 172.21.15.69:0/338624879 >> 172.21.15.69:0/338624879 conn(0x558b6adb4e00 msgr2=0x558b6adbbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 -- 172.21.15.69:0/338624879 shutdown_connections 2024-05-14T17:01:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 -- 172.21.15.69:0/338624879 wait complete. 2024-05-14T17:01:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 -- start start 2024-05-14T17:01:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.617+0000 7f665cbff740 1 Processor -- start 2024-05-14T17:01:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f665cbff740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6ae3aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f665cbff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb9210 0x558b6ae3afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f665cbff740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adc3ad0 0x558b6addaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f665cbff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f66541acc80 con 0x558b6adb9210 2024-05-14T17:01:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f665cbff740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f66541ae0f0 con 0x558b6adc3ad0 2024-05-14T17:01:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f665cbff740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66541ad350 con 0x558b6adb88f0 2024-05-14T17:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f665cbff740 10 client.0 _ll_register_callbacks cb 0x7f6644005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb9210 0x558b6ae3afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb9210 0x558b6ae3afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39094/0 (socket says 172.21.15.69:39094) 2024-05-14T17:01:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6652ffd640 1 -- 172.21.15.69:0/4294110974 learned_addr learned my addr 172.21.15.69:0/4294110974 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f66537fe640 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6ae3aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6653fff640 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adc3ad0 0x558b6addaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6652ffd640 1 -- 172.21.15.69:0/4294110974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 msgr2=0x558b6ae3aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6652ffd640 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6ae3aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6652ffd640 1 -- 172.21.15.69:0/4294110974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adc3ad0 msgr2=0x558b6addaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6652ffd640 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adc3ad0 0x558b6addaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6652ffd640 1 -- 172.21.15.69:0/4294110974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b6addb660 con 0x558b6adb9210 2024-05-14T17:01:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6653fff640 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adc3ad0 0x558b6addaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6652ffd640 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb9210 0x558b6ae3afd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f664800f010 tx=0x7f664800c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d6204fec710fdc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:18.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:01:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.618+0000 7f6632ffd640 1 -- 172.21.15.69:0/4294110974 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648029020 con 0x558b6adb9210 2024-05-14T17:01:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f6632ffd640 1 -- 172.21.15.69:0/4294110974 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f66480054e0 con 0x558b6adb9210 2024-05-14T17:01:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f665cbff740 1 -- 172.21.15.69:0/4294110974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f664c01b420 con 0x558b6adb9210 2024-05-14T17:01:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f6632ffd640 1 -- 172.21.15.69:0/4294110974 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6648034050 con 0x558b6adb9210 2024-05-14T17:01:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f665cbff740 10 client.16599 fetch_fsmap learned FSMap version 48 2024-05-14T17:01:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f665cbff740 1 -- 172.21.15.69:0/4294110974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558b6ae42330 con 0x558b6adb9210 2024-05-14T17:01:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f6632ffd640 1 -- 172.21.15.69:0/4294110974 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f664c01b420 con 0x558b6adb9210 2024-05-14T17:01:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f6632ffd640 1 -- 172.21.15.69:0/4294110974 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 48) ==== 8346+0+0 (secure 0 0 0) 0x7f66480107b0 con 0x558b6adb9210 2024-05-14T17:01:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f665cbff740 10 client.16599 fetch_fsmap finished waiting for FSMap version 48 2024-05-14T17:01:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f665cbff740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f665cbff740 20 client.16599 populate_metadata read hostname 'smithi069' 2024-05-14T17:01:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f665cbff740 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558b6ae42830 0x558b6ae44c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f665cbff740 4 client.16599 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:01:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f665cbff740 1 -- 172.21.15.69:0/4294110974 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b6ad27360 con 0x558b6ae42830 2024-05-14T17:01:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.619+0000 7f66537fe640 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558b6ae42830 0x558b6ae44c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.620+0000 7f66537fe640 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558b6ae42830 0x558b6ae44c50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=691056e6bb32c624 server_cookie=ef1e07fbfcc80a41 in_seq=0 out_seq=0 2024-05-14T17:01:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.620+0000 7f6632ffd640 10 client.16599 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f6632ffd640 1 -- 172.21.15.69:0/4294110974 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558b6ad27360 con 0x558b6ae42830 2024-05-14T17:01:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f6632ffd640 10 client.16599 handle_command_reply: tid=0 2024-05-14T17:01:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 client.16599 shutdown 2024-05-14T17:01:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 -- 172.21.15.69:0/4294110974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb9210 msgr2=0x558b6ae3afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb9210 0x558b6ae3afd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f664800f010 tx=0x7f664800c350 comp rx=0 tx=0).stop 2024-05-14T17:01:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 -- 172.21.15.69:0/4294110974 shutdown_connections 2024-05-14T17:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558b6ae42830 0x558b6ae44c50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b6adc3ad0 0x558b6addaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6adb9210 0x558b6ae3afd0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f6632ffd640 0 client.16599 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 --2- 172.21.15.69:0/4294110974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b6adb88f0 0x558b6ae3aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 -- 172.21.15.69:0/4294110974 >> 172.21.15.69:0/4294110974 conn(0x558b6adb4e00 msgr2=0x558b6adba2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 -- 172.21.15.69:0/4294110974 shutdown_connections 2024-05-14T17:01:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 1 -- 172.21.15.69:0/4294110974 wait complete. 2024-05-14T17:01:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.629+0000 7f665cbff740 20 client.16599 trim_cache size 0 max 16384 2024-05-14T17:01:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:01:20.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:01:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:01:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 -- 172.21.15.69:0/3771649921 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f663c0b2a90 msgr2=0x7f663c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f663c0b2a90 0x7f663c0b4f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f664c007e70 tx=0x7f664c004a40 comp rx=0 tx=0).stop 2024-05-14T17:01:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 -- 172.21.15.69:0/3771649921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6654102a60 msgr2=0x7f6654199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6654102a60 0x7f6654199100 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6644005940 tx=0x7f6644004270 comp rx=0 tx=0).stop 2024-05-14T17:01:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 -- 172.21.15.69:0/3771649921 shutdown_connections 2024-05-14T17:01:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f663c0b2a90 0x7f663c0b4f50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6654108a70 0x7f665419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6654103420 0x7f6654199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 --2- 172.21.15.69:0/3771649921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6654102a60 0x7f6654199100 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 -- 172.21.15.69:0/3771649921 >> 172.21.15.69:0/3771649921 conn(0x7f6654079ee0 msgr2=0x7f6654100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 -- 172.21.15.69:0/3771649921 shutdown_connections 2024-05-14T17:01:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:00:57.630+0000 7f6659ee1640 1 -- 172.21.15.69:0/3771649921 wait complete. 2024-05-14T17:01:21.043 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:09 smithi203 ceph-mon[35787]: pgmap v352: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 24 op/s 2024-05-14T17:01:21.082 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:09 smithi203 ceph-mon[35787]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T17:01:21.124 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:11 smithi203 ceph-mon[35787]: pgmap v354: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 66 KiB/s wr, 29 op/s 2024-05-14T17:01:21.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:13 smithi203 ceph-mon[35787]: pgmap v355: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-05-14T17:01:21.207 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:13 smithi203 ceph-mon[35787]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T17:01:21.252 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:15 smithi203 ceph-mon[35787]: pgmap v357: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 12 KiB/s wr, 3 op/s 2024-05-14T17:01:21.307 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:57 smithi069 ceph-mon[31507]: pgmap v346: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-05-14T17:01:21.349 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:00:59 smithi069 ceph-mon[31507]: pgmap v347: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-05-14T17:01:21.391 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 630.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:21.432 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:15 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 3. 2024-05-14T17:01:21.465 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:15 smithi203 systemd[1]: Stopped Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:01:21.507 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:15 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Consumed 1.811s CPU time. 2024-05-14T17:01:21.549 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:15 smithi203 systemd[1]: Starting Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:01:21.590 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:16 smithi203 podman[72634]: 2024-05-14 17:01:16.096416184 +0000 UTC m=+0.017521903 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:01:21.632 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:16 smithi203 podman[72634]: 2024-05-14 17:01:16.219659728 +0000 UTC m=+0.140765439 container create ac2cbd38a1eff678752634f37cf8f71f4b7260d169dc388245d516d599b96807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-14T17:01:21.674 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:16 smithi203 podman[72634]: 2024-05-14 17:01:16.487198123 +0000 UTC m=+0.408303842 container init ac2cbd38a1eff678752634f37cf8f71f4b7260d169dc388245d516d599b96807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-14T17:01:21.716 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:16 smithi203 podman[72634]: 2024-05-14 17:01:16.489719819 +0000 UTC m=+0.410825538 container start ac2cbd38a1eff678752634f37cf8f71f4b7260d169dc388245d516d599b96807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0) 2024-05-14T17:01:21.757 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:16 smithi203 ceph-mds[72654]: -- 172.21.15.203:0/543431027 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e7d849a40 con 0x559e7d81c800 2024-05-14T17:01:21.799 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:16 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: starting mds.c at 2024-05-14T17:01:21.841 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:16 smithi203 bash[72634]: ac2cbd38a1eff678752634f37cf8f71f4b7260d169dc388245d516d599b96807 2024-05-14T17:01:21.950 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:01:16 smithi203 systemd[1]: Started Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:01:21.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:09 smithi092 ceph-mon[35763]: pgmap v352: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 24 op/s 2024-05-14T17:01:22.074 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:09 smithi092 ceph-mon[35763]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T17:01:22.135 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:11 smithi092 ceph-mon[35763]: pgmap v354: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 66 KiB/s wr, 29 op/s 2024-05-14T17:01:22.174 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:13 smithi092 ceph-mon[35763]: pgmap v355: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-05-14T17:01:22.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:13 smithi092 ceph-mon[35763]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T17:01:22.241 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:15 smithi092 ceph-mon[35763]: pgmap v357: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 12 KiB/s wr, 3 op/s 2024-05-14T17:01:22.274 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:16 smithi092 ceph-mon[35763]: daemon mds.c restarted 2024-05-14T17:01:22.324 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:17 smithi092 ceph-mon[35763]: pgmap v358: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T17:01:22.358 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:17 smithi092 ceph-mon[35763]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T17:01:22.413 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:17 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6836/3631522795,v1:172.21.15.203:6837/3631522795] up:boot 2024-05-14T17:01:22.449 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:17 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:01:22.483 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:01:22.516 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:17 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:22.583 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:17 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:01:22.616 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:19 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:01:22.649 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:19 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:01:22.683 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:19 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:01:22.741 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:19 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:01:22.821 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:19 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:01:22.861 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:20 smithi092 ceph-mon[35763]: pgmap v360: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T17:01:22.900 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:01 smithi069 ceph-mon[31507]: pgmap v348: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-05-14T17:01:22.933 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:03 smithi069 ceph-mon[31507]: pgmap v349: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 67 KiB/s wr, 29 op/s 2024-05-14T17:01:22.967 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 635.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:23.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:22 smithi092 ceph-mon[35763]: pgmap v361: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 2 op/s 2024-05-14T17:01:23.033 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:16 smithi203 ceph-mon[35787]: daemon mds.c restarted 2024-05-14T17:01:23.066 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:17 smithi203 ceph-mon[35787]: pgmap v358: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T17:01:23.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:17 smithi203 ceph-mon[35787]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T17:01:23.167 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:17 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6836/3631522795,v1:172.21.15.203:6837/3631522795] up:boot 2024-05-14T17:01:23.200 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:17 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:01:23.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:01:23.267 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:17 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:23.300 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:17 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:01:23.333 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:19 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:01:23.375 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:19 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:01:23.408 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:19 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:01:23.442 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:19 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:01:23.475 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:19 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:01:23.508 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:20 smithi203 ceph-mon[35787]: pgmap v360: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T17:01:23.542 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:22 smithi203 ceph-mon[35787]: pgmap v361: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 2 op/s 2024-05-14T17:01:23.576 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:05 smithi069 ceph-mon[31507]: pgmap v350: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 62 KiB/s wr, 26 op/s 2024-05-14T17:01:23.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:07 smithi069 ceph-mon[31507]: pgmap v351: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-05-14T17:01:23.642 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:09 smithi069 ceph-mon[31507]: pgmap v352: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 24 op/s 2024-05-14T17:01:23.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:09 smithi069 ceph-mon[31507]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T17:01:23.709 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 640.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.730+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2266914216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bf4101c30 msgr2=0x7f6bf41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.730+0000 7f6bfc7da640 1 --2- 172.21.15.69:0/2266914216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bf4101c30 0x7f6bf41020b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009900 tx=0x7f6be402a410 comp rx=0 tx=0).stop 2024-05-14T17:01:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.730+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2266914216 shutdown_connections 2024-05-14T17:01:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.730+0000 7f6bfc7da640 1 --2- 172.21.15.69:0/2266914216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf41025f0 0x7f6bf410e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.730+0000 7f6bfc7da640 1 --2- 172.21.15.69:0/2266914216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bf4101c30 0x7f6bf41020b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.730+0000 7f6bfc7da640 1 --2- 172.21.15.69:0/2266914216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf4106920 0x7f6bf4106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.730+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2266914216 >> 172.21.15.69:0/2266914216 conn(0x7f6bf4078730 msgr2=0x7f6bf4078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.730+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2266914216 shutdown_connections 2024-05-14T17:01:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.730+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2266914216 wait complete. 2024-05-14T17:01:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfc7da640 1 Processor -- start 2024-05-14T17:01:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfc7da640 1 -- start start 2024-05-14T17:01:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfc7da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf4101c30 0x7f6bf4197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfc7da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41025f0 0x7f6bf41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfc7da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bf4106920 0x7f6bf41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfc7da640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf4111bc0 con 0x7f6bf4106920 2024-05-14T17:01:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfc7da640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf4111a40 con 0x7f6bf4101c30 2024-05-14T17:01:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfc7da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf4111d40 con 0x7f6bf41025f0 2024-05-14T17:01:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfad77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bf4106920 0x7f6bf41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfad77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bf4106920 0x7f6bf41987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57676/0 (socket says 172.21.15.69:57676) 2024-05-14T17:01:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfad77640 1 -- 172.21.15.69:0/2223825017 learned_addr learned my addr 172.21.15.69:0/2223825017 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfa576640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf4101c30 0x7f6bf4197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41025f0 0x7f6bf41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfad77640 1 -- 172.21.15.69:0/2223825017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41025f0 msgr2=0x7f6bf41982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfad77640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41025f0 0x7f6bf41982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfad77640 1 -- 172.21.15.69:0/2223825017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf4101c30 msgr2=0x7f6bf4197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfad77640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf4101c30 0x7f6bf4197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfad77640 1 -- 172.21.15.69:0/2223825017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf41a75e0 con 0x7f6bf4106920 2024-05-14T17:01:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41025f0 0x7f6bf41982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.731+0000 7f6bfad77640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bf4106920 0x7f6bf41987f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009fd0 tx=0x7f6bec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.732+0000 7f6beb7fe640 1 -- 172.21.15.69:0/2223825017 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec017070 con 0x7f6bf4106920 2024-05-14T17:01:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.732+0000 7f6beb7fe640 1 -- 172.21.15.69:0/2223825017 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bec003e20 con 0x7f6bf4106920 2024-05-14T17:01:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.732+0000 7f6beb7fe640 1 -- 172.21.15.69:0/2223825017 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec010460 con 0x7f6bf4106920 2024-05-14T17:01:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.732+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2223825017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf41a78d0 con 0x7f6bf4106920 2024-05-14T17:01:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.732+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2223825017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf41a7dd0 con 0x7f6bf4106920 2024-05-14T17:01:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.734+0000 7f6beb7fe640 1 -- 172.21.15.69:0/2223825017 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6bec003970 con 0x7f6bf4106920 2024-05-14T17:01:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.734+0000 7f6bff5d0740 1 Processor -- start 2024-05-14T17:01:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bff5d0740 1 -- start start 2024-05-14T17:01:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 0x56451827acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827b210 0x564518285590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518287ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5645181edc30 con 0x56451827b210 2024-05-14T17:01:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5645181dd7d0 con 0x564518285ad0 2024-05-14T17:01:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5645181e1810 con 0x56451827a8f0 2024-05-14T17:01:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bfa576640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827b210 0x564518285590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bfa576640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827b210 0x564518285590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57686/0 (socket says 172.21.15.69:57686) 2024-05-14T17:01:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bfa576640 1 -- 172.21.15.69:0/720799418 learned_addr learned my addr 172.21.15.69:0/720799418 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/720799418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518287ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/720799418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 0x56451827acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bfa576640 1 -- 172.21.15.69:0/720799418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 msgr2=0x56451827acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bfa576640 1 --2- 172.21.15.69:0/720799418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 0x56451827acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6beb7fe640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6be00b2a90 0x7f6be00b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bfa576640 1 -- 172.21.15.69:0/720799418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 msgr2=0x564518287ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bfa576640 1 --2- 172.21.15.69:0/720799418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518287ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bfa576640 1 -- 172.21.15.69:0/720799418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564517ef2210 con 0x56451827b210 2024-05-14T17:01:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/720799418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 0x56451827acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.735+0000 7f6bfad77640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6be00b2a90 0x7f6be00b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.736+0000 7f6bfa576640 1 --2- 172.21.15.69:0/720799418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827b210 0x564518285590 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000bc20 tx=0x7f6bf000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66520bed8ffaa6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.736+0000 7f6beb7fe640 1 -- 172.21.15.69:0/2223825017 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8586+0+0 (secure 0 0 0) 0x7f6bec0e24b0 con 0x7f6bf4106920 2024-05-14T17:01:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.736+0000 7f6be8ff9640 1 -- 172.21.15.69:0/720799418 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0015070 con 0x56451827b210 2024-05-14T17:01:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.736+0000 7f6be8ff9640 1 -- 172.21.15.69:0/720799418 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bf00054d0 con 0x56451827b210 2024-05-14T17:01:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.736+0000 7f6be8ff9640 1 -- 172.21.15.69:0/720799418 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0011440 con 0x56451827b210 2024-05-14T17:01:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 -- 172.21.15.69:0/720799418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827b210 msgr2=0x564518285590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/720799418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827b210 0x564518285590 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000bc20 tx=0x7f6bf000f730 comp rx=0 tx=0).stop 2024-05-14T17:01:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bfad77640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6be00b2a90 0x7f6be00b4f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0a96e0 tx=0x7f6bec004020 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 -- 172.21.15.69:0/720799418 shutdown_connections 2024-05-14T17:01:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/720799418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518287ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/720799418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827b210 0x564518285590 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/720799418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 0x56451827acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 -- 172.21.15.69:0/720799418 >> 172.21.15.69:0/720799418 conn(0x564518276e00 msgr2=0x5645182771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 -- 172.21.15.69:0/720799418 shutdown_connections 2024-05-14T17:01:27.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 -- 172.21.15.69:0/720799418 wait complete. 2024-05-14T17:01:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 -- start start 2024-05-14T17:01:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 Processor -- start 2024-05-14T17:01:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518285520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 0x564518282810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518282f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564517ec3790 con 0x56451827a8f0 2024-05-14T17:01:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564518199890 con 0x564518285ad0 2024-05-14T17:01:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5645182887a0 con 0x56451827b210 2024-05-14T17:01:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bff5d0740 10 client.0 _ll_register_callbacks cb 0x56451819cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bf9d75640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518285520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bf9d75640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518285520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57700/0 (socket says 172.21.15.69:57700) 2024-05-14T17:01:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.737+0000 7f6bf9d75640 1 -- 172.21.15.69:0/1498633968 learned_addr learned my addr 172.21.15.69:0/1498633968 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bfa576640 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 0x564518282810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518282f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bf9d75640 1 -- 172.21.15.69:0/1498633968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 msgr2=0x564518282810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 0x564518282810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bf9d75640 1 -- 172.21.15.69:0/1498633968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 msgr2=0x564518282f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518282f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bf9d75640 1 -- 172.21.15.69:0/1498633968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564517fbe620 con 0x56451827a8f0 2024-05-14T17:01:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518285520 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005ec0 tx=0x7f6be4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b630c2d19dd2b66 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bddffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:01:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bddffb640 1 -- 172.21.15.69:0/1498633968 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4045020 con 0x56451827a8f0 2024-05-14T17:01:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bddffb640 1 -- 172.21.15.69:0/1498633968 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8586+0+0 (secure 0 0 0) 0x7f6be4004450 con 0x56451827a8f0 2024-05-14T17:01:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bff5d0740 1 -- 172.21.15.69:0/1498633968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf00054d0 con 0x56451827a8f0 2024-05-14T17:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bddffb640 1 -- 172.21.15.69:0/1498633968 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6be4050050 con 0x56451827a8f0 2024-05-14T17:01:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bff5d0740 10 client.16611 fetch_fsmap learned FSMap version 48 2024-05-14T17:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bff5d0740 1 -- 172.21.15.69:0/1498633968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564518294210 con 0x56451827a8f0 2024-05-14T17:01:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.738+0000 7f6bddffb640 1 -- 172.21.15.69:0/1498633968 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bf00054d0 con 0x56451827a8f0 2024-05-14T17:01:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.739+0000 7f6bddffb640 1 -- 172.21.15.69:0/1498633968 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 48) ==== 8346+0+0 (secure 0 0 0) 0x7f6be403a050 con 0x56451827a8f0 2024-05-14T17:01:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.739+0000 7f6bff5d0740 10 client.16611 fetch_fsmap finished waiting for FSMap version 48 2024-05-14T17:01:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.739+0000 7f6bff5d0740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.739+0000 7f6bff5d0740 20 client.16611 populate_metadata read hostname 'smithi069' 2024-05-14T17:01:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.739+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564518295720 0x564518297b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.739+0000 7f6bff5d0740 4 client.16611 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:01:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.739+0000 7f6bff5d0740 1 -- 172.21.15.69:0/1498633968 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645181efb80 con 0x564518295720 2024-05-14T17:01:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.739+0000 7f6bfa576640 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564518295720 0x564518297b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.739+0000 7f6bddffb640 10 client.16611 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.739+0000 7f6bfa576640 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564518295720 0x564518297b10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be5414240aaf786 server_cookie=5b95f11a33b9a5b4 in_seq=0 out_seq=0 2024-05-14T17:01:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bddffb640 1 -- 172.21.15.69:0/1498633968 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645181efb80 con 0x564518295720 2024-05-14T17:01:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bddffb640 10 client.16611 handle_command_reply: tid=0 2024-05-14T17:01:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bff5d0740 1 client.16611 shutdown 2024-05-14T17:01:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bff5d0740 1 -- 172.21.15.69:0/1498633968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 msgr2=0x564518285520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518285520 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005ec0 tx=0x7f6be4009260 comp rx=0 tx=0).stop 2024-05-14T17:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bff5d0740 1 -- 172.21.15.69:0/1498633968 shutdown_connections 2024-05-14T17:01:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564518295720 0x564518297b10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518282f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 0x564518282810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bddffb640 0 client.16611 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/1498633968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518285520 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bff5d0740 1 -- 172.21.15.69:0/1498633968 >> 172.21.15.69:0/1498633968 conn(0x564518276e00 msgr2=0x564518276100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.740+0000 7f6bff5d0740 1 -- 172.21.15.69:0/1498633968 shutdown_connections 2024-05-14T17:01:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.741+0000 7f6bff5d0740 1 -- 172.21.15.69:0/1498633968 wait complete. 2024-05-14T17:01:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.741+0000 7f6bff5d0740 20 client.16611 trim_cache size 0 max 16384 2024-05-14T17:01:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bff5d0740 1 Processor -- start 2024-05-14T17:01:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bff5d0740 1 -- start start 2024-05-14T17:01:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518291670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 0x564518285050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518285590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41b31e0 con 0x56451827a8f0 2024-05-14T17:01:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41b0a50 con 0x564518285ad0 2024-05-14T17:01:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41ae8c0 con 0x56451827b210 2024-05-14T17:01:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bf9d75640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518291670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bf9d75640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518291670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57712/0 (socket says 172.21.15.69:57712) 2024-05-14T17:01:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bf9d75640 1 -- 172.21.15.69:0/4243954147 learned_addr learned my addr 172.21.15.69:0/4243954147 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bfa576640 1 --2- 172.21.15.69:0/4243954147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 0x564518285050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:29.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bf9d75640 1 -- 172.21.15.69:0/4243954147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 msgr2=0x564518285050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/4243954147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 0x564518285050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bf9d75640 1 -- 172.21.15.69:0/4243954147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 msgr2=0x564518285590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/4243954147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518285590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bf9d75640 1 -- 172.21.15.69:0/4243954147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56451827b040 con 0x56451827a8f0 2024-05-14T17:01:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bfa576640 1 --2- 172.21.15.69:0/4243954147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 0x564518285050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/4243954147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518291670 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009340 tx=0x7f6be402aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d3d20b69b7f76f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.756+0000 7f6bdffff640 1 -- 172.21.15.69:0/4243954147 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4045040 con 0x56451827a8f0 2024-05-14T17:01:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bdffff640 1 -- 172.21.15.69:0/4243954147 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be40054d0 con 0x56451827a8f0 2024-05-14T17:01:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bdffff640 1 -- 172.21.15.69:0/4243954147 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4035040 con 0x56451827a8f0 2024-05-14T17:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 -- 172.21.15.69:0/4243954147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 msgr2=0x564518291670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/4243954147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518291670 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009340 tx=0x7f6be402aa90 comp rx=0 tx=0).stop 2024-05-14T17:01:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 -- 172.21.15.69:0/4243954147 shutdown_connections 2024-05-14T17:01:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/4243954147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564518285ad0 0x564518285590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/4243954147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827b210 0x564518285050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/4243954147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56451827a8f0 0x564518291670 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 -- 172.21.15.69:0/4243954147 >> 172.21.15.69:0/4243954147 conn(0x564518276e00 msgr2=0x56451827cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 -- 172.21.15.69:0/4243954147 shutdown_connections 2024-05-14T17:01:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 -- 172.21.15.69:0/4243954147 wait complete. 2024-05-14T17:01:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 -- start start 2024-05-14T17:01:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.757+0000 7f6bff5d0740 1 Processor -- start 2024-05-14T17:01:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 0x56451828dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56451827b210 0x5645182890b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bff5d0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564518285ad0 0x5645182895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41aef10 con 0x564518285ad0 2024-05-14T17:01:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41b0380 con 0x56451827b210 2024-05-14T17:01:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bff5d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41af5e0 con 0x56451827a8f0 2024-05-14T17:01:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bff5d0740 10 client.0 _ll_register_callbacks cb 0x7f6be4031530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bf9d75640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564518285ad0 0x5645182895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bf9d75640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564518285ad0 0x5645182895f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57722/0 (socket says 172.21.15.69:57722) 2024-05-14T17:01:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bf9d75640 1 -- 172.21.15.69:0/3393109699 learned_addr learned my addr 172.21.15.69:0/3393109699 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bfa576640 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56451827b210 0x5645182890b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 0x56451828dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bf9d75640 1 -- 172.21.15.69:0/3393109699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 msgr2=0x56451828dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 0x56451828dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bf9d75640 1 -- 172.21.15.69:0/3393109699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56451827b210 msgr2=0x5645182890b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56451827b210 0x5645182890b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bf9d75640 1 -- 172.21.15.69:0/3393109699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564518289bb0 con 0x564518285ad0 2024-05-14T17:01:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bf9d75640 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564518285ad0 0x5645182895f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009850 tx=0x7f6be4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5367d9b6406e6cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bddffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:01:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bddffb640 1 -- 172.21.15.69:0/3393109699 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4035030 con 0x564518285ad0 2024-05-14T17:01:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bddffb640 1 -- 172.21.15.69:0/3393109699 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8586+0+0 (secure 0 0 0) 0x7f6be40035f0 con 0x564518285ad0 2024-05-14T17:01:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.758+0000 7f6bff5d0740 1 -- 172.21.15.69:0/3393109699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6be40054d0 con 0x564518285ad0 2024-05-14T17:01:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bddffb640 1 -- 172.21.15.69:0/3393109699 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6be4059040 con 0x564518285ad0 2024-05-14T17:01:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bff5d0740 10 client.16617 fetch_fsmap learned FSMap version 48 2024-05-14T17:01:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bff5d0740 1 -- 172.21.15.69:0/3393109699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5645182ffd00 con 0x564518285ad0 2024-05-14T17:01:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bddffb640 1 -- 172.21.15.69:0/3393109699 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be40054d0 con 0x564518285ad0 2024-05-14T17:01:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bddffb640 1 -- 172.21.15.69:0/3393109699 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 48) ==== 8346+0+0 (secure 0 0 0) 0x7f6be405f030 con 0x564518285ad0 2024-05-14T17:01:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bff5d0740 10 client.16617 fetch_fsmap finished waiting for FSMap version 48 2024-05-14T17:01:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bff5d0740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bff5d0740 20 client.16617 populate_metadata read hostname 'smithi069' 2024-05-14T17:01:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564518300310 0x564518302730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bff5d0740 4 client.16617 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:01:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bff5d0740 1 -- 172.21.15.69:0/3393109699 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645181e9360 con 0x564518300310 2024-05-14T17:01:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.759+0000 7f6bfa576640 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564518300310 0x564518302730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.760+0000 7f6bfa576640 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564518300310 0x564518302730 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1bfb633dbba63b server_cookie=70abe806896e2132 in_seq=0 out_seq=0 2024-05-14T17:01:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.760+0000 7f6bddffb640 10 client.16617 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bddffb640 1 -- 172.21.15.69:0/3393109699 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5645181e9360 con 0x564518300310 2024-05-14T17:01:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bddffb640 10 client.16617 handle_command_reply: tid=0 2024-05-14T17:01:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 client.16617 shutdown 2024-05-14T17:01:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 -- 172.21.15.69:0/3393109699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564518285ad0 msgr2=0x5645182895f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564518285ad0 0x5645182895f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009850 tx=0x7f6be4000bc0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 -- 172.21.15.69:0/3393109699 shutdown_connections 2024-05-14T17:01:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564518300310 0x564518302730 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564518285ad0 0x5645182895f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56451827b210 0x5645182890b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 --2- 172.21.15.69:0/3393109699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56451827a8f0 0x56451828dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bddffb640 0 client.16617 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 -- 172.21.15.69:0/3393109699 >> 172.21.15.69:0/3393109699 conn(0x564518276e00 msgr2=0x56451827d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 -- 172.21.15.69:0/3393109699 shutdown_connections 2024-05-14T17:01:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 1 -- 172.21.15.69:0/3393109699 wait complete. 2024-05-14T17:01:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.768+0000 7f6bff5d0740 20 client.16617 trim_cache size 0 max 16384 2024-05-14T17:01:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:01:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:01:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:01:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2223825017 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6be00b2a90 msgr2=0x7f6be00b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6be00b2a90 0x7f6be00b4f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0a96e0 tx=0x7f6bec004020 comp rx=0 tx=0).stop 2024-05-14T17:01:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2223825017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bf4106920 msgr2=0x7f6bf41987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bf4106920 0x7f6bf41987f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009fd0 tx=0x7f6bec00b260 comp rx=0 tx=0).stop 2024-05-14T17:01:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2223825017 shutdown_connections 2024-05-14T17:01:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6be00b2a90 0x7f6be00b4f50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bf4106920 0x7f6bf41987f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41025f0 0x7f6bf41982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 --2- 172.21.15.69:0/2223825017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf4101c30 0x7f6bf4197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2223825017 >> 172.21.15.69:0/2223825017 conn(0x7f6bf4078730 msgr2=0x7f6bf40ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2223825017 shutdown_connections 2024-05-14T17:01:32.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:10.769+0000 7f6bfc7da640 1 -- 172.21.15.69:0/2223825017 wait complete. 2024-05-14T17:01:32.733 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:11 smithi069 ceph-mon[31507]: pgmap v354: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 66 KiB/s wr, 29 op/s 2024-05-14T17:01:32.766 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:13 smithi069 ceph-mon[31507]: pgmap v355: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-05-14T17:01:32.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:13 smithi069 ceph-mon[31507]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T17:01:32.833 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 645.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:32.867 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:24 smithi092 ceph-mon[35763]: pgmap v362: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-05-14T17:01:32.899 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:26 smithi092 ceph-mon[35763]: pgmap v363: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 KiB/s wr, 6 op/s 2024-05-14T17:01:32.950 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:28 smithi092 ceph-mon[35763]: pgmap v364: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 324 KiB/s wr, 38 op/s 2024-05-14T17:01:32.983 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:30 smithi092 ceph-mon[35763]: pgmap v365: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 273 KiB/s wr, 32 op/s 2024-05-14T17:01:33.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:32 smithi092 ceph-mon[35763]: pgmap v366: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 283 KiB/s wr, 33 op/s 2024-05-14T17:01:33.049 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:24 smithi203 ceph-mon[35787]: pgmap v362: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-05-14T17:01:33.083 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:26 smithi203 ceph-mon[35787]: pgmap v363: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 KiB/s wr, 6 op/s 2024-05-14T17:01:33.116 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:28 smithi203 ceph-mon[35787]: pgmap v364: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 324 KiB/s wr, 38 op/s 2024-05-14T17:01:33.149 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:30 smithi203 ceph-mon[35787]: pgmap v365: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 273 KiB/s wr, 32 op/s 2024-05-14T17:01:33.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:32 smithi203 ceph-mon[35787]: pgmap v366: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 283 KiB/s wr, 33 op/s 2024-05-14T17:01:33.245 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:15 smithi069 ceph-mon[31507]: pgmap v357: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 12 KiB/s wr, 3 op/s 2024-05-14T17:01:33.274 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:16 smithi069 ceph-mon[31507]: daemon mds.c restarted 2024-05-14T17:01:33.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:17 smithi069 ceph-mon[31507]: pgmap v358: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T17:01:33.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:17 smithi069 ceph-mon[31507]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T17:01:33.374 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:17 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6836/3631522795,v1:172.21.15.203:6837/3631522795] up:boot 2024-05-14T17:01:33.408 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:17 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:01:33.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:01:33.475 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:17 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:33.525 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:17 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:01:33.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:19 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:01:33.625 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:19 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:01:33.658 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:19 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:01:33.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:19 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:01:33.725 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:19 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:01:33.759 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 650.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:33.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:20 smithi069 ceph-mon[31507]: pgmap v360: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T17:01:33.825 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:22 smithi069 ceph-mon[31507]: pgmap v361: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 2 op/s 2024-05-14T17:01:33.860 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.872+0000 7f0326bc1640 1 -- 172.21.15.69:0/2317893366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0320103d20 msgr2=0x7f0320104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.872+0000 7f0326bc1640 1 --2- 172.21.15.69:0/2317893366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0320103d20 0x7f0320104100 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f031c009900 tx=0x7f031c02a480 comp rx=0 tx=0).stop 2024-05-14T17:01:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.872+0000 7f0326bc1640 1 -- 172.21.15.69:0/2317893366 shutdown_connections 2024-05-14T17:01:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.872+0000 7f0326bc1640 1 --2- 172.21.15.69:0/2317893366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f032010d060 0x7f032010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.872+0000 7f0326bc1640 1 --2- 172.21.15.69:0/2317893366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0320104640 0x7f032010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.872+0000 7f0326bc1640 1 --2- 172.21.15.69:0/2317893366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0320103d20 0x7f0320104100 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.872+0000 7f0326bc1640 1 -- 172.21.15.69:0/2317893366 >> 172.21.15.69:0/2317893366 conn(0x7f0320078ad0 msgr2=0x7f0320078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.872+0000 7f0326bc1640 1 -- 172.21.15.69:0/2317893366 shutdown_connections 2024-05-14T17:01:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f0326bc1640 1 -- 172.21.15.69:0/2317893366 wait complete. 2024-05-14T17:01:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f0326bc1640 1 Processor -- start 2024-05-14T17:01:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f0326bc1640 1 -- start start 2024-05-14T17:01:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f0326bc1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0320103d20 0x7f03201991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f0326bc1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0320104640 0x7f0320199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f0326bc1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f032010d060 0x7f032019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f0326bc1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0320112c80 con 0x7f0320103d20 2024-05-14T17:01:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f0326bc1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0320112b00 con 0x7f032010d060 2024-05-14T17:01:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f0326bc1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0320112e00 con 0x7f0320104640 2024-05-14T17:01:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f032495d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0320103d20 0x7f03201991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f032495d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0320103d20 0x7f03201991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37620/0 (socket says 172.21.15.69:37620) 2024-05-14T17:01:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.873+0000 7f032495d640 1 -- 172.21.15.69:0/2354776257 learned_addr learned my addr 172.21.15.69:0/2354776257 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f0317fff640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0320104640 0x7f0320199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f032515e640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f032010d060 0x7f032019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f032495d640 1 -- 172.21.15.69:0/2354776257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0320104640 msgr2=0x7f0320199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f032495d640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0320104640 0x7f0320199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f032495d640 1 -- 172.21.15.69:0/2354776257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f032010d060 msgr2=0x7f032019daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f032495d640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f032010d060 0x7f032019daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f032495d640 1 -- 172.21.15.69:0/2354776257 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032019e1d0 con 0x7f0320103d20 2024-05-14T17:01:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f032495d640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0320103d20 0x7f03201991e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f031c005940 tx=0x7f031c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f0315ffb640 1 -- 172.21.15.69:0/2354776257 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c03f030 con 0x7f0320103d20 2024-05-14T17:01:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f0315ffb640 1 -- 172.21.15.69:0/2354776257 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c003cb0 con 0x7f0320103d20 2024-05-14T17:01:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f0326bc1640 1 -- 172.21.15.69:0/2354776257 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032019e460 con 0x7f0320103d20 2024-05-14T17:01:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f0315ffb640 1 -- 172.21.15.69:0/2354776257 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c035070 con 0x7f0320103d20 2024-05-14T17:01:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.874+0000 7f0326bc1640 1 -- 172.21.15.69:0/2354776257 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03201a5d50 con 0x7f0320103d20 2024-05-14T17:01:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f0315ffb640 1 -- 172.21.15.69:0/2354776257 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f031c042030 con 0x7f0320103d20 2024-05-14T17:01:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f03299b2740 1 Processor -- start 2024-05-14T17:01:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f03299b2740 1 -- start start 2024-05-14T17:01:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f03299b2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a58f0 0x5581557a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f03299b2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557b0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f03299b2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557b0ad0 0x5581557b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f03299b2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558155718c30 con 0x5581557a58f0 2024-05-14T17:01:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f03299b2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5581557087d0 con 0x5581557b0ad0 2024-05-14T17:01:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f03299b2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55815570c810 con 0x5581557a6210 2024-05-14T17:01:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f032515e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a58f0 0x5581557a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f032515e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a58f0 0x5581557a5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37636/0 (socket says 172.21.15.69:37636) 2024-05-14T17:01:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f032515e640 1 -- 172.21.15.69:0/3713146091 learned_addr learned my addr 172.21.15.69:0/3713146091 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f032495d640 1 --2- 172.21.15.69:0/3713146091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f0317fff640 1 --2- 172.21.15.69:0/3713146091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557b0ad0 0x5581557b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.877+0000 7f032515e640 1 -- 172.21.15.69:0/3713146091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 msgr2=0x5581557b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f032515e640 1 --2- 172.21.15.69:0/3713146091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f032515e640 1 -- 172.21.15.69:0/3713146091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557b0ad0 msgr2=0x5581557b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f032515e640 1 --2- 172.21.15.69:0/3713146091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557b0ad0 0x5581557b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f0315ffb640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03080b2a90 0x7f03080b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f032515e640 1 -- 172.21.15.69:0/3713146091 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55815541d210 con 0x5581557a58f0 2024-05-14T17:01:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f0315ffb640 1 -- 172.21.15.69:0/2354776257 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f031c102ed0 con 0x7f0320103d20 2024-05-14T17:01:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f032515e640 1 --2- 172.21.15.69:0/3713146091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a58f0 0x5581557a5cd0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0318008d20 tx=0x7f031800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8431c3e0ada0d389 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f032515e640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03080b2a90 0x7f03080b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f0306ffd640 1 -- 172.21.15.69:0/3713146091 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318014070 con 0x5581557a58f0 2024-05-14T17:01:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f0306ffd640 1 -- 172.21.15.69:0/3713146091 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0318019460 con 0x5581557a58f0 2024-05-14T17:01:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.878+0000 7f0306ffd640 1 -- 172.21.15.69:0/3713146091 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318018620 con 0x5581557a58f0 2024-05-14T17:01:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 -- 172.21.15.69:0/3713146091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a58f0 msgr2=0x5581557a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 --2- 172.21.15.69:0/3713146091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a58f0 0x5581557a5cd0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0318008d20 tx=0x7f031800e720 comp rx=0 tx=0).stop 2024-05-14T17:01:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f032515e640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03080b2a90 0x7f03080b4f50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0318008fd0 tx=0x7f0318018cf0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 -- 172.21.15.69:0/3713146091 shutdown_connections 2024-05-14T17:01:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 --2- 172.21.15.69:0/3713146091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557b0ad0 0x5581557b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 --2- 172.21.15.69:0/3713146091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 --2- 172.21.15.69:0/3713146091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a58f0 0x5581557a5cd0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 -- 172.21.15.69:0/3713146091 >> 172.21.15.69:0/3713146091 conn(0x5581557a1e00 msgr2=0x5581557a21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 -- 172.21.15.69:0/3713146091 shutdown_connections 2024-05-14T17:01:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 -- 172.21.15.69:0/3713146091 wait complete. 2024-05-14T17:01:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 -- start start 2024-05-14T17:01:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 Processor -- start 2024-05-14T17:01:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.879+0000 7f03299b2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f03299b2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557add10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f03299b2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5581553ee790 con 0x5581557b0ad0 2024-05-14T17:01:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f03299b2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5581556c4890 con 0x5581557a58f0 2024-05-14T17:01:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f03299b2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5581557b37a0 con 0x5581557a6210 2024-05-14T17:01:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f03299b2740 10 client.0 _ll_register_callbacks cb 0x5581556c7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032515e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032515e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37648/0 (socket says 172.21.15.69:37648) 2024-05-14T17:01:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032515e640 1 -- 172.21.15.69:0/1110564046 learned_addr learned my addr 172.21.15.69:0/1110564046 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f0317fff640 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032495d640 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032515e640 1 -- 172.21.15.69:0/1110564046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 msgr2=0x5581557ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032515e640 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032515e640 1 -- 172.21.15.69:0/1110564046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 msgr2=0x5581557b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032515e640 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032515e640 1 -- 172.21.15.69:0/1110564046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581554e9620 con 0x5581557b0ad0 2024-05-14T17:01:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032495d640 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f032515e640 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557add10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0318019ae0 tx=0x7f0318019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb2941fcb70105 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f02fffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:01:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f02fffff640 1 -- 172.21.15.69:0/1110564046 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318014050 con 0x5581557b0ad0 2024-05-14T17:01:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f02fffff640 1 -- 172.21.15.69:0/1110564046 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f03180183c0 con 0x5581557b0ad0 2024-05-14T17:01:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f03299b2740 1 -- 172.21.15.69:0/1110564046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0318019460 con 0x5581557b0ad0 2024-05-14T17:01:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f02fffff640 1 -- 172.21.15.69:0/1110564046 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f031803c050 con 0x5581557b0ad0 2024-05-14T17:01:37.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f03299b2740 10 client.16629 fetch_fsmap learned FSMap version 50 2024-05-14T17:01:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.880+0000 7f03299b2740 1 -- 172.21.15.69:0/1110564046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5581557bf200 con 0x5581557b0ad0 2024-05-14T17:01:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f02fffff640 1 -- 172.21.15.69:0/1110564046 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0318019460 con 0x5581557b0ad0 2024-05-14T17:01:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f02fffff640 1 -- 172.21.15.69:0/1110564046 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 50) ==== 8346+0+0 (secure 0 0 0) 0x7f031800fbb0 con 0x5581557b0ad0 2024-05-14T17:01:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f03299b2740 10 client.16629 fetch_fsmap finished waiting for FSMap version 50 2024-05-14T17:01:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f03299b2740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f03299b2740 20 client.16629 populate_metadata read hostname 'smithi069' 2024-05-14T17:01:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f03299b2740 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5581557c0660 0x5581557c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f03299b2740 4 client.16629 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:01:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f03299b2740 1 -- 172.21.15.69:0/1110564046 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55815571ab80 con 0x5581557c0660 2024-05-14T17:01:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f032495d640 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5581557c0660 0x5581557c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f02fffff640 10 client.16629 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.881+0000 7f032495d640 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5581557c0660 0x5581557c2a50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f0e0c292a7b66 server_cookie=952f578f5980228d in_seq=0 out_seq=0 2024-05-14T17:01:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.882+0000 7f02fffff640 1 -- 172.21.15.69:0/1110564046 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55815571ab80 con 0x5581557c0660 2024-05-14T17:01:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.882+0000 7f02fffff640 10 client.16629 handle_command_reply: tid=0 2024-05-14T17:01:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.882+0000 7f03299b2740 1 client.16629 shutdown 2024-05-14T17:01:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.882+0000 7f03299b2740 1 -- 172.21.15.69:0/1110564046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 msgr2=0x5581557add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.882+0000 7f03299b2740 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557add10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0318019ae0 tx=0x7f0318019000 comp rx=0 tx=0).stop 2024-05-14T17:01:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.882+0000 7f03299b2740 1 -- 172.21.15.69:0/1110564046 shutdown_connections 2024-05-14T17:01:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.882+0000 7f03299b2740 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5581557c0660 0x5581557c2a50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.883+0000 7f03299b2740 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557add10 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.883+0000 7f03299b2740 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.883+0000 7f02fffff640 0 client.16629 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.883+0000 7f03299b2740 1 --2- 172.21.15.69:0/1110564046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.883+0000 7f03299b2740 1 -- 172.21.15.69:0/1110564046 >> 172.21.15.69:0/1110564046 conn(0x5581557a1e00 msgr2=0x5581557a1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.883+0000 7f03299b2740 1 -- 172.21.15.69:0/1110564046 shutdown_connections 2024-05-14T17:01:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.883+0000 7f03299b2740 1 -- 172.21.15.69:0/1110564046 wait complete. 2024-05-14T17:01:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.883+0000 7f03299b2740 20 client.16629 trim_cache size 0 max 16384 2024-05-14T17:01:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.897+0000 7f03299b2740 1 Processor -- start 2024-05-14T17:01:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.897+0000 7f03299b2740 1 -- start start 2024-05-14T17:01:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.897+0000 7f03299b2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557bc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.897+0000 7f03299b2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a6210 0x5581557affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.897+0000 7f03299b2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557b0ad0 0x5581557c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.897+0000 7f03299b2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03201b0f20 con 0x5581557a6210 2024-05-14T17:01:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.897+0000 7f03299b2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03201ae790 con 0x5581557a58f0 2024-05-14T17:01:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.897+0000 7f03299b2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03201ac5e0 con 0x5581557b0ad0 2024-05-14T17:01:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f0317fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a6210 0x5581557affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f0317fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a6210 0x5581557affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37658/0 (socket says 172.21.15.69:37658) 2024-05-14T17:01:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f0317fff640 1 -- 172.21.15.69:0/3984077721 learned_addr learned my addr 172.21.15.69:0/3984077721 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f032515e640 1 --2- 172.21.15.69:0/3984077721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557b0ad0 0x5581557c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f032495d640 1 --2- 172.21.15.69:0/3984077721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557bc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f0317fff640 1 -- 172.21.15.69:0/3984077721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557b0ad0 msgr2=0x5581557c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f0317fff640 1 --2- 172.21.15.69:0/3984077721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557b0ad0 0x5581557c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f0317fff640 1 -- 172.21.15.69:0/3984077721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 msgr2=0x5581557bc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f0317fff640 1 --2- 172.21.15.69:0/3984077721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557bc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f0317fff640 1 -- 172.21.15.69:0/3984077721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581557abcc0 con 0x5581557a6210 2024-05-14T17:01:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f032495d640 1 --2- 172.21.15.69:0/3984077721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557bc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f0317fff640 1 --2- 172.21.15.69:0/3984077721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a6210 0x5581557affb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0320101420 tx=0x7f0310012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bc093e41176894 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f03067fc640 1 -- 172.21.15.69:0/3984077721 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031000dcb0 con 0x5581557a6210 2024-05-14T17:01:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f03067fc640 1 -- 172.21.15.69:0/3984077721 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031000de50 con 0x5581557a6210 2024-05-14T17:01:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.898+0000 7f03067fc640 1 -- 172.21.15.69:0/3984077721 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031001c420 con 0x5581557a6210 2024-05-14T17:01:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 -- 172.21.15.69:0/3984077721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a6210 msgr2=0x5581557affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 --2- 172.21.15.69:0/3984077721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a6210 0x5581557affb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0320101420 tx=0x7f0310012820 comp rx=0 tx=0).stop 2024-05-14T17:01:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 -- 172.21.15.69:0/3984077721 shutdown_connections 2024-05-14T17:01:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 --2- 172.21.15.69:0/3984077721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557b0ad0 0x5581557c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 --2- 172.21.15.69:0/3984077721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557a6210 0x5581557affb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 --2- 172.21.15.69:0/3984077721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557bc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 -- 172.21.15.69:0/3984077721 >> 172.21.15.69:0/3984077721 conn(0x5581557a1e00 msgr2=0x5581557a7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 -- 172.21.15.69:0/3984077721 shutdown_connections 2024-05-14T17:01:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 -- 172.21.15.69:0/3984077721 wait complete. 2024-05-14T17:01:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 -- start start 2024-05-14T17:01:39.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 Processor -- start 2024-05-14T17:01:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03201acc50 con 0x5581557b0ad0 2024-05-14T17:01:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03201ae0c0 con 0x5581557a58f0 2024-05-14T17:01:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.899+0000 7f03299b2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03201ad320 con 0x5581557a6210 2024-05-14T17:01:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f03299b2740 10 client.0 _ll_register_callbacks cb 0x7f03180199f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f032515e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f032515e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37670/0 (socket says 172.21.15.69:37670) 2024-05-14T17:01:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f032515e640 1 -- 172.21.15.69:0/4158760025 learned_addr learned my addr 172.21.15.69:0/4158760025 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f032495d640 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f0317fff640 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f032515e640 1 -- 172.21.15.69:0/4158760025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 msgr2=0x5581557b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f032515e640 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f032515e640 1 -- 172.21.15.69:0/4158760025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 msgr2=0x5581557b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f032515e640 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f032515e640 1 -- 172.21.15.69:0/4158760025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558155829e80 con 0x5581557b0ad0 2024-05-14T17:01:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f0317fff640 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f032515e640 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557b51b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0318008c40 tx=0x7f03180194e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f44f67914102fd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f02fffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f02fffff640 1 -- 172.21.15.69:0/4158760025 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318014040 con 0x5581557b0ad0 2024-05-14T17:01:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f02fffff640 1 -- 172.21.15.69:0/4158760025 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f0318018690 con 0x5581557b0ad0 2024-05-14T17:01:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f03299b2740 1 -- 172.21.15.69:0/4158760025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031000de50 con 0x5581557b0ad0 2024-05-14T17:01:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f02fffff640 1 -- 172.21.15.69:0/4158760025 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0318004a00 con 0x5581557b0ad0 2024-05-14T17:01:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f03299b2740 10 client.16635 fetch_fsmap learned FSMap version 50 2024-05-14T17:01:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.900+0000 7f03299b2740 1 -- 172.21.15.69:0/4158760025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55815582b0c0 con 0x5581557b0ad0 2024-05-14T17:01:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f02fffff640 1 -- 172.21.15.69:0/4158760025 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031000de50 con 0x5581557b0ad0 2024-05-14T17:01:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f02fffff640 1 -- 172.21.15.69:0/4158760025 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 50) ==== 8346+0+0 (secure 0 0 0) 0x7f0318010880 con 0x5581557b0ad0 2024-05-14T17:01:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f03299b2740 10 client.16635 fetch_fsmap finished waiting for FSMap version 50 2024-05-14T17:01:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f03299b2740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f03299b2740 20 client.16635 populate_metadata read hostname 'smithi069' 2024-05-14T17:01:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f03299b2740 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55815582b670 0x55815582da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f03299b2740 4 client.16635 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:01:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f03299b2740 1 -- 172.21.15.69:0/4158760025 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558155714360 con 0x55815582b670 2024-05-14T17:01:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f032495d640 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55815582b670 0x55815582da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f032495d640 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55815582b670 0x55815582da60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ddbbdbeb39f0d7f server_cookie=aec69468a35b202c in_seq=0 out_seq=0 2024-05-14T17:01:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.901+0000 7f02fffff640 10 client.16635 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f02fffff640 1 -- 172.21.15.69:0/4158760025 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558155714360 con 0x55815582b670 2024-05-14T17:01:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f02fffff640 10 client.16635 handle_command_reply: tid=0 2024-05-14T17:01:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 client.16635 shutdown 2024-05-14T17:01:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 -- 172.21.15.69:0/4158760025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 msgr2=0x5581557b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557b51b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0318008c40 tx=0x7f03180194e0 comp rx=0 tx=0).stop 2024-05-14T17:01:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 -- 172.21.15.69:0/4158760025 shutdown_connections 2024-05-14T17:01:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55815582b670 0x55815582da60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581557b0ad0 0x5581557b51b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f02fffff640 0 client.16635 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581557a6210 0x5581557b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 --2- 172.21.15.69:0/4158760025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581557a58f0 0x5581557b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 -- 172.21.15.69:0/4158760025 >> 172.21.15.69:0/4158760025 conn(0x5581557a1e00 msgr2=0x5581557a8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 -- 172.21.15.69:0/4158760025 shutdown_connections 2024-05-14T17:01:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.913+0000 7f03299b2740 1 -- 172.21.15.69:0/4158760025 wait complete. 2024-05-14T17:01:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f03299b2740 20 client.16635 trim_cache size 0 max 16384 2024-05-14T17:01:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:01:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:01:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:01:43.593 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:34 smithi203 ceph-mon[35787]: pgmap v367: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-05-14T17:01:43.638 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:36 smithi203 ceph-mon[35787]: pgmap v368: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.3 MiB/s wr, 328 op/s 2024-05-14T17:01:43.697 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:38 smithi203 ceph-mon[35787]: pgmap v369: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.9 MiB/s wr, 631 op/s 2024-05-14T17:01:43.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:40 smithi203 ceph-mon[35787]: pgmap v370: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.6 MiB/s wr, 602 op/s 2024-05-14T17:01:43.797 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:42 smithi203 ceph-mon[35787]: pgmap v371: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.6 MiB/s wr, 604 op/s 2024-05-14T17:01:43.872 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:34 smithi092 ceph-mon[35763]: pgmap v367: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-05-14T17:01:43.922 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:36 smithi092 ceph-mon[35763]: pgmap v368: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.3 MiB/s wr, 328 op/s 2024-05-14T17:01:43.972 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:38 smithi092 ceph-mon[35763]: pgmap v369: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.9 MiB/s wr, 631 op/s 2024-05-14T17:01:44.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:40 smithi092 ceph-mon[35763]: pgmap v370: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.6 MiB/s wr, 602 op/s 2024-05-14T17:01:44.072 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:42 smithi092 ceph-mon[35763]: pgmap v371: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.6 MiB/s wr, 604 op/s 2024-05-14T17:01:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 -- 172.21.15.69:0/2354776257 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03080b2a90 msgr2=0x7f03080b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03080b2a90 0x7f03080b4f50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0318008fd0 tx=0x7f0318018cf0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 -- 172.21.15.69:0/2354776257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0320103d20 msgr2=0x7f03201991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0320103d20 0x7f03201991e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f031c005940 tx=0x7f031c0042f0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 -- 172.21.15.69:0/2354776257 shutdown_connections 2024-05-14T17:01:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03080b2a90 0x7f03080b4f50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f032010d060 0x7f032019daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0320104640 0x7f0320199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 --2- 172.21.15.69:0/2354776257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0320103d20 0x7f03201991e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 -- 172.21.15.69:0/2354776257 >> 172.21.15.69:0/2354776257 conn(0x7f0320078ad0 msgr2=0x7f03200fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 -- 172.21.15.69:0/2354776257 shutdown_connections 2024-05-14T17:01:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:23.914+0000 7f0326bc1640 1 -- 172.21.15.69:0/2354776257 wait complete. 2024-05-14T17:01:44.814 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:24 smithi069 ceph-mon[31507]: pgmap v362: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-05-14T17:01:44.864 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 655.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:44.915 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:44 smithi203 ceph-mon[35787]: pgmap v372: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 12 MiB/s wr, 879 op/s 2024-05-14T17:01:44.964 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:44 smithi092 ceph-mon[35763]: pgmap v372: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 12 MiB/s wr, 879 op/s 2024-05-14T17:01:45.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.605+0000 7fa0ebb46700 1 -- 172.21.15.69:0/1441067426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0e4102470 msgr2=0x7fa0e41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:45.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.605+0000 7fa0ebb46700 1 --2- 172.21.15.69:0/1441067426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0e4102470 0x7fa0e41028f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009990 tx=0x7fa0d4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.605+0000 7fa0ebb46700 1 -- 172.21.15.69:0/1441067426 shutdown_connections 2024-05-14T17:01:45.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.605+0000 7fa0ebb46700 1 --2- 172.21.15.69:0/1441067426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e4102e30 0x7fa0e410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.605+0000 7fa0ebb46700 1 --2- 172.21.15.69:0/1441067426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e41084b0 0x7fa0e4108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.605+0000 7fa0ebb46700 1 --2- 172.21.15.69:0/1441067426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0e4102470 0x7fa0e41028f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.605+0000 7fa0ebb46700 1 -- 172.21.15.69:0/1441067426 >> 172.21.15.69:0/1441067426 conn(0x7fa0e40793f0 msgr2=0x7fa0e4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:45.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.605+0000 7fa0ebb46700 1 -- 172.21.15.69:0/1441067426 shutdown_connections 2024-05-14T17:01:45.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.605+0000 7fa0ebb46700 1 -- 172.21.15.69:0/1441067426 wait complete. 2024-05-14T17:01:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ebb46700 1 Processor -- start 2024-05-14T17:01:45.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ebb46700 1 -- start start 2024-05-14T17:01:45.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ebb46700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e4102470 0x7fa0e410fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:45.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ebb46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4102e30 0x7fa0e4110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:45.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ebb46700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0e41084b0 0x7fa0e4110680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ebb46700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e4113700 con 0x7fa0e41084b0 2024-05-14T17:01:45.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ebb46700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e4113580 con 0x7fa0e4102e30 2024-05-14T17:01:45.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ebb46700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e4113880 con 0x7fa0e4102470 2024-05-14T17:01:45.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ea0e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0e41084b0 0x7fa0e4110680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:45.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ea0e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0e41084b0 0x7fa0e4110680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60022/0 (socket says 172.21.15.69:60022) 2024-05-14T17:01:46.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0ea0e3700 1 -- 172.21.15.69:0/2563806153 learned_addr learned my addr 172.21.15.69:0/2563806153 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:46.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4102e30 0x7fa0e4110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:46.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.606+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e4102470 0x7fa0e410fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:46.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0ea0e3700 1 -- 172.21.15.69:0/2563806153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e4102470 msgr2=0x7fa0e410fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:46.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0ea0e3700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e4102470 0x7fa0e410fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:46.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0ea0e3700 1 -- 172.21.15.69:0/2563806153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4102e30 msgr2=0x7fa0e4110140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:46.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0ea0e3700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4102e30 0x7fa0e4110140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:46.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0ea0e3700 1 -- 172.21.15.69:0/2563806153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e41a8100 con 0x7fa0e41084b0 2024-05-14T17:01:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4102e30 0x7fa0e4110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:01:46.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0ea0e3700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0e41084b0 0x7fa0e4110680 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00ada0 tx=0x7fa0dc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:46.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0daffd700 1 -- 172.21.15.69:0/2563806153 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc017020 con 0x7fa0e41084b0 2024-05-14T17:01:46.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0daffd700 1 -- 172.21.15.69:0/2563806153 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0dc00cd80 con 0x7fa0e41084b0 2024-05-14T17:01:46.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0ebb46700 1 -- 172.21.15.69:0/2563806153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e41a83f0 con 0x7fa0e41084b0 2024-05-14T17:01:46.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0daffd700 1 -- 172.21.15.69:0/2563806153 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc005260 con 0x7fa0e41084b0 2024-05-14T17:01:46.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.607+0000 7fa0ebb46700 1 -- 172.21.15.69:0/2563806153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e41a8900 con 0x7fa0e41084b0 2024-05-14T17:01:46.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.609+0000 7fa0daffd700 1 -- 172.21.15.69:0/2563806153 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa0dc00c8d0 con 0x7fa0e41084b0 2024-05-14T17:01:46.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.610+0000 7fa0daffd700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa0d00b29f0 0x7fa0d00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:46.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.610+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa0d00b29f0 0x7fa0d00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:47.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.610+0000 7fa0daffd700 1 -- 172.21.15.69:0/2563806153 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7fa0dc0e1ce0 con 0x7fa0e41084b0 2024-05-14T17:01:47.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0f63dcb80 1 Processor -- start 2024-05-14T17:01:47.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa0d00b29f0 0x7fa0d00b4eb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009be0 tx=0x7fa0e0009260 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:47.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0f63dcb80 1 -- start start 2024-05-14T17:01:47.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322e1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:47.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322dcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:47.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322dd2c0 0x564b322dd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:47.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564b3223ae70 con 0x564b322e1420 2024-05-14T17:01:47.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b3222aa10 con 0x564b322dd2c0 2024-05-14T17:01:47.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564b3222ea50 con 0x564b322e1d40 2024-05-14T17:01:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0e98e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322e1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:47.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0e98e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322e1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60024/0 (socket says 172.21.15.69:60024) 2024-05-14T17:01:47.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0e98e2700 1 -- 172.21.15.69:0/90095986 learned_addr learned my addr 172.21.15.69:0/90095986 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:47.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322dcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:47.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0ea0e3700 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322dd2c0 0x564b322dd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:47.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0e98e2700 1 -- 172.21.15.69:0/90095986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 msgr2=0x564b322dcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:47.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322dcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0e98e2700 1 -- 172.21.15.69:0/90095986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322dd2c0 msgr2=0x564b322dd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:47.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.611+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322dd2c0 0x564b322dd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.612+0000 7fa0e98e2700 1 -- 172.21.15.69:0/90095986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b321d40b0 con 0x564b322e1420 2024-05-14T17:01:48.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.612+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322dcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:48.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.612+0000 7fa0ea0e3700 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322dd2c0 0x564b322dd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:48.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.612+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322e1800 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0011d70 tx=0x7fa0e0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea2395b1e3f2a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:48.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.612+0000 7fa0cffff700 1 -- 172.21.15.69:0/90095986 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0011da0 con 0x564b322e1420 2024-05-14T17:01:48.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.612+0000 7fa0cffff700 1 -- 172.21.15.69:0/90095986 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e00044d0 con 0x564b322e1420 2024-05-14T17:01:48.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.612+0000 7fa0cffff700 1 -- 172.21.15.69:0/90095986 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0003dc0 con 0x564b322e1420 2024-05-14T17:01:48.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/90095986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 msgr2=0x564b322e1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:48.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322e1800 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0011d70 tx=0x7fa0e0014000 comp rx=0 tx=0).stop 2024-05-14T17:01:48.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/90095986 shutdown_connections 2024-05-14T17:01:48.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322e1800 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322dd2c0 0x564b322dd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/90095986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322dcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/90095986 >> 172.21.15.69:0/90095986 conn(0x564b32209eb0 msgr2=0x564b322d74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:48.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/90095986 shutdown_connections 2024-05-14T17:01:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/90095986 wait complete. 2024-05-14T17:01:48.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 -- start start 2024-05-14T17:01:48.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.613+0000 7fa0f63dcb80 1 Processor -- start 2024-05-14T17:01:48.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322e8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:48.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1420 0x564b322e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322e6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564b321310a0 con 0x564b322dd2c0 2024-05-14T17:01:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b32192ea0 con 0x564b322e1420 2024-05-14T17:01:49.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564b322e9880 con 0x564b322e1d40 2024-05-14T17:01:49.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0f63dcb80 10 client.0 _ll_register_callbacks cb 0x564b321c5a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:49.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0e98e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322e8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0e98e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322e8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60040/0 (socket says 172.21.15.69:60040) 2024-05-14T17:01:49.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0e98e2700 1 -- 172.21.15.69:0/2091554545 learned_addr learned my addr 172.21.15.69:0/2091554545 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:49.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1420 0x564b322e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0e98e2700 1 -- 172.21.15.69:0/2091554545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 msgr2=0x564b322e6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:49.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0e98e2700 1 -- 172.21.15.69:0/2091554545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1420 msgr2=0x564b322e5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1420 0x564b322e5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0e98e2700 1 -- 172.21.15.69:0/2091554545 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3217a9d0 con 0x564b322dd2c0 2024-05-14T17:01:49.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322e8bb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009210 tx=0x7fa0e0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63cbbeaa1ba08fe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:49.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:01:49.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/2091554545 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0034060 con 0x564b322dd2c0 2024-05-14T17:01:50.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/2091554545 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7fa0e0004240 con 0x564b322dd2c0 2024-05-14T17:01:50.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.614+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/2091554545 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e00044d0 con 0x564b322dd2c0 2024-05-14T17:01:50.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/2091554545 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa0e0005050 con 0x564b322dd2c0 2024-05-14T17:01:50.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0f63dcb80 10 client.16647 fetch_fsmap learned FSMap version 50 2024-05-14T17:01:50.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/2091554545 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564b321e4860 con 0x564b322dd2c0 2024-05-14T17:01:50.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/2091554545 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e00044d0 con 0x564b322dd2c0 2024-05-14T17:01:50.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/2091554545 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 50) ==== 8346+0+0 (secure 0 0 0) 0x7fa0e002e030 con 0x564b322dd2c0 2024-05-14T17:01:50.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0f63dcb80 10 client.16647 fetch_fsmap finished waiting for FSMap version 50 2024-05-14T17:01:50.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0f63dcb80 10 client.16647 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:01:50.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0f63dcb80 20 client.16647 populate_metadata read hostname 'smithi069' 2024-05-14T17:01:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b322f6330 0x564b322f8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0f63dcb80 4 client.16647 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:01:50.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/2091554545 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b3223cdc0 con 0x564b322f6330 2024-05-14T17:01:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.615+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b322f6330 0x564b322f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:50.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.616+0000 7fa0cd7fa700 10 client.16647 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.616+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b322f6330 0x564b322f8720 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f36d5c3253ce49ee server_cookie=4cd2d00715650c6c in_seq=0 out_seq=0 2024-05-14T17:01:50.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.616+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/2091554545 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b3223cdc0 con 0x564b322f6330 2024-05-14T17:01:51.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.616+0000 7fa0cd7fa700 10 client.16647 handle_command_reply: tid=0 2024-05-14T17:01:51.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.616+0000 7fa0f63dcb80 1 client.16647 shutdown 2024-05-14T17:01:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.616+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/2091554545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 msgr2=0x564b322e8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:51.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.616+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322e8bb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009210 tx=0x7fa0e0003b90 comp rx=0 tx=0).stop 2024-05-14T17:01:51.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.617+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/2091554545 shutdown_connections 2024-05-14T17:01:51.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.617+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b322f6330 0x564b322f8720 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.617+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322e8bb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.617+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322e6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.617+0000 7fa0cd7fa700 0 client.16647 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:51.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.617+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/2091554545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1420 0x564b322e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.617+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/2091554545 >> 172.21.15.69:0/2091554545 conn(0x564b32209eb0 msgr2=0x564b322d1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:51.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.617+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/2091554545 shutdown_connections 2024-05-14T17:01:51.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.617+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/2091554545 wait complete. 2024-05-14T17:01:51.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.617+0000 7fa0f63dcb80 20 client.16647 trim_cache size 0 max 16384 2024-05-14T17:01:51.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0f63dcb80 1 Processor -- start 2024-05-14T17:01:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0f63dcb80 1 -- start start 2024-05-14T17:01:51.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322dd2c0 0x564b322f2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:52.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1d40 0x564b322fdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:52.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e41b2a30 con 0x564b322e1420 2024-05-14T17:01:52.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e41b0260 con 0x564b322e1d40 2024-05-14T17:01:52.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e41ae0b0 con 0x564b322dd2c0 2024-05-14T17:01:52.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0e90e1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:52.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0e90e1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322d20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60056/0 (socket says 172.21.15.69:60056) 2024-05-14T17:01:52.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0e90e1700 1 -- 172.21.15.69:0/3428232507 learned_addr learned my addr 172.21.15.69:0/3428232507 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:52.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.636+0000 7fa0ea0e3700 1 --2- 172.21.15.69:0/3428232507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1d40 0x564b322fdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:52.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/3428232507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322dd2c0 0x564b322f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:52.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0e90e1700 1 -- 172.21.15.69:0/3428232507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322dd2c0 msgr2=0x564b322f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:52.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/3428232507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322dd2c0 0x564b322f2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:52.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0e90e1700 1 -- 172.21.15.69:0/3428232507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1d40 msgr2=0x564b322fdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/3428232507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1d40 0x564b322fdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:52.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0e90e1700 1 -- 172.21.15.69:0/3428232507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b322d2610 con 0x564b322e1420 2024-05-14T17:01:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/3428232507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322dd2c0 0x564b322f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/3428232507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322d20d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4000c00 tx=0x7fa0d401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c94b11d399e5e88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:52.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0cf7fe700 1 -- 172.21.15.69:0/3428232507 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d402c020 con 0x564b322e1420 2024-05-14T17:01:52.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0cf7fe700 1 -- 172.21.15.69:0/3428232507 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d40056a0 con 0x564b322e1420 2024-05-14T17:01:52.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.637+0000 7fa0cf7fe700 1 -- 172.21.15.69:0/3428232507 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4028350 con 0x564b322e1420 2024-05-14T17:01:53.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3428232507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 msgr2=0x564b322d20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:53.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/3428232507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322d20d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4000c00 tx=0x7fa0d401a040 comp rx=0 tx=0).stop 2024-05-14T17:01:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3428232507 shutdown_connections 2024-05-14T17:01:53.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/3428232507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1d40 0x564b322fdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:53.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/3428232507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322dd2c0 0x564b322f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:53.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/3428232507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322e1420 0x564b322d20d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:53.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3428232507 >> 172.21.15.69:0/3428232507 conn(0x564b32209eb0 msgr2=0x564b322dc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:53.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3428232507 shutdown_connections 2024-05-14T17:01:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3428232507 wait complete. 2024-05-14T17:01:53.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 -- start start 2024-05-14T17:01:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 Processor -- start 2024-05-14T17:01:53.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:53.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1420 0x564b322eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:53.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:53.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e41ae720 con 0x564b322dd2c0 2024-05-14T17:01:53.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e41afb90 con 0x564b322e1420 2024-05-14T17:01:53.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.638+0000 7fa0f63dcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e41aedf0 con 0x564b322e1d40 2024-05-14T17:01:53.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0f63dcb80 10 client.0 _ll_register_callbacks cb 0x564b321c5a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:53.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0e98e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:54.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0e98e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322eed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60070/0 (socket says 172.21.15.69:60070) 2024-05-14T17:01:54.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0e98e2700 1 -- 172.21.15.69:0/3601634314 learned_addr learned my addr 172.21.15.69:0/3601634314 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:01:54.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1420 0x564b322eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:54.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0ea0e3700 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:54.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0e98e2700 1 -- 172.21.15.69:0/3601634314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 msgr2=0x564b322eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:54.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0e98e2700 1 -- 172.21.15.69:0/3601634314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1420 msgr2=0x564b322eb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:54.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1420 0x564b322eb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0e98e2700 1 -- 172.21.15.69:0/3601634314 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b322ebb70 con 0x564b322dd2c0 2024-05-14T17:01:54.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0e98e2700 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322eed10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0011b80 tx=0x7fa0e0003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39791b6b160a6c2e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:54.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:01:54.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/3601634314 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0034040 con 0x564b322dd2c0 2024-05-14T17:01:54.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/3601634314 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7fa0e001e420 con 0x564b322dd2c0 2024-05-14T17:01:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3601634314 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d40056a0 con 0x564b322dd2c0 2024-05-14T17:01:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/3601634314 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa0e0047050 con 0x564b322dd2c0 2024-05-14T17:01:54.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0f63dcb80 10 client.16653 fetch_fsmap learned FSMap version 50 2024-05-14T17:01:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.639+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3601634314 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564b32361580 con 0x564b322dd2c0 2024-05-14T17:01:54.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/3601634314 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d40056a0 con 0x564b322dd2c0 2024-05-14T17:01:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/3601634314 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 50) ==== 8346+0+0 (secure 0 0 0) 0x7fa0e002e070 con 0x564b322dd2c0 2024-05-14T17:01:55.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0f63dcb80 10 client.16653 fetch_fsmap finished waiting for FSMap version 50 2024-05-14T17:01:55.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0f63dcb80 10 client.16653 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:01:55.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0f63dcb80 20 client.16653 populate_metadata read hostname 'smithi069' 2024-05-14T17:01:55.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b32361c00 0x564b32364020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:55.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0f63dcb80 4 client.16653 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c837d4f6", "release": true, "await": true}] 2024-05-14T17:01:55.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3601634314 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c837d4f6", "release": true, "await": true}) -- 0x564b322365a0 con 0x564b32361c00 2024-05-14T17:01:55.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b32361c00 0x564b32364020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0e90e1700 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b32361c00 0x564b32364020 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479c2c219dbd852 server_cookie=52f3bcc8b4108724 in_seq=0 out_seq=0 2024-05-14T17:01:55.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.640+0000 7fa0cd7fa700 10 client.16653 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:55.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0cd7fa700 1 -- 172.21.15.69:0/3601634314 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564b322365a0 con 0x564b32361c00 2024-05-14T17:01:55.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0cd7fa700 10 client.16653 handle_command_reply: tid=0 2024-05-14T17:01:55.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0f63dcb80 1 client.16653 shutdown 2024-05-14T17:01:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3601634314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 msgr2=0x564b322eed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:55.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322eed10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0011b80 tx=0x7fa0e0003d60 comp rx=0 tx=0).stop 2024-05-14T17:01:55.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3601634314 shutdown_connections 2024-05-14T17:01:55.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b32361c00 0x564b32364020 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:55.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b322dd2c0 0x564b322eed10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:55.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b322e1d40 0x564b322eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:55.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0cd7fa700 0 client.16653 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:01:56.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0f63dcb80 1 --2- 172.21.15.69:0/3601634314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b322e1420 0x564b322eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:56.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.652+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3601634314 >> 172.21.15.69:0/3601634314 conn(0x564b32209eb0 msgr2=0x564b322e00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:56.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3601634314 shutdown_connections 2024-05-14T17:01:56.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0f63dcb80 1 -- 172.21.15.69:0/3601634314 wait complete. 2024-05-14T17:01:56.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0f63dcb80 20 client.16653 trim_cache size 0 max 16384 2024-05-14T17:01:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:01:56.337 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 50, 2024-05-14T17:01:56.387 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T17:01:56.446 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 36, 2024-05-14T17:01:56.496 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T17:01:56.546 INFO:teuthology.orchestra.run.smithi069.stdout: "c837d4f6": { 2024-05-14T17:01:56.596 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 36, 2024-05-14T17:01:56.679 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T17:01:56.729 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:01:56.779 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T17:01:56.829 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:01:56.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:01:56.942 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T17:01:56.988 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 137.9, 2024-05-14T17:01:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T17:01:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T17:01:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T17:01:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:01:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T17:01:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:01:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:01:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:01:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:01:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:01:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:01:58.172 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:01:58.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 -- 172.21.15.69:0/2563806153 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa0d00b29f0 msgr2=0x7fa0d00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:58.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa0d00b29f0 0x7fa0d00b4eb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009be0 tx=0x7fa0e0009260 comp rx=0 tx=0).stop 2024-05-14T17:01:58.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 -- 172.21.15.69:0/2563806153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0e41084b0 msgr2=0x7fa0e4110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:58.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0e41084b0 0x7fa0e4110680 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00ada0 tx=0x7fa0dc00c330 comp rx=0 tx=0).stop 2024-05-14T17:01:58.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 -- 172.21.15.69:0/2563806153 shutdown_connections 2024-05-14T17:01:58.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa0d00b29f0 0x7fa0d00b4eb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0e41084b0 0x7fa0e4110680 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e4102470 0x7fa0e410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 --2- 172.21.15.69:0/2563806153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4102e30 0x7fa0e4110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 -- 172.21.15.69:0/2563806153 >> 172.21.15.69:0/2563806153 conn(0x7fa0e40793f0 msgr2=0x7fa0e41004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:58.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 -- 172.21.15.69:0/2563806153 shutdown_connections 2024-05-14T17:01:58.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:01:25.653+0000 7fa0ebb46700 1 -- 172.21.15.69:0/2563806153 wait complete. 2024-05-14T17:01:58.856 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:26 smithi069 ceph-mon[31507]: pgmap v363: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 KiB/s wr, 6 op/s 2024-05-14T17:01:58.907 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:28 smithi069 ceph-mon[31507]: pgmap v364: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 324 KiB/s wr, 38 op/s 2024-05-14T17:01:58.956 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 660.0 s, 2648.7 tps, lat 563.499 ms stddev 6059.680 2024-05-14T17:01:59.007 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:30 smithi069 ceph-mon[31507]: pgmap v365: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 273 KiB/s wr, 32 op/s 2024-05-14T17:01:59.056 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:32 smithi069 ceph-mon[31507]: pgmap v366: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 283 KiB/s wr, 33 op/s 2024-05-14T17:01:59.106 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:34 smithi069 ceph-mon[31507]: pgmap v367: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-05-14T17:01:59.156 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 665.0 s, 3172.2 tps, lat 31.461 ms stddev 36.170 2024-05-14T17:01:59.206 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:46 smithi203 ceph-mon[35787]: pgmap v373: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2024-05-14T17:01:59.264 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:48 smithi203 ceph-mon[35787]: pgmap v374: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 11 MiB/s wr, 846 op/s 2024-05-14T17:01:59.314 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:50 smithi203 ceph-mon[35787]: pgmap v375: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.1 MiB/s wr, 542 op/s 2024-05-14T17:01:59.364 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:50 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:59.414 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:52 smithi203 ceph-mon[35787]: pgmap v376: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.1 MiB/s wr, 544 op/s 2024-05-14T17:01:59.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:54 smithi203 ceph-mon[35787]: pgmap v377: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.7 MiB/s wr, 807 op/s 2024-05-14T17:01:59.523 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:56 smithi203 ceph-mon[35787]: pgmap v378: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.3 MiB/s wr, 531 op/s 2024-05-14T17:01:59.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:01:58 smithi203 ceph-mon[35787]: pgmap v379: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.9 MiB/s wr, 799 op/s 2024-05-14T17:01:59.623 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:46 smithi092 ceph-mon[35763]: pgmap v373: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2024-05-14T17:01:59.681 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:48 smithi092 ceph-mon[35763]: pgmap v374: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 11 MiB/s wr, 846 op/s 2024-05-14T17:01:59.731 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:50 smithi092 ceph-mon[35763]: pgmap v375: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.1 MiB/s wr, 542 op/s 2024-05-14T17:01:59.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:50 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:59.831 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:52 smithi092 ceph-mon[35763]: pgmap v376: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.1 MiB/s wr, 544 op/s 2024-05-14T17:01:59.881 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:54 smithi092 ceph-mon[35763]: pgmap v377: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.7 MiB/s wr, 807 op/s 2024-05-14T17:01:59.931 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:56 smithi092 ceph-mon[35763]: pgmap v378: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.3 MiB/s wr, 531 op/s 2024-05-14T17:01:59.981 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:01:58 smithi092 ceph-mon[35763]: pgmap v379: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.9 MiB/s wr, 799 op/s 2024-05-14T17:02:00.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:36 smithi069 ceph-mon[31507]: pgmap v368: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.3 MiB/s wr, 328 op/s 2024-05-14T17:02:00.081 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c837d4f6', total seconds elapsed: 112.2 2024-05-14T17:02:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.049+0000 7f7b3dd9e640 1 -- 172.21.15.69:0/4049309432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b38107790 msgr2=0x7f7b38107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.049+0000 7f7b3dd9e640 1 --2- 172.21.15.69:0/4049309432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b38107790 0x7f7b38107b70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009ca0 tx=0x7f7b2802a8f0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.054+0000 7f7b3dd9e640 1 -- 172.21.15.69:0/4049309432 shutdown_connections 2024-05-14T17:02:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.054+0000 7f7b3dd9e640 1 --2- 172.21.15.69:0/4049309432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b38102140 0x7f7b3810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.054+0000 7f7b3dd9e640 1 --2- 172.21.15.69:0/4049309432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38101780 0x7f7b38101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.054+0000 7f7b3dd9e640 1 --2- 172.21.15.69:0/4049309432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b38107790 0x7f7b38107b70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.054+0000 7f7b3dd9e640 1 -- 172.21.15.69:0/4049309432 >> 172.21.15.69:0/4049309432 conn(0x7f7b380fd2b0 msgr2=0x7f7b380ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.062+0000 7f7b3dd9e640 1 -- 172.21.15.69:0/4049309432 shutdown_connections 2024-05-14T17:02:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.063+0000 7f7b3dd9e640 1 -- 172.21.15.69:0/4049309432 wait complete. 2024-05-14T17:02:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.064+0000 7f7b3dd9e640 1 Processor -- start 2024-05-14T17:02:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.064+0000 7f7b3dd9e640 1 -- start start 2024-05-14T17:02:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.064+0000 7f7b3dd9e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b38101780 0x7f7b38117040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.064+0000 7f7b3dd9e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38102140 0x7f7b38117580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.064+0000 7f7b3dd9e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b38107790 0x7f7b38112150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.064+0000 7f7b3dd9e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38110a30 con 0x7f7b38101780 2024-05-14T17:02:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.064+0000 7f7b3dd9e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7b381108b0 con 0x7f7b38107790 2024-05-14T17:02:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.064+0000 7f7b3dd9e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38110bb0 con 0x7f7b38102140 2024-05-14T17:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.064+0000 7f7b3d59d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b38107790 0x7f7b38112150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.065+0000 7f7b3d59d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b38107790 0x7f7b38112150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:59072/0 (socket says 172.21.15.69:59072) 2024-05-14T17:02:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.065+0000 7f7b3d59d640 1 -- 172.21.15.69:0/258714339 learned_addr learned my addr 172.21.15.69:0/258714339 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.065+0000 7f7b3d59d640 1 -- 172.21.15.69:0/258714339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38102140 msgr2=0x7f7b38117580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.065+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38102140 0x7f7b38117580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.065+0000 7f7b3d59d640 1 -- 172.21.15.69:0/258714339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b38101780 msgr2=0x7f7b38117040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.065+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b38101780 0x7f7b38117040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.065+0000 7f7b3d59d640 1 -- 172.21.15.69:0/258714339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b38112a10 con 0x7f7b38107790 2024-05-14T17:02:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.065+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b38107790 0x7f7b38112150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000b3f0 tx=0x7f7b3000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.067+0000 7f7b35ffb640 1 -- 172.21.15.69:0/258714339 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30015020 con 0x7f7b38107790 2024-05-14T17:02:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.068+0000 7f7b35ffb640 1 -- 172.21.15.69:0/258714339 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b30011ce0 con 0x7f7b38107790 2024-05-14T17:02:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.068+0000 7f7b35ffb640 1 -- 172.21.15.69:0/258714339 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30010dc0 con 0x7f7b38107790 2024-05-14T17:02:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.068+0000 7f7b3dd9e640 1 -- 172.21.15.69:0/258714339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b38112d00 con 0x7f7b38107790 2024-05-14T17:02:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.068+0000 7f7b3dd9e640 1 -- 172.21.15.69:0/258714339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b3811ef40 con 0x7f7b38107790 2024-05-14T17:02:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.071+0000 7f7b41dfa740 1 Processor -- start 2024-05-14T17:02:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.072+0000 7f7b41dfa740 1 -- start start 2024-05-14T17:02:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.072+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 0x55f370e5e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.072+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e5f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.072+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e7aa50 0x55f370e7ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.072+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f370dd1c30 con 0x55f370e7aa50 2024-05-14T17:02:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.072+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f370dc17d0 con 0x55f370e5eeb0 2024-05-14T17:02:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.072+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f370dc5810 con 0x55f370e603a0 2024-05-14T17:02:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.075+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 0x55f370e5e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.075+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 0x55f370e5e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:40772/0 (socket says 172.21.15.69:40772) 2024-05-14T17:02:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.075+0000 7f7b37fff640 1 -- 172.21.15.69:0/2726787880 learned_addr learned my addr 172.21.15.69:0/2726787880 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.075+0000 7f7b3cd9c640 1 --2- 172.21.15.69:0/2726787880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e5f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.075+0000 7f7b37fff640 1 -- 172.21.15.69:0/2726787880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 msgr2=0x55f370e5f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.075+0000 7f7b37fff640 1 --2- 172.21.15.69:0/2726787880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e5f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.075+0000 7f7b37fff640 1 -- 172.21.15.69:0/2726787880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e7aa50 msgr2=0x55f370e7ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.075+0000 7f7b37fff640 1 --2- 172.21.15.69:0/2726787880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e7aa50 0x55f370e7ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.076+0000 7f7b37fff640 1 -- 172.21.15.69:0/2726787880 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f370ad6210 con 0x55f370e603a0 2024-05-14T17:02:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.076+0000 7f7b37fff640 1 --2- 172.21.15.69:0/2726787880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 0x55f370e5e970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b20001760 tx=0x7f7b20001c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e045edb89cdfd904 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.078+0000 7f7b35ffb640 1 -- 172.21.15.69:0/258714339 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7b30018020 con 0x7f7b38107790 2024-05-14T17:02:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.078+0000 7f7b3dd9e640 1 -- 172.21.15.69:0/2726787880 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20004be0 con 0x55f370e603a0 2024-05-14T17:02:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.078+0000 7f7b3dd9e640 1 -- 172.21.15.69:0/2726787880 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b20008e30 con 0x55f370e603a0 2024-05-14T17:02:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.078+0000 7f7b3dd9e640 1 -- 172.21.15.69:0/2726787880 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000e920 con 0x55f370e603a0 2024-05-14T17:02:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.080+0000 7f7b35ffb640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7b240b2b60 0x7f7b240b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.080+0000 7f7b35ffb640 1 -- 172.21.15.69:0/258714339 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f7b300e1c30 con 0x7f7b38107790 2024-05-14T17:02:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.081+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2726787880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 msgr2=0x55f370e5e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.081+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2726787880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 0x55f370e5e970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b20001760 tx=0x7f7b20001c20 comp rx=0 tx=0).stop 2024-05-14T17:02:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.081+0000 7f7b3cd9c640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7b240b2b60 0x7f7b240b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.082+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2726787880 shutdown_connections 2024-05-14T17:02:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.082+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2726787880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e7aa50 0x55f370e7ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.082+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2726787880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e5f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.082+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2726787880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 0x55f370e5e970 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.082+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2726787880 >> 172.21.15.69:0/2726787880 conn(0x55f370e5b2b0 msgr2=0x55f370e5b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.083+0000 7f7b3cd9c640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7b240b2b60 0x7f7b240b5020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7b28006010 tx=0x7f7b2802adf0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.085+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2726787880 shutdown_connections 2024-05-14T17:02:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.088+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2726787880 wait complete. 2024-05-14T17:02:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.088+0000 7f7b41dfa740 1 -- start start 2024-05-14T17:02:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.088+0000 7f7b41dfa740 1 Processor -- start 2024-05-14T17:02:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.089+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e5eeb0 0x55f370e6bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.089+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 0x55f370e6c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.089+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e7aa50 0x55f370e66ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.089+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f370aa7790 con 0x55f370e5eeb0 2024-05-14T17:02:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.089+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f370d7d890 con 0x55f370e7aa50 2024-05-14T17:02:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.089+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f370e61e00 con 0x55f370e603a0 2024-05-14T17:02:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.089+0000 7f7b41dfa740 10 client.0 _ll_register_callbacks cb 0x55f370d80d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.089+0000 7f7b3d59d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e7aa50 0x55f370e66ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.089+0000 7f7b3d59d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e7aa50 0x55f370e66ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:59084/0 (socket says 172.21.15.69:59084) 2024-05-14T17:02:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.089+0000 7f7b3d59d640 1 -- 172.21.15.69:0/2659194395 learned_addr learned my addr 172.21.15.69:0/2659194395 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.092+0000 7f7b3d59d640 1 -- 172.21.15.69:0/2659194395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 msgr2=0x55f370e6c080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T17:02:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.092+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 0x55f370e6c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.092+0000 7f7b3d59d640 1 -- 172.21.15.69:0/2659194395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e5eeb0 msgr2=0x55f370e6bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.092+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e5eeb0 0x55f370e6bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.092+0000 7f7b3d59d640 1 -- 172.21.15.69:0/2659194395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f370ba2620 con 0x55f370e7aa50 2024-05-14T17:02:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.092+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e7aa50 0x55f370e66ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b300a5050 tx=0x7f7b300a8c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87766bb4c181d0c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.092+0000 7f7b17fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:02:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.092+0000 7f7b17fff640 1 -- 172.21.15.69:0/2659194395 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b300c2070 con 0x55f370e7aa50 2024-05-14T17:02:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.093+0000 7f7b17fff640 1 -- 172.21.15.69:0/2659194395 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f7b300a5260 con 0x55f370e7aa50 2024-05-14T17:02:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.093+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2659194395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b20008e30 con 0x55f370e7aa50 2024-05-14T17:02:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.093+0000 7f7b17fff640 1 -- 172.21.15.69:0/2659194395 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7b300b1e30 con 0x55f370e7aa50 2024-05-14T17:02:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.093+0000 7f7b41dfa740 10 client.26084 fetch_fsmap learned FSMap version 50 2024-05-14T17:02:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.093+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2659194395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55f370e75a90 con 0x55f370e7aa50 2024-05-14T17:02:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.094+0000 7f7b17fff640 1 -- 172.21.15.69:0/2659194395 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b20008e30 con 0x55f370e7aa50 2024-05-14T17:02:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.094+0000 7f7b17fff640 1 -- 172.21.15.69:0/2659194395 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 50) ==== 8346+0+0 (secure 0 0 0) 0x7f7b30022060 con 0x55f370e7aa50 2024-05-14T17:02:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.094+0000 7f7b41dfa740 10 client.26084 fetch_fsmap finished waiting for FSMap version 50 2024-05-14T17:02:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.094+0000 7f7b41dfa740 10 client.26084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.094+0000 7f7b41dfa740 20 client.26084 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.094+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f370e76fa0 0x55f370e79440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.094+0000 7f7b41dfa740 4 client.26084 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.094+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2659194395 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f370dd3b80 con 0x55f370e76fa0 2024-05-14T17:02:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.094+0000 7f7b3cd9c640 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f370e76fa0 0x55f370e79440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.095+0000 7f7b17fff640 10 client.26084 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.095+0000 7f7b3cd9c640 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f370e76fa0 0x55f370e79440 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=587451679748dfbf server_cookie=76e26bb73a93cbbc in_seq=0 out_seq=0 2024-05-14T17:02:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.095+0000 7f7b17fff640 1 -- 172.21.15.69:0/2659194395 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f370dd3b80 con 0x55f370e76fa0 2024-05-14T17:02:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.095+0000 7f7b17fff640 10 client.26084 handle_command_reply: tid=0 2024-05-14T17:02:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.096+0000 7f7b41dfa740 1 client.26084 shutdown 2024-05-14T17:02:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.096+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2659194395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e7aa50 msgr2=0x55f370e66ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.096+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e7aa50 0x55f370e66ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b300a5050 tx=0x7f7b300a8c20 comp rx=0 tx=0).stop 2024-05-14T17:02:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.096+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2659194395 shutdown_connections 2024-05-14T17:02:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.096+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f370e76fa0 0x55f370e79440 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.096+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e7aa50 0x55f370e66ea0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.096+0000 7f7b17fff640 0 client.26084 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.096+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e603a0 0x55f370e6c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.096+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2659194395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e5eeb0 0x55f370e6bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.096+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2659194395 >> 172.21.15.69:0/2659194395 conn(0x55f370e5b2b0 msgr2=0x55f370e61120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.097+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2659194395 shutdown_connections 2024-05-14T17:02:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.097+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2659194395 wait complete. 2024-05-14T17:02:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.097+0000 7f7b41dfa740 20 client.26084 trim_cache size 0 max 16384 2024-05-14T17:02:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b41dfa740 1 Processor -- start 2024-05-14T17:02:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b41dfa740 1 -- start start 2024-05-14T17:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e7a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e66000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 0x55f370e5a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b381219f0 con 0x55f370e603a0 2024-05-14T17:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3811f260 con 0x55f370e5eeb0 2024-05-14T17:02:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38072a30 con 0x55f370e7aa50 2024-05-14T17:02:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e66000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e66000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37238/0 (socket says 172.21.15.69:37238) 2024-05-14T17:02:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b37fff640 1 -- 172.21.15.69:0/2318357903 learned_addr learned my addr 172.21.15.69:0/2318357903 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b3cd9c640 1 --2- 172.21.15.69:0/2318357903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e7a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/2318357903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 0x55f370e5a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b37fff640 1 -- 172.21.15.69:0/2318357903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 msgr2=0x55f370e5a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b37fff640 1 --2- 172.21.15.69:0/2318357903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 0x55f370e5a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b37fff640 1 -- 172.21.15.69:0/2318357903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 msgr2=0x55f370e7a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.122+0000 7f7b37fff640 1 --2- 172.21.15.69:0/2318357903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e7a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.123+0000 7f7b37fff640 1 -- 172.21.15.69:0/2318357903 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f370e5c4c0 con 0x55f370e603a0 2024-05-14T17:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.123+0000 7f7b37fff640 1 --2- 172.21.15.69:0/2318357903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e66000 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000fb0 tx=0x7f7b20000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f3dad7797cf0a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.123+0000 7f7b2e7fc640 1 -- 172.21.15.69:0/2318357903 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20005cf0 con 0x55f370e603a0 2024-05-14T17:02:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.123+0000 7f7b2e7fc640 1 -- 172.21.15.69:0/2318357903 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b20008e30 con 0x55f370e603a0 2024-05-14T17:02:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.123+0000 7f7b2e7fc640 1 -- 172.21.15.69:0/2318357903 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000e940 con 0x55f370e603a0 2024-05-14T17:02:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2318357903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 msgr2=0x55f370e66000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2318357903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e66000 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000fb0 tx=0x7f7b20000fe0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2318357903 shutdown_connections 2024-05-14T17:02:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2318357903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 0x55f370e5a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2318357903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e66000 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/2318357903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e7a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2318357903 >> 172.21.15.69:0/2318357903 conn(0x55f370e5b2b0 msgr2=0x55f370e59b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2318357903 shutdown_connections 2024-05-14T17:02:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 -- 172.21.15.69:0/2318357903 wait complete. 2024-05-14T17:02:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 -- start start 2024-05-14T17:02:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.124+0000 7f7b41dfa740 1 Processor -- start 2024-05-14T17:02:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e72c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b41dfa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 0x55f370e73150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b380730a0 con 0x55f370e603a0 2024-05-14T17:02:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38074510 con 0x55f370e5eeb0 2024-05-14T17:02:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b41dfa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38073770 con 0x55f370e7aa50 2024-05-14T17:02:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b41dfa740 10 client.0 _ll_register_callbacks cb 0x7f7b300a8ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e72c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e72c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37242/0 (socket says 172.21.15.69:37242) 2024-05-14T17:02:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b37fff640 1 -- 172.21.15.69:0/4265562690 learned_addr learned my addr 172.21.15.69:0/4265562690 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 0x55f370e73150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b3d59d640 1 -- 172.21.15.69:0/4265562690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 msgr2=0x55f370e726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b3d59d640 1 -- 172.21.15.69:0/4265562690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 msgr2=0x55f370e72c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e72c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.125+0000 7f7b3d59d640 1 -- 172.21.15.69:0/4265562690 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f370f05310 con 0x55f370e7aa50 2024-05-14T17:02:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.126+0000 7f7b17fff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T17:02:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.126+0000 7f7b3d59d640 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 0x55f370e73150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b300a8cd0 tx=0x7f7b300a8460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc240f35d638dc61 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.126+0000 7f7b17fff640 1 -- 172.21.15.69:0/4265562690 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b300c2050 con 0x55f370e7aa50 2024-05-14T17:02:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.126+0000 7f7b17fff640 1 -- 172.21.15.69:0/4265562690 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f7b300b9070 con 0x55f370e7aa50 2024-05-14T17:02:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.127+0000 7f7b41dfa740 1 -- 172.21.15.69:0/4265562690 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b20008e30 con 0x55f370e7aa50 2024-05-14T17:02:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.127+0000 7f7b17fff640 1 -- 172.21.15.69:0/4265562690 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7b300c2050 con 0x55f370e7aa50 2024-05-14T17:02:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.127+0000 7f7b41dfa740 10 client.25981 fetch_fsmap learned FSMap version 50 2024-05-14T17:02:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.127+0000 7f7b41dfa740 1 -- 172.21.15.69:0/4265562690 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55f370f06620 con 0x55f370e7aa50 2024-05-14T17:02:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.127+0000 7f7b17fff640 1 -- 172.21.15.69:0/4265562690 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b20008e30 con 0x55f370e7aa50 2024-05-14T17:02:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.128+0000 7f7b17fff640 1 -- 172.21.15.69:0/4265562690 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 50) ==== 8346+0+0 (secure 0 0 0) 0x7f7b300a49e0 con 0x55f370e7aa50 2024-05-14T17:02:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.128+0000 7f7b41dfa740 10 client.25981 fetch_fsmap finished waiting for FSMap version 50 2024-05-14T17:02:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.128+0000 7f7b41dfa740 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.128+0000 7f7b41dfa740 20 client.25981 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.128+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f370f06be0 0x55f370f09000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.128+0000 7f7b41dfa740 4 client.25981 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:02:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.128+0000 7f7b41dfa740 1 -- 172.21.15.69:0/4265562690 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f370dcd360 con 0x55f370f06be0 2024-05-14T17:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.128+0000 7f7b3cd9c640 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f370f06be0 0x55f370f09000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.128+0000 7f7b17fff640 10 client.25981 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.128+0000 7f7b3cd9c640 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f370f06be0 0x55f370f09000 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae0a9fda3ee182ba server_cookie=bfed20c75254623a in_seq=0 out_seq=0 2024-05-14T17:02:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.151+0000 7f7b17fff640 1 -- 172.21.15.69:0/4265562690 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f370dcd360 con 0x55f370f06be0 2024-05-14T17:02:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.151+0000 7f7b17fff640 10 client.25981 handle_command_reply: tid=0 2024-05-14T17:02:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.151+0000 7f7b41dfa740 1 client.25981 shutdown 2024-05-14T17:02:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.151+0000 7f7b41dfa740 1 -- 172.21.15.69:0/4265562690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 msgr2=0x55f370e73150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.151+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 0x55f370e73150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b300a8cd0 tx=0x7f7b300a8460 comp rx=0 tx=0).stop 2024-05-14T17:02:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.151+0000 7f7b41dfa740 1 -- 172.21.15.69:0/4265562690 shutdown_connections 2024-05-14T17:02:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.151+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55f370f06be0 0x55f370f09000 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.152+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f370e7aa50 0x55f370e73150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.152+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f370e603a0 0x55f370e72c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.152+0000 7f7b41dfa740 1 --2- 172.21.15.69:0/4265562690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f370e5eeb0 0x55f370e726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.152+0000 7f7b17fff640 0 client.25981 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.152+0000 7f7b41dfa740 1 -- 172.21.15.69:0/4265562690 >> 172.21.15.69:0/4265562690 conn(0x55f370e5b2b0 msgr2=0x55f370e59b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.152+0000 7f7b41dfa740 1 -- 172.21.15.69:0/4265562690 shutdown_connections 2024-05-14T17:02:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.152+0000 7f7b41dfa740 1 -- 172.21.15.69:0/4265562690 wait complete. 2024-05-14T17:02:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.152+0000 7f7b41dfa740 20 client.25981 trim_cache size 0 max 16384 2024-05-14T17:02:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:02:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:02:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:02:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.152+0000 7f7b2f7fe640 1 -- 172.21.15.69:0/258714339 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7b240b2b60 msgr2=0x7f7b240b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.152+0000 7f7b2f7fe640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7b240b2b60 0x7f7b240b5020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7b28006010 tx=0x7f7b2802adf0 comp rx=0 tx=0).stop 2024-05-14T17:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.153+0000 7f7b2f7fe640 1 -- 172.21.15.69:0/258714339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b38107790 msgr2=0x7f7b38112150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.153+0000 7f7b2f7fe640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b38107790 0x7f7b38112150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000b3f0 tx=0x7f7b3000b8b0 comp rx=0 tx=0).stop 2024-05-14T17:02:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.153+0000 7f7b2f7fe640 1 -- 172.21.15.69:0/258714339 shutdown_connections 2024-05-14T17:02:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.153+0000 7f7b2f7fe640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7b240b2b60 0x7f7b240b5020 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.153+0000 7f7b2f7fe640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b38107790 0x7f7b38112150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.153+0000 7f7b2f7fe640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38102140 0x7f7b38117580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.153+0000 7f7b2f7fe640 1 --2- 172.21.15.69:0/258714339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b38101780 0x7f7b38117040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.153+0000 7f7b2f7fe640 1 -- 172.21.15.69:0/258714339 >> 172.21.15.69:0/258714339 conn(0x7f7b380fd2b0 msgr2=0x7f7b380fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.153+0000 7f7b2f7fe640 1 -- 172.21.15.69:0/258714339 shutdown_connections 2024-05-14T17:02:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:37.153+0000 7f7b2f7fe640 1 -- 172.21.15.69:0/258714339 wait complete. 2024-05-14T17:02:11.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:38 smithi069 ceph-mon[31507]: pgmap v369: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.9 MiB/s wr, 631 op/s 2024-05-14T17:02:12.025 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 670.0 s, 2191.0 tps, lat 45.660 ms stddev 50.368 2024-05-14T17:02:12.059 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:40 smithi069 ceph-mon[31507]: pgmap v370: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.6 MiB/s wr, 602 op/s 2024-05-14T17:02:12.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:42 smithi069 ceph-mon[31507]: pgmap v371: 97 pgs: 97 active+clean; 4.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.6 MiB/s wr, 604 op/s 2024-05-14T17:02:12.124 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:44 smithi069 ceph-mon[31507]: pgmap v372: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 12 MiB/s wr, 879 op/s 2024-05-14T17:02:12.158 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 675.0 s, 2154.6 tps, lat 46.275 ms stddev 47.210 2024-05-14T17:02:12.191 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:00 smithi092 ceph-mon[35763]: pgmap v380: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.3 MiB/s wr, 534 op/s 2024-05-14T17:02:12.224 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:02 smithi092 ceph-mon[35763]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-05-14T17:02:12.265 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:04 smithi092 ceph-mon[35763]: pgmap v382: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.4 MiB/s wr, 827 op/s 2024-05-14T17:02:12.299 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:05 smithi092 ceph-mon[35763]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T17:02:12.332 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:06 smithi092 ceph-mon[35763]: pgmap v384: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 675 op/s 2024-05-14T17:02:12.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:08 smithi092 ceph-mon[35763]: pgmap v385: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-05-14T17:02:12.412 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:09 smithi092 ceph-mon[35763]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T17:02:12.441 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:09 smithi092 ceph-mon[35763]: pgmap v387: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.8 MiB/s wr, 644 op/s 2024-05-14T17:02:12.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:00 smithi203 ceph-mon[35787]: pgmap v380: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.3 MiB/s wr, 534 op/s 2024-05-14T17:02:12.532 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:02 smithi203 ceph-mon[35787]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-05-14T17:02:12.566 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:04 smithi203 ceph-mon[35787]: pgmap v382: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.4 MiB/s wr, 827 op/s 2024-05-14T17:02:12.599 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:05 smithi203 ceph-mon[35787]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T17:02:12.633 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:06 smithi203 ceph-mon[35787]: pgmap v384: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 675 op/s 2024-05-14T17:02:12.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:08 smithi203 ceph-mon[35787]: pgmap v385: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-05-14T17:02:12.699 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:09 smithi203 ceph-mon[35787]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T17:02:12.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:09 smithi203 ceph-mon[35787]: pgmap v387: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.8 MiB/s wr, 644 op/s 2024-05-14T17:02:12.767 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:46 smithi069 ceph-mon[31507]: pgmap v373: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2024-05-14T17:02:12.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:48 smithi069 ceph-mon[31507]: pgmap v374: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 11 MiB/s wr, 846 op/s 2024-05-14T17:02:12.834 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:11 smithi092 ceph-mon[35763]: pgmap v388: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-05-14T17:02:12.867 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:11 smithi203 ceph-mon[35787]: pgmap v388: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-05-14T17:02:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.275+0000 7f4a344d1640 1 -- 172.21.15.69:0/719948915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a2c10f770 msgr2=0x7f4a2c111b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.275+0000 7f4a344d1640 1 --2- 172.21.15.69:0/719948915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a2c10f770 0x7f4a2c111b60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009900 tx=0x7f4a2802a480 comp rx=0 tx=0).stop 2024-05-14T17:02:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 -- 172.21.15.69:0/719948915 shutdown_connections 2024-05-14T17:02:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 --2- 172.21.15.69:0/719948915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a2c10f770 0x7f4a2c111b60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 --2- 172.21.15.69:0/719948915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a2c06b570 0x7f4a2c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 --2- 172.21.15.69:0/719948915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a2c06ac50 0x7f4a2c06b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 -- 172.21.15.69:0/719948915 >> 172.21.15.69:0/719948915 conn(0x7f4a2c078830 msgr2=0x7f4a2c078c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 -- 172.21.15.69:0/719948915 shutdown_connections 2024-05-14T17:02:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 -- 172.21.15.69:0/719948915 wait complete. 2024-05-14T17:02:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 Processor -- start 2024-05-14T17:02:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 -- start start 2024-05-14T17:02:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a2c06ac50 0x7f4a2c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a2c06b570 0x7f4a2c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a2c10f770 0x7f4a2c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c114f30 con 0x7f4a2c06b570 2024-05-14T17:02:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c114db0 con 0x7f4a2c10f770 2024-05-14T17:02:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.276+0000 7f4a344d1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c1150b0 con 0x7f4a2c06ac50 2024-05-14T17:02:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a31a6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a2c06b570 0x7f4a2c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a3226d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a2c06ac50 0x7f4a2c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a32a6e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a2c10f770 0x7f4a2c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a31a6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a2c06b570 0x7f4a2c19b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56670/0 (socket says 172.21.15.69:56670) 2024-05-14T17:02:13.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a31a6c640 1 -- 172.21.15.69:0/652974787 learned_addr learned my addr 172.21.15.69:0/652974787 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a32a6e640 1 -- 172.21.15.69:0/652974787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a2c06ac50 msgr2=0x7f4a2c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a32a6e640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a2c06ac50 0x7f4a2c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a32a6e640 1 -- 172.21.15.69:0/652974787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a2c06b570 msgr2=0x7f4a2c19b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a32a6e640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a2c06b570 0x7f4a2c19b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a32a6e640 1 -- 172.21.15.69:0/652974787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2c1a02f0 con 0x7f4a2c10f770 2024-05-14T17:02:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a2c06b570 0x7f4a2c19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.277+0000 7f4a32a6e640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a2c10f770 0x7f4a2c19fbc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009230 tx=0x7f4a28003ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.278+0000 7f4a1f7fe640 1 -- 172.21.15.69:0/652974787 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2803f070 con 0x7f4a2c10f770 2024-05-14T17:02:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.278+0000 7f4a1f7fe640 1 -- 172.21.15.69:0/652974787 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a28031b50 con 0x7f4a2c10f770 2024-05-14T17:02:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.278+0000 7f4a1f7fe640 1 -- 172.21.15.69:0/652974787 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2803f070 con 0x7f4a2c10f770 2024-05-14T17:02:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.278+0000 7f4a344d1640 1 -- 172.21.15.69:0/652974787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2c1a0580 con 0x7f4a2c10f770 2024-05-14T17:02:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.278+0000 7f4a344d1640 1 -- 172.21.15.69:0/652974787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a2c1a09f0 con 0x7f4a2c10f770 2024-05-14T17:02:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.281+0000 7f4a371ee740 1 Processor -- start 2024-05-14T17:02:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.282+0000 7f4a1f7fe640 1 -- 172.21.15.69:0/652974787 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4a28042070 con 0x7f4a2c10f770 2024-05-14T17:02:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.282+0000 7f4a371ee740 1 -- start start 2024-05-14T17:02:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.282+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4ef8f0 0x564b1a4efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.282+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4f0210 0x564b1a4fa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.282+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 0x564b1a4fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.282+0000 7f4a371ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564b1a462c30 con 0x564b1a4f0210 2024-05-14T17:02:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.282+0000 7f4a371ee740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b1a4527d0 con 0x564b1a4ef8f0 2024-05-14T17:02:14.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.282+0000 7f4a371ee740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564b1a456810 con 0x564b1a4faad0 2024-05-14T17:02:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a3226d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4f0210 0x564b1a4fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a3226d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4f0210 0x564b1a4fa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56672/0 (socket says 172.21.15.69:56672) 2024-05-14T17:02:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a3226d640 1 -- 172.21.15.69:0/3337348763 learned_addr learned my addr 172.21.15.69:0/3337348763 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:14.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a31a6c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 0x564b1a4fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/3337348763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4ef8f0 0x564b1a4efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a1f7fe640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4a180b2e00 0x7f4a180b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a31a6c640 1 -- 172.21.15.69:0/3337348763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4ef8f0 msgr2=0x564b1a4efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/3337348763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4ef8f0 0x564b1a4efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a31a6c640 1 -- 172.21.15.69:0/3337348763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4f0210 msgr2=0x564b1a4fa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/3337348763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4f0210 0x564b1a4fa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a31a6c640 1 -- 172.21.15.69:0/3337348763 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b1a167210 con 0x564b1a4faad0 2024-05-14T17:02:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.284+0000 7f4a3226d640 1 --2- 172.21.15.69:0/3337348763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4f0210 0x564b1a4fa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.283+0000 7f4a1f7fe640 1 -- 172.21.15.69:0/652974787 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f4a280314a0 con 0x7f4a2c10f770 2024-05-14T17:02:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.284+0000 7f4a32a6e640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4a180b2e00 0x7f4a180b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.284+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/3337348763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 0x564b1a4fcec0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c102d10 tx=0x7f4a24009a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8448586d425783a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.285+0000 7f4a1cbfe640 1 -- 172.21.15.69:0/3337348763 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24011d30 con 0x564b1a4faad0 2024-05-14T17:02:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.285+0000 7f4a32a6e640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4a180b2e00 0x7f4a180b52c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4a280c6880 tx=0x7f4a2802a480 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.285+0000 7f4a1cbfe640 1 -- 172.21.15.69:0/3337348763 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a2401f870 con 0x564b1a4faad0 2024-05-14T17:02:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.285+0000 7f4a1cbfe640 1 -- 172.21.15.69:0/3337348763 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24011d30 con 0x564b1a4faad0 2024-05-14T17:02:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 -- 172.21.15.69:0/3337348763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 msgr2=0x564b1a4fcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:15.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 --2- 172.21.15.69:0/3337348763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 0x564b1a4fcec0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c102d10 tx=0x7f4a24009a30 comp rx=0 tx=0).stop 2024-05-14T17:02:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 -- 172.21.15.69:0/3337348763 shutdown_connections 2024-05-14T17:02:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 --2- 172.21.15.69:0/3337348763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 0x564b1a4fcec0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 --2- 172.21.15.69:0/3337348763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4f0210 0x564b1a4fa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 --2- 172.21.15.69:0/3337348763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4ef8f0 0x564b1a4efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 -- 172.21.15.69:0/3337348763 >> 172.21.15.69:0/3337348763 conn(0x564b1a4ebe00 msgr2=0x564b1a4ec1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 -- 172.21.15.69:0/3337348763 shutdown_connections 2024-05-14T17:02:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 -- 172.21.15.69:0/3337348763 wait complete. 2024-05-14T17:02:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 -- start start 2024-05-14T17:02:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 Processor -- start 2024-05-14T17:02:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.286+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 0x564b1a4fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a4f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 0x564b1a4f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a371ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564b1a138790 con 0x564b1a4faad0 2024-05-14T17:02:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a371ee740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b1a40e890 con 0x564b1a4f0210 2024-05-14T17:02:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a371ee740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564b1a4fd7a0 con 0x564b1a4ef8f0 2024-05-14T17:02:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a371ee740 10 client.0 _ll_register_callbacks cb 0x564b1a411d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a31a6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 0x564b1a4f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a31a6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 0x564b1a4f7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56676/0 (socket says 172.21.15.69:56676) 2024-05-14T17:02:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a31a6c640 1 -- 172.21.15.69:0/593879566 learned_addr learned my addr 172.21.15.69:0/593879566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 0x564b1a4fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a3226d640 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a4f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a31a6c640 1 -- 172.21.15.69:0/593879566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 msgr2=0x564b1a4fa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 0x564b1a4fa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a31a6c640 1 -- 172.21.15.69:0/593879566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 msgr2=0x564b1a4f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a4f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a31a6c640 1 -- 172.21.15.69:0/593879566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b1a233620 con 0x564b1a4faad0 2024-05-14T17:02:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 0x564b1a4f7c00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4a24012010 tx=0x7f4a24018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8090983627b1f107 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:02:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a15ffb640 1 -- 172.21.15.69:0/593879566 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24011d30 con 0x564b1a4faad0 2024-05-14T17:02:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a15ffb640 1 -- 172.21.15.69:0/593879566 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f4a2401c070 con 0x564b1a4faad0 2024-05-14T17:02:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.287+0000 7f4a371ee740 1 -- 172.21.15.69:0/593879566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2401f870 con 0x564b1a4faad0 2024-05-14T17:02:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a15ffb640 1 -- 172.21.15.69:0/593879566 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f4a24004a40 con 0x564b1a4faad0 2024-05-14T17:02:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a371ee740 10 client.16671 fetch_fsmap learned FSMap version 51 2024-05-14T17:02:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a371ee740 1 -- 172.21.15.69:0/593879566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564b1a509200 con 0x564b1a4faad0 2024-05-14T17:02:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a15ffb640 1 -- 172.21.15.69:0/593879566 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a2401f870 con 0x564b1a4faad0 2024-05-14T17:02:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a15ffb640 1 -- 172.21.15.69:0/593879566 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f4a2402c020 con 0x564b1a4faad0 2024-05-14T17:02:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a371ee740 10 client.16671 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:02:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a371ee740 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a371ee740 20 client.16671 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a371ee740 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b1a50a5f0 0x564b1a50ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a371ee740 4 client.16671 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.288+0000 7f4a371ee740 1 -- 172.21.15.69:0/593879566 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b1a464b80 con 0x564b1a50a5f0 2024-05-14T17:02:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.289+0000 7f4a3226d640 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b1a50a5f0 0x564b1a50ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.289+0000 7f4a3226d640 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b1a50a5f0 0x564b1a50ca50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf64ec9f387fb07 server_cookie=94874745a8958e7c in_seq=0 out_seq=0 2024-05-14T17:02:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.289+0000 7f4a15ffb640 10 client.16671 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a15ffb640 1 -- 172.21.15.69:0/593879566 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b1a464b80 con 0x564b1a50a5f0 2024-05-14T17:02:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a15ffb640 10 client.16671 handle_command_reply: tid=0 2024-05-14T17:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 client.16671 shutdown 2024-05-14T17:02:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 -- 172.21.15.69:0/593879566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 msgr2=0x564b1a4f7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 0x564b1a4f7c00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4a24012010 tx=0x7f4a24018040 comp rx=0 tx=0).stop 2024-05-14T17:02:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 -- 172.21.15.69:0/593879566 shutdown_connections 2024-05-14T17:02:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b1a50a5f0 0x564b1a50ca50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 0x564b1a4f7c00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a4f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 --2- 172.21.15.69:0/593879566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 0x564b1a4fa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 -- 172.21.15.69:0/593879566 >> 172.21.15.69:0/593879566 conn(0x564b1a4ebe00 msgr2=0x564b1a4eb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a15ffb640 0 client.16671 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 -- 172.21.15.69:0/593879566 shutdown_connections 2024-05-14T17:02:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 1 -- 172.21.15.69:0/593879566 wait complete. 2024-05-14T17:02:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.290+0000 7f4a371ee740 20 client.16671 trim_cache size 0 max 16384 2024-05-14T17:02:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.311+0000 7f4a371ee740 1 Processor -- start 2024-05-14T17:02:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.311+0000 7f4a371ee740 1 -- start start 2024-05-14T17:02:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.311+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4ef8f0 0x564b1a4f6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.311+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a5066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.311+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 0x564b1a4f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.311+0000 7f4a371ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a100110a0 con 0x564b1a4ef8f0 2024-05-14T17:02:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.311+0000 7f4a371ee740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1000e8e0 con 0x564b1a4f0210 2024-05-14T17:02:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.311+0000 7f4a371ee740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1000c730 con 0x564b1a4faad0 2024-05-14T17:02:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.311+0000 7f4a31a6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4ef8f0 0x564b1a4f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a31a6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4ef8f0 0x564b1a4f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56692/0 (socket says 172.21.15.69:56692) 2024-05-14T17:02:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a31a6c640 1 -- 172.21.15.69:0/2015312227 learned_addr learned my addr 172.21.15.69:0/2015312227 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a3226d640 1 --2- 172.21.15.69:0/2015312227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a5066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/2015312227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 0x564b1a4f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a3226d640 1 -- 172.21.15.69:0/2015312227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 msgr2=0x564b1a4f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a3226d640 1 --2- 172.21.15.69:0/2015312227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 0x564b1a4f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a3226d640 1 -- 172.21.15.69:0/2015312227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4ef8f0 msgr2=0x564b1a4f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a3226d640 1 --2- 172.21.15.69:0/2015312227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4ef8f0 0x564b1a4f6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a3226d640 1 -- 172.21.15.69:0/2015312227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b1a4f0bb0 con 0x564b1a4f0210 2024-05-14T17:02:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/2015312227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4ef8f0 0x564b1a4f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:17.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a3226d640 1 --2- 172.21.15.69:0/2015312227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a5066d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x564b1a50d290 tx=0x7f4a200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a02bff88d0855b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a17fff640 1 -- 172.21.15.69:0/2015312227 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20025070 con 0x564b1a4f0210 2024-05-14T17:02:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a17fff640 1 -- 172.21.15.69:0/2015312227 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a20003d20 con 0x564b1a4f0210 2024-05-14T17:02:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.312+0000 7f4a17fff640 1 -- 172.21.15.69:0/2015312227 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20010720 con 0x564b1a4f0210 2024-05-14T17:02:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 -- 172.21.15.69:0/2015312227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 msgr2=0x564b1a5066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 --2- 172.21.15.69:0/2015312227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a5066d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x564b1a50d290 tx=0x7f4a200046c0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 -- 172.21.15.69:0/2015312227 shutdown_connections 2024-05-14T17:02:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 --2- 172.21.15.69:0/2015312227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4faad0 0x564b1a4f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 --2- 172.21.15.69:0/2015312227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a5066d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 --2- 172.21.15.69:0/2015312227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4ef8f0 0x564b1a4f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 -- 172.21.15.69:0/2015312227 >> 172.21.15.69:0/2015312227 conn(0x564b1a4ebe00 msgr2=0x564b1a4f2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 -- 172.21.15.69:0/2015312227 shutdown_connections 2024-05-14T17:02:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 -- 172.21.15.69:0/2015312227 wait complete. 2024-05-14T17:02:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 -- start start 2024-05-14T17:02:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.313+0000 7f4a371ee740 1 Processor -- start 2024-05-14T17:02:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 0x564b1a4fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a4fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a371ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 0x564b1a4fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a371ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1000cda0 con 0x564b1a4faad0 2024-05-14T17:02:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a371ee740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1000e210 con 0x564b1a4f0210 2024-05-14T17:02:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a371ee740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1000d470 con 0x564b1a4ef8f0 2024-05-14T17:02:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a371ee740 10 client.0 _ll_register_callbacks cb 0x7f4a2401fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a31a6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 0x564b1a4fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a31a6c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 0x564b1a4fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a3226d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a4fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a31a6c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 0x564b1a4fda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:54774/0 (socket says 172.21.15.69:54774) 2024-05-14T17:02:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a31a6c640 1 -- 172.21.15.69:0/2827565286 learned_addr learned my addr 172.21.15.69:0/2827565286 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a31a6c640 1 -- 172.21.15.69:0/2827565286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 msgr2=0x564b1a4fdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a4fdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a31a6c640 1 -- 172.21.15.69:0/2827565286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 msgr2=0x564b1a4fe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 0x564b1a4fe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a31a6c640 1 -- 172.21.15.69:0/2827565286 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b1a572fc0 con 0x564b1a4ef8f0 2024-05-14T17:02:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a3226d640 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a4fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a31a6c640 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 0x564b1a4fda10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a24012010 tx=0x7f4a24013dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14936edee45a8038 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T17:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a15ffb640 1 -- 172.21.15.69:0/2827565286 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24011d00 con 0x564b1a4ef8f0 2024-05-14T17:02:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a371ee740 1 -- 172.21.15.69:0/2827565286 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a20003d20 con 0x564b1a4ef8f0 2024-05-14T17:02:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.314+0000 7f4a15ffb640 1 -- 172.21.15.69:0/2827565286 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f4a24009040 con 0x564b1a4ef8f0 2024-05-14T17:02:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a15ffb640 1 -- 172.21.15.69:0/2827565286 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f4a24013020 con 0x564b1a4ef8f0 2024-05-14T17:02:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a371ee740 10 client.25999 fetch_fsmap learned FSMap version 51 2024-05-14T17:02:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a371ee740 1 -- 172.21.15.69:0/2827565286 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564b1a5742b0 con 0x564b1a4ef8f0 2024-05-14T17:02:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a15ffb640 1 -- 172.21.15.69:0/2827565286 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a20003d20 con 0x564b1a4ef8f0 2024-05-14T17:02:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a15ffb640 1 -- 172.21.15.69:0/2827565286 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f4a2402c040 con 0x564b1a4ef8f0 2024-05-14T17:02:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a371ee740 10 client.25999 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:02:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a371ee740 10 client.25999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a371ee740 20 client.25999 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a371ee740 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b1a574860 0x564b1a576c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a371ee740 4 client.25999 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:02:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a371ee740 1 -- 172.21.15.69:0/2827565286 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b1a45e360 con 0x564b1a574860 2024-05-14T17:02:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a3226d640 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b1a574860 0x564b1a576c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a15ffb640 10 client.25999 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.315+0000 7f4a3226d640 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b1a574860 0x564b1a576c80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcac4bd2349e77aa server_cookie=f8c1f0a326370b10 in_seq=0 out_seq=0 2024-05-14T17:02:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a15ffb640 1 -- 172.21.15.69:0/2827565286 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564b1a45e360 con 0x564b1a574860 2024-05-14T17:02:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a15ffb640 10 client.25999 handle_command_reply: tid=0 2024-05-14T17:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 client.25999 shutdown 2024-05-14T17:02:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 -- 172.21.15.69:0/2827565286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 msgr2=0x564b1a4fda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 0x564b1a4fda10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a24012010 tx=0x7f4a24013dc0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 -- 172.21.15.69:0/2827565286 shutdown_connections 2024-05-14T17:02:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564b1a574860 0x564b1a576c80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b1a4faad0 0x564b1a4fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a15ffb640 0 client.25999 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b1a4f0210 0x564b1a4fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 --2- 172.21.15.69:0/2827565286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b1a4ef8f0 0x564b1a4fda10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 -- 172.21.15.69:0/2827565286 >> 172.21.15.69:0/2827565286 conn(0x564b1a4ebe00 msgr2=0x564b1a4f2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 -- 172.21.15.69:0/2827565286 shutdown_connections 2024-05-14T17:02:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 1 -- 172.21.15.69:0/2827565286 wait complete. 2024-05-14T17:02:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a371ee740 20 client.25999 trim_cache size 0 max 16384 2024-05-14T17:02:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:02:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:02:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:02:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a344d1640 1 -- 172.21.15.69:0/652974787 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4a180b2e00 msgr2=0x7f4a180b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a344d1640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4a180b2e00 0x7f4a180b52c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4a280c6880 tx=0x7f4a2802a480 comp rx=0 tx=0).stop 2024-05-14T17:02:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.340+0000 7f4a344d1640 1 -- 172.21.15.69:0/652974787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a2c10f770 msgr2=0x7f4a2c19fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.341+0000 7f4a344d1640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a2c10f770 0x7f4a2c19fbc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009230 tx=0x7f4a28003ae0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.341+0000 7f4a344d1640 1 -- 172.21.15.69:0/652974787 shutdown_connections 2024-05-14T17:02:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.341+0000 7f4a344d1640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4a180b2e00 0x7f4a180b52c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.341+0000 7f4a344d1640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a2c10f770 0x7f4a2c19fbc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.341+0000 7f4a344d1640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a2c06b570 0x7f4a2c19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.341+0000 7f4a344d1640 1 --2- 172.21.15.69:0/652974787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a2c06ac50 0x7f4a2c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.341+0000 7f4a344d1640 1 -- 172.21.15.69:0/652974787 >> 172.21.15.69:0/652974787 conn(0x7f4a2c078830 msgr2=0x7f4a2c110ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.341+0000 7f4a344d1640 1 -- 172.21.15.69:0/652974787 shutdown_connections 2024-05-14T17:02:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:01:50.341+0000 7f4a344d1640 1 -- 172.21.15.69:0/652974787 wait complete. 2024-05-14T17:02:21.057 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 680.0 s, 1818.6 tps, lat 54.582 ms stddev 55.693 2024-05-14T17:02:21.116 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:50 smithi069 ceph-mon[31507]: pgmap v375: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.1 MiB/s wr, 542 op/s 2024-05-14T17:02:21.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:50 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:02:21.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:52 smithi069 ceph-mon[31507]: pgmap v376: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.1 MiB/s wr, 544 op/s 2024-05-14T17:02:21.224 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:54 smithi069 ceph-mon[31507]: pgmap v377: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.7 MiB/s wr, 807 op/s 2024-05-14T17:02:21.258 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 685.0 s, 2112.6 tps, lat 47.796 ms stddev 62.257 2024-05-14T17:02:21.291 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:56 smithi069 ceph-mon[31507]: pgmap v378: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.3 MiB/s wr, 531 op/s 2024-05-14T17:02:21.324 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:01:58 smithi069 ceph-mon[31507]: pgmap v379: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.9 MiB/s wr, 799 op/s 2024-05-14T17:02:21.358 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 690.0 s, 1866.4 tps, lat 52.579 ms stddev 54.358 2024-05-14T17:02:21.391 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:00 smithi069 ceph-mon[31507]: pgmap v380: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.3 MiB/s wr, 534 op/s 2024-05-14T17:02:21.424 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:02 smithi069 ceph-mon[31507]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-05-14T17:02:21.460 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:13 smithi092 ceph-mon[35763]: pgmap v389: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 1.95k op/s 2024-05-14T17:02:21.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:15 smithi092 ceph-mon[35763]: pgmap v390: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.66k op/s 2024-05-14T17:02:21.524 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:17 smithi092 ceph-mon[35763]: pgmap v391: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.84k op/s 2024-05-14T17:02:21.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:19 smithi092 ceph-mon[35763]: pgmap v392: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2024-05-14T17:02:21.591 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:19 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:02:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:19 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:02:21.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:19 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:02:21.699 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:20 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:02:21.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:13 smithi203 ceph-mon[35787]: pgmap v389: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 1.95k op/s 2024-05-14T17:02:21.766 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:15 smithi203 ceph-mon[35787]: pgmap v390: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.66k op/s 2024-05-14T17:02:21.800 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:17 smithi203 ceph-mon[35787]: pgmap v391: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.84k op/s 2024-05-14T17:02:21.833 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:19 smithi203 ceph-mon[35787]: pgmap v392: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2024-05-14T17:02:21.866 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:19 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:02:21.900 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:19 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:02:21.933 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:19 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:02:21.966 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:20 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:02:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.442+0000 7f1ab87af640 1 -- 172.21.15.69:0/2493976862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ab0102370 msgr2=0x7f1ab010e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.442+0000 7f1ab87af640 1 --2- 172.21.15.69:0/2493976862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ab0102370 0x7f1ab010e830 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009cd0 tx=0x7f1aa802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:02:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.442+0000 7f1ab87af640 1 -- 172.21.15.69:0/2493976862 shutdown_connections 2024-05-14T17:02:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.442+0000 7f1ab87af640 1 --2- 172.21.15.69:0/2493976862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ab0102370 0x7f1ab010e830 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.442+0000 7f1ab87af640 1 --2- 172.21.15.69:0/2493976862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ab01019b0 0x7f1ab0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.442+0000 7f1ab87af640 1 --2- 172.21.15.69:0/2493976862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab0107970 0x7f1ab0107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.442+0000 7f1ab87af640 1 -- 172.21.15.69:0/2493976862 >> 172.21.15.69:0/2493976862 conn(0x7f1ab0078aa0 msgr2=0x7f1ab0078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.442+0000 7f1ab87af640 1 -- 172.21.15.69:0/2493976862 shutdown_connections 2024-05-14T17:02:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.442+0000 7f1ab87af640 1 -- 172.21.15.69:0/2493976862 wait complete. 2024-05-14T17:02:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.443+0000 7f1ab87af640 1 Processor -- start 2024-05-14T17:02:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab87af640 1 -- start start 2024-05-14T17:02:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab87af640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ab01019b0 0x7f1ab01991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab87af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ab0102370 0x7f1ab0199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab87af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab0107970 0x7f1ab019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab87af640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0111c50 con 0x7f1ab01019b0 2024-05-14T17:02:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab87af640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0111ad0 con 0x7f1ab0107970 2024-05-14T17:02:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab87af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0111dd0 con 0x7f1ab0102370 2024-05-14T17:02:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab6d4c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab0107970 0x7f1ab019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab6d4c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab0107970 0x7f1ab019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:45952/0 (socket says 172.21.15.69:45952) 2024-05-14T17:02:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab6d4c640 1 -- 172.21.15.69:0/1771321803 learned_addr learned my addr 172.21.15.69:0/1771321803 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab654b640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ab01019b0 0x7f1ab01991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ab0102370 0x7f1ab0199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab6d4c640 1 -- 172.21.15.69:0/1771321803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ab0102370 msgr2=0x7f1ab0199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab6d4c640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ab0102370 0x7f1ab0199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab6d4c640 1 -- 172.21.15.69:0/1771321803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ab01019b0 msgr2=0x7f1ab01991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab6d4c640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ab01019b0 0x7f1ab01991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.444+0000 7f1ab6d4c640 1 -- 172.21.15.69:0/1771321803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab019e1e0 con 0x7f1ab0107970 2024-05-14T17:02:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.445+0000 7f1ab6d4c640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab0107970 0x7f1ab019dab0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8005e30 tx=0x7f1aa8003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.445+0000 7f1aa77fe640 1 -- 172.21.15.69:0/1771321803 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8004040 con 0x7f1ab0107970 2024-05-14T17:02:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.445+0000 7f1ab87af640 1 -- 172.21.15.69:0/1771321803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab019e470 con 0x7f1ab0107970 2024-05-14T17:02:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.445+0000 7f1aa77fe640 1 -- 172.21.15.69:0/1771321803 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa8004690 con 0x7f1ab0107970 2024-05-14T17:02:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.445+0000 7f1aa77fe640 1 -- 172.21.15.69:0/1771321803 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8030e80 con 0x7f1ab0107970 2024-05-14T17:02:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.445+0000 7f1ab87af640 1 -- 172.21.15.69:0/1771321803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab01a5d60 con 0x7f1ab0107970 2024-05-14T17:02:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.448+0000 7f1aa77fe640 1 -- 172.21.15.69:0/1771321803 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1aa80305a0 con 0x7f1ab0107970 2024-05-14T17:02:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.448+0000 7f1abb5a6740 1 Processor -- start 2024-05-14T17:02:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1abb5a6740 1 -- start start 2024-05-14T17:02:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1aa77fe640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a9c0b2a10 0x7f1a9c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1aa77fe640 1 -- 172.21.15.69:0/1771321803 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f1aa802e030 con 0x7f1ab0107970 2024-05-14T17:02:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a9c0b2a10 0x7f1a9c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5628f3337c30 con 0x5628f33d3ee0 2024-05-14T17:02:23.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5628f33277d0 con 0x5628f33c50f0 2024-05-14T17:02:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5628f332b810 con 0x5628f33c47d0 2024-05-14T17:02:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1ab5d4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1ab6d4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1ab5d4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:45954/0 (socket says 172.21.15.69:45954) 2024-05-14T17:02:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1ab6d4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48318/0 (socket says 172.21.15.69:48318) 2024-05-14T17:02:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/3875067024 learned_addr learned my addr 172.21.15.69:0/3875067024 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.449+0000 7f1ab654b640 1 --2- 172.21.15.69:0/3875067024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1ab654b640 1 -- 172.21.15.69:0/3875067024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 msgr2=0x5628f33d39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1ab654b640 1 --2- 172.21.15.69:0/3875067024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1ab654b640 1 -- 172.21.15.69:0/3875067024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 msgr2=0x5628f33d62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1ab654b640 1 --2- 172.21.15.69:0/3875067024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1ab654b640 1 -- 172.21.15.69:0/3875067024 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628f303c210 con 0x5628f33c47d0 2024-05-14T17:02:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1ab6d4c640 1 --2- 172.21.15.69:0/3875067024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/3875067024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a9c0b2a10 0x7f1a9c0b4ed0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0013570 tx=0x7f1aa0013970 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:24.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1ab654b640 1 --2- 172.21.15.69:0/3875067024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33c4bb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009c20 tx=0x7f1aac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d421629a6a6c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1aa4ff9640 1 -- 172.21.15.69:0/3875067024 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac004050 con 0x5628f33c47d0 2024-05-14T17:02:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1aa4ff9640 1 -- 172.21.15.69:0/3875067024 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aac0041f0 con 0x5628f33c47d0 2024-05-14T17:02:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.450+0000 7f1aa4ff9640 1 -- 172.21.15.69:0/3875067024 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac019680 con 0x5628f33c47d0 2024-05-14T17:02:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.451+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3875067024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 msgr2=0x5628f33c4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.451+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/3875067024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33c4bb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009c20 tx=0x7f1aac011810 comp rx=0 tx=0).stop 2024-05-14T17:02:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.451+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3875067024 shutdown_connections 2024-05-14T17:02:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.451+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/3875067024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.451+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/3875067024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.451+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/3875067024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33c4bb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.451+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3875067024 >> 172.21.15.69:0/3875067024 conn(0x5628f33c0e00 msgr2=0x5628f33c11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.452+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3875067024 shutdown_connections 2024-05-14T17:02:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.452+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3875067024 wait complete. 2024-05-14T17:02:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.452+0000 7f1abb5a6740 1 -- start start 2024-05-14T17:02:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.452+0000 7f1abb5a6740 1 Processor -- start 2024-05-14T17:02:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.452+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.452+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.452+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5628f300d790 con 0x5628f33d3ee0 2024-05-14T17:02:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5628f32e3890 con 0x5628f33c50f0 2024-05-14T17:02:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5628f33d6bb0 con 0x5628f33c47d0 2024-05-14T17:02:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab6d4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:25.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab654b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab6d4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48326/0 (socket says 172.21.15.69:48326) 2024-05-14T17:02:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab5d4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab6d4c640 1 -- 172.21.15.69:0/278323953 learned_addr learned my addr 172.21.15.69:0/278323953 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1abb5a6740 10 client.0 _ll_register_callbacks cb 0x5628f32e6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/278323953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 msgr2=0x5628f33d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33d3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/278323953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 msgr2=0x5628f33d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/278323953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628f3108620 con 0x5628f33c50f0 2024-05-14T17:02:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab654b640 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d0b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0005e00 tx=0x7f1aa0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2660d48f24c34900 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1a99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:02:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1a99ffb640 1 -- 172.21.15.69:0/278323953 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0003d40 con 0x5628f33c50f0 2024-05-14T17:02:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.453+0000 7f1a99ffb640 1 -- 172.21.15.69:0/278323953 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f1aa0005020 con 0x5628f33c50f0 2024-05-14T17:02:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1abb5a6740 1 -- 172.21.15.69:0/278323953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aac0041f0 con 0x5628f33c50f0 2024-05-14T17:02:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1a99ffb640 1 -- 172.21.15.69:0/278323953 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1aa0005220 con 0x5628f33c50f0 2024-05-14T17:02:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1abb5a6740 10 client.26117 fetch_fsmap learned FSMap version 51 2024-05-14T17:02:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1abb5a6740 1 -- 172.21.15.69:0/278323953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5628f33e2680 con 0x5628f33c50f0 2024-05-14T17:02:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1a99ffb640 1 -- 172.21.15.69:0/278323953 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aac0041f0 con 0x5628f33c50f0 2024-05-14T17:02:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1a99ffb640 1 -- 172.21.15.69:0/278323953 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f1aa001f070 con 0x5628f33c50f0 2024-05-14T17:02:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1abb5a6740 10 client.26117 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:02:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1abb5a6740 10 client.26117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1abb5a6740 20 client.26117 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628f33e3b90 0x5628f33e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1abb5a6740 4 client.26117 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1abb5a6740 1 -- 172.21.15.69:0/278323953 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628f3339b80 con 0x5628f33e3b90 2024-05-14T17:02:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.454+0000 7f1ab654b640 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628f33e3b90 0x5628f33e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.455+0000 7f1a99ffb640 10 client.26117 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.455+0000 7f1ab654b640 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628f33e3b90 0x5628f33e5f80 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d092645933adf328 server_cookie=2e65ab8b702cabb7 in_seq=0 out_seq=0 2024-05-14T17:02:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1a99ffb640 1 -- 172.21.15.69:0/278323953 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628f3339b80 con 0x5628f33e3b90 2024-05-14T17:02:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1a99ffb640 10 client.26117 handle_command_reply: tid=0 2024-05-14T17:02:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1abb5a6740 1 client.26117 shutdown 2024-05-14T17:02:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1abb5a6740 1 -- 172.21.15.69:0/278323953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 msgr2=0x5628f33d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d0b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0005e00 tx=0x7f1aa0004730 comp rx=0 tx=0).stop 2024-05-14T17:02:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1abb5a6740 1 -- 172.21.15.69:0/278323953 shutdown_connections 2024-05-14T17:02:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628f33e3b90 0x5628f33e5f80 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33d3ee0 0x5628f33d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c50f0 0x5628f33d0b50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/278323953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1abb5a6740 1 -- 172.21.15.69:0/278323953 >> 172.21.15.69:0/278323953 conn(0x5628f33c0e00 msgr2=0x5628f33c6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.456+0000 7f1a99ffb640 0 client.26117 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.457+0000 7f1abb5a6740 1 -- 172.21.15.69:0/278323953 shutdown_connections 2024-05-14T17:02:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.457+0000 7f1abb5a6740 1 -- 172.21.15.69:0/278323953 wait complete. 2024-05-14T17:02:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.457+0000 7f1abb5a6740 20 client.26117 trim_cache size 0 max 16384 2024-05-14T17:02:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.473+0000 7f1abb5a6740 1 Processor -- start 2024-05-14T17:02:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.473+0000 7f1abb5a6740 1 -- start start 2024-05-14T17:02:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.473+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33dfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.473+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.473+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33d3ee0 0x5628f33eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.473+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01aece0 con 0x5628f33c50f0 2024-05-14T17:02:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.473+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01ac520 con 0x5628f33d3ee0 2024-05-14T17:02:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.473+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01aa370 con 0x5628f33c47d0 2024-05-14T17:02:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab5d4a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab5d4a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48332/0 (socket says 172.21.15.69:48332) 2024-05-14T17:02:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/4111023125 learned_addr learned my addr 172.21.15.69:0/4111023125 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab654b640 1 --2- 172.21.15.69:0/4111023125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33dfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab6d4c640 1 --2- 172.21.15.69:0/4111023125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33d3ee0 0x5628f33eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/4111023125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 msgr2=0x5628f33dfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/4111023125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33dfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/4111023125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33d3ee0 msgr2=0x5628f33eb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/4111023125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33d3ee0 0x5628f33eb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/4111023125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628f33c0c30 con 0x5628f33c50f0 2024-05-14T17:02:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/4111023125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33d3390 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0004f70 tx=0x7f1aa0028450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a69b1e738994d5a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1a9bfff640 1 -- 172.21.15.69:0/4111023125 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0035020 con 0x5628f33c50f0 2024-05-14T17:02:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1a9bfff640 1 -- 172.21.15.69:0/4111023125 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa0005000 con 0x5628f33c50f0 2024-05-14T17:02:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.474+0000 7f1a9bfff640 1 -- 172.21.15.69:0/4111023125 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0004a00 con 0x5628f33c50f0 2024-05-14T17:02:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 -- 172.21.15.69:0/4111023125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 msgr2=0x5628f33d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/4111023125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33d3390 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0004f70 tx=0x7f1aa0028450 comp rx=0 tx=0).stop 2024-05-14T17:02:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 -- 172.21.15.69:0/4111023125 shutdown_connections 2024-05-14T17:02:27.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/4111023125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33d3ee0 0x5628f33eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/4111023125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33d3390 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/4111023125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33c47d0 0x5628f33dfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 -- 172.21.15.69:0/4111023125 >> 172.21.15.69:0/4111023125 conn(0x5628f33c0e00 msgr2=0x5628f33cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 -- 172.21.15.69:0/4111023125 shutdown_connections 2024-05-14T17:02:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 -- 172.21.15.69:0/4111023125 wait complete. 2024-05-14T17:02:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 -- start start 2024-05-14T17:02:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.475+0000 7f1abb5a6740 1 Processor -- start 2024-05-14T17:02:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c47d0 0x5628f33dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1abb5a6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33d3ee0 0x5628f33cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01aa9e0 con 0x5628f33c50f0 2024-05-14T17:02:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01abe50 con 0x5628f33c47d0 2024-05-14T17:02:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1abb5a6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01ab0b0 con 0x5628f33d3ee0 2024-05-14T17:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1abb5a6740 10 client.0 _ll_register_callbacks cb 0x7f1aa0004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab5d4a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab5d4a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48336/0 (socket says 172.21.15.69:48336) 2024-05-14T17:02:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/3495287554 learned_addr learned my addr 172.21.15.69:0/3495287554 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab654b640 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c47d0 0x5628f33dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab6d4c640 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33d3ee0 0x5628f33cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/3495287554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33d3ee0 msgr2=0x5628f33cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33d3ee0 0x5628f33cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/3495287554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c47d0 msgr2=0x5628f33dc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c47d0 0x5628f33dc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab5d4a640 1 -- 172.21.15.69:0/3495287554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628f33cde40 con 0x5628f33c50f0 2024-05-14T17:02:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1ab5d4a640 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33cd3c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009b70 tx=0x7f1aa00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2566816443d83c91 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1a99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:02:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1a99ffb640 1 -- 172.21.15.69:0/3495287554 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00045d0 con 0x5628f33c50f0 2024-05-14T17:02:28.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1a99ffb640 1 -- 172.21.15.69:0/3495287554 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8498+0+0 (secure 0 0 0) 0x7f1aa0004770 con 0x5628f33c50f0 2024-05-14T17:02:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3495287554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa0005000 con 0x5628f33c50f0 2024-05-14T17:02:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1a99ffb640 1 -- 172.21.15.69:0/3495287554 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1aa0049050 con 0x5628f33c50f0 2024-05-14T17:02:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1abb5a6740 10 client.16695 fetch_fsmap learned FSMap version 51 2024-05-14T17:02:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.476+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3495287554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5628f344e090 con 0x5628f33c50f0 2024-05-14T17:02:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1a99ffb640 1 -- 172.21.15.69:0/3495287554 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa0005000 con 0x5628f33c50f0 2024-05-14T17:02:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1a99ffb640 1 -- 172.21.15.69:0/3495287554 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f1aa00288c0 con 0x5628f33c50f0 2024-05-14T17:02:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1abb5a6740 10 client.16695 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:02:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1abb5a6740 10 client.16695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1abb5a6740 20 client.16695 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628f344e770 0x5628f3450b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1abb5a6740 4 client.16695 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:02:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3495287554 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628f3333360 con 0x5628f344e770 2024-05-14T17:02:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1ab654b640 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628f344e770 0x5628f3450b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1a99ffb640 10 client.16695 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.477+0000 7f1ab654b640 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628f344e770 0x5628f3450b90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1da624f696f17477 server_cookie=d7ef24bdcaf146e8 in_seq=0 out_seq=0 2024-05-14T17:02:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1a99ffb640 1 -- 172.21.15.69:0/3495287554 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5628f3333360 con 0x5628f344e770 2024-05-14T17:02:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1a99ffb640 10 client.16695 handle_command_reply: tid=0 2024-05-14T17:02:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 client.16695 shutdown 2024-05-14T17:02:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3495287554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 msgr2=0x5628f33cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33cd3c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009b70 tx=0x7f1aa00096c0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3495287554 shutdown_connections 2024-05-14T17:02:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628f344e770 0x5628f3450b90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f33d3ee0 0x5628f33cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5628f33c50f0 0x5628f33cd3c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1a99ffb640 0 client.16695 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 --2- 172.21.15.69:0/3495287554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628f33c47d0 0x5628f33dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3495287554 >> 172.21.15.69:0/3495287554 conn(0x5628f33c0e00 msgr2=0x5628f33bf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3495287554 shutdown_connections 2024-05-14T17:02:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.634+0000 7f1abb5a6740 1 -- 172.21.15.69:0/3495287554 wait complete. 2024-05-14T17:02:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1abb5a6740 20 client.16695 trim_cache size 0 max 16384 2024-05-14T17:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:02:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 -- 172.21.15.69:0/1771321803 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a9c0b2a10 msgr2=0x7f1a9c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a9c0b2a10 0x7f1a9c0b4ed0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0013570 tx=0x7f1aa0013970 comp rx=0 tx=0).stop 2024-05-14T17:02:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 -- 172.21.15.69:0/1771321803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab0107970 msgr2=0x7f1ab019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab0107970 0x7f1ab019dab0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8005e30 tx=0x7f1aa8003970 comp rx=0 tx=0).stop 2024-05-14T17:02:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 -- 172.21.15.69:0/1771321803 shutdown_connections 2024-05-14T17:02:30.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1a9c0b2a10 0x7f1a9c0b4ed0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab0107970 0x7f1ab019dab0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ab0102370 0x7f1ab0199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 --2- 172.21.15.69:0/1771321803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ab01019b0 0x7f1ab01991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 -- 172.21.15.69:0/1771321803 >> 172.21.15.69:0/1771321803 conn(0x7f1ab0078aa0 msgr2=0x7f1ab006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 -- 172.21.15.69:0/1771321803 shutdown_connections 2024-05-14T17:02:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:03.635+0000 7f1ab87af640 1 -- 172.21.15.69:0/1771321803 wait complete. 2024-05-14T17:02:30.424 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:04 smithi069 ceph-mon[31507]: pgmap v382: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.4 MiB/s wr, 827 op/s 2024-05-14T17:02:30.457 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 695.0 s, 178.4 tps, lat 523.886 ms stddev 330.871 2024-05-14T17:02:30.491 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:05 smithi069 ceph-mon[31507]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T17:02:30.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:06 smithi069 ceph-mon[31507]: pgmap v384: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 675 op/s 2024-05-14T17:02:30.557 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:08 smithi069 ceph-mon[31507]: pgmap v385: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-05-14T17:02:30.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:09 smithi069 ceph-mon[31507]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T17:02:30.648 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:09 smithi069 ceph-mon[31507]: pgmap v387: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.8 MiB/s wr, 644 op/s 2024-05-14T17:02:30.682 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 700.0 s, 875.6 tps, lat 123.975 ms stddev 251.469 2024-05-14T17:02:30.720 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:11 smithi069 ceph-mon[31507]: pgmap v388: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-05-14T17:02:30.749 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:13 smithi069 ceph-mon[31507]: pgmap v389: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 1.95k op/s 2024-05-14T17:02:30.783 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 705.0 s, 2548.2 tps, lat 38.818 ms stddev 43.524 2024-05-14T17:02:30.816 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:21 smithi092 ceph-mon[35763]: pgmap v393: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 1.54k op/s 2024-05-14T17:02:30.849 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:23 smithi092 ceph-mon[35763]: pgmap v394: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 33 MiB/s wr, 1.80k op/s 2024-05-14T17:02:30.882 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:25 smithi092 ceph-mon[35763]: pgmap v395: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.6 MiB/s wr, 565 op/s 2024-05-14T17:02:30.915 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:27 smithi092 ceph-mon[35763]: pgmap v396: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.3 MiB/s wr, 842 op/s 2024-05-14T17:02:30.949 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:29 smithi092 ceph-mon[35763]: pgmap v397: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.4 MiB/s wr, 553 op/s 2024-05-14T17:02:31.008 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:21 smithi203 ceph-mon[35787]: pgmap v393: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 1.54k op/s 2024-05-14T17:02:31.050 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:23 smithi203 ceph-mon[35787]: pgmap v394: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 33 MiB/s wr, 1.80k op/s 2024-05-14T17:02:31.082 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:25 smithi203 ceph-mon[35787]: pgmap v395: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.6 MiB/s wr, 565 op/s 2024-05-14T17:02:31.115 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:27 smithi203 ceph-mon[35787]: pgmap v396: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.3 MiB/s wr, 842 op/s 2024-05-14T17:02:31.149 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:29 smithi203 ceph-mon[35787]: pgmap v397: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.4 MiB/s wr, 553 op/s 2024-05-14T17:02:31.182 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:15 smithi069 ceph-mon[31507]: pgmap v390: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.66k op/s 2024-05-14T17:02:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.770+0000 7f1dab623640 1 -- 172.21.15.69:0/876111042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da41045e0 msgr2=0x7f1da410cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.770+0000 7f1dab623640 1 --2- 172.21.15.69:0/876111042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da41045e0 0x7f1da410cac0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009900 tx=0x7f1d9402a480 comp rx=0 tx=0).stop 2024-05-14T17:02:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.770+0000 7f1dab623640 1 -- 172.21.15.69:0/876111042 shutdown_connections 2024-05-14T17:02:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.770+0000 7f1dab623640 1 --2- 172.21.15.69:0/876111042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da410d090 0x7f1da410f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.770+0000 7f1dab623640 1 --2- 172.21.15.69:0/876111042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da41045e0 0x7f1da410cac0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.770+0000 7f1dab623640 1 --2- 172.21.15.69:0/876111042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da4103cc0 0x7f1da41040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.770+0000 7f1dab623640 1 -- 172.21.15.69:0/876111042 >> 172.21.15.69:0/876111042 conn(0x7f1da4078b00 msgr2=0x7f1da4078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.770+0000 7f1dab623640 1 -- 172.21.15.69:0/876111042 shutdown_connections 2024-05-14T17:02:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.770+0000 7f1dab623640 1 -- 172.21.15.69:0/876111042 wait complete. 2024-05-14T17:02:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1dab623640 1 Processor -- start 2024-05-14T17:02:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1dab623640 1 -- start start 2024-05-14T17:02:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1dab623640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da4103cc0 0x7f1da41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1dab623640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da41045e0 0x7f1da41996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1dab623640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da410d090 0x7f1da419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1dab623640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4112ce0 con 0x7f1da4103cc0 2024-05-14T17:02:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1dab623640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4112b60 con 0x7f1da41045e0 2024-05-14T17:02:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1dab623640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4112e60 con 0x7f1da410d090 2024-05-14T17:02:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1da93bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da4103cc0 0x7f1da41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1da8bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da41045e0 0x7f1da41996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1da93bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da4103cc0 0x7f1da41991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56204/0 (socket says 172.21.15.69:56204) 2024-05-14T17:02:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1da8bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da41045e0 0x7f1da41996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:57020/0 (socket says 172.21.15.69:57020) 2024-05-14T17:02:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1da93bf640 1 -- 172.21.15.69:0/3560153463 learned_addr learned my addr 172.21.15.69:0/3560153463 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.771+0000 7f1da9bc0640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da410d090 0x7f1da419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1da93bf640 1 -- 172.21.15.69:0/3560153463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da410d090 msgr2=0x7f1da419da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1da93bf640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da410d090 0x7f1da419da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1da93bf640 1 -- 172.21.15.69:0/3560153463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da41045e0 msgr2=0x7f1da41996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1da93bf640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da41045e0 0x7f1da41996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1da93bf640 1 -- 172.21.15.69:0/3560153463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da419e190 con 0x7f1da4103cc0 2024-05-14T17:02:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1da8bbe640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da41045e0 0x7f1da41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1da93bf640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da4103cc0 0x7f1da41991a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1da000e9f0 tx=0x7f1da000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1d9a7fc640 1 -- 172.21.15.69:0/3560153463 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da000cd80 con 0x7f1da4103cc0 2024-05-14T17:02:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1d9a7fc640 1 -- 172.21.15.69:0/3560153463 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da00054d0 con 0x7f1da4103cc0 2024-05-14T17:02:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1d9a7fc640 1 -- 172.21.15.69:0/3560153463 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0010430 con 0x7f1da4103cc0 2024-05-14T17:02:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1dab623640 1 -- 172.21.15.69:0/3560153463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da419e480 con 0x7f1da4103cc0 2024-05-14T17:02:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.772+0000 7f1dab623640 1 -- 172.21.15.69:0/3560153463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da41a5d10 con 0x7f1da4103cc0 2024-05-14T17:02:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.776+0000 7f1dae41d740 1 Processor -- start 2024-05-14T17:02:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.776+0000 7f1d9a7fc640 1 -- 172.21.15.69:0/3560153463 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1da0005020 con 0x7f1da4103cc0 2024-05-14T17:02:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.776+0000 7f1dae41d740 1 -- start start 2024-05-14T17:02:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.776+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cbe30 0x55b3f86ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.776+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.776+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86e0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.776+0000 7f1dae41d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b3f863dc30 con 0x55b3f86cbe30 2024-05-14T17:02:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.776+0000 7f1dae41d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b3f862d7d0 con 0x55b3f86cacf0 2024-05-14T17:02:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.776+0000 7f1dae41d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b3f8631810 con 0x55b3f86de170 2024-05-14T17:02:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da9bc0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cbe30 0x55b3f86ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da93bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da93bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:57034/0 (socket says 172.21.15.69:57034) 2024-05-14T17:02:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da93bf640 1 -- 172.21.15.69:0/1261170531 learned_addr learned my addr 172.21.15.69:0/1261170531 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da8bbe640 1 --2- 172.21.15.69:0/1261170531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da93bf640 1 -- 172.21.15.69:0/1261170531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 msgr2=0x55b3f86e0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da93bf640 1 --2- 172.21.15.69:0/1261170531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86e0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da93bf640 1 -- 172.21.15.69:0/1261170531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cbe30 msgr2=0x55b3f86ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da93bf640 1 --2- 172.21.15.69:0/1261170531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cbe30 0x55b3f86ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da93bf640 1 -- 172.21.15.69:0/1261170531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f8342210 con 0x55b3f86cacf0 2024-05-14T17:02:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da93bf640 1 --2- 172.21.15.69:0/1261170531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86cb170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1da00e0090 tx=0x7f1da00e6ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95ac986295e1dfb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1d9a7fc640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d900b2a10 0x7f1d900b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1d8f7fe640 1 -- 172.21.15.69:0/1261170531 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da00f6070 con 0x55b3f86cacf0 2024-05-14T17:02:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1da9bc0640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d900b2a10 0x7f1d900b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1d8f7fe640 1 -- 172.21.15.69:0/1261170531 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da00a97c0 con 0x55b3f86cacf0 2024-05-14T17:02:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.777+0000 7f1d8f7fe640 1 -- 172.21.15.69:0/1261170531 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da009d860 con 0x55b3f86cacf0 2024-05-14T17:02:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1d9a7fc640 1 -- 172.21.15.69:0/3560153463 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f1da00e29e0 con 0x7f1da4103cc0 2024-05-14T17:02:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1dae41d740 1 -- 172.21.15.69:0/1261170531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 msgr2=0x55b3f86cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1da9bc0640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d900b2a10 0x7f1d900b4ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c005fd0 tx=0x7f1d9c009500 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1dae41d740 1 --2- 172.21.15.69:0/1261170531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86cb170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1da00e0090 tx=0x7f1da00e6ab0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1dae41d740 1 -- 172.21.15.69:0/1261170531 shutdown_connections 2024-05-14T17:02:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1dae41d740 1 --2- 172.21.15.69:0/1261170531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86e0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1dae41d740 1 --2- 172.21.15.69:0/1261170531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86cb170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1dae41d740 1 --2- 172.21.15.69:0/1261170531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cbe30 0x55b3f86ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1dae41d740 1 -- 172.21.15.69:0/1261170531 >> 172.21.15.69:0/1261170531 conn(0x55b3f86c6e00 msgr2=0x55b3f86c71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1dae41d740 1 -- 172.21.15.69:0/1261170531 shutdown_connections 2024-05-14T17:02:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1dae41d740 1 -- 172.21.15.69:0/1261170531 wait complete. 2024-05-14T17:02:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.778+0000 7f1dae41d740 1 -- start start 2024-05-14T17:02:33.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1dae41d740 1 Processor -- start 2024-05-14T17:02:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 0x55b3f86ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1dae41d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b3f8313790 con 0x55b3f86cacf0 2024-05-14T17:02:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1dae41d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b3f85e9890 con 0x55b3f86cbe30 2024-05-14T17:02:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1dae41d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b3f86e0f70 con 0x55b3f86de170 2024-05-14T17:02:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da93bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1dae41d740 10 client.0 _ll_register_callbacks cb 0x55b3f85ecd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da93bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86d0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56222/0 (socket says 172.21.15.69:56222) 2024-05-14T17:02:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da93bf640 1 -- 172.21.15.69:0/3952790753 learned_addr learned my addr 172.21.15.69:0/3952790753 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da9bc0640 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da8bbe640 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 0x55b3f86ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da93bf640 1 -- 172.21.15.69:0/3952790753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 msgr2=0x55b3f86ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da93bf640 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da93bf640 1 -- 172.21.15.69:0/3952790753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 msgr2=0x55b3f86ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da93bf640 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 0x55b3f86ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da93bf640 1 -- 172.21.15.69:0/3952790753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f840e620 con 0x55b3f86cacf0 2024-05-14T17:02:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1da93bf640 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86d0ef0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1da00a6f20 tx=0x7f1da00a9a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d3ea7d6c41bfd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.779+0000 7f1d8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:02:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.780+0000 7f1d8cff9640 1 -- 172.21.15.69:0/3952790753 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da009f1c0 con 0x55b3f86cacf0 2024-05-14T17:02:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.780+0000 7f1d8cff9640 1 -- 172.21.15.69:0/3952790753 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f1da00a62a0 con 0x55b3f86cacf0 2024-05-14T17:02:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.780+0000 7f1dae41d740 1 -- 172.21.15.69:0/3952790753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da00a97c0 con 0x55b3f86cacf0 2024-05-14T17:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.780+0000 7f1d8cff9640 1 -- 172.21.15.69:0/3952790753 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1da001f340 con 0x55b3f86cacf0 2024-05-14T17:02:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.780+0000 7f1dae41d740 10 client.16707 fetch_fsmap learned FSMap version 51 2024-05-14T17:02:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.780+0000 7f1dae41d740 1 -- 172.21.15.69:0/3952790753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b3f86eca30 con 0x55b3f86cacf0 2024-05-14T17:02:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.780+0000 7f1d8cff9640 1 -- 172.21.15.69:0/3952790753 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da00a97c0 con 0x55b3f86cacf0 2024-05-14T17:02:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.780+0000 7f1d8cff9640 1 -- 172.21.15.69:0/3952790753 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f1da00a5f40 con 0x55b3f86cacf0 2024-05-14T17:02:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.781+0000 7f1dae41d740 10 client.16707 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:02:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.781+0000 7f1dae41d740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.781+0000 7f1dae41d740 20 client.16707 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.781+0000 7f1dae41d740 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b3f86edd80 0x55b3f86f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.781+0000 7f1dae41d740 4 client.16707 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.781+0000 7f1dae41d740 1 -- 172.21.15.69:0/3952790753 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3f863fb80 con 0x55b3f86edd80 2024-05-14T17:02:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.781+0000 7f1da8bbe640 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b3f86edd80 0x55b3f86f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1da8bbe640 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b3f86edd80 0x55b3f86f0170 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9365109152b3a14b server_cookie=e315faa6e3a162b2 in_seq=0 out_seq=0 2024-05-14T17:02:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1d8cff9640 10 client.16707 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1d8cff9640 1 -- 172.21.15.69:0/3952790753 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b3f863fb80 con 0x55b3f86edd80 2024-05-14T17:02:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1d8cff9640 10 client.16707 handle_command_reply: tid=0 2024-05-14T17:02:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1dae41d740 1 client.16707 shutdown 2024-05-14T17:02:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1dae41d740 1 -- 172.21.15.69:0/3952790753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 msgr2=0x55b3f86d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1dae41d740 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86d0ef0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1da00a6f20 tx=0x7f1da00a9a80 comp rx=0 tx=0).stop 2024-05-14T17:02:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1dae41d740 1 -- 172.21.15.69:0/3952790753 shutdown_connections 2024-05-14T17:02:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1dae41d740 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b3f86edd80 0x55b3f86f0170 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1dae41d740 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1dae41d740 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 0x55b3f86ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1dae41d740 1 --2- 172.21.15.69:0/3952790753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86d0ef0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1d8cff9640 0 client.16707 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.782+0000 7f1dae41d740 1 -- 172.21.15.69:0/3952790753 >> 172.21.15.69:0/3952790753 conn(0x55b3f86c6e00 msgr2=0x55b3f86ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.783+0000 7f1dae41d740 1 -- 172.21.15.69:0/3952790753 shutdown_connections 2024-05-14T17:02:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.783+0000 7f1dae41d740 1 -- 172.21.15.69:0/3952790753 wait complete. 2024-05-14T17:02:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.783+0000 7f1dae41d740 20 client.16707 trim_cache size 0 max 16384 2024-05-14T17:02:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.800+0000 7f1dae41d740 1 Processor -- start 2024-05-14T17:02:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.800+0000 7f1dae41d740 1 -- start start 2024-05-14T17:02:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.800+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.800+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 0x55b3f86d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86da150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1dae41d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41aee20 con 0x55b3f86cacf0 2024-05-14T17:02:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1dae41d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41ac660 con 0x55b3f86cbe30 2024-05-14T17:02:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1dae41d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41aa4b0 con 0x55b3f86de170 2024-05-14T17:02:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da8bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 0x55b3f86d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da8bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 0x55b3f86d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:57050/0 (socket says 172.21.15.69:57050) 2024-05-14T17:02:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da8bbe640 1 -- 172.21.15.69:0/2031254185 learned_addr learned my addr 172.21.15.69:0/2031254185 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da93bf640 1 --2- 172.21.15.69:0/2031254185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da9bc0640 1 --2- 172.21.15.69:0/2031254185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86da150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da8bbe640 1 -- 172.21.15.69:0/2031254185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 msgr2=0x55b3f86da150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da8bbe640 1 --2- 172.21.15.69:0/2031254185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86da150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da8bbe640 1 -- 172.21.15.69:0/2031254185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 msgr2=0x55b3f86e9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da8bbe640 1 --2- 172.21.15.69:0/2031254185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86e9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da8bbe640 1 -- 172.21.15.69:0/2031254185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f86ceb40 con 0x55b3f86cbe30 2024-05-14T17:02:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da93bf640 1 --2- 172.21.15.69:0/2031254185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86e9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.801+0000 7f1da8bbe640 1 --2- 172.21.15.69:0/2031254185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 0x55b3f86d0a90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d94032fd0 tx=0x7f1d94005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d7d6ad6d42331b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1d8effd640 1 -- 172.21.15.69:0/2031254185 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94035040 con 0x55b3f86cbe30 2024-05-14T17:02:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1d8effd640 1 -- 172.21.15.69:0/2031254185 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9404b070 con 0x55b3f86cbe30 2024-05-14T17:02:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1d8effd640 1 -- 172.21.15.69:0/2031254185 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940049d0 con 0x55b3f86cbe30 2024-05-14T17:02:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1dae41d740 1 -- 172.21.15.69:0/2031254185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 msgr2=0x55b3f86d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1dae41d740 1 --2- 172.21.15.69:0/2031254185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 0x55b3f86d0a90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d94032fd0 tx=0x7f1d94005b50 comp rx=0 tx=0).stop 2024-05-14T17:02:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1dae41d740 1 -- 172.21.15.69:0/2031254185 shutdown_connections 2024-05-14T17:02:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1dae41d740 1 --2- 172.21.15.69:0/2031254185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86de170 0x55b3f86da150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1dae41d740 1 --2- 172.21.15.69:0/2031254185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cbe30 0x55b3f86d0a90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1dae41d740 1 --2- 172.21.15.69:0/2031254185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86cacf0 0x55b3f86e9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1dae41d740 1 -- 172.21.15.69:0/2031254185 >> 172.21.15.69:0/2031254185 conn(0x55b3f86c6e00 msgr2=0x55b3f86cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1dae41d740 1 -- 172.21.15.69:0/2031254185 shutdown_connections 2024-05-14T17:02:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.802+0000 7f1dae41d740 1 -- 172.21.15.69:0/2031254185 wait complete. 2024-05-14T17:02:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.803+0000 7f1dae41d740 1 -- start start 2024-05-14T17:02:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.803+0000 7f1dae41d740 1 Processor -- start 2024-05-14T17:02:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.803+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86e6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.803+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86cbe30 0x55b3f86e6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.803+0000 7f1dae41d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86de170 0x55b3f86db510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.803+0000 7f1dae41d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41aab20 con 0x55b3f86de170 2024-05-14T17:02:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.803+0000 7f1dae41d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41abf90 con 0x55b3f86cacf0 2024-05-14T17:02:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.803+0000 7f1dae41d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41ab1f0 con 0x55b3f86cbe30 2024-05-14T17:02:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.803+0000 7f1dae41d740 10 client.0 _ll_register_callbacks cb 0x7f1da0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.804+0000 7f1da93bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86e6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.804+0000 7f1da93bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86e6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:57064/0 (socket says 172.21.15.69:57064) 2024-05-14T17:02:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.804+0000 7f1da93bf640 1 -- 172.21.15.69:0/726837582 learned_addr learned my addr 172.21.15.69:0/726837582 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.804+0000 7f1da93bf640 1 -- 172.21.15.69:0/726837582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86cbe30 msgr2=0x55b3f86e6740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:02:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.804+0000 7f1da93bf640 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86cbe30 0x55b3f86e6740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.804+0000 7f1da93bf640 1 -- 172.21.15.69:0/726837582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86de170 msgr2=0x55b3f86db510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:02:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.804+0000 7f1da93bf640 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86de170 0x55b3f86db510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.804+0000 7f1da93bf640 1 -- 172.21.15.69:0/726837582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f86dbc80 con 0x55b3f86cacf0 2024-05-14T17:02:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.804+0000 7f1da93bf640 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86e6200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1da00e0720 tx=0x7f1da00095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb2501b1e8caf5f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.804+0000 7f1d8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:02:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1d8cff9640 1 -- 172.21.15.69:0/726837582 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da00a5440 con 0x55b3f86cacf0 2024-05-14T17:02:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1d8cff9640 1 -- 172.21.15.69:0/726837582 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f1da00a62a0 con 0x55b3f86cacf0 2024-05-14T17:02:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1dae41d740 1 -- 172.21.15.69:0/726837582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9404b070 con 0x55b3f86cacf0 2024-05-14T17:02:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1d8cff9640 1 -- 172.21.15.69:0/726837582 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1da00a9600 con 0x55b3f86cacf0 2024-05-14T17:02:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1dae41d740 10 client.26141 fetch_fsmap learned FSMap version 51 2024-05-14T17:02:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1dae41d740 1 -- 172.21.15.69:0/726837582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b3f86d2350 con 0x55b3f86cacf0 2024-05-14T17:02:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1d8cff9640 1 -- 172.21.15.69:0/726837582 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9404b070 con 0x55b3f86cacf0 2024-05-14T17:02:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1d8cff9640 1 -- 172.21.15.69:0/726837582 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f1da00b2680 con 0x55b3f86cacf0 2024-05-14T17:02:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1dae41d740 10 client.26141 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1dae41d740 10 client.26141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1dae41d740 20 client.26141 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1dae41d740 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b3f86d2830 0x55b3f86d4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1dae41d740 4 client.26141 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:02:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.805+0000 7f1dae41d740 1 -- 172.21.15.69:0/726837582 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3f8639360 con 0x55b3f86d2830 2024-05-14T17:02:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.806+0000 7f1da8bbe640 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b3f86d2830 0x55b3f86d4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.806+0000 7f1d8cff9640 10 client.26141 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.806+0000 7f1da8bbe640 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b3f86d2830 0x55b3f86d4ca0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510da16ae78bc3ab server_cookie=de9763da4fdf2179 in_seq=0 out_seq=0 2024-05-14T17:02:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.819+0000 7f1d8cff9640 1 -- 172.21.15.69:0/726837582 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b3f8639360 con 0x55b3f86d2830 2024-05-14T17:02:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.819+0000 7f1d8cff9640 10 client.26141 handle_command_reply: tid=0 2024-05-14T17:02:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.819+0000 7f1dae41d740 1 client.26141 shutdown 2024-05-14T17:02:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.819+0000 7f1dae41d740 1 -- 172.21.15.69:0/726837582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 msgr2=0x55b3f86e6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.819+0000 7f1dae41d740 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86e6200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1da00e0720 tx=0x7f1da00095b0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.819+0000 7f1dae41d740 1 -- 172.21.15.69:0/726837582 shutdown_connections 2024-05-14T17:02:39.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.819+0000 7f1dae41d740 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b3f86d2830 0x55b3f86d4ca0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.819+0000 7f1dae41d740 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f86de170 0x55b3f86db510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1d8cff9640 0 client.26141 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dae41d740 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b3f86cbe30 0x55b3f86e6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dae41d740 1 --2- 172.21.15.69:0/726837582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3f86cacf0 0x55b3f86e6200 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dae41d740 1 -- 172.21.15.69:0/726837582 >> 172.21.15.69:0/726837582 conn(0x55b3f86c6e00 msgr2=0x55b3f86dd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dae41d740 1 -- 172.21.15.69:0/726837582 shutdown_connections 2024-05-14T17:02:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dae41d740 1 -- 172.21.15.69:0/726837582 wait complete. 2024-05-14T17:02:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dae41d740 20 client.26141 trim_cache size 0 max 16384 2024-05-14T17:02:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:02:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:02:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:02:40.232 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-14T17:02:40.274 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.65 2024-05-14T17:02:40.306 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:02:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 -- 172.21.15.69:0/3560153463 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d900b2a10 msgr2=0x7f1d900b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d900b2a10 0x7f1d900b4ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c005fd0 tx=0x7f1d9c009500 comp rx=0 tx=0).stop 2024-05-14T17:02:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 -- 172.21.15.69:0/3560153463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da4103cc0 msgr2=0x7f1da41991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da4103cc0 0x7f1da41991a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1da000e9f0 tx=0x7f1da000edf0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 -- 172.21.15.69:0/3560153463 shutdown_connections 2024-05-14T17:02:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d900b2a10 0x7f1d900b4ed0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da410d090 0x7f1da419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da41045e0 0x7f1da41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 --2- 172.21.15.69:0/3560153463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da4103cc0 0x7f1da41991a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 -- 172.21.15.69:0/3560153463 >> 172.21.15.69:0/3560153463 conn(0x7f1da4078b00 msgr2=0x7f1da40fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 -- 172.21.15.69:0/3560153463 shutdown_connections 2024-05-14T17:02:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:16.820+0000 7f1dab623640 1 -- 172.21.15.69:0/3560153463 wait complete. 2024-05-14T17:02:40.815 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:31 smithi092 ceph-mon[35763]: pgmap v398: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-05-14T17:02:40.848 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:33 smithi092 ceph-mon[35763]: pgmap v399: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.5 MiB/s wr, 877 op/s 2024-05-14T17:02:40.881 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:35 smithi092 ceph-mon[35763]: pgmap v400: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.8 MiB/s wr, 603 op/s 2024-05-14T17:02:40.914 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:37 smithi092 ceph-mon[35763]: pgmap v401: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.7 MiB/s wr, 896 op/s 2024-05-14T17:02:40.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:39 smithi092 ceph-mon[35763]: pgmap v402: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.0 MiB/s wr, 619 op/s 2024-05-14T17:02:40.981 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:31 smithi203 ceph-mon[35787]: pgmap v398: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-05-14T17:02:41.048 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:33 smithi203 ceph-mon[35787]: pgmap v399: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.5 MiB/s wr, 877 op/s 2024-05-14T17:02:41.081 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:35 smithi203 ceph-mon[35787]: pgmap v400: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.8 MiB/s wr, 603 op/s 2024-05-14T17:02:41.115 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:37 smithi203 ceph-mon[35787]: pgmap v401: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.7 MiB/s wr, 896 op/s 2024-05-14T17:02:41.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:39 smithi203 ceph-mon[35787]: pgmap v402: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.0 MiB/s wr, 619 op/s 2024-05-14T17:02:41.182 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:17 smithi069 ceph-mon[31507]: pgmap v391: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.84k op/s 2024-05-14T17:02:41.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:19 smithi069 ceph-mon[31507]: pgmap v392: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2024-05-14T17:02:41.248 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:19 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:02:41.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:19 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:02:41.315 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:19 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:02:41.390 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 710.0 s, 2398.4 tps, lat 41.883 ms stddev 44.981 2024-05-14T17:02:41.423 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:20 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:02:41.464 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:21 smithi069 ceph-mon[31507]: pgmap v393: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 1.54k op/s 2024-05-14T17:02:41.490 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:23 smithi069 ceph-mon[31507]: pgmap v394: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 33 MiB/s wr, 1.80k op/s 2024-05-14T17:02:41.524 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 715.0 s, 2422.4 tps, lat 41.420 ms stddev 44.557 2024-05-14T17:02:41.557 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:25 smithi069 ceph-mon[31507]: pgmap v395: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.6 MiB/s wr, 565 op/s 2024-05-14T17:02:41.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:27 smithi069 ceph-mon[31507]: pgmap v396: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.3 MiB/s wr, 842 op/s 2024-05-14T17:02:41.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.958+0000 7f236f845640 1 -- 172.21.15.69:0/3517319891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368103410 msgr2=0x7f236810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.958+0000 7f236f845640 1 --2- 172.21.15.69:0/3517319891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368103410 0x7f236810f920 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2360009cd0 tx=0x7f236002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.958+0000 7f236f845640 1 -- 172.21.15.69:0/3517319891 shutdown_connections 2024-05-14T17:02:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.958+0000 7f236f845640 1 --2- 172.21.15.69:0/3517319891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368103410 0x7f236810f920 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.958+0000 7f236f845640 1 --2- 172.21.15.69:0/3517319891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2368102a50 0x7f2368102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.958+0000 7f236f845640 1 --2- 172.21.15.69:0/3517319891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2368108a60 0x7f2368108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.958+0000 7f236f845640 1 -- 172.21.15.69:0/3517319891 >> 172.21.15.69:0/3517319891 conn(0x7f23680fe500 msgr2=0x7f2368100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.958+0000 7f236f845640 1 -- 172.21.15.69:0/3517319891 shutdown_connections 2024-05-14T17:02:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.958+0000 7f236f845640 1 -- 172.21.15.69:0/3517319891 wait complete. 2024-05-14T17:02:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236f845640 1 Processor -- start 2024-05-14T17:02:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236f845640 1 -- start start 2024-05-14T17:02:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236f845640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2368102a50 0x7f236810bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236f845640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2368103410 0x7f236810c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236f845640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368108a60 0x7f236810c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236f845640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2368075a20 con 0x7f2368108a60 2024-05-14T17:02:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236f845640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f23680758a0 con 0x7f2368103410 2024-05-14T17:02:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236f845640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2368075ba0 con 0x7f2368102a50 2024-05-14T17:02:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236dde2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368108a60 0x7f236810c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236dde2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368108a60 0x7f236810c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41666/0 (socket says 172.21.15.69:41666) 2024-05-14T17:02:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236cde0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2368103410 0x7f236810c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236dde2640 1 -- 172.21.15.69:0/2127820235 learned_addr learned my addr 172.21.15.69:0/2127820235 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236d5e1640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2368102a50 0x7f236810bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236cde0640 1 -- 172.21.15.69:0/2127820235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2368102a50 msgr2=0x7f236810bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236cde0640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2368102a50 0x7f236810bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236cde0640 1 -- 172.21.15.69:0/2127820235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368108a60 msgr2=0x7f236810c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236cde0640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368108a60 0x7f236810c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236cde0640 1 -- 172.21.15.69:0/2127820235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23681ab9e0 con 0x7f2368103410 2024-05-14T17:02:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.959+0000 7f236dde2640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368108a60 0x7f236810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.960+0000 7f236cde0640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2368103410 0x7f236810c0b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f235800ea20 tx=0x7f235800eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.960+0000 7f235e7fc640 1 -- 172.21.15.69:0/2127820235 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235800cc40 con 0x7f2368103410 2024-05-14T17:02:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.960+0000 7f235e7fc640 1 -- 172.21.15.69:0/2127820235 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f235800cde0 con 0x7f2368103410 2024-05-14T17:02:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.960+0000 7f236f845640 1 -- 172.21.15.69:0/2127820235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23681abcd0 con 0x7f2368103410 2024-05-14T17:02:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.960+0000 7f235e7fc640 1 -- 172.21.15.69:0/2127820235 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23580187a0 con 0x7f2368103410 2024-05-14T17:02:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.960+0000 7f236f845640 1 -- 172.21.15.69:0/2127820235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23681ac1d0 con 0x7f2368103410 2024-05-14T17:02:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.962+0000 7f23721fd740 1 Processor -- start 2024-05-14T17:02:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.962+0000 7f235e7fc640 1 -- 172.21.15.69:0/2127820235 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2358010a50 con 0x7f2368103410 2024-05-14T17:02:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f23721fd740 1 -- start start 2024-05-14T17:02:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f23721fd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824ef4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f23721fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824eff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f23721fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824effad0 0x55b824f01ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f23721fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b824e67c30 con 0x55b824effad0 2024-05-14T17:02:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f23721fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b824e577d0 con 0x55b824ef5210 2024-05-14T17:02:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f23721fd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b824e5b810 con 0x55b824ef48f0 2024-05-14T17:02:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f236dde2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824ef4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f236dde2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824ef4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:41190/0 (socket says 172.21.15.69:41190) 2024-05-14T17:02:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f236dde2640 1 -- 172.21.15.69:0/2360891192 learned_addr learned my addr 172.21.15.69:0/2360891192 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f236cde0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824effad0 0x55b824f01ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.963+0000 7f236d5e1640 1 --2- 172.21.15.69:0/2360891192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824eff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f236d5e1640 1 -- 172.21.15.69:0/2360891192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 msgr2=0x55b824ef4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f236d5e1640 1 --2- 172.21.15.69:0/2360891192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824ef4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f236d5e1640 1 -- 172.21.15.69:0/2360891192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824effad0 msgr2=0x55b824f01ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f236d5e1640 1 --2- 172.21.15.69:0/2360891192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824effad0 0x55b824f01ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f236d5e1640 1 -- 172.21.15.69:0/2360891192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b824b6c210 con 0x55b824ef5210 2024-05-14T17:02:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f235e7fc640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f23540b2a60 0x7f23540b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f236dde2640 1 --2- 172.21.15.69:0/2360891192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824ef4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f236d5e1640 1 --2- 172.21.15.69:0/2360891192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824eff590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f23640045b0 tx=0x7f236400bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=694fa43db212a68b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f235e7fc640 1 -- 172.21.15.69:0/2127820235 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f23580e2850 con 0x7f2368103410 2024-05-14T17:02:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f236dde2640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f23540b2a60 0x7f23540b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f23537fe640 1 -- 172.21.15.69:0/2360891192 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364003bd0 con 0x55b824ef5210 2024-05-14T17:02:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f23537fe640 1 -- 172.21.15.69:0/2360891192 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2364015070 con 0x55b824ef5210 2024-05-14T17:02:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.964+0000 7f23537fe640 1 -- 172.21.15.69:0/2360891192 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364004c30 con 0x55b824ef5210 2024-05-14T17:02:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 -- 172.21.15.69:0/2360891192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 msgr2=0x55b824eff590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f236dde2640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f23540b2a60 0x7f23540b4f20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2360004ad0 tx=0x7f2360004a60 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 --2- 172.21.15.69:0/2360891192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824eff590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f23640045b0 tx=0x7f236400bb20 comp rx=0 tx=0).stop 2024-05-14T17:02:44.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 -- 172.21.15.69:0/2360891192 shutdown_connections 2024-05-14T17:02:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 --2- 172.21.15.69:0/2360891192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824effad0 0x55b824f01ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 --2- 172.21.15.69:0/2360891192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824eff590 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 --2- 172.21.15.69:0/2360891192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824ef4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 -- 172.21.15.69:0/2360891192 >> 172.21.15.69:0/2360891192 conn(0x55b824ef0e00 msgr2=0x55b824ef11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 -- 172.21.15.69:0/2360891192 shutdown_connections 2024-05-14T17:02:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 -- 172.21.15.69:0/2360891192 wait complete. 2024-05-14T17:02:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 -- start start 2024-05-14T17:02:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.965+0000 7f23721fd740 1 Processor -- start 2024-05-14T17:02:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f23721fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef48f0 0x55b824eff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f23721fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef5210 0x55b824efc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f23721fd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 0x55b824efcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f23721fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b824b3d790 con 0x55b824ef5210 2024-05-14T17:02:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f23721fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b824e13890 con 0x55b824ef48f0 2024-05-14T17:02:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f23721fd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b824f027a0 con 0x55b824effad0 2024-05-14T17:02:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f23721fd740 10 client.0 _ll_register_callbacks cb 0x55b824e16d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236cde0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef5210 0x55b824efc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236cde0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef5210 0x55b824efc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41694/0 (socket says 172.21.15.69:41694) 2024-05-14T17:02:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236cde0640 1 -- 172.21.15.69:0/520813818 learned_addr learned my addr 172.21.15.69:0/520813818 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236d5e1640 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef48f0 0x55b824eff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236dde2640 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 0x55b824efcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236cde0640 1 -- 172.21.15.69:0/520813818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 msgr2=0x55b824efcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236cde0640 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 0x55b824efcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236cde0640 1 -- 172.21.15.69:0/520813818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef48f0 msgr2=0x55b824eff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236cde0640 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef48f0 0x55b824eff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236cde0640 1 -- 172.21.15.69:0/520813818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b824c38620 con 0x55b824ef5210 2024-05-14T17:02:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236dde2640 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 0x55b824efcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:02:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f236cde0640 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef5210 0x55b824efc720 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f23580a6e40 tx=0x7f2358009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c957c780213bc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.966+0000 7f2350ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:02:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f2350ff9640 1 -- 172.21.15.69:0/520813818 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23580a9340 con 0x55b824ef5210 2024-05-14T17:02:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f2350ff9640 1 -- 172.21.15.69:0/520813818 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f23580a9040 con 0x55b824ef5210 2024-05-14T17:02:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f23721fd740 1 -- 172.21.15.69:0/520813818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2364015070 con 0x55b824ef5210 2024-05-14T17:02:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f2350ff9640 1 -- 172.21.15.69:0/520813818 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f23580a5360 con 0x55b824ef5210 2024-05-14T17:02:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f23721fd740 10 client.16722 fetch_fsmap learned FSMap version 51 2024-05-14T17:02:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f23721fd740 1 -- 172.21.15.69:0/520813818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b824f0e270 con 0x55b824ef5210 2024-05-14T17:02:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f2350ff9640 1 -- 172.21.15.69:0/520813818 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2364015070 con 0x55b824ef5210 2024-05-14T17:02:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f2350ff9640 1 -- 172.21.15.69:0/520813818 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f23580a5040 con 0x55b824ef5210 2024-05-14T17:02:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f23721fd740 10 client.16722 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:02:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f23721fd740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f23721fd740 20 client.16722 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f23721fd740 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b824f0f780 0x55b824f11b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f23721fd740 4 client.16722 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f23721fd740 1 -- 172.21.15.69:0/520813818 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b824e69b80 con 0x55b824f0f780 2024-05-14T17:02:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.967+0000 7f236d5e1640 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b824f0f780 0x55b824f11b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.968+0000 7f2350ff9640 10 client.16722 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.968+0000 7f236d5e1640 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b824f0f780 0x55b824f11b70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301382a7e1ad11 server_cookie=c263465b7db680d5 in_seq=0 out_seq=0 2024-05-14T17:02:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.969+0000 7f2350ff9640 1 -- 172.21.15.69:0/520813818 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b824e69b80 con 0x55b824f0f780 2024-05-14T17:02:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.969+0000 7f2350ff9640 10 client.16722 handle_command_reply: tid=0 2024-05-14T17:02:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.969+0000 7f23721fd740 1 client.16722 shutdown 2024-05-14T17:02:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.969+0000 7f23721fd740 1 -- 172.21.15.69:0/520813818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef5210 msgr2=0x55b824efc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.969+0000 7f23721fd740 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef5210 0x55b824efc720 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f23580a6e40 tx=0x7f2358009310 comp rx=0 tx=0).stop 2024-05-14T17:02:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.969+0000 7f23721fd740 1 -- 172.21.15.69:0/520813818 shutdown_connections 2024-05-14T17:02:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.969+0000 7f23721fd740 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b824f0f780 0x55b824f11b70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.969+0000 7f23721fd740 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 0x55b824efcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.969+0000 7f23721fd740 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef5210 0x55b824efc720 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.969+0000 7f2350ff9640 0 client.16722 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.970+0000 7f23721fd740 1 --2- 172.21.15.69:0/520813818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef48f0 0x55b824eff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.970+0000 7f23721fd740 1 -- 172.21.15.69:0/520813818 >> 172.21.15.69:0/520813818 conn(0x55b824ef0e00 msgr2=0x55b824ef0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.970+0000 7f23721fd740 1 -- 172.21.15.69:0/520813818 shutdown_connections 2024-05-14T17:02:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.970+0000 7f23721fd740 1 -- 172.21.15.69:0/520813818 wait complete. 2024-05-14T17:02:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.970+0000 7f23721fd740 20 client.16722 trim_cache size 0 max 16384 2024-05-14T17:02:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.988+0000 7f23721fd740 1 Processor -- start 2024-05-14T17:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.988+0000 7f23721fd740 1 -- start start 2024-05-14T17:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.989+0000 7f23721fd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824f17460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.989+0000 7f23721fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824f179a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.989+0000 7f23721fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824effad0 0x55b824ef9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.989+0000 7f23721fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f23681b75c0 con 0x55b824effad0 2024-05-14T17:02:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.989+0000 7f23721fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f23681b4e00 con 0x55b824ef5210 2024-05-14T17:02:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.989+0000 7f23721fd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f23681b2c70 con 0x55b824ef48f0 2024-05-14T17:02:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.989+0000 7f236d5e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824f17460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.989+0000 7f236d5e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824f17460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:41208/0 (socket says 172.21.15.69:41208) 2024-05-14T17:02:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.989+0000 7f236d5e1640 1 -- 172.21.15.69:0/2619679791 learned_addr learned my addr 172.21.15.69:0/2619679791 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.989+0000 7f236dde2640 1 --2- 172.21.15.69:0/2619679791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824effad0 0x55b824ef9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.990+0000 7f236d5e1640 1 -- 172.21.15.69:0/2619679791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 msgr2=0x55b824f179a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.990+0000 7f236d5e1640 1 --2- 172.21.15.69:0/2619679791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824f179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.990+0000 7f236d5e1640 1 -- 172.21.15.69:0/2619679791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824effad0 msgr2=0x55b824ef9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.990+0000 7f236d5e1640 1 --2- 172.21.15.69:0/2619679791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824effad0 0x55b824ef9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.990+0000 7f236d5e1640 1 -- 172.21.15.69:0/2619679791 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b824ef5040 con 0x55b824ef48f0 2024-05-14T17:02:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.990+0000 7f236d5e1640 1 --2- 172.21.15.69:0/2619679791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824f17460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f23640045b0 tx=0x7f236400b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16b29a1bf47d3907 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.990+0000 7f2352ffd640 1 -- 172.21.15.69:0/2619679791 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364003bd0 con 0x55b824ef48f0 2024-05-14T17:02:47.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.990+0000 7f2352ffd640 1 -- 172.21.15.69:0/2619679791 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2364015070 con 0x55b824ef48f0 2024-05-14T17:02:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.990+0000 7f2352ffd640 1 -- 172.21.15.69:0/2619679791 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23640049c0 con 0x55b824ef48f0 2024-05-14T17:02:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 -- 172.21.15.69:0/2619679791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 msgr2=0x55b824f17460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 --2- 172.21.15.69:0/2619679791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824f17460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f23640045b0 tx=0x7f236400b390 comp rx=0 tx=0).stop 2024-05-14T17:02:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 -- 172.21.15.69:0/2619679791 shutdown_connections 2024-05-14T17:02:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 --2- 172.21.15.69:0/2619679791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824effad0 0x55b824ef9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 --2- 172.21.15.69:0/2619679791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824f179a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 --2- 172.21.15.69:0/2619679791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824ef48f0 0x55b824f17460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 -- 172.21.15.69:0/2619679791 >> 172.21.15.69:0/2619679791 conn(0x55b824ef0e00 msgr2=0x55b824f0a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 -- 172.21.15.69:0/2619679791 shutdown_connections 2024-05-14T17:02:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 -- 172.21.15.69:0/2619679791 wait complete. 2024-05-14T17:02:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 -- start start 2024-05-14T17:02:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.991+0000 7f23721fd740 1 Processor -- start 2024-05-14T17:02:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f23721fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef48f0 0x55b824f080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f23721fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824f05450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f23721fd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 0x55b824f05b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f23721fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f23681b32c0 con 0x55b824ef48f0 2024-05-14T17:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f23721fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f23681b4730 con 0x55b824ef5210 2024-05-14T17:02:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f23721fd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f23681b3990 con 0x55b824effad0 2024-05-14T17:02:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f23721fd740 10 client.0 _ll_register_callbacks cb 0x7f23580a5a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236d5e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef48f0 0x55b824f080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236d5e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef48f0 0x55b824f080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41712/0 (socket says 172.21.15.69:41712) 2024-05-14T17:02:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236d5e1640 1 -- 172.21.15.69:0/1082226581 learned_addr learned my addr 172.21.15.69:0/1082226581 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236cde0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824f05450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236dde2640 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 0x55b824f05b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236d5e1640 1 -- 172.21.15.69:0/1082226581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 msgr2=0x55b824f05b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236d5e1640 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 0x55b824f05b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236d5e1640 1 -- 172.21.15.69:0/1082226581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 msgr2=0x55b824f05450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236d5e1640 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824f05450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236d5e1640 1 -- 172.21.15.69:0/1082226581 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b824f79020 con 0x55b824ef48f0 2024-05-14T17:02:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236cde0640 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824f05450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.992+0000 7f236dde2640 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 0x55b824f05b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:02:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.993+0000 7f236d5e1640 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef48f0 0x55b824f080c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2364006010 tx=0x7f236400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca87059922e84975 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.993+0000 7f2350ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:02:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.993+0000 7f2350ff9640 1 -- 172.21.15.69:0/1082226581 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236402c020 con 0x55b824ef48f0 2024-05-14T17:02:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.993+0000 7f2350ff9640 1 -- 172.21.15.69:0/1082226581 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f2364005230 con 0x55b824ef48f0 2024-05-14T17:02:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.993+0000 7f23721fd740 1 -- 172.21.15.69:0/1082226581 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2364015070 con 0x55b824ef48f0 2024-05-14T17:02:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.993+0000 7f2350ff9640 1 -- 172.21.15.69:0/1082226581 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2364037050 con 0x55b824ef48f0 2024-05-14T17:02:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.993+0000 7f23721fd740 10 client.16728 fetch_fsmap learned FSMap version 51 2024-05-14T17:02:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.993+0000 7f23721fd740 1 -- 172.21.15.69:0/1082226581 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b824f7a1d0 con 0x55b824ef48f0 2024-05-14T17:02:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.994+0000 7f2350ff9640 1 -- 172.21.15.69:0/1082226581 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2364015070 con 0x55b824ef48f0 2024-05-14T17:02:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.994+0000 7f2350ff9640 1 -- 172.21.15.69:0/1082226581 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f2364028190 con 0x55b824ef48f0 2024-05-14T17:02:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.994+0000 7f23721fd740 10 client.16728 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:02:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.994+0000 7f23721fd740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.994+0000 7f23721fd740 20 client.16728 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.994+0000 7f23721fd740 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b824f7a6f0 0x55b824f7cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.994+0000 7f23721fd740 4 client.16728 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:02:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.994+0000 7f23721fd740 1 -- 172.21.15.69:0/1082226581 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b824e63360 con 0x55b824f7a6f0 2024-05-14T17:02:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.994+0000 7f236cde0640 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b824f7a6f0 0x55b824f7cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.994+0000 7f2350ff9640 10 client.16728 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:29.995+0000 7f236cde0640 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b824f7a6f0 0x55b824f7cae0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95f1bea20c70662d server_cookie=2fc17693cd9ab711 in_seq=0 out_seq=0 2024-05-14T17:02:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f2350ff9640 1 -- 172.21.15.69:0/1082226581 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b824e63360 con 0x55b824f7a6f0 2024-05-14T17:02:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f2350ff9640 10 client.16728 handle_command_reply: tid=0 2024-05-14T17:02:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 client.16728 shutdown 2024-05-14T17:02:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 -- 172.21.15.69:0/1082226581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef48f0 msgr2=0x55b824f080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef48f0 0x55b824f080c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2364006010 tx=0x7f236400bf10 comp rx=0 tx=0).stop 2024-05-14T17:02:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 -- 172.21.15.69:0/1082226581 shutdown_connections 2024-05-14T17:02:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55b824f7a6f0 0x55b824f7cae0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b824effad0 0x55b824f05b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b824ef5210 0x55b824f05450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 --2- 172.21.15.69:0/1082226581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b824ef48f0 0x55b824f080c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 -- 172.21.15.69:0/1082226581 >> 172.21.15.69:0/1082226581 conn(0x55b824ef0e00 msgr2=0x55b824f0a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f2350ff9640 0 client.16728 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 -- 172.21.15.69:0/1082226581 shutdown_connections 2024-05-14T17:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 1 -- 172.21.15.69:0/1082226581 wait complete. 2024-05-14T17:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.019+0000 7f23721fd740 20 client.16728 trim_cache size 0 max 16384 2024-05-14T17:02:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:02:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:02:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:02:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 -- 172.21.15.69:0/2127820235 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f23540b2a60 msgr2=0x7f23540b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f23540b2a60 0x7f23540b4f20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2360004ad0 tx=0x7f2360004a60 comp rx=0 tx=0).stop 2024-05-14T17:02:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 -- 172.21.15.69:0/2127820235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2368103410 msgr2=0x7f236810c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2368103410 0x7f236810c0b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f235800ea20 tx=0x7f235800eee0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 -- 172.21.15.69:0/2127820235 shutdown_connections 2024-05-14T17:02:51.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f23540b2a60 0x7f23540b4f20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368108a60 0x7f236810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2368103410 0x7f236810c0b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 --2- 172.21.15.69:0/2127820235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2368102a50 0x7f236810bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 -- 172.21.15.69:0/2127820235 >> 172.21.15.69:0/2127820235 conn(0x7f23680fe500 msgr2=0x7f2368106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 -- 172.21.15.69:0/2127820235 shutdown_connections 2024-05-14T17:02:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:30.020+0000 7f236f845640 1 -- 172.21.15.69:0/2127820235 wait complete. 2024-05-14T17:02:51.628 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:29 smithi069 ceph-mon[31507]: pgmap v397: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.4 MiB/s wr, 553 op/s 2024-05-14T17:02:51.678 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 720.0 s, 1899.4 tps, lat 52.641 ms stddev 57.368 2024-05-14T17:02:51.729 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:31 smithi069 ceph-mon[31507]: pgmap v398: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-05-14T17:02:51.778 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:33 smithi069 ceph-mon[31507]: pgmap v399: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.5 MiB/s wr, 877 op/s 2024-05-14T17:02:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 725.0 s, 2579.8 tps, lat 38.839 ms stddev 43.496 2024-05-14T17:02:51.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:35 smithi069 ceph-mon[31507]: pgmap v400: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.8 MiB/s wr, 603 op/s 2024-05-14T17:02:51.928 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:37 smithi069 ceph-mon[31507]: pgmap v401: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.7 MiB/s wr, 896 op/s 2024-05-14T17:02:51.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:39 smithi069 ceph-mon[31507]: pgmap v402: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.0 MiB/s wr, 619 op/s 2024-05-14T17:02:52.046 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 730.0 s, 1556.4 tps, lat 63.848 ms stddev 119.778 2024-05-14T17:02:52.096 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:41 smithi069 ceph-mon[31507]: pgmap v403: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.3 MiB/s wr, 630 op/s 2024-05-14T17:02:52.157 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:41 smithi092 ceph-mon[35763]: pgmap v403: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.3 MiB/s wr, 630 op/s 2024-05-14T17:02:52.203 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:43 smithi092 ceph-mon[35763]: pgmap v404: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 29 MiB/s wr, 1.93k op/s 2024-05-14T17:02:52.253 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:44 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:02:52.316 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:45 smithi092 ceph-mon[35763]: pgmap v405: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 1.60k op/s 2024-05-14T17:02:52.362 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:47 smithi092 ceph-mon[35763]: pgmap v406: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 28 MiB/s wr, 1.87k op/s 2024-05-14T17:02:52.412 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:49 smithi092 ceph-mon[35763]: pgmap v407: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2024-05-14T17:02:52.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:41 smithi203 ceph-mon[35787]: pgmap v403: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.3 MiB/s wr, 630 op/s 2024-05-14T17:02:52.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:43 smithi203 ceph-mon[35787]: pgmap v404: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 29 MiB/s wr, 1.93k op/s 2024-05-14T17:02:52.562 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:44 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:02:52.612 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:45 smithi203 ceph-mon[35787]: pgmap v405: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 1.60k op/s 2024-05-14T17:02:52.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:47 smithi203 ceph-mon[35787]: pgmap v406: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 28 MiB/s wr, 1.87k op/s 2024-05-14T17:02:52.712 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:49 smithi203 ceph-mon[35787]: pgmap v407: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2024-05-14T17:02:52.763 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:51 smithi092 ceph-mon[35763]: pgmap v408: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2024-05-14T17:02:52.812 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:51 smithi203 ceph-mon[35787]: pgmap v408: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2024-05-14T17:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.143+0000 7f86bbce1640 1 -- 172.21.15.69:0/2244212486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86b4106ed0 msgr2=0x7f86b4107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.143+0000 7f86bbce1640 1 --2- 172.21.15.69:0/2244212486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86b4106ed0 0x7f86b4107380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009cd0 tx=0x7f86ac02a9c0 comp rx=0 tx=0).stop 2024-05-14T17:02:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.143+0000 7f86bbce1640 1 -- 172.21.15.69:0/2244212486 shutdown_connections 2024-05-14T17:02:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.143+0000 7f86bbce1640 1 --2- 172.21.15.69:0/2244212486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86b4106ed0 0x7f86b4107380 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.143+0000 7f86bbce1640 1 --2- 172.21.15.69:0/2244212486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b41023f0 0x7f86b41068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.144+0000 7f86bbce1640 1 --2- 172.21.15.69:0/2244212486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86b4101ad0 0x7f86b4101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.144+0000 7f86bbce1640 1 -- 172.21.15.69:0/2244212486 >> 172.21.15.69:0/2244212486 conn(0x7f86b4078a70 msgr2=0x7f86b4078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.144+0000 7f86bbce1640 1 -- 172.21.15.69:0/2244212486 shutdown_connections 2024-05-14T17:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.144+0000 7f86bbce1640 1 -- 172.21.15.69:0/2244212486 wait complete. 2024-05-14T17:02:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.144+0000 7f86bbce1640 1 Processor -- start 2024-05-14T17:02:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.144+0000 7f86bbce1640 1 -- start start 2024-05-14T17:02:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86bbce1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86b4101ad0 0x7f86b41a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86bbce1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86b41023f0 0x7f86b41a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86bbce1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b4106ed0 0x7f86b419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86bbce1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86b41193d0 con 0x7f86b41023f0 2024-05-14T17:02:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86b927c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86b41023f0 0x7f86b41a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86b927c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86b41023f0 0x7f86b41a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45652/0 (socket says 172.21.15.69:45652) 2024-05-14T17:02:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86bbce1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86b4119250 con 0x7f86b4106ed0 2024-05-14T17:02:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86bbce1640 1 -- 172.21.15.69:0/481179507 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f86b4119550 con 0x7f86b4101ad0 2024-05-14T17:02:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86b927c640 1 -- 172.21.15.69:0/481179507 learned_addr learned my addr 172.21.15.69:0/481179507 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86b4101ad0 0x7f86b41a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.145+0000 7f86ba27e640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b4106ed0 0x7f86b419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86ba27e640 1 -- 172.21.15.69:0/481179507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86b4101ad0 msgr2=0x7f86b41a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86ba27e640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86b4101ad0 0x7f86b41a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86ba27e640 1 -- 172.21.15.69:0/481179507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86b41023f0 msgr2=0x7f86b41a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86ba27e640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86b41023f0 0x7f86b41a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86ba27e640 1 -- 172.21.15.69:0/481179507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b419ffd0 con 0x7f86b4106ed0 2024-05-14T17:02:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86b4101ad0 0x7f86b41a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86ba27e640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b4106ed0 0x7f86b419f710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86ac02afb0 tx=0x7f86ac02fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86aaffd640 1 -- 172.21.15.69:0/481179507 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac034030 con 0x7f86b4106ed0 2024-05-14T17:02:55.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86bbce1640 1 -- 172.21.15.69:0/481179507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b41a0260 con 0x7f86b4106ed0 2024-05-14T17:02:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86aaffd640 1 -- 172.21.15.69:0/481179507 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ac030b10 con 0x7f86b4106ed0 2024-05-14T17:02:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.146+0000 7f86bbce1640 1 -- 172.21.15.69:0/481179507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b41ac5d0 con 0x7f86b4106ed0 2024-05-14T17:02:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.147+0000 7f86aaffd640 1 -- 172.21.15.69:0/481179507 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac004760 con 0x7f86b4106ed0 2024-05-14T17:02:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.150+0000 7f86bea01740 1 Processor -- start 2024-05-14T17:02:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.150+0000 7f86aaffd640 1 -- 172.21.15.69:0/481179507 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f86ac037020 con 0x7f86b4106ed0 2024-05-14T17:02:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.150+0000 7f86bea01740 1 -- start start 2024-05-14T17:02:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.150+0000 7f86bea01740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380a48f0 0x55bd380a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.150+0000 7f86bea01740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.150+0000 7f86bea01740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 0x55bd380b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.150+0000 7f86bea01740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bd38017c30 con 0x55bd380afad0 2024-05-14T17:02:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.150+0000 7f86bea01740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bd380077d0 con 0x55bd380a5210 2024-05-14T17:02:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.150+0000 7f86bea01740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bd3800b810 con 0x55bd380a48f0 2024-05-14T17:02:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b9a7d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b9a7d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:46542/0 (socket says 172.21.15.69:46542) 2024-05-14T17:02:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b9a7d640 1 -- 172.21.15.69:0/965510898 learned_addr learned my addr 172.21.15.69:0/965510898 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b927c640 1 --2- 172.21.15.69:0/965510898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 0x55bd380b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b927c640 1 --2- 172.21.15.69:0/965510898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380a48f0 0x55bd380a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b9a7d640 1 -- 172.21.15.69:0/965510898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380a48f0 msgr2=0x55bd380a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/965510898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380a48f0 0x55bd380a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b9a7d640 1 -- 172.21.15.69:0/965510898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 msgr2=0x55bd380b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/965510898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 0x55bd380b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b9a7d640 1 -- 172.21.15.69:0/965510898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd37d1c210 con 0x55bd380a5210 2024-05-14T17:02:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86aaffd640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f86a00b2b60 0x7f86a00b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86aaffd640 1 -- 172.21.15.69:0/481179507 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f86ac103450 con 0x7f86b4106ed0 2024-05-14T17:02:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/965510898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380af590 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009e70 tx=0x7f86b000e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14942f90472d0808 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.151+0000 7f86b927c640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f86a00b2b60 0x7f86a00b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.152+0000 7f869ffff640 1 -- 172.21.15.69:0/965510898 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0017070 con 0x55bd380a5210 2024-05-14T17:02:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.152+0000 7f869ffff640 1 -- 172.21.15.69:0/965510898 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86b0013920 con 0x55bd380a5210 2024-05-14T17:02:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.152+0000 7f869ffff640 1 -- 172.21.15.69:0/965510898 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0012870 con 0x55bd380a5210 2024-05-14T17:02:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 -- 172.21.15.69:0/965510898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 msgr2=0x55bd380af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86b927c640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f86a00b2b60 0x7f86a00b5020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55bd380a2220 tx=0x7f86a4004700 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 --2- 172.21.15.69:0/965510898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380af590 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009e70 tx=0x7f86b000e5a0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 -- 172.21.15.69:0/965510898 shutdown_connections 2024-05-14T17:02:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 --2- 172.21.15.69:0/965510898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 0x55bd380b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 --2- 172.21.15.69:0/965510898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380af590 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 --2- 172.21.15.69:0/965510898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380a48f0 0x55bd380a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 -- 172.21.15.69:0/965510898 >> 172.21.15.69:0/965510898 conn(0x55bd380a0e00 msgr2=0x55bd380a11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 -- 172.21.15.69:0/965510898 shutdown_connections 2024-05-14T17:02:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 -- 172.21.15.69:0/965510898 wait complete. 2024-05-14T17:02:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 -- start start 2024-05-14T17:02:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.153+0000 7f86bea01740 1 Processor -- start 2024-05-14T17:02:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86bea01740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380a48f0 0x55bd380af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86bea01740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86bea01740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 0x55bd380acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86bea01740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bd37ced790 con 0x55bd380a48f0 2024-05-14T17:02:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86bea01740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bd37fc3890 con 0x55bd380a5210 2024-05-14T17:02:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86bea01740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bd380b27a0 con 0x55bd380afad0 2024-05-14T17:02:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86bea01740 10 client.0 _ll_register_callbacks cb 0x55bd37fc6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86b9a7d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380a48f0 0x55bd380af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86b9a7d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380a48f0 0x55bd380af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45662/0 (socket says 172.21.15.69:45662) 2024-05-14T17:02:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86b9a7d640 1 -- 172.21.15.69:0/2920709573 learned_addr learned my addr 172.21.15.69:0/2920709573 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:02:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86b927c640 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86ba27e640 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 0x55bd380acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86b927c640 1 -- 172.21.15.69:0/2920709573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 msgr2=0x55bd380acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86b927c640 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 0x55bd380acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86b927c640 1 -- 172.21.15.69:0/2920709573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380a48f0 msgr2=0x55bd380af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86b927c640 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380a48f0 0x55bd380af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86b927c640 1 -- 172.21.15.69:0/2920709573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd37de8620 con 0x55bd380a5210 2024-05-14T17:02:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f86b927c640 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380ac6d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55bd380a1f90 tx=0x7f86a4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb1abdee44efc626 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.154+0000 7f869d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:02:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f869d7fa640 1 -- 172.21.15.69:0/2920709573 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a401e050 con 0x55bd380a5210 2024-05-14T17:02:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f869d7fa640 1 -- 172.21.15.69:0/2920709573 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f86a4015c40 con 0x55bd380a5210 2024-05-14T17:02:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f86bea01740 1 -- 172.21.15.69:0/2920709573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86b0013920 con 0x55bd380a5210 2024-05-14T17:02:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f869d7fa640 1 -- 172.21.15.69:0/2920709573 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f86a4015e40 con 0x55bd380a5210 2024-05-14T17:02:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f86bea01740 10 client.26165 fetch_fsmap learned FSMap version 51 2024-05-14T17:02:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f86bea01740 1 -- 172.21.15.69:0/2920709573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55bd380be210 con 0x55bd380a5210 2024-05-14T17:02:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f869d7fa640 1 -- 172.21.15.69:0/2920709573 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86b0013920 con 0x55bd380a5210 2024-05-14T17:02:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f869d7fa640 1 -- 172.21.15.69:0/2920709573 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f86a4015040 con 0x55bd380a5210 2024-05-14T17:02:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f86bea01740 10 client.26165 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:02:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f86bea01740 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:59.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f86bea01740 20 client.26165 populate_metadata read hostname 'smithi069' 2024-05-14T17:02:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f86bea01740 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bd380bf830 0x55bd380c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f86bea01740 4 client.26165 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.155+0000 7f86bea01740 1 -- 172.21.15.69:0/2920709573 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd38019b80 con 0x55bd380bf830 2024-05-14T17:02:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.156+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bd380bf830 0x55bd380c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.156+0000 7f869d7fa640 10 client.26165 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:02:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.156+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bd380bf830 0x55bd380c1cb0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e27ce3fb62f6f server_cookie=3f42404fb0104b50 in_seq=0 out_seq=0 2024-05-14T17:02:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.157+0000 7f869d7fa640 1 -- 172.21.15.69:0/2920709573 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd38019b80 con 0x55bd380bf830 2024-05-14T17:02:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.157+0000 7f869d7fa640 10 client.26165 handle_command_reply: tid=0 2024-05-14T17:02:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.157+0000 7f86bea01740 1 client.26165 shutdown 2024-05-14T17:02:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.157+0000 7f86bea01740 1 -- 172.21.15.69:0/2920709573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 msgr2=0x55bd380ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.157+0000 7f86bea01740 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380ac6d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55bd380a1f90 tx=0x7f86a4004770 comp rx=0 tx=0).stop 2024-05-14T17:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.158+0000 7f86bea01740 1 -- 172.21.15.69:0/2920709573 shutdown_connections 2024-05-14T17:02:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.158+0000 7f86bea01740 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bd380bf830 0x55bd380c1cb0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.158+0000 7f869d7fa640 0 client.26165 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.158+0000 7f86bea01740 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 0x55bd380acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.158+0000 7f86bea01740 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a5210 0x55bd380ac6d0 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55bd380a1f90 tx=0x7f86a4004770 comp rx=0 tx=0).stop 2024-05-14T17:03:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.158+0000 7f86bea01740 1 --2- 172.21.15.69:0/2920709573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380a48f0 0x55bd380af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.158+0000 7f86bea01740 1 -- 172.21.15.69:0/2920709573 >> 172.21.15.69:0/2920709573 conn(0x55bd380a0e00 msgr2=0x55bd380a0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.158+0000 7f86bea01740 1 -- 172.21.15.69:0/2920709573 shutdown_connections 2024-05-14T17:03:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.158+0000 7f86bea01740 1 -- 172.21.15.69:0/2920709573 wait complete. 2024-05-14T17:03:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.158+0000 7f86bea01740 20 client.26165 trim_cache size 0 max 16384 2024-05-14T17:03:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.175+0000 7f86bea01740 1 Processor -- start 2024-05-14T17:03:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.175+0000 7f86bea01740 1 -- start start 2024-05-14T17:03:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86bea01740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380a48f0 0x55bd380bb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86bea01740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 0x55bd380c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86bea01740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380bf830 0x55bd380c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86bea01740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8698011130 con 0x55bd380afad0 2024-05-14T17:03:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86bea01740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f869800e970 con 0x55bd380bf830 2024-05-14T17:03:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86bea01740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f869800c7e0 con 0x55bd380a48f0 2024-05-14T17:03:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86b927c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 0x55bd380c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86b927c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 0x55bd380c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45678/0 (socket says 172.21.15.69:45678) 2024-05-14T17:03:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86b927c640 1 -- 172.21.15.69:0/4080090818 learned_addr learned my addr 172.21.15.69:0/4080090818 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86ba27e640 1 --2- 172.21.15.69:0/4080090818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380bf830 0x55bd380c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/4080090818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380a48f0 0x55bd380bb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86ba27e640 1 -- 172.21.15.69:0/4080090818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380a48f0 msgr2=0x55bd380bb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86ba27e640 1 --2- 172.21.15.69:0/4080090818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380a48f0 0x55bd380bb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86ba27e640 1 -- 172.21.15.69:0/4080090818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 msgr2=0x55bd380c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86ba27e640 1 --2- 172.21.15.69:0/4080090818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 0x55bd380c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.176+0000 7f86ba27e640 1 -- 172.21.15.69:0/4080090818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd380a0c30 con 0x55bd380bf830 2024-05-14T17:03:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.177+0000 7f86ba27e640 1 --2- 172.21.15.69:0/4080090818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380bf830 0x55bd380c7b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f86ac100b30 tx=0x7f86ac0c66b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72a4df99823ffb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.177+0000 7f869f7fe640 1 -- 172.21.15.69:0/4080090818 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0cadc0 con 0x55bd380bf830 2024-05-14T17:03:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.177+0000 7f869f7fe640 1 -- 172.21.15.69:0/4080090818 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ac02c040 con 0x55bd380bf830 2024-05-14T17:03:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.177+0000 7f869f7fe640 1 -- 172.21.15.69:0/4080090818 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0c6950 con 0x55bd380bf830 2024-05-14T17:03:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.178+0000 7f86bea01740 1 -- 172.21.15.69:0/4080090818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380bf830 msgr2=0x55bd380c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.178+0000 7f86bea01740 1 --2- 172.21.15.69:0/4080090818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380bf830 0x55bd380c7b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f86ac100b30 tx=0x7f86ac0c66b0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.178+0000 7f86bea01740 1 -- 172.21.15.69:0/4080090818 shutdown_connections 2024-05-14T17:03:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.178+0000 7f86bea01740 1 --2- 172.21.15.69:0/4080090818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380bf830 0x55bd380c7b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.178+0000 7f86bea01740 1 --2- 172.21.15.69:0/4080090818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380afad0 0x55bd380c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.178+0000 7f86bea01740 1 --2- 172.21.15.69:0/4080090818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380a48f0 0x55bd380bb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.178+0000 7f86bea01740 1 -- 172.21.15.69:0/4080090818 >> 172.21.15.69:0/4080090818 conn(0x55bd380a0e00 msgr2=0x55bd380a0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.178+0000 7f86bea01740 1 -- 172.21.15.69:0/4080090818 shutdown_connections 2024-05-14T17:03:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.178+0000 7f86bea01740 1 -- 172.21.15.69:0/4080090818 wait complete. 2024-05-14T17:03:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.178+0000 7f86bea01740 1 -- start start 2024-05-14T17:03:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86bea01740 1 Processor -- start 2024-05-14T17:03:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86bea01740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a48f0 0x55bd380b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86bea01740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 0x55bd380b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86bea01740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380bf830 0x55bd38129180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86bea01740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f869800ce30 con 0x55bd380bf830 2024-05-14T17:03:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86bea01740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f869800e2a0 con 0x55bd380a48f0 2024-05-14T17:03:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86bea01740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f869800d500 con 0x55bd380afad0 2024-05-14T17:03:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86bea01740 10 client.0 _ll_register_callbacks cb 0x7f86a4018880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86b9a7d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a48f0 0x55bd380b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86b9a7d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a48f0 0x55bd380b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:46564/0 (socket says 172.21.15.69:46564) 2024-05-14T17:03:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86b9a7d640 1 -- 172.21.15.69:0/3979439055 learned_addr learned my addr 172.21.15.69:0/3979439055 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86ba27e640 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380bf830 0x55bd38129180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.179+0000 7f86b927c640 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 0x55bd380b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f86b9a7d640 1 -- 172.21.15.69:0/3979439055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 msgr2=0x55bd380b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 0x55bd380b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f86b9a7d640 1 -- 172.21.15.69:0/3979439055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380bf830 msgr2=0x55bd38129180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380bf830 0x55bd38129180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f86b9a7d640 1 -- 172.21.15.69:0/3979439055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd380b66f0 con 0x55bd380a48f0 2024-05-14T17:03:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f86b927c640 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 0x55bd380b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f86b9a7d640 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a48f0 0x55bd380b5920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009c00 tx=0x7f86b0012510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34e1fbb66d21dfcc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f869d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:03:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f869d7fa640 1 -- 172.21.15.69:0/3979439055 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0017040 con 0x55bd380a48f0 2024-05-14T17:03:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f869d7fa640 1 -- 172.21.15.69:0/3979439055 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f86b0005dd0 con 0x55bd380a48f0 2024-05-14T17:03:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f86bea01740 1 -- 172.21.15.69:0/3979439055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ac02c040 con 0x55bd380a48f0 2024-05-14T17:03:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f869d7fa640 1 -- 172.21.15.69:0/3979439055 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f86b0013630 con 0x55bd380a48f0 2024-05-14T17:03:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f86bea01740 10 client.26171 fetch_fsmap learned FSMap version 51 2024-05-14T17:03:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.180+0000 7f86bea01740 1 -- 172.21.15.69:0/3979439055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55bd3812a7a0 con 0x55bd380a48f0 2024-05-14T17:03:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.181+0000 7f869d7fa640 1 -- 172.21.15.69:0/3979439055 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ac02c040 con 0x55bd380a48f0 2024-05-14T17:03:03.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.181+0000 7f869d7fa640 1 -- 172.21.15.69:0/3979439055 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 51) ==== 8350+0+0 (secure 0 0 0) 0x7f86b0026040 con 0x55bd380a48f0 2024-05-14T17:03:03.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.181+0000 7f86bea01740 10 client.26171 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:03:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.181+0000 7f86bea01740 10 client.26171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.181+0000 7f86bea01740 20 client.26171 populate_metadata read hostname 'smithi069' 2024-05-14T17:03:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.181+0000 7f86bea01740 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bd3812abf0 0x55bd3812d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.181+0000 7f86bea01740 4 client.26171 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:03:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.181+0000 7f86bea01740 1 -- 172.21.15.69:0/3979439055 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd38013360 con 0x55bd3812abf0 2024-05-14T17:03:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.182+0000 7f86b927c640 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bd3812abf0 0x55bd3812d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.182+0000 7f869d7fa640 10 client.26171 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.182+0000 7f86b927c640 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bd3812abf0 0x55bd3812d010 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d597a9e259da19c server_cookie=ec126703280c1427 in_seq=0 out_seq=0 2024-05-14T17:03:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f869d7fa640 1 -- 172.21.15.69:0/3979439055 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bd38013360 con 0x55bd3812abf0 2024-05-14T17:03:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f869d7fa640 10 client.26171 handle_command_reply: tid=0 2024-05-14T17:03:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 client.26171 shutdown 2024-05-14T17:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 -- 172.21.15.69:0/3979439055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a48f0 msgr2=0x55bd380b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a48f0 0x55bd380b5920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009c00 tx=0x7f86b0012510 comp rx=0 tx=0).stop 2024-05-14T17:03:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 -- 172.21.15.69:0/3979439055 shutdown_connections 2024-05-14T17:03:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bd3812abf0 0x55bd3812d010 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd380bf830 0x55bd38129180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bd380afad0 0x55bd380b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 --2- 172.21.15.69:0/3979439055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd380a48f0 0x55bd380b5920 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 -- 172.21.15.69:0/3979439055 >> 172.21.15.69:0/3979439055 conn(0x55bd380a0e00 msgr2=0x55bd380ba6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f869d7fa640 0 client.26171 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 -- 172.21.15.69:0/3979439055 shutdown_connections 2024-05-14T17:03:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.202+0000 7f86bea01740 1 -- 172.21.15.69:0/3979439055 wait complete. 2024-05-14T17:03:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bea01740 20 client.26171 trim_cache size 0 max 16384 2024-05-14T17:03:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:03:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:03:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:03:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 -- 172.21.15.69:0/481179507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f86a00b2b60 msgr2=0x7f86a00b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f86a00b2b60 0x7f86a00b5020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55bd380a2220 tx=0x7f86a4004700 comp rx=0 tx=0).stop 2024-05-14T17:03:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 -- 172.21.15.69:0/481179507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b4106ed0 msgr2=0x7f86b419f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b4106ed0 0x7f86b419f710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86ac02afb0 tx=0x7f86ac02fb20 comp rx=0 tx=0).stop 2024-05-14T17:03:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 -- 172.21.15.69:0/481179507 shutdown_connections 2024-05-14T17:03:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f86a00b2b60 0x7f86a00b5020 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b4106ed0 0x7f86b419f710 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86b41023f0 0x7f86b41a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 --2- 172.21.15.69:0/481179507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86b4101ad0 0x7f86b41a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 -- 172.21.15.69:0/481179507 >> 172.21.15.69:0/481179507 conn(0x7f86b4078a70 msgr2=0x7f86b4104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 -- 172.21.15.69:0/481179507 shutdown_connections 2024-05-14T17:03:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:43.203+0000 7f86bbce1640 1 -- 172.21.15.69:0/481179507 wait complete. 2024-05-14T17:03:04.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:43 smithi069 ceph-mon[31507]: pgmap v404: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 29 MiB/s wr, 1.93k op/s 2024-05-14T17:03:04.754 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:44 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:03:04.796 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 735.0 s, 1658.4 tps, lat 60.570 ms stddev 64.399 2024-05-14T17:03:04.856 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:45 smithi069 ceph-mon[31507]: pgmap v405: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 1.60k op/s 2024-05-14T17:03:04.895 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:47 smithi069 ceph-mon[31507]: pgmap v406: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 28 MiB/s wr, 1.87k op/s 2024-05-14T17:03:04.937 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:49 smithi069 ceph-mon[31507]: pgmap v407: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2024-05-14T17:03:04.979 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 740.0 s, 2315.4 tps, lat 43.014 ms stddev 50.543 2024-05-14T17:03:05.013 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:53 smithi092 ceph-mon[35763]: pgmap v409: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 27 MiB/s wr, 1.84k op/s 2024-05-14T17:03:05.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:54 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1806318035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:03:05.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:55 smithi092 ceph-mon[35763]: pgmap v410: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-05-14T17:03:05.129 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:57 smithi092 ceph-mon[35763]: pgmap v411: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 786 op/s 2024-05-14T17:03:05.170 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:02:59 smithi092 ceph-mon[35763]: pgmap v412: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-05-14T17:03:05.204 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:01 smithi092 ceph-mon[35763]: pgmap v413: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-05-14T17:03:05.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:03 smithi092 ceph-mon[35763]: pgmap v414: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 20 MiB/s wr, 1.85k op/s 2024-05-14T17:03:05.271 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:53 smithi203 ceph-mon[35787]: pgmap v409: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 27 MiB/s wr, 1.84k op/s 2024-05-14T17:03:05.404 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:54 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1806318035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:03:05.437 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:55 smithi203 ceph-mon[35787]: pgmap v410: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-05-14T17:03:05.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:57 smithi203 ceph-mon[35787]: pgmap v411: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 786 op/s 2024-05-14T17:03:05.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:02:59 smithi203 ceph-mon[35787]: pgmap v412: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-05-14T17:03:05.571 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:01 smithi203 ceph-mon[35787]: pgmap v413: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-05-14T17:03:05.612 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:03 smithi203 ceph-mon[35787]: pgmap v414: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 20 MiB/s wr, 1.85k op/s 2024-05-14T17:03:05.647 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:51 smithi069 ceph-mon[31507]: pgmap v408: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2024-05-14T17:03:05.680 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:05.713 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:04 smithi092 ceph-mon[35763]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T17:03:05.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:04 smithi203 ceph-mon[35787]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T17:03:05.780 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:53 smithi069 ceph-mon[31507]: pgmap v409: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 27 MiB/s wr, 1.84k op/s 2024-05-14T17:03:05.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.287+0000 7f6b936f9700 1 -- 172.21.15.69:0/1301008912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c103850 msgr2=0x7f6b8c103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:05.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.287+0000 7f6b936f9700 1 --2- 172.21.15.69:0/1301008912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c103850 0x7f6b8c103c30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009990 tx=0x7f6b88009ca0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.287+0000 7f6b936f9700 1 -- 172.21.15.69:0/1301008912 shutdown_connections 2024-05-14T17:03:05.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.287+0000 7f6b936f9700 1 --2- 172.21.15.69:0/1301008912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b8c108920 0x7f6b8c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.287+0000 7f6b936f9700 1 --2- 172.21.15.69:0/1301008912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c103850 0x7f6b8c103c30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.287+0000 7f6b936f9700 1 --2- 172.21.15.69:0/1301008912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b8c104200 0x7f6b8c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.287+0000 7f6b936f9700 1 -- 172.21.15.69:0/1301008912 >> 172.21.15.69:0/1301008912 conn(0x7f6b8c07a5d0 msgr2=0x7f6b8c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.287+0000 7f6b936f9700 1 -- 172.21.15.69:0/1301008912 shutdown_connections 2024-05-14T17:03:06.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.287+0000 7f6b936f9700 1 -- 172.21.15.69:0/1301008912 wait complete. 2024-05-14T17:03:06.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b936f9700 1 Processor -- start 2024-05-14T17:03:06.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b936f9700 1 -- start start 2024-05-14T17:03:06.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b936f9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b8c103850 0x7f6b8c077430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:06.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b936f9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b8c104200 0x7f6b8c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:06.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b936f9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108920 0x7f6b8c075ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:06.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b936f9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c113750 con 0x7f6b8c104200 2024-05-14T17:03:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b936f9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1135d0 con 0x7f6b8c103850 2024-05-14T17:03:06.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b936f9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1138d0 con 0x7f6b8c108920 2024-05-14T17:03:06.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b91c96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108920 0x7f6b8c075ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:06.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b91495700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b8c103850 0x7f6b8c077430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:06.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b91c96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108920 0x7f6b8c075ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:56102/0 (socket says 172.21.15.69:56102) 2024-05-14T17:03:06.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b91c96700 1 -- 172.21.15.69:0/1806318035 learned_addr learned my addr 172.21.15.69:0/1806318035 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:06.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b90c94700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b8c104200 0x7f6b8c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:06.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b91c96700 1 -- 172.21.15.69:0/1806318035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b8c103850 msgr2=0x7f6b8c077430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:06.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b91c96700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b8c103850 0x7f6b8c077430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b91c96700 1 -- 172.21.15.69:0/1806318035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b8c104200 msgr2=0x7f6b8c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:06.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b91c96700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b8c104200 0x7f6b8c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.288+0000 7f6b91c96700 1 -- 172.21.15.69:0/1806318035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c1ac5f0 con 0x7f6b8c108920 2024-05-14T17:03:06.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.289+0000 7f6b91c96700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108920 0x7f6b8c075ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400ef30 tx=0x7f6b8400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:06.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.289+0000 7f6b827fc700 1 -- 172.21.15.69:0/1806318035 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840092e0 con 0x7f6b8c108920 2024-05-14T17:03:06.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.289+0000 7f6b827fc700 1 -- 172.21.15.69:0/1806318035 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84004bb0 con 0x7f6b8c108920 2024-05-14T17:03:06.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.289+0000 7f6b936f9700 1 -- 172.21.15.69:0/1806318035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c1ac820 con 0x7f6b8c108920 2024-05-14T17:03:07.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.289+0000 7f6b827fc700 1 -- 172.21.15.69:0/1806318035 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84004280 con 0x7f6b8c108920 2024-05-14T17:03:07.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.289+0000 7f6b936f9700 1 -- 172.21.15.69:0/1806318035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1acc70 con 0x7f6b8c108920 2024-05-14T17:03:07.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.290+0000 7f6b936f9700 1 -- 172.21.15.69:0/1806318035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b8c10bb60 con 0x7f6b8c108920 2024-05-14T17:03:07.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.292+0000 7f6b827fc700 1 -- 172.21.15.69:0/1806318035 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6b8401b030 con 0x7f6b8c108920 2024-05-14T17:03:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.293+0000 7f6b827fc700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6b780b2a70 0x7f6b780b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:07.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.293+0000 7f6b827fc700 1 -- 172.21.15.69:0/1806318035 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f6b840e20a0 con 0x7f6b8c108920 2024-05-14T17:03:07.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.293+0000 7f6b91495700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6b780b2a70 0x7f6b780b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.293+0000 7f6b91495700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6b780b2a70 0x7f6b780b4f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800b790 tx=0x7f6b8800b260 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:07.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.298+0000 7f6b827fc700 1 -- 172.21.15.69:0/1806318035 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b84022020 con 0x7f6b8c108920 2024-05-14T17:03:07.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.700+0000 7f6b936f9700 1 -- 172.21.15.69:0/1806318035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b8c104cb0 con 0x7f6b8c108920 2024-05-14T17:03:07.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.701+0000 7f6b827fc700 1 -- 172.21.15.69:0/1806318035 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11863 (secure 0 0 0) 0x7f6b84010040 con 0x7f6b8c108920 2024-05-14T17:03:07.515 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:03:07.565 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:02:43.717961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3631522795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3631522795},{"type":"v1","addr":"172.21.15.203:6837","nonce":3631522795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T17:03:07.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 -- 172.21.15.69:0/1806318035 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6b780b2a70 msgr2=0x7f6b780b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:07.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6b780b2a70 0x7f6b780b4f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800b790 tx=0x7f6b8800b260 comp rx=0 tx=0).stop 2024-05-14T17:03:07.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 -- 172.21.15.69:0/1806318035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108920 msgr2=0x7f6b8c075ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:07.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108920 0x7f6b8c075ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400ef30 tx=0x7f6b8400c3a0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 -- 172.21.15.69:0/1806318035 shutdown_connections 2024-05-14T17:03:07.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108920 0x7f6b8c075ea0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b8c103850 0x7f6b8c077430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6b780b2a70 0x7f6b780b4f30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 --2- 172.21.15.69:0/1806318035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b8c104200 0x7f6b8c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 -- 172.21.15.69:0/1806318035 >> 172.21.15.69:0/1806318035 conn(0x7f6b8c07a5d0 msgr2=0x7f6b8c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:08.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 -- 172.21.15.69:0/1806318035 shutdown_connections 2024-05-14T17:03:08.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:02:54.704+0000 7f6b936f9700 1 -- 172.21.15.69:0/1806318035 wait complete. 2024-05-14T17:03:08.281 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 52 2024-05-14T17:03:08.332 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:54 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1806318035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:03:08.382 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 745.0 s, 1420.0 tps, lat 70.157 ms stddev 81.167 2024-05-14T17:03:08.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:05 smithi203 ceph-mon[35787]: pgmap v416: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.88k op/s 2024-05-14T17:03:08.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:05 smithi092 ceph-mon[35763]: pgmap v416: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.88k op/s 2024-05-14T17:03:08.548 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:07 smithi092 ceph-mon[35763]: pgmap v417: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.93k op/s 2024-05-14T17:03:08.600 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 149.65 --await-for 45.0' 2024-05-14T17:03:08.649 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 149.65 --await-for 45.0 2024-05-14T17:03:08.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:55 smithi069 ceph-mon[31507]: pgmap v410: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-05-14T17:03:08.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:07 smithi203 ceph-mon[35787]: pgmap v417: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.93k op/s 2024-05-14T17:03:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.334+0000 7ff19da6d640 1 -- 172.21.15.69:0/1514035100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff198107940 msgr2=0x7ff198107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.334+0000 7ff19da6d640 1 --2- 172.21.15.69:0/1514035100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff198107940 0x7ff198107d20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff190009ca0 tx=0x7ff19002a8d0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.335+0000 7ff19da6d640 1 -- 172.21.15.69:0/1514035100 shutdown_connections 2024-05-14T17:03:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.335+0000 7ff19da6d640 1 --2- 172.21.15.69:0/1514035100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff198102340 0x7ff19810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.335+0000 7ff19da6d640 1 --2- 172.21.15.69:0/1514035100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff198101980 0x7ff198101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.335+0000 7ff19da6d640 1 --2- 172.21.15.69:0/1514035100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff198107940 0x7ff198107d20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.335+0000 7ff19da6d640 1 -- 172.21.15.69:0/1514035100 >> 172.21.15.69:0/1514035100 conn(0x7ff198078ad0 msgr2=0x7ff198078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.335+0000 7ff19da6d640 1 -- 172.21.15.69:0/1514035100 shutdown_connections 2024-05-14T17:03:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.335+0000 7ff19da6d640 1 -- 172.21.15.69:0/1514035100 wait complete. 2024-05-14T17:03:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.336+0000 7ff19da6d640 1 Processor -- start 2024-05-14T17:03:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.338+0000 7ff19da6d640 1 -- start start 2024-05-14T17:03:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.338+0000 7ff19da6d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff198101980 0x7ff1981a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.338+0000 7ff19da6d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff198102340 0x7ff1981a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.338+0000 7ff19da6d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff198107940 0x7ff19819f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.338+0000 7ff19da6d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff198111c70 con 0x7ff198107940 2024-05-14T17:03:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.338+0000 7ff19da6d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff198111af0 con 0x7ff198102340 2024-05-14T17:03:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.338+0000 7ff19da6d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff198111df0 con 0x7ff198101980 2024-05-14T17:03:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.339+0000 7ff197fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff198107940 0x7ff19819f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.339+0000 7ff197fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff198107940 0x7ff19819f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34818/0 (socket says 172.21.15.69:34818) 2024-05-14T17:03:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.339+0000 7ff197fff640 1 -- 172.21.15.69:0/2659509913 learned_addr learned my addr 172.21.15.69:0/2659509913 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.340+0000 7ff1977fe640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff198101980 0x7ff1981a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.340+0000 7ff197fff640 1 -- 172.21.15.69:0/2659509913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff198101980 msgr2=0x7ff1981a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.340+0000 7ff197fff640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff198101980 0x7ff1981a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.340+0000 7ff197fff640 1 -- 172.21.15.69:0/2659509913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff198102340 msgr2=0x7ff1981a5b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T17:03:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.340+0000 7ff197fff640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff198102340 0x7ff1981a5b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.340+0000 7ff197fff640 1 -- 172.21.15.69:0/2659509913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1981a0180 con 0x7ff198107940 2024-05-14T17:03:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.340+0000 7ff1977fe640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff198101980 0x7ff1981a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.341+0000 7ff197fff640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff198107940 0x7ff19819f950 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff18800b3f0 tx=0x7ff18800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.342+0000 7ff194ff9640 1 -- 172.21.15.69:0/2659509913 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188015020 con 0x7ff198107940 2024-05-14T17:03:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.342+0000 7ff194ff9640 1 -- 172.21.15.69:0/2659509913 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff188011ce0 con 0x7ff198107940 2024-05-14T17:03:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.342+0000 7ff19da6d640 1 -- 172.21.15.69:0/2659509913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1981a0470 con 0x7ff198107940 2024-05-14T17:03:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.342+0000 7ff194ff9640 1 -- 172.21.15.69:0/2659509913 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188010e50 con 0x7ff198107940 2024-05-14T17:03:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.342+0000 7ff19da6d640 1 -- 172.21.15.69:0/2659509913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1981ac500 con 0x7ff198107940 2024-05-14T17:03:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.345+0000 7ff194ff9640 1 -- 172.21.15.69:0/2659509913 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff188018020 con 0x7ff198107940 2024-05-14T17:03:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.345+0000 7ff1a0782740 1 Processor -- start 2024-05-14T17:03:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.345+0000 7ff1a0782740 1 -- start start 2024-05-14T17:03:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff194ff9640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff1800b2d00 0x7ff1800b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605ef8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 0x55d605f03590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 0x55d605f05ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff1977fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605ef8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff196ffd640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff1800b2d00 0x7ff1800b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff1977fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605ef8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:52750/0 (socket says 172.21.15.69:52750) 2024-05-14T17:03:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff1977fe640 1 -- 172.21.15.69:0/3604859876 learned_addr learned my addr 172.21.15.69:0/3604859876 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff197fff640 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 0x55d605f05ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff196ffd640 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 0x55d605f03590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff1a0782740 1 -- 172.21.15.69:0/3604859876 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d605e6bc30 con 0x55d605f03ad0 2024-05-14T17:03:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff194ff9640 1 -- 172.21.15.69:0/2659509913 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7ff1880e1de0 con 0x7ff198107940 2024-05-14T17:03:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff1a0782740 1 -- 172.21.15.69:0/3604859876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d605e5b7d0 con 0x55d605ef88f0 2024-05-14T17:03:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.346+0000 7ff1a0782740 1 -- 172.21.15.69:0/3604859876 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d605e5f810 con 0x55d605ef9210 2024-05-14T17:03:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.347+0000 7ff1977fe640 1 -- 172.21.15.69:0/3604859876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 msgr2=0x55d605f03590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.347+0000 7ff1977fe640 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 0x55d605f03590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.347+0000 7ff1977fe640 1 -- 172.21.15.69:0/3604859876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 msgr2=0x55d605f05ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.347+0000 7ff1977fe640 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 0x55d605f05ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.347+0000 7ff1977fe640 1 -- 172.21.15.69:0/3604859876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d605b70210 con 0x55d605ef88f0 2024-05-14T17:03:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.347+0000 7ff196ffd640 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 0x55d605f03590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.347+0000 7ff197fff640 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 0x55d605f05ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.347+0000 7ff196ffd640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff1800b2d00 0x7ff1800b51c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009460 tx=0x7ff18c0099a0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.347+0000 7ff1977fe640 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605ef8cd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff190035000 tx=0x7ff1900098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a186e472b7fea98 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.347+0000 7ff17dffb640 1 -- 172.21.15.69:0/3604859876 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190031d70 con 0x55d605ef88f0 2024-05-14T17:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.348+0000 7ff17dffb640 1 -- 172.21.15.69:0/3604859876 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff190036040 con 0x55d605ef88f0 2024-05-14T17:03:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.348+0000 7ff17dffb640 1 -- 172.21.15.69:0/3604859876 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190004280 con 0x55d605ef88f0 2024-05-14T17:03:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.348+0000 7ff1a0782740 1 -- 172.21.15.69:0/3604859876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 msgr2=0x55d605ef8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.348+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605ef8cd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff190035000 tx=0x7ff1900098b0 comp rx=0 tx=0).stop 2024-05-14T17:03:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.349+0000 7ff1a0782740 1 -- 172.21.15.69:0/3604859876 shutdown_connections 2024-05-14T17:03:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.349+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 0x55d605f05ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.349+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 0x55d605f03590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.349+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3604859876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605ef8cd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.349+0000 7ff1a0782740 1 -- 172.21.15.69:0/3604859876 >> 172.21.15.69:0/3604859876 conn(0x55d605ef4e00 msgr2=0x55d605ef51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.349+0000 7ff1a0782740 1 -- 172.21.15.69:0/3604859876 shutdown_connections 2024-05-14T17:03:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.351+0000 7ff1a0782740 1 -- 172.21.15.69:0/3604859876 wait complete. 2024-05-14T17:03:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff1a0782740 1 -- start start 2024-05-14T17:03:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff1a0782740 1 Processor -- start 2024-05-14T17:03:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f03430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 0x55d605f00720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605f03ad0 0x55d605f00c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff1a0782740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d605b41790 con 0x55d605ef9210 2024-05-14T17:03:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff1a0782740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d605e17890 con 0x55d605ef88f0 2024-05-14T17:03:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff1a0782740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d605f06740 con 0x55d605f03ad0 2024-05-14T17:03:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff1a0782740 10 client.0 _ll_register_callbacks cb 0x55d605e1ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff1977fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f03430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.352+0000 7ff196ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 0x55d605f00720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff1977fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f03430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:52762/0 (socket says 172.21.15.69:52762) 2024-05-14T17:03:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff1977fe640 1 -- 172.21.15.69:0/2207795620 learned_addr learned my addr 172.21.15.69:0/2207795620 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff197fff640 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605f03ad0 0x55d605f00c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff1977fe640 1 -- 172.21.15.69:0/2207795620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605f03ad0 msgr2=0x55d605f00c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff1977fe640 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605f03ad0 0x55d605f00c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff1977fe640 1 -- 172.21.15.69:0/2207795620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 msgr2=0x55d605f00720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff1977fe640 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 0x55d605f00720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:13.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff1977fe640 1 -- 172.21.15.69:0/2207795620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d605c3c620 con 0x55d605ef88f0 2024-05-14T17:03:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff1977fe640 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f03430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff19002adf0 tx=0x7ff19002a8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f4c701ce9b0aff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:03:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff176ffd640 1 -- 172.21.15.69:0/2207795620 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190031460 con 0x55d605ef88f0 2024-05-14T17:03:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff176ffd640 1 -- 172.21.15.69:0/2207795620 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7ff190031600 con 0x55d605ef88f0 2024-05-14T17:03:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.353+0000 7ff1a0782740 1 -- 172.21.15.69:0/2207795620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff190036040 con 0x55d605ef88f0 2024-05-14T17:03:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff176ffd640 1 -- 172.21.15.69:0/2207795620 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff19005c050 con 0x55d605ef88f0 2024-05-14T17:03:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff1a0782740 10 client.26186 fetch_fsmap learned FSMap version 52 2024-05-14T17:03:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff1a0782740 1 -- 172.21.15.69:0/2207795620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55d605f11f30 con 0x55d605ef88f0 2024-05-14T17:03:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff176ffd640 1 -- 172.21.15.69:0/2207795620 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff190036040 con 0x55d605ef88f0 2024-05-14T17:03:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff176ffd640 1 -- 172.21.15.69:0/2207795620 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7ff1900305f0 con 0x55d605ef88f0 2024-05-14T17:03:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff1a0782740 10 client.26186 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:03:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff1a0782740 10 client.26186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff1a0782740 20 client.26186 populate_metadata read hostname 'smithi069' 2024-05-14T17:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff1a0782740 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d605f13280 0x55d605f15670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff1a0782740 4 client.26186 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff1a0782740 1 -- 172.21.15.69:0/2207795620 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d605e6db80 con 0x55d605f13280 2024-05-14T17:03:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.354+0000 7ff196ffd640 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d605f13280 0x55d605f15670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.355+0000 7ff176ffd640 10 client.26186 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.355+0000 7ff196ffd640 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d605f13280 0x55d605f15670 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c5000147f42ecb server_cookie=c2471a8e0b27aed1 in_seq=0 out_seq=0 2024-05-14T17:03:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.355+0000 7ff176ffd640 1 -- 172.21.15.69:0/2207795620 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d605e6db80 con 0x55d605f13280 2024-05-14T17:03:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.355+0000 7ff176ffd640 10 client.26186 handle_command_reply: tid=0 2024-05-14T17:03:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.355+0000 7ff1a0782740 1 client.26186 shutdown 2024-05-14T17:03:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.355+0000 7ff1a0782740 1 -- 172.21.15.69:0/2207795620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 msgr2=0x55d605f03430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.355+0000 7ff1a0782740 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f03430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff19002adf0 tx=0x7ff19002a8d0 comp rx=0 tx=0).stop 2024-05-14T17:03:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.356+0000 7ff1a0782740 1 -- 172.21.15.69:0/2207795620 shutdown_connections 2024-05-14T17:03:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.356+0000 7ff1a0782740 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d605f13280 0x55d605f15670 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.356+0000 7ff176ffd640 0 client.26186 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.356+0000 7ff1a0782740 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605f03ad0 0x55d605f00c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.356+0000 7ff1a0782740 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 0x55d605f00720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.356+0000 7ff1a0782740 1 --2- 172.21.15.69:0/2207795620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f03430 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.356+0000 7ff1a0782740 1 -- 172.21.15.69:0/2207795620 >> 172.21.15.69:0/2207795620 conn(0x55d605ef4e00 msgr2=0x55d605ef4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.356+0000 7ff1a0782740 1 -- 172.21.15.69:0/2207795620 shutdown_connections 2024-05-14T17:03:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.356+0000 7ff1a0782740 1 -- 172.21.15.69:0/2207795620 wait complete. 2024-05-14T17:03:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.356+0000 7ff1a0782740 20 client.26186 trim_cache size 0 max 16384 2024-05-14T17:03:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.378+0000 7ff1a0782740 1 Processor -- start 2024-05-14T17:03:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.378+0000 7ff1a0782740 1 -- start start 2024-05-14T17:03:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.378+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f02f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.378+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 0x55d605efa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.378+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 0x55d605f1ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff1a0782740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1981b75c0 con 0x55d605f03ad0 2024-05-14T17:03:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff1a0782740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1981b4e00 con 0x55d605ef88f0 2024-05-14T17:03:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff1a0782740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff1981b2c70 con 0x55d605ef9210 2024-05-14T17:03:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff1977fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f02f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff1977fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f02f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:52764/0 (socket says 172.21.15.69:52764) 2024-05-14T17:03:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff197fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 0x55d605f1ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff1977fe640 1 -- 172.21.15.69:0/3272909352 learned_addr learned my addr 172.21.15.69:0/3272909352 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff196ffd640 1 --2- 172.21.15.69:0/3272909352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 0x55d605efa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff197fff640 1 -- 172.21.15.69:0/3272909352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 msgr2=0x55d605efa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff197fff640 1 --2- 172.21.15.69:0/3272909352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 0x55d605efa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff197fff640 1 -- 172.21.15.69:0/3272909352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 msgr2=0x55d605f02f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff197fff640 1 --2- 172.21.15.69:0/3272909352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f02f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.379+0000 7ff197fff640 1 -- 172.21.15.69:0/3272909352 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d605f159e0 con 0x55d605f03ad0 2024-05-14T17:03:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.380+0000 7ff197fff640 1 --2- 172.21.15.69:0/3272909352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 0x55d605f1ac70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff1880df5d0 tx=0x7ff1880a8dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5da1dd7ae55f6c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.380+0000 7ff17d7fa640 1 -- 172.21.15.69:0/3272909352 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1880a9160 con 0x55d605f03ad0 2024-05-14T17:03:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.380+0000 7ff17d7fa640 1 -- 172.21.15.69:0/3272909352 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1880ad510 con 0x55d605f03ad0 2024-05-14T17:03:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.380+0000 7ff17d7fa640 1 -- 172.21.15.69:0/3272909352 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1880df740 con 0x55d605f03ad0 2024-05-14T17:03:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.381+0000 7ff1a0782740 1 -- 172.21.15.69:0/3272909352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 msgr2=0x55d605f1ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.381+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3272909352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 0x55d605f1ac70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff1880df5d0 tx=0x7ff1880a8dc0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.381+0000 7ff1a0782740 1 -- 172.21.15.69:0/3272909352 shutdown_connections 2024-05-14T17:03:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.381+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3272909352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605f03ad0 0x55d605f1ac70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.381+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3272909352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef9210 0x55d605efa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.381+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3272909352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605ef88f0 0x55d605f02f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.381+0000 7ff1a0782740 1 -- 172.21.15.69:0/3272909352 >> 172.21.15.69:0/3272909352 conn(0x55d605ef4e00 msgr2=0x55d605efa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.381+0000 7ff1a0782740 1 -- 172.21.15.69:0/3272909352 shutdown_connections 2024-05-14T17:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.381+0000 7ff1a0782740 1 -- 172.21.15.69:0/3272909352 wait complete. 2024-05-14T17:03:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.381+0000 7ff1a0782740 1 -- start start 2024-05-14T17:03:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1a0782740 1 Processor -- start 2024-05-14T17:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef88f0 0x55d605f0bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 0x55d605f06fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1a0782740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605f03ad0 0x55d605f07510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1a0782740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1981b32c0 con 0x55d605ef9210 2024-05-14T17:03:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1a0782740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1981b4730 con 0x55d605f03ad0 2024-05-14T17:03:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1a0782740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff1981b3990 con 0x55d605ef88f0 2024-05-14T17:03:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1977fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef88f0 0x55d605f0bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1977fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef88f0 0x55d605f0bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:53994/0 (socket says 172.21.15.69:53994) 2024-05-14T17:03:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1977fe640 1 -- 172.21.15.69:0/3087696114 learned_addr learned my addr 172.21.15.69:0/3087696114 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff196ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 0x55d605f06fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff1a0782740 10 client.0 _ll_register_callbacks cb 0x7ff190009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.382+0000 7ff197fff640 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605f03ad0 0x55d605f07510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff1977fe640 1 -- 172.21.15.69:0/3087696114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605f03ad0 msgr2=0x55d605f07510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff1977fe640 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605f03ad0 0x55d605f07510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff1977fe640 1 -- 172.21.15.69:0/3087696114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 msgr2=0x55d605f06fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff1977fe640 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 0x55d605f06fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff1977fe640 1 -- 172.21.15.69:0/3087696114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d605f07a80 con 0x55d605ef88f0 2024-05-14T17:03:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff197fff640 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605f03ad0 0x55d605f07510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff196ffd640 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 0x55d605f06fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff1977fe640 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef88f0 0x55d605f0bc60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff190030a40 tx=0x7ff190030800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da10faeab952391a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T17:03:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff176ffd640 1 -- 172.21.15.69:0/3087696114 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190031af0 con 0x55d605ef88f0 2024-05-14T17:03:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff176ffd640 1 -- 172.21.15.69:0/3087696114 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7ff190030e40 con 0x55d605ef88f0 2024-05-14T17:03:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff1a0782740 1 -- 172.21.15.69:0/3087696114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1880ad510 con 0x55d605ef88f0 2024-05-14T17:03:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff176ffd640 1 -- 172.21.15.69:0/3087696114 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff19006d050 con 0x55d605ef88f0 2024-05-14T17:03:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff1a0782740 10 client.26083 fetch_fsmap learned FSMap version 52 2024-05-14T17:03:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.383+0000 7ff1a0782740 1 -- 172.21.15.69:0/3087696114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55d605f7d590 con 0x55d605ef88f0 2024-05-14T17:03:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.384+0000 7ff176ffd640 1 -- 172.21.15.69:0/3087696114 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff190036040 con 0x55d605ef88f0 2024-05-14T17:03:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.384+0000 7ff176ffd640 1 -- 172.21.15.69:0/3087696114 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7ff190068030 con 0x55d605ef88f0 2024-05-14T17:03:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.385+0000 7ff1a0782740 10 client.26083 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:03:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.385+0000 7ff1a0782740 10 client.26083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.385+0000 7ff1a0782740 20 client.26083 populate_metadata read hostname 'smithi069' 2024-05-14T17:03:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.385+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d605f13280 0x55d605f7fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.385+0000 7ff1a0782740 4 client.26083 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:03:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.385+0000 7ff1a0782740 1 -- 172.21.15.69:0/3087696114 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff1880ad510 con 0x55d605f13280 2024-05-14T17:03:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.387+0000 7ff196ffd640 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d605f13280 0x55d605f7fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.387+0000 7ff196ffd640 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d605f13280 0x55d605f7fd70 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09e5b21166afce5 server_cookie=2601668a33e0ec2c in_seq=0 out_seq=0 2024-05-14T17:03:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.387+0000 7ff176ffd640 10 client.26083 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff176ffd640 1 -- 172.21.15.69:0/3087696114 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7ff1880ad510 con 0x55d605f13280 2024-05-14T17:03:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff176ffd640 10 client.26083 handle_command_reply: tid=0 2024-05-14T17:03:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff1a0782740 1 client.26083 shutdown 2024-05-14T17:03:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff1a0782740 1 -- 172.21.15.69:0/3087696114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef88f0 msgr2=0x55d605f0bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef88f0 0x55d605f0bc60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff190030a40 tx=0x7ff190030800 comp rx=0 tx=0).stop 2024-05-14T17:03:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff1a0782740 1 -- 172.21.15.69:0/3087696114 shutdown_connections 2024-05-14T17:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d605f13280 0x55d605f7fd70 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d605f03ad0 0x55d605f07510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff176ffd640 0 client.26083 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d605ef9210 0x55d605f06fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff1a0782740 1 --2- 172.21.15.69:0/3087696114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d605ef88f0 0x55d605f0bc60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.399+0000 7ff1a0782740 1 -- 172.21.15.69:0/3087696114 >> 172.21.15.69:0/3087696114 conn(0x55d605ef4e00 msgr2=0x55d605efab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.400+0000 7ff1a0782740 1 -- 172.21.15.69:0/3087696114 shutdown_connections 2024-05-14T17:03:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.400+0000 7ff1a0782740 1 -- 172.21.15.69:0/3087696114 wait complete. 2024-05-14T17:03:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.400+0000 7ff1a0782740 20 client.26083 trim_cache size 0 max 16384 2024-05-14T17:03:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:03:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:03:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:03:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 -- 172.21.15.69:0/2659509913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff1800b2d00 msgr2=0x7ff1800b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff1800b2d00 0x7ff1800b51c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009460 tx=0x7ff18c0099a0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 -- 172.21.15.69:0/2659509913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff198107940 msgr2=0x7ff19819f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff198107940 0x7ff19819f950 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff18800b3f0 tx=0x7ff18800b8b0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 -- 172.21.15.69:0/2659509913 shutdown_connections 2024-05-14T17:03:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff1800b2d00 0x7ff1800b51c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff198107940 0x7ff19819f950 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff198102340 0x7ff1981a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 --2- 172.21.15.69:0/2659509913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff198101980 0x7ff1981a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 -- 172.21.15.69:0/2659509913 >> 172.21.15.69:0/2659509913 conn(0x7ff198078ad0 msgr2=0x7ff19806a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 -- 172.21.15.69:0/2659509913 shutdown_connections 2024-05-14T17:03:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:02:56.401+0000 7ff19da6d640 1 -- 172.21.15.69:0/2659509913 wait complete. 2024-05-14T17:03:21.864 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:57 smithi069 ceph-mon[31507]: pgmap v411: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 786 op/s 2024-05-14T17:03:21.913 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:02:59 smithi069 ceph-mon[31507]: pgmap v412: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-05-14T17:03:21.981 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 750.0 s, 1805.6 tps, lat 55.473 ms stddev 58.330 2024-05-14T17:03:22.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:09 smithi092 ceph-mon[35763]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T17:03:22.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:09 smithi092 ceph-mon[35763]: pgmap v419: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 26 MiB/s wr, 2.41k op/s 2024-05-14T17:03:22.139 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:11 smithi092 ceph-mon[35763]: pgmap v420: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 418 op/s 2024-05-14T17:03:22.197 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:13 smithi092 ceph-mon[35763]: pgmap v421: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.0 MiB/s wr, 737 op/s 2024-05-14T17:03:22.247 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:15 smithi092 ceph-mon[35763]: pgmap v422: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-05-14T17:03:22.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:17 smithi092 ceph-mon[35763]: pgmap v423: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 616 op/s 2024-05-14T17:03:22.347 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:19 smithi092 ceph-mon[35763]: pgmap v424: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 612 op/s 2024-05-14T17:03:22.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:20 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:22.447 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:09 smithi203 ceph-mon[35787]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T17:03:22.497 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:09 smithi203 ceph-mon[35787]: pgmap v419: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 26 MiB/s wr, 2.41k op/s 2024-05-14T17:03:22.547 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:11 smithi203 ceph-mon[35787]: pgmap v420: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 418 op/s 2024-05-14T17:03:22.597 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:13 smithi203 ceph-mon[35787]: pgmap v421: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.0 MiB/s wr, 737 op/s 2024-05-14T17:03:22.647 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:15 smithi203 ceph-mon[35787]: pgmap v422: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-05-14T17:03:22.697 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:17 smithi203 ceph-mon[35787]: pgmap v423: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 616 op/s 2024-05-14T17:03:22.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:19 smithi203 ceph-mon[35787]: pgmap v424: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 612 op/s 2024-05-14T17:03:22.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:20 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:22.864 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:01 smithi069 ceph-mon[31507]: pgmap v413: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-05-14T17:03:22.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:03 smithi069 ceph-mon[31507]: pgmap v414: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 20 MiB/s wr, 1.85k op/s 2024-05-14T17:03:22.965 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 755.0 s, 1273.0 tps, lat 78.215 ms stddev 132.608 2024-05-14T17:03:23.015 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:21 smithi092 ceph-mon[35763]: pgmap v425: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 513 op/s 2024-05-14T17:03:23.068 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:21 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:23.139 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:21 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:23.189 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:21 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:03:23.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:21 smithi203 ceph-mon[35787]: pgmap v425: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 513 op/s 2024-05-14T17:03:23.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:21 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:23.339 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:21 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:23.389 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:21 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:03:23.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:04 smithi069 ceph-mon[31507]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T17:03:23.489 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:05 smithi069 ceph-mon[31507]: pgmap v416: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.88k op/s 2024-05-14T17:03:23.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:07 smithi069 ceph-mon[31507]: pgmap v417: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.93k op/s 2024-05-14T17:03:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.510+0000 7f71eb445640 1 -- 172.21.15.69:0/2334119531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71e4101cc0 msgr2=0x7f71e4105d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.510+0000 7f71eb445640 1 --2- 172.21.15.69:0/2334119531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71e4101cc0 0x7f71e4105d00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009900 tx=0x7f71d402a410 comp rx=0 tx=0).stop 2024-05-14T17:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.511+0000 7f71eb445640 1 -- 172.21.15.69:0/2334119531 shutdown_connections 2024-05-14T17:03:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.511+0000 7f71eb445640 1 --2- 172.21.15.69:0/2334119531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e41063e0 0x7f71e410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.511+0000 7f71eb445640 1 --2- 172.21.15.69:0/2334119531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71e4101cc0 0x7f71e4105d00 secure :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009900 tx=0x7f71d402a410 comp rx=0 tx=0).stop 2024-05-14T17:03:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.511+0000 7f71eb445640 1 --2- 172.21.15.69:0/2334119531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71e4101310 0x7f71e41016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.511+0000 7f71eb445640 1 -- 172.21.15.69:0/2334119531 >> 172.21.15.69:0/2334119531 conn(0x7f71e4077ee0 msgr2=0x7f71e40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.511+0000 7f71eb445640 1 -- 172.21.15.69:0/2334119531 shutdown_connections 2024-05-14T17:03:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.511+0000 7f71eb445640 1 -- 172.21.15.69:0/2334119531 wait complete. 2024-05-14T17:03:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71eb445640 1 Processor -- start 2024-05-14T17:03:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71eb445640 1 -- start start 2024-05-14T17:03:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71eb445640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71e4101310 0x7f71e4199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71eb445640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71e41063e0 0x7f71e4199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:24.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71eb445640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e419dce0 0x7f71e419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71eb445640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4110ac0 con 0x7f71e4101310 2024-05-14T17:03:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71eb445640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4110940 con 0x7f71e419dce0 2024-05-14T17:03:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71eb445640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4110c40 con 0x7f71e41063e0 2024-05-14T17:03:24.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e91e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71e4101310 0x7f71e4199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e99e2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e419dce0 0x7f71e419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e91e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71e4101310 0x7f71e4199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37148/0 (socket says 172.21.15.69:37148) 2024-05-14T17:03:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e91e1640 1 -- 172.21.15.69:0/3665586291 learned_addr learned my addr 172.21.15.69:0/3665586291 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e89e0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71e41063e0 0x7f71e4199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e99e2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e419dce0 0x7f71e419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:41624/0 (socket says 172.21.15.69:41624) 2024-05-14T17:03:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e99e2640 1 -- 172.21.15.69:0/3665586291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71e41063e0 msgr2=0x7f71e4199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e99e2640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71e41063e0 0x7f71e4199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e99e2640 1 -- 172.21.15.69:0/3665586291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71e4101310 msgr2=0x7f71e4199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e99e2640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71e4101310 0x7f71e4199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e99e2640 1 -- 172.21.15.69:0/3665586291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e419e7d0 con 0x7f71e419dce0 2024-05-14T17:03:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e91e1640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71e4101310 0x7f71e4199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.512+0000 7f71e89e0640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71e41063e0 0x7f71e4199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.513+0000 7f71e99e2640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e419dce0 0x7f71e419e190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00e970 tx=0x7f71dc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.513+0000 7f71da7fc640 1 -- 172.21.15.69:0/3665586291 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc00ccc0 con 0x7f71e419dce0 2024-05-14T17:03:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.513+0000 7f71eb445640 1 -- 172.21.15.69:0/3665586291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e419eac0 con 0x7f71e419dce0 2024-05-14T17:03:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.513+0000 7f71da7fc640 1 -- 172.21.15.69:0/3665586291 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71dc0035f0 con 0x7f71e419dce0 2024-05-14T17:03:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.513+0000 7f71da7fc640 1 -- 172.21.15.69:0/3665586291 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc00ce60 con 0x7f71e419dce0 2024-05-14T17:03:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.513+0000 7f71eb445640 1 -- 172.21.15.69:0/3665586291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e41a6230 con 0x7f71e419dce0 2024-05-14T17:03:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.516+0000 7f71ee239740 1 Processor -- start 2024-05-14T17:03:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.516+0000 7f71da7fc640 1 -- 172.21.15.69:0/3665586291 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f71dc004020 con 0x7f71e419dce0 2024-05-14T17:03:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71ee239740 1 -- start start 2024-05-14T17:03:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71ee239740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71ee239740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71ee239740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a9d20 0x55cd128aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71ee239740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cd1280ec30 con 0x55cd128a5610 2024-05-14T17:03:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71ee239740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cd127fe7d0 con 0x55cd128a5fc0 2024-05-14T17:03:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71ee239740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cd12802810 con 0x55cd128a9d20 2024-05-14T17:03:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e89e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e89e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37152/0 (socket says 172.21.15.69:37152) 2024-05-14T17:03:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e89e0640 1 -- 172.21.15.69:0/265653632 learned_addr learned my addr 172.21.15.69:0/265653632 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e91e1640 1 --2- 172.21.15.69:0/265653632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e89e0640 1 --2- 172.21.15.69:0/265653632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a9d20 0x55cd128aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e89e0640 1 -- 172.21.15.69:0/265653632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a9d20 msgr2=0x55cd128aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e89e0640 1 --2- 172.21.15.69:0/265653632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a9d20 0x55cd128aa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e89e0640 1 -- 172.21.15.69:0/265653632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 msgr2=0x55cd128a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e89e0640 1 --2- 172.21.15.69:0/265653632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e89e0640 1 -- 172.21.15.69:0/265653632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd12513210 con 0x55cd128a5610 2024-05-14T17:03:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.517+0000 7f71e91e1640 1 --2- 172.21.15.69:0/265653632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.518+0000 7f71e89e0640 1 --2- 172.21.15.69:0/265653632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a59f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f71d4031440 tx=0x7f71d4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c2d65932033367 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.518+0000 7f71da7fc640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f71d00b2a10 0x7f71d00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.518+0000 7f71cf7fe640 1 -- 172.21.15.69:0/265653632 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4044020 con 0x55cd128a5610 2024-05-14T17:03:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.518+0000 7f71e91e1640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f71d00b2a10 0x7f71d00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.518+0000 7f71da7fc640 1 -- 172.21.15.69:0/3665586291 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f71dc0e27d0 con 0x7f71e419dce0 2024-05-14T17:03:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.518+0000 7f71cf7fe640 1 -- 172.21.15.69:0/265653632 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d4005040 con 0x55cd128a5610 2024-05-14T17:03:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.518+0000 7f71cf7fe640 1 -- 172.21.15.69:0/265653632 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4035040 con 0x55cd128a5610 2024-05-14T17:03:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71ee239740 1 -- 172.21.15.69:0/265653632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 msgr2=0x55cd128a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71e91e1640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f71d00b2a10 0x7f71d00b4ed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55cd128a6400 tx=0x7f71e0010040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71ee239740 1 --2- 172.21.15.69:0/265653632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a59f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f71d4031440 tx=0x7f71d4000bc0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71ee239740 1 -- 172.21.15.69:0/265653632 shutdown_connections 2024-05-14T17:03:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71ee239740 1 --2- 172.21.15.69:0/265653632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a9d20 0x55cd128aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71ee239740 1 --2- 172.21.15.69:0/265653632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71ee239740 1 --2- 172.21.15.69:0/265653632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a59f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71ee239740 1 -- 172.21.15.69:0/265653632 >> 172.21.15.69:0/265653632 conn(0x55cd128978b0 msgr2=0x55cd12897c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71ee239740 1 -- 172.21.15.69:0/265653632 shutdown_connections 2024-05-14T17:03:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71ee239740 1 -- 172.21.15.69:0/265653632 wait complete. 2024-05-14T17:03:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.519+0000 7f71ee239740 1 -- start start 2024-05-14T17:03:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.520+0000 7f71ee239740 1 Processor -- start 2024-05-14T17:03:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.520+0000 7f71ee239740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 0x55cd128b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.520+0000 7f71ee239740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.520+0000 7f71ee239740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a9d20 0x55cd128aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.520+0000 7f71ee239740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cd124e4790 con 0x55cd128a9d20 2024-05-14T17:03:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.520+0000 7f71ee239740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cd127ba890 con 0x55cd128a5fc0 2024-05-14T17:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.520+0000 7f71ee239740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cd128b1f10 con 0x55cd128a5610 2024-05-14T17:03:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.520+0000 7f71ee239740 10 client.0 _ll_register_callbacks cb 0x55cd127bdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71e91e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 0x55cd128b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71e91e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 0x55cd128b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:52786/0 (socket says 172.21.15.69:52786) 2024-05-14T17:03:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71e91e1640 1 -- 172.21.15.69:0/206978338 learned_addr learned my addr 172.21.15.69:0/206978338 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71e99e2640 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a9d20 0x55cd128aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71e91e1640 1 -- 172.21.15.69:0/206978338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 msgr2=0x55cd128ae5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:03:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71e91e1640 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128ae5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71e91e1640 1 -- 172.21.15.69:0/206978338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a9d20 msgr2=0x55cd128aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71e91e1640 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a9d20 0x55cd128aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71e91e1640 1 -- 172.21.15.69:0/206978338 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd125df620 con 0x55cd128a5610 2024-05-14T17:03:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71e99e2640 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a9d20 0x55cd128aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.521+0000 7f71ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T17:03:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.522+0000 7f71e91e1640 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 0x55cd128b12c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71e001cc20 tx=0x7f71e0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed77fff88cafed0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.522+0000 7f71ccff9640 1 -- 172.21.15.69:0/206978338 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e002a060 con 0x55cd128a5610 2024-05-14T17:03:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.522+0000 7f71ccff9640 1 -- 172.21.15.69:0/206978338 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f71e0009040 con 0x55cd128a5610 2024-05-14T17:03:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.522+0000 7f71ee239740 1 -- 172.21.15.69:0/206978338 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d4005040 con 0x55cd128a5610 2024-05-14T17:03:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.522+0000 7f71ccff9640 1 -- 172.21.15.69:0/206978338 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f71e001c0d0 con 0x55cd128a5610 2024-05-14T17:03:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.522+0000 7f71ee239740 10 client.26095 fetch_fsmap learned FSMap version 52 2024-05-14T17:03:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.522+0000 7f71ee239740 1 -- 172.21.15.69:0/206978338 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55cd1289e310 con 0x55cd128a5610 2024-05-14T17:03:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.522+0000 7f71ccff9640 1 -- 172.21.15.69:0/206978338 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d4005040 con 0x55cd128a5610 2024-05-14T17:03:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.522+0000 7f71ccff9640 1 -- 172.21.15.69:0/206978338 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f71e000b040 con 0x55cd128a5610 2024-05-14T17:03:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.523+0000 7f71ee239740 10 client.26095 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:03:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.523+0000 7f71ee239740 10 client.26095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.523+0000 7f71ee239740 20 client.26095 populate_metadata read hostname 'smithi069' 2024-05-14T17:03:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.523+0000 7f71ee239740 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd1289f820 0x55cd128a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.523+0000 7f71ee239740 4 client.26095 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.523+0000 7f71ee239740 1 -- 172.21.15.69:0/206978338 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd12810b80 con 0x55cd1289f820 2024-05-14T17:03:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.523+0000 7f71e89e0640 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd1289f820 0x55cd128a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.523+0000 7f71ccff9640 10 client.26095 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.523+0000 7f71e89e0640 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd1289f820 0x55cd128a1cc0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35fcb2ff6aa53708 server_cookie=3229e60cf0d5dd83 in_seq=0 out_seq=0 2024-05-14T17:03:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ccff9640 1 -- 172.21.15.69:0/206978338 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cd12810b80 con 0x55cd1289f820 2024-05-14T17:03:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ccff9640 10 client.26095 handle_command_reply: tid=0 2024-05-14T17:03:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 client.26095 shutdown 2024-05-14T17:03:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 -- 172.21.15.69:0/206978338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 msgr2=0x55cd128b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:29.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 0x55cd128b12c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71e001cc20 tx=0x7f71e0004ab0 comp rx=0 tx=0).stop 2024-05-14T17:03:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 -- 172.21.15.69:0/206978338 shutdown_connections 2024-05-14T17:03:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd1289f820 0x55cd128a1cc0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a9d20 0x55cd128aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ccff9640 0 client.26095 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 --2- 172.21.15.69:0/206978338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 0x55cd128b12c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 -- 172.21.15.69:0/206978338 >> 172.21.15.69:0/206978338 conn(0x55cd128978b0 msgr2=0x55cd128a4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 -- 172.21.15.69:0/206978338 shutdown_connections 2024-05-14T17:03:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 1 -- 172.21.15.69:0/206978338 wait complete. 2024-05-14T17:03:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.524+0000 7f71ee239740 20 client.26095 trim_cache size 0 max 16384 2024-05-14T17:03:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71ee239740 1 Processor -- start 2024-05-14T17:03:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71ee239740 1 -- start start 2024-05-14T17:03:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71ee239740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd1289f820 0x55cd128a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71ee239740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71ee239740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128a8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71ee239740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41af3a0 con 0x55cd128a5610 2024-05-14T17:03:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71ee239740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41acbe0 con 0x55cd128a5fc0 2024-05-14T17:03:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71ee239740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41aaa50 con 0x55cd1289f820 2024-05-14T17:03:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71e89e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71e89e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37168/0 (socket says 172.21.15.69:37168) 2024-05-14T17:03:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71e89e0640 1 -- 172.21.15.69:0/2134289256 learned_addr learned my addr 172.21.15.69:0/2134289256 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71e99e2640 1 --2- 172.21.15.69:0/2134289256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128a8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.545+0000 7f71e91e1640 1 --2- 172.21.15.69:0/2134289256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd1289f820 0x55cd128a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.546+0000 7f71e89e0640 1 -- 172.21.15.69:0/2134289256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd1289f820 msgr2=0x55cd128a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.546+0000 7f71e89e0640 1 --2- 172.21.15.69:0/2134289256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd1289f820 0x55cd128a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.546+0000 7f71e89e0640 1 -- 172.21.15.69:0/2134289256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 msgr2=0x55cd128a8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.546+0000 7f71e89e0640 1 --2- 172.21.15.69:0/2134289256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128a8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.546+0000 7f71e89e0640 1 -- 172.21.15.69:0/2134289256 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd128ad560 con 0x55cd128a5610 2024-05-14T17:03:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.546+0000 7f71e99e2640 1 --2- 172.21.15.69:0/2134289256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128a8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.546+0000 7f71e89e0640 1 --2- 172.21.15.69:0/2134289256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a6c00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009900 tx=0x7f71d402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69af70a8535e3d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.546+0000 7f71ceffd640 1 -- 172.21.15.69:0/2134289256 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4044040 con 0x55cd128a5610 2024-05-14T17:03:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.546+0000 7f71ceffd640 1 -- 172.21.15.69:0/2134289256 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d4005500 con 0x55cd128a5610 2024-05-14T17:03:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.546+0000 7f71ceffd640 1 -- 172.21.15.69:0/2134289256 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4035040 con 0x55cd128a5610 2024-05-14T17:03:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.547+0000 7f71ee239740 1 -- 172.21.15.69:0/2134289256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 msgr2=0x55cd128a6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.547+0000 7f71ee239740 1 --2- 172.21.15.69:0/2134289256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a6c00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009900 tx=0x7f71d402a410 comp rx=0 tx=0).stop 2024-05-14T17:03:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.547+0000 7f71ee239740 1 -- 172.21.15.69:0/2134289256 shutdown_connections 2024-05-14T17:03:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.547+0000 7f71ee239740 1 --2- 172.21.15.69:0/2134289256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128a8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.547+0000 7f71ee239740 1 --2- 172.21.15.69:0/2134289256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd128a5610 0x55cd128a6c00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.548+0000 7f71ee239740 1 --2- 172.21.15.69:0/2134289256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd1289f820 0x55cd128a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.548+0000 7f71ee239740 1 -- 172.21.15.69:0/2134289256 >> 172.21.15.69:0/2134289256 conn(0x55cd128978b0 msgr2=0x55cd128a64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.548+0000 7f71ee239740 1 -- 172.21.15.69:0/2134289256 shutdown_connections 2024-05-14T17:03:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.548+0000 7f71ee239740 1 -- 172.21.15.69:0/2134289256 wait complete. 2024-05-14T17:03:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71ee239740 1 -- start start 2024-05-14T17:03:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71ee239740 1 Processor -- start 2024-05-14T17:03:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71ee239740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd1289f820 0x55cd128d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71ee239740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 0x55cd128d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71ee239740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128b5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71ee239740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41ab0a0 con 0x55cd1289f820 2024-05-14T17:03:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71ee239740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41ac510 con 0x55cd128a5fc0 2024-05-14T17:03:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71ee239740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41ab770 con 0x55cd128a5610 2024-05-14T17:03:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71ee239740 10 client.0 _ll_register_callbacks cb 0x7f71e001c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71e91e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd1289f820 0x55cd128d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71e99e2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71e99e2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:41644/0 (socket says 172.21.15.69:41644) 2024-05-14T17:03:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71e99e2640 1 -- 172.21.15.69:0/923249694 learned_addr learned my addr 172.21.15.69:0/923249694 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.549+0000 7f71e89e0640 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 0x55cd128d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71e99e2640 1 -- 172.21.15.69:0/923249694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 msgr2=0x55cd128d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71e99e2640 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 0x55cd128d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71e99e2640 1 -- 172.21.15.69:0/923249694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd1289f820 msgr2=0x55cd128d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71e99e2640 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd1289f820 0x55cd128d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71e99e2640 1 -- 172.21.15.69:0/923249694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd128b5db0 con 0x55cd128a5fc0 2024-05-14T17:03:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71e91e1640 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd1289f820 0x55cd128d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71e99e2640 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128b5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0a97a0 tx=0x7f71dc003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a95e75b04ba26e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:03:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71ccff9640 1 -- 172.21.15.69:0/923249694 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0a95b0 con 0x55cd128a5fc0 2024-05-14T17:03:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71ccff9640 1 -- 172.21.15.69:0/923249694 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f71dc0a6090 con 0x55cd128a5fc0 2024-05-14T17:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71ee239740 1 -- 172.21.15.69:0/923249694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d4005500 con 0x55cd128a5fc0 2024-05-14T17:03:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71ccff9640 1 -- 172.21.15.69:0/923249694 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f71dc0aa1a0 con 0x55cd128a5fc0 2024-05-14T17:03:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71ee239740 10 client.26207 fetch_fsmap learned FSMap version 52 2024-05-14T17:03:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71ee239740 1 -- 172.21.15.69:0/923249694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55cd12932480 con 0x55cd128a5fc0 2024-05-14T17:03:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.550+0000 7f71ccff9640 1 -- 172.21.15.69:0/923249694 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d4005500 con 0x55cd128a5fc0 2024-05-14T17:03:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.551+0000 7f71ccff9640 1 -- 172.21.15.69:0/923249694 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f71dc0a8e20 con 0x55cd128a5fc0 2024-05-14T17:03:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.551+0000 7f71ee239740 10 client.26207 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:03:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.551+0000 7f71ee239740 10 client.26207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.551+0000 7f71ee239740 20 client.26207 populate_metadata read hostname 'smithi069' 2024-05-14T17:03:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.551+0000 7f71ee239740 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd128a9d20 0x55cd12934c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.551+0000 7f71ee239740 4 client.26207 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:03:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.551+0000 7f71ee239740 1 -- 172.21.15.69:0/923249694 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd1280a360 con 0x55cd128a9d20 2024-05-14T17:03:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.551+0000 7f71e91e1640 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd128a9d20 0x55cd12934c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.551+0000 7f71ccff9640 10 client.26207 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.551+0000 7f71e91e1640 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd128a9d20 0x55cd12934c90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe39e6789db060e3 server_cookie=c5fed206a12564f6 in_seq=0 out_seq=0 2024-05-14T17:03:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ccff9640 1 -- 172.21.15.69:0/923249694 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cd1280a360 con 0x55cd128a9d20 2024-05-14T17:03:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ccff9640 10 client.26207 handle_command_reply: tid=0 2024-05-14T17:03:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 client.26207 shutdown 2024-05-14T17:03:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 -- 172.21.15.69:0/923249694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 msgr2=0x55cd128b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128b5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0a97a0 tx=0x7f71dc003cd0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 -- 172.21.15.69:0/923249694 shutdown_connections 2024-05-14T17:03:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd128a9d20 0x55cd12934c90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd128a5fc0 0x55cd128b5870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd128a5610 0x55cd128d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ccff9640 0 client.26207 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 --2- 172.21.15.69:0/923249694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd1289f820 0x55cd128d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 -- 172.21.15.69:0/923249694 >> 172.21.15.69:0/923249694 conn(0x55cd128978b0 msgr2=0x55cd12897100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 -- 172.21.15.69:0/923249694 shutdown_connections 2024-05-14T17:03:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.564+0000 7f71ee239740 1 -- 172.21.15.69:0/923249694 wait complete. 2024-05-14T17:03:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.565+0000 7f71ee239740 20 client.26207 trim_cache size 0 max 16384 2024-05-14T17:03:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:03:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:03:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:03:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.571+0000 7f71eb445640 1 -- 172.21.15.69:0/3665586291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f71d00b2a10 msgr2=0x7f71d00b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.571+0000 7f71eb445640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f71d00b2a10 0x7f71d00b4ed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55cd128a6400 tx=0x7f71e0010040 comp rx=0 tx=0).stop 2024-05-14T17:03:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.571+0000 7f71eb445640 1 -- 172.21.15.69:0/3665586291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e419dce0 msgr2=0x7f71e419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.571+0000 7f71eb445640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e419dce0 0x7f71e419e190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00e970 tx=0x7f71dc00ee30 comp rx=0 tx=0).stop 2024-05-14T17:03:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.575+0000 7f71eb445640 1 -- 172.21.15.69:0/3665586291 shutdown_connections 2024-05-14T17:03:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.575+0000 7f71eb445640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f71d00b2a10 0x7f71d00b4ed0 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55cd128a6400 tx=0x7f71e0010040 comp rx=0 tx=0).stop 2024-05-14T17:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.575+0000 7f71eb445640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e419dce0 0x7f71e419e190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.575+0000 7f71eb445640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71e41063e0 0x7f71e4199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.575+0000 7f71eb445640 1 --2- 172.21.15.69:0/3665586291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71e4101310 0x7f71e4199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.575+0000 7f71eb445640 1 -- 172.21.15.69:0/3665586291 >> 172.21.15.69:0/3665586291 conn(0x7f71e4077ee0 msgr2=0x7f71e40ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.579+0000 7f71eb445640 1 -- 172.21.15.69:0/3665586291 shutdown_connections 2024-05-14T17:03:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:09.581+0000 7f71eb445640 1 -- 172.21.15.69:0/3665586291 wait complete. 2024-05-14T17:03:35.881 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:23 smithi092 ceph-mon[35763]: pgmap v426: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.3 MiB/s wr, 774 op/s 2024-05-14T17:03:35.974 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:25 smithi092 ceph-mon[35763]: pgmap v427: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-14T17:03:36.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:27 smithi092 ceph-mon[35763]: pgmap v428: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.0 MiB/s wr, 757 op/s 2024-05-14T17:03:36.072 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:29 smithi092 ceph-mon[35763]: pgmap v429: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.8 MiB/s wr, 510 op/s 2024-05-14T17:03:36.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:31 smithi092 ceph-mon[35763]: pgmap v430: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.8 MiB/s wr, 510 op/s 2024-05-14T17:03:36.172 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:33 smithi092 ceph-mon[35763]: pgmap v431: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.5 MiB/s wr, 588 op/s 2024-05-14T17:03:36.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:23 smithi203 ceph-mon[35787]: pgmap v426: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.3 MiB/s wr, 774 op/s 2024-05-14T17:03:36.272 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:25 smithi203 ceph-mon[35787]: pgmap v427: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-14T17:03:36.322 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:27 smithi203 ceph-mon[35787]: pgmap v428: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.0 MiB/s wr, 757 op/s 2024-05-14T17:03:36.372 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:29 smithi203 ceph-mon[35787]: pgmap v429: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.8 MiB/s wr, 510 op/s 2024-05-14T17:03:36.422 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:31 smithi203 ceph-mon[35787]: pgmap v430: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.8 MiB/s wr, 510 op/s 2024-05-14T17:03:36.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:33 smithi203 ceph-mon[35787]: pgmap v431: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.5 MiB/s wr, 588 op/s 2024-05-14T17:03:36.523 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:09 smithi069 ceph-mon[31507]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T17:03:36.572 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:09 smithi069 ceph-mon[31507]: pgmap v419: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 26 MiB/s wr, 2.41k op/s 2024-05-14T17:03:36.631 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 760.0 s, 1901.0 tps, lat 52.997 ms stddev 60.353 2024-05-14T17:03:36.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:11 smithi069 ceph-mon[31507]: pgmap v420: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 418 op/s 2024-05-14T17:03:36.756 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:13 smithi069 ceph-mon[31507]: pgmap v421: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.0 MiB/s wr, 737 op/s 2024-05-14T17:03:36.806 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 765.0 s, 1537.2 tps, lat 64.828 ms stddev 68.326 2024-05-14T17:03:36.856 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:35 smithi092 ceph-mon[35763]: pgmap v432: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 3.0 MiB/s wr, 327 op/s 2024-05-14T17:03:36.906 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:35 smithi203 ceph-mon[35787]: pgmap v432: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 3.0 MiB/s wr, 327 op/s 2024-05-14T17:03:36.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:15 smithi069 ceph-mon[31507]: pgmap v422: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-05-14T17:03:37.006 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:17 smithi069 ceph-mon[31507]: pgmap v423: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 616 op/s 2024-05-14T17:03:37.056 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:19 smithi069 ceph-mon[31507]: pgmap v424: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 612 op/s 2024-05-14T17:03:37.107 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 770.0 s, 1976.8 tps, lat 50.491 ms stddev 58.508 2024-05-14T17:03:37.156 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:20 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:37.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:21 smithi069 ceph-mon[31507]: pgmap v425: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 513 op/s 2024-05-14T17:03:37.256 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:21 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:37.306 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:21 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:37.356 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:21 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:03:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.736+0000 7f17fc237640 1 -- 172.21.15.69:0/2674319612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4111620 msgr2=0x7f17f4109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.736+0000 7f17fc237640 1 --2- 172.21.15.69:0/2674319612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4111620 0x7f17f4109f90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f17e001a940 tx=0x7f17e003b520 comp rx=0 tx=0).stop 2024-05-14T17:03:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.737+0000 7f17fc237640 1 -- 172.21.15.69:0/2674319612 shutdown_connections 2024-05-14T17:03:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.737+0000 7f17fc237640 1 --2- 172.21.15.69:0/2674319612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f411c230 0x7f17f411e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.737+0000 7f17fc237640 1 --2- 172.21.15.69:0/2674319612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4111620 0x7f17f4109f90 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.737+0000 7f17fc237640 1 --2- 172.21.15.69:0/2674319612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f4111240 0x7f17f4109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.737+0000 7f17fc237640 1 -- 172.21.15.69:0/2674319612 >> 172.21.15.69:0/2674319612 conn(0x7f17f406db20 msgr2=0x7f17f406df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:37.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.742+0000 7f17fc237640 1 -- 172.21.15.69:0/2674319612 shutdown_connections 2024-05-14T17:03:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.744+0000 7f17fc237640 1 -- 172.21.15.69:0/2674319612 wait complete. 2024-05-14T17:03:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.744+0000 7f17fc237640 1 Processor -- start 2024-05-14T17:03:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.746+0000 7f17fc237640 1 -- start start 2024-05-14T17:03:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.746+0000 7f17fc237640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4111240 0x7f17f4117970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.746+0000 7f17fc237640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f411c230 0x7f17f4117eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.746+0000 7f17fc237640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f411b220 0x7f17f411b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.746+0000 7f17fc237640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17f4114320 con 0x7f17f411b220 2024-05-14T17:03:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.746+0000 7f17fc237640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41141a0 con 0x7f17f411c230 2024-05-14T17:03:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.746+0000 7f17fc237640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41144a0 con 0x7f17f4111240 2024-05-14T17:03:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.746+0000 7f17f9fd3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4111240 0x7f17f4117970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.746+0000 7f17f9fd3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4111240 0x7f17f4117970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:40526/0 (socket says 172.21.15.69:40526) 2024-05-14T17:03:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.746+0000 7f17f9fd3640 1 -- 172.21.15.69:0/2823408213 learned_addr learned my addr 172.21.15.69:0/2823408213 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.747+0000 7f17f9fd3640 1 -- 172.21.15.69:0/2823408213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f411c230 msgr2=0x7f17f4117eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.747+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f411c230 0x7f17f4117eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.747+0000 7f17f9fd3640 1 -- 172.21.15.69:0/2823408213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f411b220 msgr2=0x7f17f411b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.747+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f411b220 0x7f17f411b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.747+0000 7f17f9fd3640 1 -- 172.21.15.69:0/2823408213 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f411bd10 con 0x7f17f4111240 2024-05-14T17:03:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.747+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4111240 0x7f17f4117970 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17e800ba50 tx=0x7f17e800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.747+0000 7f17f2ffd640 1 -- 172.21.15.69:0/2823408213 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8019070 con 0x7f17f4111240 2024-05-14T17:03:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.747+0000 7f17fc237640 1 -- 172.21.15.69:0/2823408213 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f411c000 con 0x7f17f4111240 2024-05-14T17:03:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.747+0000 7f17f2ffd640 1 -- 172.21.15.69:0/2823408213 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e800cc10 con 0x7f17f4111240 2024-05-14T17:03:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.748+0000 7f17fc237640 1 -- 172.21.15.69:0/2823408213 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f41ac5a0 con 0x7f17f4111240 2024-05-14T17:03:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.748+0000 7f17f2ffd640 1 -- 172.21.15.69:0/2823408213 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8019070 con 0x7f17f4111240 2024-05-14T17:03:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.751+0000 7f17ff02c740 1 Processor -- start 2024-05-14T17:03:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17ff02c740 1 -- start start 2024-05-14T17:03:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 0x555bd4231cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd423c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd423eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17ff02c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555bd41a4c30 con 0x555bd423cad0 2024-05-14T17:03:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17ff02c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555bd41947d0 con 0x555bd42318f0 2024-05-14T17:03:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17ff02c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555bd4198810 con 0x555bd4232210 2024-05-14T17:03:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17f97d2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd423eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17f97d2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd423eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35418/0 (socket says 172.21.15.69:35418) 2024-05-14T17:03:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17f97d2640 1 -- 172.21.15.69:0/2678042888 learned_addr learned my addr 172.21.15.69:0/2678042888 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.752+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/2678042888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 0x555bd4231cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.753+0000 7f17f2ffd640 1 -- 172.21.15.69:0/2823408213 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f17e8014440 con 0x7f17f4111240 2024-05-14T17:03:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.754+0000 7f17f2ffd640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17e40b2ed0 0x7f17e40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.754+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/2678042888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd423c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.754+0000 7f17f2ffd640 1 -- 172.21.15.69:0/2823408213 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f17e800ca10 con 0x7f17f4111240 2024-05-14T17:03:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.755+0000 7f17f97d2640 1 -- 172.21.15.69:0/2678042888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 msgr2=0x555bd423c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.755+0000 7f17f97d2640 1 --2- 172.21.15.69:0/2678042888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd423c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.755+0000 7f17f97d2640 1 -- 172.21.15.69:0/2678042888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 msgr2=0x555bd4231cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.755+0000 7f17f97d2640 1 --2- 172.21.15.69:0/2678042888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 0x555bd4231cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.755+0000 7f17f97d2640 1 -- 172.21.15.69:0/2678042888 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bd3ea9210 con 0x555bd423cad0 2024-05-14T17:03:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.756+0000 7f17f97d2640 1 --2- 172.21.15.69:0/2678042888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd423eec0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f17e003ba90 tx=0x7f17e0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b307fc7c56857652 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.756+0000 7f17dffff640 1 -- 172.21.15.69:0/2678042888 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0050070 con 0x555bd423cad0 2024-05-14T17:03:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.756+0000 7f17dffff640 1 -- 172.21.15.69:0/2678042888 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e00047c0 con 0x555bd423cad0 2024-05-14T17:03:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.756+0000 7f17dffff640 1 -- 172.21.15.69:0/2678042888 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0040770 con 0x555bd423cad0 2024-05-14T17:03:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.757+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17e40b2ed0 0x7f17e40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.757+0000 7f17ff02c740 1 -- 172.21.15.69:0/2678042888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 msgr2=0x555bd423eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.757+0000 7f17ff02c740 1 --2- 172.21.15.69:0/2678042888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd423eec0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f17e003ba90 tx=0x7f17e0005c00 comp rx=0 tx=0).stop 2024-05-14T17:03:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.758+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17e40b2ed0 0x7f17e40b5390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f17ec005d00 tx=0x7f17ec005c90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.758+0000 7f17ff02c740 1 -- 172.21.15.69:0/2678042888 shutdown_connections 2024-05-14T17:03:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.758+0000 7f17ff02c740 1 --2- 172.21.15.69:0/2678042888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd423eec0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.758+0000 7f17ff02c740 1 --2- 172.21.15.69:0/2678042888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd423c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.758+0000 7f17ff02c740 1 --2- 172.21.15.69:0/2678042888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 0x555bd4231cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.758+0000 7f17ff02c740 1 -- 172.21.15.69:0/2678042888 >> 172.21.15.69:0/2678042888 conn(0x555bd422de00 msgr2=0x555bd422e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.758+0000 7f17ff02c740 1 -- 172.21.15.69:0/2678042888 shutdown_connections 2024-05-14T17:03:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.758+0000 7f17ff02c740 1 -- 172.21.15.69:0/2678042888 wait complete. 2024-05-14T17:03:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.759+0000 7f17ff02c740 1 -- start start 2024-05-14T17:03:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.759+0000 7f17ff02c740 1 Processor -- start 2024-05-14T17:03:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd42318f0 0x555bd423c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd4239750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17f9fd3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd42318f0 0x555bd423c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17f9fd3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd42318f0 0x555bd423c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35430/0 (socket says 172.21.15.69:35430) 2024-05-14T17:03:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17f9fd3640 1 -- 172.21.15.69:0/1101155742 learned_addr learned my addr 172.21.15.69:0/1101155742 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17ff02c740 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4239cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555bd3e7a790 con 0x555bd42318f0 2024-05-14T17:03:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555bd4150890 con 0x555bd423cad0 2024-05-14T17:03:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555bd423f740 con 0x555bd4232210 2024-05-14T17:03:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4239cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.760+0000 7f17f97d2640 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd4239750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17fa7d4640 1 -- 172.21.15.69:0/1101155742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 msgr2=0x555bd4239750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd4239750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17fa7d4640 1 -- 172.21.15.69:0/1101155742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd42318f0 msgr2=0x555bd423c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd42318f0 0x555bd423c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17fa7d4640 1 -- 172.21.15.69:0/1101155742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bd3f75620 con 0x555bd423cad0 2024-05-14T17:03:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4239cd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f17ec003870 tx=0x7f17ec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28bafe1a7615b03d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:03:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17ff02c740 10 client.0 _ll_register_callbacks cb 0x555bd4153d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17dd7fa640 1 -- 172.21.15.69:0/1101155742 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec023020 con 0x555bd423cad0 2024-05-14T17:03:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17dd7fa640 1 -- 172.21.15.69:0/1101155742 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f17ec011990 con 0x555bd423cad0 2024-05-14T17:03:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.761+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555bd41a6b80 con 0x555bd423cad0 2024-05-14T17:03:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.762+0000 7f17dd7fa640 1 -- 172.21.15.69:0/1101155742 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f17ec02e050 con 0x555bd423cad0 2024-05-14T17:03:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.762+0000 7f17ff02c740 10 client.26213 fetch_fsmap learned FSMap version 52 2024-05-14T17:03:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.762+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({fsmap=52}) -- 0x555bd424ae00 con 0x555bd423cad0 2024-05-14T17:03:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.762+0000 7f17dd7fa640 1 -- 172.21.15.69:0/1101155742 <== mon.2 v2:172.21.15.92:3300/0 4 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f17ec00f040 con 0x555bd423cad0 2024-05-14T17:03:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.762+0000 7f17ff02c740 10 client.26213 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:03:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.762+0000 7f17ff02c740 10 client.26213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.762+0000 7f17ff02c740 20 client.26213 populate_metadata read hostname 'smithi069' 2024-05-14T17:03:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.762+0000 7f17ff02c740 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555bd424c6c0 0x555bd424eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.762+0000 7f17ff02c740 4 client.26213 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.762+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bd41a6b80 con 0x555bd424c6c0 2024-05-14T17:03:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.763+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555bd424c6c0 0x555bd424eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.763+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555bd424c6c0 0x555bd424eab0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fb5a41f3d7f08ca server_cookie=15f380f6f3620491 in_seq=0 out_seq=0 2024-05-14T17:03:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.763+0000 7f17dd7fa640 10 client.26213 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17dd7fa640 1 -- 172.21.15.69:0/1101155742 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555bd41a6b80 con 0x555bd424c6c0 2024-05-14T17:03:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17dd7fa640 10 client.26213 handle_command_reply: tid=0 2024-05-14T17:03:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 client.26213 shutdown 2024-05-14T17:03:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 msgr2=0x555bd4239cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4239cd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f17ec003870 tx=0x7f17ec0046c0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 shutdown_connections 2024-05-14T17:03:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555bd424c6c0 0x555bd424eab0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4239cd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd4239750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 --2- 172.21.15.69:0/1101155742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd42318f0 0x555bd423c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 >> 172.21.15.69:0/1101155742 conn(0x555bd422de00 msgr2=0x555bd422d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17dd7fa640 0 client.26213 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 shutdown_connections 2024-05-14T17:03:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 1 -- 172.21.15.69:0/1101155742 wait complete. 2024-05-14T17:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.764+0000 7f17ff02c740 20 client.26213 trim_cache size 0 max 16384 2024-05-14T17:03:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17ff02c740 1 Processor -- start 2024-05-14T17:03:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17ff02c740 1 -- start start 2024-05-14T17:03:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 0x555bd4248610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd4254050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd4254590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17ff02c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41b7540 con 0x555bd423cad0 2024-05-14T17:03:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17ff02c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41b4d80 con 0x555bd42318f0 2024-05-14T17:03:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17ff02c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41b2bf0 con 0x555bd4232210 2024-05-14T17:03:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17f9fd3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 0x555bd4248610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17fa7d4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd4254590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17f9fd3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 0x555bd4248610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:48816/0 (socket says 172.21.15.69:48816) 2024-05-14T17:03:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17fa7d4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd4254590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35440/0 (socket says 172.21.15.69:35440) 2024-05-14T17:03:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17f9fd3640 1 -- 172.21.15.69:0/4051745564 learned_addr learned my addr 172.21.15.69:0/4051745564 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.785+0000 7f17f97d2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd4254050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.786+0000 7f17f9fd3640 1 -- 172.21.15.69:0/4051745564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 msgr2=0x555bd4254050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.786+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/4051745564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd4254050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.786+0000 7f17f9fd3640 1 -- 172.21.15.69:0/4051745564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 msgr2=0x555bd4254590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.786+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/4051745564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd4254590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.786+0000 7f17f9fd3640 1 -- 172.21.15.69:0/4051745564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bd4239fa0 con 0x555bd42318f0 2024-05-14T17:03:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.786+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/4051745564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd4254590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.786+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/4051745564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 0x555bd4248610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17e80a5780 tx=0x7f17e800e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eeb78824ebdfb5e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.786+0000 7f17df7fe640 1 -- 172.21.15.69:0/4051745564 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80c1070 con 0x555bd42318f0 2024-05-14T17:03:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.786+0000 7f17df7fe640 1 -- 172.21.15.69:0/4051745564 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e800cc10 con 0x555bd42318f0 2024-05-14T17:03:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.786+0000 7f17df7fe640 1 -- 172.21.15.69:0/4051745564 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8019040 con 0x555bd42318f0 2024-05-14T17:03:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 -- 172.21.15.69:0/4051745564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 msgr2=0x555bd4248610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 --2- 172.21.15.69:0/4051745564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 0x555bd4248610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17e80a5780 tx=0x7f17e800e040 comp rx=0 tx=0).stop 2024-05-14T17:03:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 -- 172.21.15.69:0/4051745564 shutdown_connections 2024-05-14T17:03:44.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 --2- 172.21.15.69:0/4051745564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd423cad0 0x555bd4254590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 --2- 172.21.15.69:0/4051745564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd4232210 0x555bd4254050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 --2- 172.21.15.69:0/4051745564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd42318f0 0x555bd4248610 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 -- 172.21.15.69:0/4051745564 >> 172.21.15.69:0/4051745564 conn(0x555bd422de00 msgr2=0x555bd422c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 -- 172.21.15.69:0/4051745564 shutdown_connections 2024-05-14T17:03:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 -- 172.21.15.69:0/4051745564 wait complete. 2024-05-14T17:03:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 -- start start 2024-05-14T17:03:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.787+0000 7f17ff02c740 1 Processor -- start 2024-05-14T17:03:44.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd42318f0 0x555bd4244db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd4232210 0x555bd4242140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17ff02c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4242680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17ff02c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41b3240 con 0x555bd4232210 2024-05-14T17:03:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17ff02c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41b46b0 con 0x555bd423cad0 2024-05-14T17:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17ff02c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41b3910 con 0x555bd42318f0 2024-05-14T17:03:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17ff02c740 10 client.0 _ll_register_callbacks cb 0x7f17ec004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17fa7d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4242680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17f97d2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd4232210 0x555bd4242140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17fa7d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4242680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:48826/0 (socket says 172.21.15.69:48826) 2024-05-14T17:03:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.788+0000 7f17fa7d4640 1 -- 172.21.15.69:0/2157011588 learned_addr learned my addr 172.21.15.69:0/2157011588 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.789+0000 7f17fa7d4640 1 -- 172.21.15.69:0/2157011588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd42318f0 msgr2=0x555bd4244db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T17:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.789+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd42318f0 0x555bd4244db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.789+0000 7f17fa7d4640 1 -- 172.21.15.69:0/2157011588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd4232210 msgr2=0x555bd4242140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.789+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd4232210 0x555bd4242140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.789+0000 7f17fa7d4640 1 -- 172.21.15.69:0/2157011588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bd4242d70 con 0x555bd423cad0 2024-05-14T17:03:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.789+0000 7f17fa7d4640 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4242680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00b3f0 tx=0x7f17ec011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a5d12711f598122 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.789+0000 7f17dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:03:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.789+0000 7f17dd7fa640 1 -- 172.21.15.69:0/2157011588 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec023040 con 0x555bd423cad0 2024-05-14T17:03:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.789+0000 7f17dd7fa640 1 -- 172.21.15.69:0/2157011588 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f17ec033480 con 0x555bd423cad0 2024-05-14T17:03:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.789+0000 7f17ff02c740 1 -- 172.21.15.69:0/2157011588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e800cc10 con 0x555bd423cad0 2024-05-14T17:03:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.790+0000 7f17dd7fa640 1 -- 172.21.15.69:0/2157011588 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f17ec042050 con 0x555bd423cad0 2024-05-14T17:03:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.790+0000 7f17ff02c740 10 client.26219 fetch_fsmap learned FSMap version 52 2024-05-14T17:03:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.790+0000 7f17ff02c740 1 -- 172.21.15.69:0/2157011588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x555bd42b6b40 con 0x555bd423cad0 2024-05-14T17:03:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.790+0000 7f17dd7fa640 1 -- 172.21.15.69:0/2157011588 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e800cc10 con 0x555bd423cad0 2024-05-14T17:03:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.790+0000 7f17dd7fa640 1 -- 172.21.15.69:0/2157011588 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f17ec03a020 con 0x555bd423cad0 2024-05-14T17:03:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.791+0000 7f17ff02c740 10 client.26219 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:03:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.791+0000 7f17ff02c740 10 client.26219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.791+0000 7f17ff02c740 20 client.26219 populate_metadata read hostname 'smithi069' 2024-05-14T17:03:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.791+0000 7f17ff02c740 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555bd42b7320 0x555bd42b9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.791+0000 7f17ff02c740 4 client.26219 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:03:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.791+0000 7f17ff02c740 1 -- 172.21.15.69:0/2157011588 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bd41a0360 con 0x555bd42b7320 2024-05-14T17:03:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.791+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555bd42b7320 0x555bd42b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.791+0000 7f17dd7fa640 10 client.26219 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.791+0000 7f17f9fd3640 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555bd42b7320 0x555bd42b9740 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5ae19ce7ba47d3 server_cookie=87913c4781d659c5 in_seq=0 out_seq=0 2024-05-14T17:03:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17dd7fa640 1 -- 172.21.15.69:0/2157011588 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555bd41a0360 con 0x555bd42b7320 2024-05-14T17:03:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17dd7fa640 10 client.26219 handle_command_reply: tid=0 2024-05-14T17:03:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17ff02c740 1 client.26219 shutdown 2024-05-14T17:03:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17ff02c740 1 -- 172.21.15.69:0/2157011588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 msgr2=0x555bd4242680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17ff02c740 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4242680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00b3f0 tx=0x7f17ec011000 comp rx=0 tx=0).stop 2024-05-14T17:03:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17ff02c740 1 -- 172.21.15.69:0/2157011588 shutdown_connections 2024-05-14T17:03:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17ff02c740 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555bd42b7320 0x555bd42b9740 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17ff02c740 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bd423cad0 0x555bd4242680 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17ff02c740 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555bd4232210 0x555bd4242140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17dd7fa640 0 client.26219 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17ff02c740 1 --2- 172.21.15.69:0/2157011588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555bd42318f0 0x555bd4244db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.803+0000 7f17ff02c740 1 -- 172.21.15.69:0/2157011588 >> 172.21.15.69:0/2157011588 conn(0x555bd422de00 msgr2=0x555bd422d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17ff02c740 1 -- 172.21.15.69:0/2157011588 shutdown_connections 2024-05-14T17:03:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17ff02c740 1 -- 172.21.15.69:0/2157011588 wait complete. 2024-05-14T17:03:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17ff02c740 20 client.26219 trim_cache size 0 max 16384 2024-05-14T17:03:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:03:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:03:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:03:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 -- 172.21.15.69:0/2823408213 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17e40b2ed0 msgr2=0x7f17e40b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17e40b2ed0 0x7f17e40b5390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f17ec005d00 tx=0x7f17ec005c90 comp rx=0 tx=0).stop 2024-05-14T17:03:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 -- 172.21.15.69:0/2823408213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4111240 msgr2=0x7f17f4117970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4111240 0x7f17f4117970 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17e800ba50 tx=0x7f17e800bf10 comp rx=0 tx=0).stop 2024-05-14T17:03:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 -- 172.21.15.69:0/2823408213 shutdown_connections 2024-05-14T17:03:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17e40b2ed0 0x7f17e40b5390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f411b220 0x7f17f411b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f411c230 0x7f17f4117eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 --2- 172.21.15.69:0/2823408213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4111240 0x7f17f4117970 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 -- 172.21.15.69:0/2823408213 >> 172.21.15.69:0/2823408213 conn(0x7f17f406db20 msgr2=0x7f17f411c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 -- 172.21.15.69:0/2823408213 shutdown_connections 2024-05-14T17:03:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:22.804+0000 7f17fc237640 1 -- 172.21.15.69:0/2823408213 wait complete. 2024-05-14T17:03:47.223 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:47.255 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:23 smithi069 ceph-mon[31507]: pgmap v426: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.3 MiB/s wr, 774 op/s 2024-05-14T17:03:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 775.0 s, 1522.6 tps, lat 65.793 ms stddev 71.595 2024-05-14T17:03:47.321 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:37 smithi092 ceph-mon[35763]: pgmap v433: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-05-14T17:03:47.359 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:39 smithi092 ceph-mon[35763]: pgmap v434: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 765 KiB/s wr, 86 op/s 2024-05-14T17:03:47.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:41 smithi092 ceph-mon[35763]: pgmap v435: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 765 KiB/s wr, 86 op/s 2024-05-14T17:03:47.430 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:43 smithi092 ceph-mon[35763]: pgmap v436: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-05-14T17:03:47.492 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:45 smithi092 ceph-mon[35763]: pgmap v437: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T17:03:47.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:37 smithi203 ceph-mon[35787]: pgmap v433: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-05-14T17:03:47.580 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:39 smithi203 ceph-mon[35787]: pgmap v434: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 765 KiB/s wr, 86 op/s 2024-05-14T17:03:47.613 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:41 smithi203 ceph-mon[35787]: pgmap v435: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 765 KiB/s wr, 86 op/s 2024-05-14T17:03:47.647 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:43 smithi203 ceph-mon[35787]: pgmap v436: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-05-14T17:03:47.680 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:45 smithi203 ceph-mon[35787]: pgmap v437: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T17:03:47.714 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:25 smithi069 ceph-mon[31507]: pgmap v427: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-14T17:03:47.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.179+0000 7f8efac11700 1 -- 172.21.15.69:0/2227495633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ef41084c0 msgr2=0x7f8ef41088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:47.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.179+0000 7f8efac11700 1 --2- 172.21.15.69:0/2227495633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ef41084c0 0x7f8ef41088a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009990 tx=0x7f8ee8009ca0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.179+0000 7f8efac11700 1 -- 172.21.15.69:0/2227495633 shutdown_connections 2024-05-14T17:03:47.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.179+0000 7f8efac11700 1 --2- 172.21.15.69:0/2227495633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ef41084c0 0x7f8ef41088a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.179+0000 7f8efac11700 1 --2- 172.21.15.69:0/2227495633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef4103600 0x7f8ef410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.179+0000 7f8efac11700 1 --2- 172.21.15.69:0/2227495633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ef4102c40 0x7f8ef41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.179+0000 7f8efac11700 1 -- 172.21.15.69:0/2227495633 >> 172.21.15.69:0/2227495633 conn(0x7f8ef4078ae0 msgr2=0x7f8ef40ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:48.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.179+0000 7f8efac11700 1 -- 172.21.15.69:0/2227495633 shutdown_connections 2024-05-14T17:03:48.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.179+0000 7f8efac11700 1 -- 172.21.15.69:0/2227495633 wait complete. 2024-05-14T17:03:48.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.180+0000 7f8efac11700 1 Processor -- start 2024-05-14T17:03:48.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.180+0000 7f8efac11700 1 -- start start 2024-05-14T17:03:48.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8efac11700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ef4102c40 0x7f8ef4197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:48.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8efac11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef4103600 0x7f8ef4197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8efac11700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ef41084c0 0x7f8ef4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:48.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8efac11700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4111500 con 0x7f8ef4102c40 2024-05-14T17:03:48.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8efac11700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4111380 con 0x7f8ef4103600 2024-05-14T17:03:48.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8efac11700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4111680 con 0x7f8ef41084c0 2024-05-14T17:03:48.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef4103600 0x7f8ef4197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:48.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef91ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ef41084c0 0x7f8ef4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:48.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef4103600 0x7f8ef4197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:58544/0 (socket says 172.21.15.69:58544) 2024-05-14T17:03:48.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef3fff700 1 -- 172.21.15.69:0/2564478848 learned_addr learned my addr 172.21.15.69:0/2564478848 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:48.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ef4102c40 0x7f8ef4197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:48.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef3fff700 1 -- 172.21.15.69:0/2564478848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ef41084c0 msgr2=0x7f8ef4198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:48.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef3fff700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ef41084c0 0x7f8ef4198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef3fff700 1 -- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ef4102c40 msgr2=0x7f8ef4197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:48.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef3fff700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ef4102c40 0x7f8ef4197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef3fff700 1 -- 172.21.15.69:0/2564478848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef41ac370 con 0x7f8ef4103600 2024-05-14T17:03:48.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.181+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ef4102c40 0x7f8ef4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:48.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.184+0000 7f8ef3fff700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef4103600 0x7f8ef4197c40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000d950 tx=0x7f8ee000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:48.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.184+0000 7f8ef1ffb700 1 -- 172.21.15.69:0/2564478848 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0016070 con 0x7f8ef4103600 2024-05-14T17:03:48.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.184+0000 7f8ef1ffb700 1 -- 172.21.15.69:0/2564478848 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee0009df0 con 0x7f8ef4103600 2024-05-14T17:03:49.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.184+0000 7f8ef1ffb700 1 -- 172.21.15.69:0/2564478848 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000f620 con 0x7f8ef4103600 2024-05-14T17:03:49.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.184+0000 7f8efac11700 1 -- 172.21.15.69:0/2564478848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef41ac660 con 0x7f8ef4103600 2024-05-14T17:03:49.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.184+0000 7f8efac11700 1 -- 172.21.15.69:0/2564478848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef41acb50 con 0x7f8ef4103600 2024-05-14T17:03:49.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.187+0000 7f8ef1ffb700 1 -- 172.21.15.69:0/2564478848 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8ee0019070 con 0x7f8ef4103600 2024-05-14T17:03:49.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.187+0000 7f8ef1ffb700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8ee40b2bc0 0x7f8ee40b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.187+0000 7f8ef1ffb700 1 -- 172.21.15.69:0/2564478848 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f8ee00e2140 con 0x7f8ef4103600 2024-05-14T17:03:49.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.188+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8ee40b2bc0 0x7f8ee40b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:49.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.188+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8ee40b2bc0 0x7f8ee40b5080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800b2e0 tx=0x7f8ee801a040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:49.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.188+0000 7f8f054a7b80 1 Processor -- start 2024-05-14T17:03:49.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.188+0000 7f8f054a7b80 1 -- start start 2024-05-14T17:03:49.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.188+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95ae40 0x555eeb95b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:49.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.188+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb955550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:49.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.188+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 0x555eeb955ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.188+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555eeb8b3e30 con 0x555eeb95b760 2024-05-14T17:03:49.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.188+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555eeb8a39d0 con 0x555eeb95ae40 2024-05-14T17:03:49.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.188+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555eeb8a7a10 con 0x555eeb955b70 2024-05-14T17:03:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb955550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:49.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb955550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52618/0 (socket says 172.21.15.69:52618) 2024-05-14T17:03:49.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef3fff700 1 -- 172.21.15.69:0/122360943 learned_addr learned my addr 172.21.15.69:0/122360943 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:49.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef91ae700 1 --2- 172.21.15.69:0/122360943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 0x555eeb955ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:49.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/122360943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95ae40 0x555eeb95b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:49.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef3fff700 1 -- 172.21.15.69:0/122360943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 msgr2=0x555eeb955ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:49.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef3fff700 1 --2- 172.21.15.69:0/122360943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 0x555eeb955ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef3fff700 1 -- 172.21.15.69:0/122360943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95ae40 msgr2=0x555eeb95b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:49.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef3fff700 1 --2- 172.21.15.69:0/122360943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95ae40 0x555eeb95b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef3fff700 1 -- 172.21.15.69:0/122360943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eeb7f3a70 con 0x555eeb95b760 2024-05-14T17:03:49.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/122360943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95ae40 0x555eeb95b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:50.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8ef3fff700 1 --2- 172.21.15.69:0/122360943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb955550 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00a90b0 tx=0x7f8ee00a9b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b31c43d73cee1e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:50.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8edeffd700 1 -- 172.21.15.69:0/122360943 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0020070 con 0x555eeb95b760 2024-05-14T17:03:50.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8edeffd700 1 -- 172.21.15.69:0/122360943 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee00ad760 con 0x555eeb95b760 2024-05-14T17:03:50.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.189+0000 7f8edeffd700 1 -- 172.21.15.69:0/122360943 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00a9870 con 0x555eeb95b760 2024-05-14T17:03:50.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.190+0000 7f8f054a7b80 1 -- 172.21.15.69:0/122360943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 msgr2=0x555eeb955550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:50.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.190+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/122360943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb955550 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00a90b0 tx=0x7f8ee00a9b30 comp rx=0 tx=0).stop 2024-05-14T17:03:50.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.190+0000 7f8f054a7b80 1 -- 172.21.15.69:0/122360943 shutdown_connections 2024-05-14T17:03:50.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.190+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/122360943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 0x555eeb955ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:50.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.190+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/122360943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95ae40 0x555eeb95b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:50.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.190+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/122360943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb955550 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:50.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.190+0000 7f8f054a7b80 1 -- 172.21.15.69:0/122360943 >> 172.21.15.69:0/122360943 conn(0x555eeb8803f0 msgr2=0x555eeb94fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:50.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.190+0000 7f8f054a7b80 1 -- 172.21.15.69:0/122360943 shutdown_connections 2024-05-14T17:03:50.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.190+0000 7f8f054a7b80 1 -- 172.21.15.69:0/122360943 wait complete. 2024-05-14T17:03:50.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.191+0000 7f8f054a7b80 1 -- start start 2024-05-14T17:03:50.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.191+0000 7f8f054a7b80 1 Processor -- start 2024-05-14T17:03:50.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.191+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb955b70 0x555eeb9614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:50.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.191+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 0x555eeb95e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:50.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.191+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb95ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.191+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555eeb7aa140 con 0x555eeb95b760 2024-05-14T17:03:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.191+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555eeb845960 con 0x555eeb955b70 2024-05-14T17:03:50.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.191+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555eeb9620e0 con 0x555eeb95ae40 2024-05-14T17:03:50.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8f054a7b80 10 client.0 _ll_register_callbacks cb 0x555eeb83a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 0x555eeb95e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:50.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef91ae700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb95ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:50.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 0x555eeb95e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:56844/0 (socket says 172.21.15.69:56844) 2024-05-14T17:03:51.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef3fff700 1 -- 172.21.15.69:0/4255403099 learned_addr learned my addr 172.21.15.69:0/4255403099 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:51.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb955b70 0x555eeb9614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:51.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef3fff700 1 -- 172.21.15.69:0/4255403099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb955b70 msgr2=0x555eeb9614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:51.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef3fff700 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb955b70 0x555eeb9614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:51.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef3fff700 1 -- 172.21.15.69:0/4255403099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 msgr2=0x555eeb95ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:51.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef3fff700 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb95ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:51.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef3fff700 1 -- 172.21.15.69:0/4255403099 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eeb842830 con 0x555eeb95ae40 2024-05-14T17:03:51.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef91ae700 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb95ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ef3fff700 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 0x555eeb95e840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0000c00 tx=0x7f8ee00096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18eabcea5286f67f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:51.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ed3fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T17:03:51.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ed3fff700 1 -- 172.21.15.69:0/4255403099 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00c2020 con 0x555eeb95ae40 2024-05-14T17:03:51.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8f054a7b80 1 -- 172.21.15.69:0/4255403099 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee00ad760 con 0x555eeb95ae40 2024-05-14T17:03:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.192+0000 7f8ed3fff700 1 -- 172.21.15.69:0/4255403099 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f8ee00a9820 con 0x555eeb95ae40 2024-05-14T17:03:51.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8ed3fff700 1 -- 172.21.15.69:0/4255403099 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8ee0020070 con 0x555eeb95ae40 2024-05-14T17:03:51.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8f054a7b80 10 client.26128 fetch_fsmap learned FSMap version 52 2024-05-14T17:03:51.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8f054a7b80 1 -- 172.21.15.69:0/4255403099 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x555eeb96c830 con 0x555eeb95ae40 2024-05-14T17:03:51.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8ed3fff700 1 -- 172.21.15.69:0/4255403099 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee00ad760 con 0x555eeb95ae40 2024-05-14T17:03:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8ed3fff700 1 -- 172.21.15.69:0/4255403099 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f8ee00a8610 con 0x555eeb95ae40 2024-05-14T17:03:52.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8f054a7b80 10 client.26128 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:03:52.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8f054a7b80 10 client.26128 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:03:52.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8f054a7b80 20 client.26128 populate_metadata read hostname 'smithi069' 2024-05-14T17:03:52.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555eeb96ed00 0x555eeb9710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:52.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8f054a7b80 4 client.26128 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:52.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8f054a7b80 1 -- 172.21.15.69:0/4255403099 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eeb921500 con 0x555eeb96ed00 2024-05-14T17:03:52.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.193+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555eeb96ed00 0x555eeb9710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:52.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.194+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555eeb96ed00 0x555eeb9710f0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d4f045a483ae8c server_cookie=deaaf57a590c19cd in_seq=0 out_seq=0 2024-05-14T17:03:52.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.194+0000 7f8ed3fff700 10 client.26128 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:52.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.194+0000 7f8ed3fff700 1 -- 172.21.15.69:0/4255403099 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555eeb921500 con 0x555eeb96ed00 2024-05-14T17:03:52.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.194+0000 7f8ed3fff700 10 client.26128 handle_command_reply: tid=0 2024-05-14T17:03:52.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.194+0000 7f8f054a7b80 1 client.26128 shutdown 2024-05-14T17:03:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 1 -- 172.21.15.69:0/4255403099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 msgr2=0x555eeb95e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:52.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 0x555eeb95e840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0000c00 tx=0x7f8ee00096c0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 1 -- 172.21.15.69:0/4255403099 shutdown_connections 2024-05-14T17:03:52.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555eeb96ed00 0x555eeb9710f0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb95ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb955b70 0x555eeb9614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/4255403099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 0x555eeb95e840 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8ed3fff700 0 client.26128 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 1 -- 172.21.15.69:0/4255403099 >> 172.21.15.69:0/4255403099 conn(0x555eeb8803f0 msgr2=0x555eeb94a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:52.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 1 -- 172.21.15.69:0/4255403099 shutdown_connections 2024-05-14T17:03:52.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 1 -- 172.21.15.69:0/4255403099 wait complete. 2024-05-14T17:03:52.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.195+0000 7f8f054a7b80 20 client.26128 trim_cache size 0 max 16384 2024-05-14T17:03:52.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.224+0000 7f8f054a7b80 1 Processor -- start 2024-05-14T17:03:52.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.224+0000 7f8f054a7b80 1 -- start start 2024-05-14T17:03:52.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.224+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb955b70 0x555eeb96aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:52.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.224+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 0x555eeb95d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:53.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.224+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb94a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:53.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41b6da0 con 0x555eeb95b760 2024-05-14T17:03:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41b45a0 con 0x555eeb955b70 2024-05-14T17:03:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41b2410 con 0x555eeb95ae40 2024-05-14T17:03:53.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef91ae700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb94a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:53.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef91ae700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb94a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52636/0 (socket says 172.21.15.69:52636) 2024-05-14T17:03:53.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef91ae700 1 -- 172.21.15.69:0/3055486797 learned_addr learned my addr 172.21.15.69:0/3055486797 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:53.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef3fff700 1 --2- 172.21.15.69:0/3055486797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 0x555eeb95d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:53.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/3055486797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb955b70 0x555eeb96aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:53.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef91ae700 1 -- 172.21.15.69:0/3055486797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 msgr2=0x555eeb95d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:53.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef91ae700 1 --2- 172.21.15.69:0/3055486797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 0x555eeb95d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef91ae700 1 -- 172.21.15.69:0/3055486797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb955b70 msgr2=0x555eeb96aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:53.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef91ae700 1 --2- 172.21.15.69:0/3055486797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb955b70 0x555eeb96aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef91ae700 1 -- 172.21.15.69:0/3055486797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eeb94aec0 con 0x555eeb95b760 2024-05-14T17:03:53.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.225+0000 7f8ef91ae700 1 --2- 172.21.15.69:0/3055486797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb94a980 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8eec014c50 tx=0x7f8eec009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e077d14bb2bfdab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:53.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.226+0000 7f8ede7fc700 1 -- 172.21.15.69:0/3055486797 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec003e10 con 0x555eeb95b760 2024-05-14T17:03:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.226+0000 7f8ede7fc700 1 -- 172.21.15.69:0/3055486797 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec017d30 con 0x555eeb95b760 2024-05-14T17:03:53.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.226+0000 7f8ede7fc700 1 -- 172.21.15.69:0/3055486797 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec01f3e0 con 0x555eeb95b760 2024-05-14T17:03:53.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3055486797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 msgr2=0x555eeb94a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/3055486797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb94a980 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8eec014c50 tx=0x7f8eec009920 comp rx=0 tx=0).stop 2024-05-14T17:03:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3055486797 shutdown_connections 2024-05-14T17:03:53.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/3055486797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95b760 0x555eeb94a980 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/3055486797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb955b70 0x555eeb96aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/3055486797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb95ae40 0x555eeb95d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3055486797 >> 172.21.15.69:0/3055486797 conn(0x555eeb8803f0 msgr2=0x555eeb9599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:53.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3055486797 shutdown_connections 2024-05-14T17:03:53.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3055486797 wait complete. 2024-05-14T17:03:53.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 -- start start 2024-05-14T17:03:53.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.227+0000 7f8f054a7b80 1 Processor -- start 2024-05-14T17:03:53.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 0x555eeb967550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:53.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95ae40 0x555eeb967a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:53.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8f054a7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95b760 0x555eeb9643e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:54.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41b2a60 con 0x555eeb95ae40 2024-05-14T17:03:54.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41b3ed0 con 0x555eeb95b760 2024-05-14T17:03:54.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8f054a7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41b3130 con 0x555eeb955b70 2024-05-14T17:03:54.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8f054a7b80 10 client.0 _ll_register_callbacks cb 0x555eeb83a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:54.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95ae40 0x555eeb967a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:54.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef91ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95b760 0x555eeb9643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:54.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef91ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95b760 0x555eeb9643e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:58572/0 (socket says 172.21.15.69:58572) 2024-05-14T17:03:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef91ae700 1 -- 172.21.15.69:0/3002604212 learned_addr learned my addr 172.21.15.69:0/3002604212 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:54.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 0x555eeb967550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:54.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef91ae700 1 -- 172.21.15.69:0/3002604212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 msgr2=0x555eeb967550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:54.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef91ae700 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 0x555eeb967550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:54.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef91ae700 1 -- 172.21.15.69:0/3002604212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95ae40 msgr2=0x555eeb967a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:54.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef91ae700 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95ae40 0x555eeb967a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:54.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef91ae700 1 -- 172.21.15.69:0/3002604212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eeb964c70 con 0x555eeb95b760 2024-05-14T17:03:54.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.228+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 0x555eeb967550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:54.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8ef91ae700 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95b760 0x555eeb9643e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009ec0 tx=0x7f8eec0048c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dae6bc53d914dd4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:54.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8ed3fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:03:54.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8ed3fff700 1 -- 172.21.15.69:0/3002604212 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec017a90 con 0x555eeb95b760 2024-05-14T17:03:54.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8ed3fff700 1 -- 172.21.15.69:0/3002604212 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f8eec020440 con 0x555eeb95b760 2024-05-14T17:03:54.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3002604212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eec017d30 con 0x555eeb95b760 2024-05-14T17:03:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8ed3fff700 1 -- 172.21.15.69:0/3002604212 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8eec036050 con 0x555eeb95b760 2024-05-14T17:03:54.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8f054a7b80 10 client.26237 fetch_fsmap learned FSMap version 52 2024-05-14T17:03:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3002604212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x555eeb9d9e50 con 0x555eeb95b760 2024-05-14T17:03:54.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8ed3fff700 1 -- 172.21.15.69:0/3002604212 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec02d070 con 0x555eeb95b760 2024-05-14T17:03:54.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8ed3fff700 1 -- 172.21.15.69:0/3002604212 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f8eec01e070 con 0x555eeb95b760 2024-05-14T17:03:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8f054a7b80 10 client.26237 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:03:54.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8f054a7b80 10 client.26237 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:03:54.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8f054a7b80 20 client.26237 populate_metadata read hostname 'smithi069' 2024-05-14T17:03:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555eeb9da4f0 0x555eeb9dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:54.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8f054a7b80 4 client.26237 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.65, "await_for": 45.0}] 2024-05-14T17:03:54.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.229+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3002604212 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.65, "await_for": 45.0}) -- 0x7f8eec017d30 con 0x555eeb9da4f0 2024-05-14T17:03:54.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.230+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555eeb9da4f0 0x555eeb9dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:55.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.230+0000 7f8ef89ad700 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555eeb9da4f0 0x555eeb9dc910 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c82a142a647ef198 server_cookie=d240004806a854f7 in_seq=0 out_seq=0 2024-05-14T17:03:55.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.230+0000 7f8ed3fff700 10 client.26237 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:55.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8ed3fff700 1 -- 172.21.15.69:0/3002604212 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f8eec017d30 con 0x555eeb9da4f0 2024-05-14T17:03:55.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8ed3fff700 10 client.26237 handle_command_reply: tid=0 2024-05-14T17:03:55.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 client.26237 shutdown 2024-05-14T17:03:55.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3002604212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95b760 msgr2=0x555eeb9643e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:55.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95b760 0x555eeb9643e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009ec0 tx=0x7f8eec0048c0 comp rx=0 tx=0).stop 2024-05-14T17:03:55.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3002604212 shutdown_connections 2024-05-14T17:03:55.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x555eeb9da4f0 0x555eeb9dc910 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:55.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555eeb95b760 0x555eeb9643e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eeb955b70 0x555eeb967550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:55.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8ed3fff700 0 client.26237 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:55.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 --2- 172.21.15.69:0/3002604212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555eeb95ae40 0x555eeb967a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:55.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3002604212 >> 172.21.15.69:0/3002604212 conn(0x555eeb8803f0 msgr2=0x555eeb9599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:55.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3002604212 shutdown_connections 2024-05-14T17:03:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 1 -- 172.21.15.69:0/3002604212 wait complete. 2024-05-14T17:03:55.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.416+0000 7f8f054a7b80 20 client.26237 trim_cache size 0 max 16384 2024-05-14T17:03:55.537 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:03:55.562 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 52, 2024-05-14T17:03:55.587 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T17:03:55.623 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 38, 2024-05-14T17:03:55.682 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T17:03:55.704 INFO:teuthology.orchestra.run.smithi069.stdout: "a8922838": { 2024-05-14T17:03:55.729 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 38, 2024-05-14T17:03:55.754 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T17:03:55.787 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:03:55.812 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T17:03:55.837 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:03:55.862 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:03:55.887 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T17:03:55.912 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 149.7, 2024-05-14T17:03:55.937 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T17:03:55.962 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T17:03:55.987 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T17:03:56.012 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:03:56.054 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T17:03:56.079 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:03:56.106 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:03:56.157 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:03:56.179 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:03:56.204 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:03:56.254 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:03:56.279 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:03:56.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 -- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8ee40b2bc0 msgr2=0x7f8ee40b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:56.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8ee40b2bc0 0x7f8ee40b5080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800b2e0 tx=0x7f8ee801a040 comp rx=0 tx=0).stop 2024-05-14T17:03:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 -- 172.21.15.69:0/2564478848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef4103600 msgr2=0x7f8ef4197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:56.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef4103600 0x7f8ef4197c40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000d950 tx=0x7f8ee000dc60 comp rx=0 tx=0).stop 2024-05-14T17:03:56.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 -- 172.21.15.69:0/2564478848 shutdown_connections 2024-05-14T17:03:56.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8ee40b2bc0 0x7f8ee40b5080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ef4102c40 0x7f8ef4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ef41084c0 0x7f8ef4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 --2- 172.21.15.69:0/2564478848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef4103600 0x7f8ef4197c40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 -- 172.21.15.69:0/2564478848 >> 172.21.15.69:0/2564478848 conn(0x7f8ef4078ae0 msgr2=0x7f8ef4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:56.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 -- 172.21.15.69:0/2564478848 shutdown_connections 2024-05-14T17:03:56.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:03:26.417+0000 7f8efac11700 1 -- 172.21.15.69:0/2564478848 wait complete. 2024-05-14T17:03:56.629 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:27 smithi069 ceph-mon[31507]: pgmap v428: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.0 MiB/s wr, 757 op/s 2024-05-14T17:03:56.646 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:29 smithi069 ceph-mon[31507]: pgmap v429: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.8 MiB/s wr, 510 op/s 2024-05-14T17:03:56.713 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 780.0 s, 285.6 tps, lat 63.302 ms stddev 66.259 2024-05-14T17:03:56.739 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a8922838', quiesce took 76.5 seconds. Will release after: -46.85 2024-05-14T17:03:56.755 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:31 smithi069 ceph-mon[31507]: pgmap v430: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.8 MiB/s wr, 510 op/s 2024-05-14T17:03:56.772 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:33 smithi069 ceph-mon[31507]: pgmap v431: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.5 MiB/s wr, 588 op/s 2024-05-14T17:03:56.797 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 785.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:03:56.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:47 smithi092 ceph-mon[35763]: pgmap v438: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T17:03:56.830 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:49 smithi092 ceph-mon[35763]: pgmap v439: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-14T17:03:56.855 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:51 smithi092 ceph-mon[35763]: pgmap v440: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-14T17:03:56.872 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:53 smithi092 ceph-mon[35763]: pgmap v441: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T17:03:56.897 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:54 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:03:56.913 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:55 smithi092 ceph-mon[35763]: pgmap v442: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 49 KiB/s wr, 12 op/s 2024-05-14T17:03:56.930 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:47 smithi203 ceph-mon[35787]: pgmap v438: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T17:03:56.958 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:49 smithi203 ceph-mon[35787]: pgmap v439: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-14T17:03:56.980 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:51 smithi203 ceph-mon[35787]: pgmap v440: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-14T17:03:57.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:53 smithi203 ceph-mon[35787]: pgmap v441: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T17:03:57.065 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:54 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:03:57.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:55 smithi203 ceph-mon[35787]: pgmap v442: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 49 KiB/s wr, 12 op/s 2024-05-14T17:03:57.123 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:03:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.905+0000 7f8c8c072640 1 -- 172.21.15.69:0/1559292788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c841033f0 msgr2=0x7f8c8410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.905+0000 7f8c8c072640 1 --2- 172.21.15.69:0/1559292788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c841033f0 0x7f8c8410f900 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009cd0 tx=0x7f8c7c02a8e0 comp rx=0 tx=0).stop 2024-05-14T17:03:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.905+0000 7f8c8c072640 1 -- 172.21.15.69:0/1559292788 shutdown_connections 2024-05-14T17:03:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.905+0000 7f8c8c072640 1 --2- 172.21.15.69:0/1559292788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c841033f0 0x7f8c8410f900 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.905+0000 7f8c8c072640 1 --2- 172.21.15.69:0/1559292788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c84102a30 0x7f8c84102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.905+0000 7f8c8c072640 1 --2- 172.21.15.69:0/1559292788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c84108a40 0x7f8c84108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.905+0000 7f8c8c072640 1 -- 172.21.15.69:0/1559292788 >> 172.21.15.69:0/1559292788 conn(0x7f8c840fe500 msgr2=0x7f8c84100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.905+0000 7f8c8c072640 1 -- 172.21.15.69:0/1559292788 shutdown_connections 2024-05-14T17:03:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.905+0000 7f8c8c072640 1 -- 172.21.15.69:0/1559292788 wait complete. 2024-05-14T17:03:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c8c072640 1 Processor -- start 2024-05-14T17:03:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c8c072640 1 -- start start 2024-05-14T17:03:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c8c072640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84102a30 0x7f8c84199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c8c072640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841033f0 0x7f8c84199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c8c072640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c84108a40 0x7f8c8419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c8c072640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84112d00 con 0x7f8c84102a30 2024-05-14T17:03:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c8c072640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84112b80 con 0x7f8c841033f0 2024-05-14T17:03:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c8c072640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84112e80 con 0x7f8c84108a40 2024-05-14T17:03:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c89e0e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84102a30 0x7f8c84199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c89e0e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84102a30 0x7f8c84199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51928/0 (socket says 172.21.15.69:51928) 2024-05-14T17:03:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c89e0e640 1 -- 172.21.15.69:0/4213193264 learned_addr learned my addr 172.21.15.69:0/4213193264 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.906+0000 7f8c8960d640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841033f0 0x7f8c84199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c84108a40 0x7f8c8419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c89e0e640 1 -- 172.21.15.69:0/4213193264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c84108a40 msgr2=0x7f8c8419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c84108a40 0x7f8c8419d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c89e0e640 1 -- 172.21.15.69:0/4213193264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841033f0 msgr2=0x7f8c84199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841033f0 0x7f8c84199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c89e0e640 1 -- 172.21.15.69:0/4213193264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8419e0f0 con 0x7f8c84102a30 2024-05-14T17:03:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c8960d640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841033f0 0x7f8c84199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c84108a40 0x7f8c8419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84102a30 0x7f8c84199100 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c920 tx=0x7f8c7400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c82ffd640 1 -- 172.21.15.69:0/4213193264 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74007c70 con 0x7f8c84102a30 2024-05-14T17:03:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c82ffd640 1 -- 172.21.15.69:0/4213193264 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c74007e10 con 0x7f8c84102a30 2024-05-14T17:03:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c8c072640 1 -- 172.21.15.69:0/4213193264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8419e3e0 con 0x7f8c84102a30 2024-05-14T17:03:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c82ffd640 1 -- 172.21.15.69:0/4213193264 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74018820 con 0x7f8c84102a30 2024-05-14T17:03:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.907+0000 7f8c8c072640 1 -- 172.21.15.69:0/4213193264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c841a5c70 con 0x7f8c84102a30 2024-05-14T17:03:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.910+0000 7f8c82ffd640 1 -- 172.21.15.69:0/4213193264 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8c7400f450 con 0x7f8c84102a30 2024-05-14T17:03:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.910+0000 7f8c8ed8e740 1 Processor -- start 2024-05-14T17:03:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.910+0000 7f8c8ed8e740 1 -- start start 2024-05-14T17:03:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.910+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.910+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84cb210 0x5649a84d5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.910+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84d5ad0 0x5649a84d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.910+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5649a843dc30 con 0x5649a84ca8f0 2024-05-14T17:03:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.910+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5649a842d7d0 con 0x5649a84d5ad0 2024-05-14T17:03:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.910+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5649a8431810 con 0x5649a84cb210 2024-05-14T17:03:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.910+0000 7f8c8a60f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c8a60f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51930/0 (socket says 172.21.15.69:51930) 2024-05-14T17:03:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c8a60f640 1 -- 172.21.15.69:0/4122692098 learned_addr learned my addr 172.21.15.69:0/4122692098 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c8960d640 1 --2- 172.21.15.69:0/4122692098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84d5ad0 0x5649a84d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/4122692098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84cb210 0x5649a84d5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c8a60f640 1 -- 172.21.15.69:0/4122692098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84cb210 msgr2=0x5649a84d5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/4122692098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84cb210 0x5649a84d5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c8a60f640 1 -- 172.21.15.69:0/4122692098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84d5ad0 msgr2=0x5649a84d7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/4122692098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84d5ad0 0x5649a84d7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c82ffd640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8c6c0b2a10 0x7f8c6c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c8a60f640 1 -- 172.21.15.69:0/4122692098 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a8142210 con 0x5649a84ca8f0 2024-05-14T17:03:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/4122692098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84cb210 0x5649a84d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c82ffd640 1 -- 172.21.15.69:0/4213193264 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f8c740e27c0 con 0x7f8c84102a30 2024-05-14T17:03:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8c6c0b2a10 0x7f8c6c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/4122692098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84cacd0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c005e30 tx=0x7f8c7c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc81b1badbafe7f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c6bfff640 1 -- 172.21.15.69:0/4122692098 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c031cb0 con 0x5649a84ca8f0 2024-05-14T17:03:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c6bfff640 1 -- 172.21.15.69:0/4122692098 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7c036040 con 0x5649a84ca8f0 2024-05-14T17:03:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.911+0000 7f8c6bfff640 1 -- 172.21.15.69:0/4122692098 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c030e80 con 0x5649a84ca8f0 2024-05-14T17:03:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.912+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/4122692098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 msgr2=0x5649a84cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.912+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8c6c0b2a10 0x7f8c6c0b4ed0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c048000 tx=0x7f8c7c04b000 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.912+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/4122692098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84cacd0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c005e30 tx=0x7f8c7c009a20 comp rx=0 tx=0).stop 2024-05-14T17:03:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.912+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/4122692098 shutdown_connections 2024-05-14T17:03:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.912+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/4122692098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84d5ad0 0x5649a84d7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.912+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/4122692098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84cb210 0x5649a84d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.912+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/4122692098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84cacd0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.912+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/4122692098 >> 172.21.15.69:0/4122692098 conn(0x5649a84c6e00 msgr2=0x5649a84c71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.912+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/4122692098 shutdown_connections 2024-05-14T17:03:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.912+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/4122692098 wait complete. 2024-05-14T17:03:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8ed8e740 1 -- start start 2024-05-14T17:03:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8ed8e740 1 Processor -- start 2024-05-14T17:03:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84d5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84d5ad0 0x5649a84d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5649a8113790 con 0x5649a84ca8f0 2024-05-14T17:03:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5649a83e9890 con 0x5649a84cb210 2024-05-14T17:03:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5649a84d8740 con 0x5649a84d5ad0 2024-05-14T17:03:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8ed8e740 10 client.0 _ll_register_callbacks cb 0x5649a83ecd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c89e0e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84d5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8960d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8960d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:49776/0 (socket says 172.21.15.69:49776) 2024-05-14T17:03:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.913+0000 7f8c8960d640 1 -- 172.21.15.69:0/3717230307 learned_addr learned my addr 172.21.15.69:0/3717230307 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:03:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84d5ad0 0x5649a84d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c8960d640 1 -- 172.21.15.69:0/3717230307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84d5ad0 msgr2=0x5649a84d2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c8960d640 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84d5ad0 0x5649a84d2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c8960d640 1 -- 172.21.15.69:0/3717230307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 msgr2=0x5649a84d5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c8960d640 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84d5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c8960d640 1 -- 172.21.15.69:0/3717230307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a820e620 con 0x5649a84cb210 2024-05-14T17:03:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84d5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c8960d640 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d26f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800bf00 tx=0x7f8c780117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cbdcb952e0b1b52 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c697fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:03:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c697fa640 1 -- 172.21.15.69:0/3717230307 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78012650 con 0x5649a84cb210 2024-05-14T17:03:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c697fa640 1 -- 172.21.15.69:0/3717230307 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f8c780127f0 con 0x5649a84cb210 2024-05-14T17:03:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.914+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3717230307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7c036040 con 0x5649a84cb210 2024-05-14T17:03:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c697fa640 1 -- 172.21.15.69:0/3717230307 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8c780129f0 con 0x5649a84cb210 2024-05-14T17:03:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c8ed8e740 10 client.26246 fetch_fsmap learned FSMap version 52 2024-05-14T17:03:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3717230307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5649a84e4110 con 0x5649a84cb210 2024-05-14T17:03:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c697fa640 1 -- 172.21.15.69:0/3717230307 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7c036040 con 0x5649a84cb210 2024-05-14T17:03:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c697fa640 1 -- 172.21.15.69:0/3717230307 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f8c78023410 con 0x5649a84cb210 2024-05-14T17:03:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c8ed8e740 10 client.26246 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:03:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c8ed8e740 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c8ed8e740 20 client.26246 populate_metadata read hostname 'smithi069' 2024-05-14T17:03:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5649a84e5710 0x5649a84e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c8ed8e740 4 client.26246 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3717230307 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649a843fb80 con 0x5649a84e5710 2024-05-14T17:03:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.915+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5649a84e5710 0x5649a84e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.916+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5649a84e5710 0x5649a84e7bd0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2a5b4cd4f6dde48 server_cookie=ae5cb0e5555352e2 in_seq=0 out_seq=0 2024-05-14T17:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.916+0000 7f8c697fa640 10 client.26246 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:03:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.916+0000 7f8c697fa640 1 -- 172.21.15.69:0/3717230307 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5649a843fb80 con 0x5649a84e5710 2024-05-14T17:03:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.916+0000 7f8c697fa640 10 client.26246 handle_command_reply: tid=0 2024-05-14T17:03:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.916+0000 7f8c8ed8e740 1 client.26246 shutdown 2024-05-14T17:03:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.916+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3717230307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 msgr2=0x5649a84d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.916+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d26f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800bf00 tx=0x7f8c780117b0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.917+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3717230307 shutdown_connections 2024-05-14T17:03:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.917+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5649a84e5710 0x5649a84e7bd0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.917+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84d5ad0 0x5649a84d2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.917+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d26f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.917+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/3717230307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84ca8f0 0x5649a84d5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.917+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3717230307 >> 172.21.15.69:0/3717230307 conn(0x5649a84c6e00 msgr2=0x5649a84c6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.917+0000 7f8c697fa640 0 client.26246 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.917+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3717230307 shutdown_connections 2024-05-14T17:04:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.917+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3717230307 wait complete. 2024-05-14T17:04:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.917+0000 7f8c8ed8e740 20 client.26246 trim_cache size 0 max 16384 2024-05-14T17:04:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8ed8e740 1 Processor -- start 2024-05-14T17:04:00.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8ed8e740 1 -- start start 2024-05-14T17:04:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84ca8f0 0x5649a84e1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841aed10 con 0x5649a84d5ad0 2024-05-14T17:04:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ac550 con 0x5649a84cb210 2024-05-14T17:04:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841aa3e0 con 0x5649a84ca8f0 2024-05-14T17:04:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8a60f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8a60f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51950/0 (socket says 172.21.15.69:51950) 2024-05-14T17:04:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8a60f640 1 -- 172.21.15.69:0/3707874066 learned_addr learned my addr 172.21.15.69:0/3707874066 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/3707874066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84ca8f0 0x5649a84e1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.932+0000 7f8c8960d640 1 --2- 172.21.15.69:0/3707874066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.933+0000 7f8c8a60f640 1 -- 172.21.15.69:0/3707874066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84ca8f0 msgr2=0x5649a84e1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.933+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/3707874066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84ca8f0 0x5649a84e1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.933+0000 7f8c8a60f640 1 -- 172.21.15.69:0/3707874066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 msgr2=0x5649a84d16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.933+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/3707874066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.933+0000 7f8c8a60f640 1 -- 172.21.15.69:0/3707874066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a84c2830 con 0x5649a84d5ad0 2024-05-14T17:04:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.933+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/3707874066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d4f30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c005e30 tx=0x7f8c7c052000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f8d5ab9147ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.935+0000 7f8c6b7fe640 1 -- 172.21.15.69:0/3707874066 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c067070 con 0x5649a84d5ad0 2024-05-14T17:04:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.935+0000 7f8c6b7fe640 1 -- 172.21.15.69:0/3707874066 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7c036040 con 0x5649a84d5ad0 2024-05-14T17:04:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.935+0000 7f8c6b7fe640 1 -- 172.21.15.69:0/3707874066 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c030c70 con 0x5649a84d5ad0 2024-05-14T17:04:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3707874066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 msgr2=0x5649a84d4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/3707874066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d4f30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c005e30 tx=0x7f8c7c052000 comp rx=0 tx=0).stop 2024-05-14T17:04:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3707874066 shutdown_connections 2024-05-14T17:04:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/3707874066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d4f30 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/3707874066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/3707874066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84ca8f0 0x5649a84e1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3707874066 >> 172.21.15.69:0/3707874066 conn(0x5649a84c6e00 msgr2=0x5649a84d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3707874066 shutdown_connections 2024-05-14T17:04:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/3707874066 wait complete. 2024-05-14T17:04:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 -- start start 2024-05-14T17:04:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.936+0000 7f8c8ed8e740 1 Processor -- start 2024-05-14T17:04:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84ca8f0 0x5649a84ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8ed8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841aaa30 con 0x5649a84d5ad0 2024-05-14T17:04:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841abe80 con 0x5649a84cb210 2024-05-14T17:04:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8ed8e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ab0e0 con 0x5649a84ca8f0 2024-05-14T17:04:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8ed8e740 10 client.0 _ll_register_callbacks cb 0x7f8c7800bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8a60f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8a60f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51952/0 (socket says 172.21.15.69:51952) 2024-05-14T17:04:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8a60f640 1 -- 172.21.15.69:0/1690772414 learned_addr learned my addr 172.21.15.69:0/1690772414 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8960d640 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84ca8f0 0x5649a84ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8a60f640 1 -- 172.21.15.69:0/1690772414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84ca8f0 msgr2=0x5649a84ddce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84ca8f0 0x5649a84ddce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8a60f640 1 -- 172.21.15.69:0/1690772414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 msgr2=0x5649a84d9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8a60f640 1 -- 172.21.15.69:0/1690772414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a84d9b80 con 0x5649a84d5ad0 2024-05-14T17:04:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c8a60f640 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d9590 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c049040 tx=0x7f8c7c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdc92834cbcb747 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.937+0000 7f8c697fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c697fa640 1 -- 172.21.15.69:0/1690772414 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c067040 con 0x5649a84d5ad0 2024-05-14T17:04:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c697fa640 1 -- 172.21.15.69:0/1690772414 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f8c7c004690 con 0x5649a84d5ad0 2024-05-14T17:04:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/1690772414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7c036040 con 0x5649a84d5ad0 2024-05-14T17:04:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c697fa640 1 -- 172.21.15.69:0/1690772414 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c0305a0 con 0x5649a84d5ad0 2024-05-14T17:04:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c8ed8e740 10 client.16836 fetch_fsmap learned FSMap version 52 2024-05-14T17:04:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/1690772414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5649a854fe60 con 0x5649a84d5ad0 2024-05-14T17:04:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c697fa640 1 -- 172.21.15.69:0/1690772414 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7c036040 con 0x5649a84d5ad0 2024-05-14T17:04:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c697fa640 1 -- 172.21.15.69:0/1690772414 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f8c7c030070 con 0x5649a84d5ad0 2024-05-14T17:04:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c8ed8e740 10 client.16836 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:04:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c8ed8e740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c8ed8e740 20 client.16836 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5649a8550100 0x5649a8552520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c8ed8e740 4 client.16836 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/1690772414 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649a8439360 con 0x5649a8550100 2024-05-14T17:04:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.938+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5649a8550100 0x5649a8552520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.939+0000 7f8c89e0e640 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5649a8550100 0x5649a8552520 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f1599adc9e440e6 server_cookie=fbd336cbe01c0f6a in_seq=0 out_seq=0 2024-05-14T17:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.939+0000 7f8c697fa640 10 client.16836 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c697fa640 1 -- 172.21.15.69:0/1690772414 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5649a8439360 con 0x5649a8550100 2024-05-14T17:04:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c697fa640 10 client.16836 handle_command_reply: tid=0 2024-05-14T17:04:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 client.16836 shutdown 2024-05-14T17:04:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/1690772414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 msgr2=0x5649a84d9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d9590 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c049040 tx=0x7f8c7c005c50 comp rx=0 tx=0).stop 2024-05-14T17:04:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/1690772414 shutdown_connections 2024-05-14T17:04:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5649a8550100 0x5649a8552520 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5649a84d5ad0 0x5649a84d9590 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649a84cb210 0x5649a84d9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c697fa640 0 client.16836 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 --2- 172.21.15.69:0/1690772414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5649a84ca8f0 0x5649a84ddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/1690772414 >> 172.21.15.69:0/1690772414 conn(0x5649a84c6e00 msgr2=0x5649a84cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/1690772414 shutdown_connections 2024-05-14T17:04:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 1 -- 172.21.15.69:0/1690772414 wait complete. 2024-05-14T17:04:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.960+0000 7f8c8ed8e740 20 client.16836 trim_cache size 0 max 16384 2024-05-14T17:04:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:04:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:04:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:04:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 -- 172.21.15.69:0/4213193264 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8c6c0b2a10 msgr2=0x7f8c6c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8c6c0b2a10 0x7f8c6c0b4ed0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c048000 tx=0x7f8c7c04b000 comp rx=0 tx=0).stop 2024-05-14T17:04:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 -- 172.21.15.69:0/4213193264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84102a30 msgr2=0x7f8c84199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84102a30 0x7f8c84199100 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c920 tx=0x7f8c7400cde0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 -- 172.21.15.69:0/4213193264 shutdown_connections 2024-05-14T17:04:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8c6c0b2a10 0x7f8c6c0b4ed0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c84108a40 0x7f8c8419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841033f0 0x7f8c84199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 --2- 172.21.15.69:0/4213193264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84102a30 0x7f8c84199100 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 -- 172.21.15.69:0/4213193264 >> 172.21.15.69:0/4213193264 conn(0x7f8c840fe500 msgr2=0x7f8c840fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 -- 172.21.15.69:0/4213193264 shutdown_connections 2024-05-14T17:04:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:35.961+0000 7f8c8c072640 1 -- 172.21.15.69:0/4213193264 wait complete. 2024-05-14T17:04:02.613 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:35 smithi069 ceph-mon[31507]: pgmap v432: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 3.0 MiB/s wr, 327 op/s 2024-05-14T17:04:02.629 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:37 smithi069 ceph-mon[31507]: pgmap v433: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-05-14T17:04:02.654 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:39 smithi069 ceph-mon[31507]: pgmap v434: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 765 KiB/s wr, 86 op/s 2024-05-14T17:04:02.680 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 790.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:02.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:41 smithi069 ceph-mon[31507]: pgmap v435: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 765 KiB/s wr, 86 op/s 2024-05-14T17:04:02.713 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:43 smithi069 ceph-mon[31507]: pgmap v436: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-05-14T17:04:02.730 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 795.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:02.774 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:57 smithi203 ceph-mon[35787]: pgmap v443: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-05-14T17:04:02.796 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:03:59 smithi203 ceph-mon[35787]: pgmap v444: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:04:02.819 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:01 smithi203 ceph-mon[35787]: pgmap v445: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:04:02.880 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:57 smithi092 ceph-mon[35763]: pgmap v443: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-05-14T17:04:02.896 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:03:59 smithi092 ceph-mon[35763]: pgmap v444: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:04:02.913 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:01 smithi092 ceph-mon[35763]: pgmap v445: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:04:02.930 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:45 smithi069 ceph-mon[31507]: pgmap v437: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T17:04:02.955 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:47 smithi069 ceph-mon[31507]: pgmap v438: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T17:04:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.057+0000 7f2e4ad11640 1 -- 172.21.15.69:0/2076804480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44102340 msgr2=0x7f2e4410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.057+0000 7f2e4ad11640 1 --2- 172.21.15.69:0/2076804480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44102340 0x7f2e4410e800 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009930 tx=0x7f2e3c02a420 comp rx=0 tx=0).stop 2024-05-14T17:04:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.057+0000 7f2e4ad11640 1 -- 172.21.15.69:0/2076804480 shutdown_connections 2024-05-14T17:04:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.057+0000 7f2e4ad11640 1 --2- 172.21.15.69:0/2076804480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44102340 0x7f2e4410e800 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.057+0000 7f2e4ad11640 1 --2- 172.21.15.69:0/2076804480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e44101980 0x7f2e44101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.057+0000 7f2e4ad11640 1 --2- 172.21.15.69:0/2076804480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e44107940 0x7f2e44107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.057+0000 7f2e4ad11640 1 -- 172.21.15.69:0/2076804480 >> 172.21.15.69:0/2076804480 conn(0x7f2e44078ad0 msgr2=0x7f2e44078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.057+0000 7f2e4ad11640 1 -- 172.21.15.69:0/2076804480 shutdown_connections 2024-05-14T17:04:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.057+0000 7f2e4ad11640 1 -- 172.21.15.69:0/2076804480 wait complete. 2024-05-14T17:04:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.057+0000 7f2e4ad11640 1 Processor -- start 2024-05-14T17:04:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e4ad11640 1 -- start start 2024-05-14T17:04:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e4ad11640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101980 0x7f2e441991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e4ad11640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e44102340 0x7f2e441996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e4ad11640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e44107940 0x7f2e4419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e4ad11640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44111c00 con 0x7f2e44101980 2024-05-14T17:04:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e4ad11640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44111a80 con 0x7f2e44107940 2024-05-14T17:04:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e4ad11640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44111d80 con 0x7f2e44102340 2024-05-14T17:04:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e48aad640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101980 0x7f2e441991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e48aad640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101980 0x7f2e441991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58496/0 (socket says 172.21.15.69:58496) 2024-05-14T17:04:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e48aad640 1 -- 172.21.15.69:0/1482698355 learned_addr learned my addr 172.21.15.69:0/1482698355 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e492ae640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e44107940 0x7f2e4419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e48aad640 1 -- 172.21.15.69:0/1482698355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e44102340 msgr2=0x7f2e441996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e48aad640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e44102340 0x7f2e441996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e48aad640 1 -- 172.21.15.69:0/1482698355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e44107940 msgr2=0x7f2e4419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e48aad640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e44107940 0x7f2e4419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e48aad640 1 -- 172.21.15.69:0/1482698355 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4419e1a0 con 0x7f2e44101980 2024-05-14T17:04:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e48aad640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101980 0x7f2e441991b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00b4a0 tx=0x7f2e2c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.058+0000 7f2e41ffb640 1 -- 172.21.15.69:0/1482698355 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c014070 con 0x7f2e44101980 2024-05-14T17:04:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.059+0000 7f2e41ffb640 1 -- 172.21.15.69:0/1482698355 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2c010ac0 con 0x7f2e44101980 2024-05-14T17:04:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.059+0000 7f2e4ad11640 1 -- 172.21.15.69:0/1482698355 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4419e490 con 0x7f2e44101980 2024-05-14T17:04:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.059+0000 7f2e41ffb640 1 -- 172.21.15.69:0/1482698355 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c017c30 con 0x7f2e44101980 2024-05-14T17:04:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.059+0000 7f2e4ad11640 1 -- 172.21.15.69:0/1482698355 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a5d20 con 0x7f2e44101980 2024-05-14T17:04:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.061+0000 7f2e41ffb640 1 -- 172.21.15.69:0/1482698355 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2e2c017070 con 0x7f2e44101980 2024-05-14T17:04:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.061+0000 7f2e4da21740 1 Processor -- start 2024-05-14T17:04:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e4da21740 1 -- start start 2024-05-14T17:04:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d9310210 0x5633d931a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d931aad0 0x5633d931cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e4da21740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5633d9282c30 con 0x5633d930f8f0 2024-05-14T17:04:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e4da21740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5633d92727d0 con 0x5633d931aad0 2024-05-14T17:04:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e4da21740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5633d9276810 con 0x5633d9310210 2024-05-14T17:04:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e492ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e492ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58504/0 (socket says 172.21.15.69:58504) 2024-05-14T17:04:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e492ae640 1 -- 172.21.15.69:0/2309130333 learned_addr learned my addr 172.21.15.69:0/2309130333 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e41ffb640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f2e300b2a90 0x7f2e300b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e43fff640 1 --2- 172.21.15.69:0/2309130333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d931aad0 0x5633d931cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e48aad640 1 --2- 172.21.15.69:0/2309130333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d9310210 0x5633d931a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e492ae640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f2e300b2a90 0x7f2e300b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e492ae640 1 -- 172.21.15.69:0/2309130333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d9310210 msgr2=0x5633d931a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e41ffb640 1 -- 172.21.15.69:0/1482698355 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f2e2c0e1920 con 0x7f2e44101980 2024-05-14T17:04:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e492ae640 1 --2- 172.21.15.69:0/2309130333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d9310210 0x5633d931a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e492ae640 1 -- 172.21.15.69:0/2309130333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d931aad0 msgr2=0x5633d931cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e492ae640 1 --2- 172.21.15.69:0/2309130333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d931aad0 0x5633d931cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e492ae640 1 -- 172.21.15.69:0/2309130333 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d8f87210 con 0x5633d930f8f0 2024-05-14T17:04:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.062+0000 7f2e48aad640 1 --2- 172.21.15.69:0/2309130333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d9310210 0x5633d931a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.063+0000 7f2e492ae640 1 --2- 172.21.15.69:0/2309130333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930fcd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005ab0 tx=0x7f2e3c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7894f7d695e42b6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.063+0000 7f2e3affd640 1 -- 172.21.15.69:0/2309130333 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c04b020 con 0x5633d930f8f0 2024-05-14T17:04:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.063+0000 7f2e3affd640 1 -- 172.21.15.69:0/2309130333 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e3c0044c0 con 0x5633d930f8f0 2024-05-14T17:04:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.063+0000 7f2e3affd640 1 -- 172.21.15.69:0/2309130333 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c035070 con 0x5633d930f8f0 2024-05-14T17:04:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.063+0000 7f2e4da21740 1 -- 172.21.15.69:0/2309130333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 msgr2=0x5633d930fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.063+0000 7f2e492ae640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f2e300b2a90 0x7f2e300b4f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c049000 tx=0x7f2e3c048000 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.063+0000 7f2e4da21740 1 --2- 172.21.15.69:0/2309130333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930fcd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005ab0 tx=0x7f2e3c0317e0 comp rx=0 tx=0).stop 2024-05-14T17:04:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 -- 172.21.15.69:0/2309130333 shutdown_connections 2024-05-14T17:04:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 --2- 172.21.15.69:0/2309130333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d931aad0 0x5633d931cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 --2- 172.21.15.69:0/2309130333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d9310210 0x5633d931a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:04.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 --2- 172.21.15.69:0/2309130333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930fcd0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 -- 172.21.15.69:0/2309130333 >> 172.21.15.69:0/2309130333 conn(0x5633d930be00 msgr2=0x5633d930c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 -- 172.21.15.69:0/2309130333 shutdown_connections 2024-05-14T17:04:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 -- 172.21.15.69:0/2309130333 wait complete. 2024-05-14T17:04:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 -- start start 2024-05-14T17:04:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 Processor -- start 2024-05-14T17:04:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d930f8f0 0x5633d931a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d9310210 0x5633d9317730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d931aad0 0x5633d9317c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5633d8f58790 con 0x5633d9310210 2024-05-14T17:04:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5633d922e890 con 0x5633d930f8f0 2024-05-14T17:04:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5633d931d7a0 con 0x5633d931aad0 2024-05-14T17:04:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.064+0000 7f2e4da21740 10 client.0 _ll_register_callbacks cb 0x5633d9231d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e43fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d9310210 0x5633d9317730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e43fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d9310210 0x5633d9317730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58520/0 (socket says 172.21.15.69:58520) 2024-05-14T17:04:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e43fff640 1 -- 172.21.15.69:0/2125951690 learned_addr learned my addr 172.21.15.69:0/2125951690 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e48aad640 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d930f8f0 0x5633d931a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e492ae640 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d931aad0 0x5633d9317c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e43fff640 1 -- 172.21.15.69:0/2125951690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d931aad0 msgr2=0x5633d9317c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e43fff640 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d931aad0 0x5633d9317c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e43fff640 1 -- 172.21.15.69:0/2125951690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d930f8f0 msgr2=0x5633d931a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e43fff640 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d930f8f0 0x5633d931a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e43fff640 1 -- 172.21.15.69:0/2125951690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d9053620 con 0x5633d9310210 2024-05-14T17:04:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e48aad640 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d930f8f0 0x5633d931a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e492ae640 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d931aad0 0x5633d9317c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e43fff640 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d9310210 0x5633d9317730 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400f9e0 tx=0x7f2e3400fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b5b26e6ed7966d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e23fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e23fff640 1 -- 172.21.15.69:0/2125951690 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400ddc0 con 0x5633d9310210 2024-05-14T17:04:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e23fff640 1 -- 172.21.15.69:0/2125951690 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f2e34004b20 con 0x5633d9310210 2024-05-14T17:04:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e4da21740 1 -- 172.21.15.69:0/2125951690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633d9284b80 con 0x5633d9310210 2024-05-14T17:04:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e23fff640 1 -- 172.21.15.69:0/2125951690 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2e34004d20 con 0x5633d9310210 2024-05-14T17:04:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e4da21740 10 client.16848 fetch_fsmap learned FSMap version 52 2024-05-14T17:04:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e4da21740 1 -- 172.21.15.69:0/2125951690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5633d9329260 con 0x5633d9310210 2024-05-14T17:04:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.065+0000 7f2e23fff640 1 -- 172.21.15.69:0/2125951690 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5633d9284b80 con 0x5633d9310210 2024-05-14T17:04:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.066+0000 7f2e23fff640 1 -- 172.21.15.69:0/2125951690 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f2e340204a0 con 0x5633d9310210 2024-05-14T17:04:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.066+0000 7f2e4da21740 10 client.16848 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:04:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.066+0000 7f2e4da21740 10 client.16848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.066+0000 7f2e4da21740 20 client.16848 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.066+0000 7f2e4da21740 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5633d932a880 0x5633d932ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.066+0000 7f2e4da21740 4 client.16848 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.066+0000 7f2e4da21740 1 -- 172.21.15.69:0/2125951690 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633d927e360 con 0x5633d932a880 2024-05-14T17:04:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.066+0000 7f2e48aad640 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5633d932a880 0x5633d932ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.066+0000 7f2e48aad640 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5633d932a880 0x5633d932ccc0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1765eb9c6b078a64 server_cookie=87b51de168a75e6e in_seq=0 out_seq=0 2024-05-14T17:04:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.066+0000 7f2e23fff640 10 client.16848 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e23fff640 1 -- 172.21.15.69:0/2125951690 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5633d927e360 con 0x5633d932a880 2024-05-14T17:04:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e23fff640 10 client.16848 handle_command_reply: tid=0 2024-05-14T17:04:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e4da21740 1 client.16848 shutdown 2024-05-14T17:04:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e4da21740 1 -- 172.21.15.69:0/2125951690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d9310210 msgr2=0x5633d9317730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e4da21740 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d9310210 0x5633d9317730 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400f9e0 tx=0x7f2e3400fea0 comp rx=0 tx=0).stop 2024-05-14T17:04:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e4da21740 1 -- 172.21.15.69:0/2125951690 shutdown_connections 2024-05-14T17:04:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e4da21740 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5633d932a880 0x5633d932ccc0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e4da21740 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d931aad0 0x5633d9317c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e4da21740 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d9310210 0x5633d9317730 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e23fff640 0 client.16848 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e4da21740 1 --2- 172.21.15.69:0/2125951690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d930f8f0 0x5633d931a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.067+0000 7f2e4da21740 1 -- 172.21.15.69:0/2125951690 >> 172.21.15.69:0/2125951690 conn(0x5633d930be00 msgr2=0x5633d930b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.068+0000 7f2e4da21740 1 -- 172.21.15.69:0/2125951690 shutdown_connections 2024-05-14T17:04:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.068+0000 7f2e4da21740 1 -- 172.21.15.69:0/2125951690 wait complete. 2024-05-14T17:04:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.068+0000 7f2e4da21740 20 client.16848 trim_cache size 0 max 16384 2024-05-14T17:04:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e4da21740 1 Processor -- start 2024-05-14T17:04:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e4da21740 1 -- start start 2024-05-14T17:04:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d9310210 0x5633d9319f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d931aad0 0x5633d93325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e4da21740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b1030 con 0x5633d930f8f0 2024-05-14T17:04:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e4da21740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441ae870 con 0x5633d9310210 2024-05-14T17:04:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e4da21740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441ac6e0 con 0x5633d931aad0 2024-05-14T17:04:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e48aad640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e48aad640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58534/0 (socket says 172.21.15.69:58534) 2024-05-14T17:04:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e48aad640 1 -- 172.21.15.69:0/391517955 learned_addr learned my addr 172.21.15.69:0/391517955 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e48aad640 1 -- 172.21.15.69:0/391517955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d931aad0 msgr2=0x5633d93325d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e48aad640 1 --2- 172.21.15.69:0/391517955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d931aad0 0x5633d93325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e48aad640 1 -- 172.21.15.69:0/391517955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d9310210 msgr2=0x5633d9319f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e48aad640 1 --2- 172.21.15.69:0/391517955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d9310210 0x5633d9319f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e48aad640 1 -- 172.21.15.69:0/391517955 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d930f0f0 con 0x5633d930f8f0 2024-05-14T17:04:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.083+0000 7f2e48aad640 1 --2- 172.21.15.69:0/391517955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930b880 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0df220 tx=0x7f2e2c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491150f4fb70175e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e3a7fc640 1 -- 172.21.15.69:0/391517955 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c01e070 con 0x5633d930f8f0 2024-05-14T17:04:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e3a7fc640 1 -- 172.21.15.69:0/391517955 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2c0ad0b0 con 0x5633d930f8f0 2024-05-14T17:04:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e3a7fc640 1 -- 172.21.15.69:0/391517955 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c0a7f60 con 0x5633d930f8f0 2024-05-14T17:04:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e4da21740 1 -- 172.21.15.69:0/391517955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 msgr2=0x5633d930b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e4da21740 1 --2- 172.21.15.69:0/391517955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930b880 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0df220 tx=0x7f2e2c00bf90 comp rx=0 tx=0).stop 2024-05-14T17:04:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e4da21740 1 -- 172.21.15.69:0/391517955 shutdown_connections 2024-05-14T17:04:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e4da21740 1 --2- 172.21.15.69:0/391517955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d931aad0 0x5633d93325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e4da21740 1 --2- 172.21.15.69:0/391517955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d9310210 0x5633d9319f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e4da21740 1 --2- 172.21.15.69:0/391517955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d930f8f0 0x5633d930b880 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e4da21740 1 -- 172.21.15.69:0/391517955 >> 172.21.15.69:0/391517955 conn(0x5633d930be00 msgr2=0x5633d9311b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e4da21740 1 -- 172.21.15.69:0/391517955 shutdown_connections 2024-05-14T17:04:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.084+0000 7f2e4da21740 1 -- 172.21.15.69:0/391517955 wait complete. 2024-05-14T17:04:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e4da21740 1 -- start start 2024-05-14T17:04:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e4da21740 1 Processor -- start 2024-05-14T17:04:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d930f8f0 0x5633d931e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d9310210 0x5633d931eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e4da21740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d931aad0 0x5633d9394180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e4da21740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441acd30 con 0x5633d931aad0 2024-05-14T17:04:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e4da21740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441ae1a0 con 0x5633d9310210 2024-05-14T17:04:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e4da21740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441ad400 con 0x5633d930f8f0 2024-05-14T17:04:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e4da21740 10 client.0 _ll_register_callbacks cb 0x7f2e3400f5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e492ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d931aad0 0x5633d9394180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e492ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d931aad0 0x5633d9394180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58550/0 (socket says 172.21.15.69:58550) 2024-05-14T17:04:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e492ae640 1 -- 172.21.15.69:0/1610279344 learned_addr learned my addr 172.21.15.69:0/1610279344 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e43fff640 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d9310210 0x5633d931eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e48aad640 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d930f8f0 0x5633d931e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e492ae640 1 -- 172.21.15.69:0/1610279344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d930f8f0 msgr2=0x5633d931e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e492ae640 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d930f8f0 0x5633d931e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e492ae640 1 -- 172.21.15.69:0/1610279344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d9310210 msgr2=0x5633d931eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e492ae640 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d9310210 0x5633d931eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.085+0000 7f2e492ae640 1 -- 172.21.15.69:0/1610279344 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d93946c0 con 0x5633d931aad0 2024-05-14T17:04:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e48aad640 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d930f8f0 0x5633d931e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e492ae640 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d931aad0 0x5633d9394180 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c04a040 tx=0x7f2e3c004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ed76d1d0738ee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e23fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e23fff640 1 -- 172.21.15.69:0/1610279344 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c058070 con 0x5633d931aad0 2024-05-14T17:04:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e23fff640 1 -- 172.21.15.69:0/1610279344 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7f2e3c005220 con 0x5633d931aad0 2024-05-14T17:04:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e4da21740 1 -- 172.21.15.69:0/1610279344 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e2c0ad0b0 con 0x5633d931aad0 2024-05-14T17:04:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e23fff640 1 -- 172.21.15.69:0/1610279344 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2e3c064050 con 0x5633d931aad0 2024-05-14T17:04:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e4da21740 10 client.16854 fetch_fsmap learned FSMap version 52 2024-05-14T17:04:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e4da21740 1 -- 172.21.15.69:0/1610279344 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5633d9395890 con 0x5633d931aad0 2024-05-14T17:04:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e23fff640 1 -- 172.21.15.69:0/1610279344 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2c0ad0b0 con 0x5633d931aad0 2024-05-14T17:04:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e23fff640 1 -- 172.21.15.69:0/1610279344 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f2e3c005490 con 0x5633d931aad0 2024-05-14T17:04:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e4da21740 10 client.16854 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:04:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e4da21740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e4da21740 20 client.16854 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e4da21740 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5633d9395b10 0x5633d9397f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e4da21740 4 client.16854 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.086+0000 7f2e4da21740 1 -- 172.21.15.69:0/1610279344 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633d91f85f0 con 0x5633d9395b10 2024-05-14T17:04:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.087+0000 7f2e48aad640 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5633d9395b10 0x5633d9397f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.087+0000 7f2e23fff640 10 client.16854 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.087+0000 7f2e48aad640 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5633d9395b10 0x5633d9397f00 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89da6b811e286f3c server_cookie=ddd9c5b491223960 in_seq=0 out_seq=0 2024-05-14T17:04:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e23fff640 1 -- 172.21.15.69:0/1610279344 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5633d91f85f0 con 0x5633d9395b10 2024-05-14T17:04:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e23fff640 10 client.16854 handle_command_reply: tid=0 2024-05-14T17:04:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 client.16854 shutdown 2024-05-14T17:04:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 -- 172.21.15.69:0/1610279344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d931aad0 msgr2=0x5633d9394180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d931aad0 0x5633d9394180 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c04a040 tx=0x7f2e3c004b10 comp rx=0 tx=0).stop 2024-05-14T17:04:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 -- 172.21.15.69:0/1610279344 shutdown_connections 2024-05-14T17:04:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5633d9395b10 0x5633d9397f00 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633d931aad0 0x5633d9394180 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633d9310210 0x5633d931eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e23fff640 0 client.16854 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 --2- 172.21.15.69:0/1610279344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d930f8f0 0x5633d931e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 -- 172.21.15.69:0/1610279344 >> 172.21.15.69:0/1610279344 conn(0x5633d930be00 msgr2=0x5633d93112a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 -- 172.21.15.69:0/1610279344 shutdown_connections 2024-05-14T17:04:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 1 -- 172.21.15.69:0/1610279344 wait complete. 2024-05-14T17:04:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.101+0000 7f2e4da21740 20 client.16854 trim_cache size 0 max 16384 2024-05-14T17:04:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:04:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:04:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:04:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 -- 172.21.15.69:0/1482698355 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f2e300b2a90 msgr2=0x7f2e300b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f2e300b2a90 0x7f2e300b4f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c049000 tx=0x7f2e3c048000 comp rx=0 tx=0).stop 2024-05-14T17:04:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 -- 172.21.15.69:0/1482698355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101980 msgr2=0x7f2e441991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101980 0x7f2e441991b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00b4a0 tx=0x7f2e2c00b960 comp rx=0 tx=0).stop 2024-05-14T17:04:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 -- 172.21.15.69:0/1482698355 shutdown_connections 2024-05-14T17:04:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f2e300b2a90 0x7f2e300b4f50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e44107940 0x7f2e4419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e44102340 0x7f2e441996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 --2- 172.21.15.69:0/1482698355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101980 0x7f2e441991b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 -- 172.21.15.69:0/1482698355 >> 172.21.15.69:0/1482698355 conn(0x7f2e44078ad0 msgr2=0x7f2e4406a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 -- 172.21.15.69:0/1482698355 shutdown_connections 2024-05-14T17:04:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:03:49.102+0000 7f2e4ad11640 1 -- 172.21.15.69:0/1482698355 wait complete. 2024-05-14T17:04:10.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:49 smithi069 ceph-mon[31507]: pgmap v439: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-14T17:04:10.295 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 800.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:10.345 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:51 smithi069 ceph-mon[31507]: pgmap v440: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-14T17:04:10.379 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:53 smithi069 ceph-mon[31507]: pgmap v441: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T17:04:10.413 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 805.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:10.446 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: *** Caught signal (Segmentation fault) ** 2024-05-14T17:04:10.495 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: in thread 7f843c06e700 thread_name:md_log_replay 2024-05-14T17:04:10.529 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:04:10.584 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f844d430d20] 2024-05-14T17:04:10.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2: /usr/bin/ceph-mds(+0x54ac02) [0x559e7af8cc02] 2024-05-14T17:04:10.654 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559e7af903b3] 2024-05-14T17:04:10.706 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 4: (MDLog::_replay_thread()+0x617) [0x559e7afeb197] 2024-05-14T17:04:10.737 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559e7aff4a11] 2024-05-14T17:04:10.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f844d4261ca] 2024-05-14T17:04:10.804 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 7: clone() 2024-05-14T17:04:10.837 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2024-05-14T17:04:03.725+0000 7f843c06e700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:04:10.871 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: in thread 7f843c06e700 thread_name:md_log_replay 2024-05-14T17:04:10.904 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2024-05-14T17:04:10.937 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:04:10.971 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f844d430d20] 2024-05-14T17:04:11.004 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2: /usr/bin/ceph-mds(+0x54ac02) [0x559e7af8cc02] 2024-05-14T17:04:11.038 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559e7af903b3] 2024-05-14T17:04:11.096 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 4: (MDLog::_replay_thread()+0x617) [0x559e7afeb197] 2024-05-14T17:04:11.129 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559e7aff4a11] 2024-05-14T17:04:11.163 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f844d4261ca] 2024-05-14T17:04:11.211 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 7: clone() 2024-05-14T17:04:11.246 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:04:11.279 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2024-05-14T17:04:11.329 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 0> 2024-05-14T17:04:03.725+0000 7f843c06e700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:04:11.365 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: in thread 7f843c06e700 thread_name:md_log_replay 2024-05-14T17:04:11.396 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2024-05-14T17:04:11.429 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:04:11.463 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f844d430d20] 2024-05-14T17:04:11.496 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2: /usr/bin/ceph-mds(+0x54ac02) [0x559e7af8cc02] 2024-05-14T17:04:11.530 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559e7af903b3] 2024-05-14T17:04:11.563 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 4: (MDLog::_replay_thread()+0x617) [0x559e7afeb197] 2024-05-14T17:04:11.596 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559e7aff4a11] 2024-05-14T17:04:11.630 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f844d4261ca] 2024-05-14T17:04:11.671 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 7: clone() 2024-05-14T17:04:11.705 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:04:11.738 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2024-05-14T17:04:11.772 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 0> 2024-05-14T17:04:03.725+0000 7f843c06e700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:04:11.819 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: in thread 7f843c06e700 thread_name:md_log_replay 2024-05-14T17:04:11.855 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2024-05-14T17:04:11.888 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:04:11.939 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f844d430d20] 2024-05-14T17:04:11.972 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2: /usr/bin/ceph-mds(+0x54ac02) [0x559e7af8cc02] 2024-05-14T17:04:12.005 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559e7af903b3] 2024-05-14T17:04:12.039 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 4: (MDLog::_replay_thread()+0x617) [0x559e7afeb197] 2024-05-14T17:04:12.072 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559e7aff4a11] 2024-05-14T17:04:12.105 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f844d4261ca] 2024-05-14T17:04:12.138 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 7: clone() 2024-05-14T17:04:12.172 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:04:12.205 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[72650]: 2024-05-14T17:04:12.247 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:03 smithi203 podman[73272]: 2024-05-14 17:04:03.786074693 +0000 UTC m=+0.020305513 container died ac2cbd38a1eff678752634f37cf8f71f4b7260d169dc388245d516d599b96807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-14T17:04:12.281 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:03 smithi203 ceph-mon[35787]: pgmap v446: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 70 KiB/s wr, 30 op/s 2024-05-14T17:04:12.314 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:54 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:12.347 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:55 smithi069 ceph-mon[31507]: pgmap v442: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 49 KiB/s wr, 12 op/s 2024-05-14T17:04:12.380 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:57 smithi069 ceph-mon[31507]: pgmap v443: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-05-14T17:04:12.414 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 810.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:12.447 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:03 smithi092 ceph-mon[35763]: pgmap v446: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 70 KiB/s wr, 30 op/s 2024-05-14T17:04:12.480 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:04 smithi092 ceph-mon[35763]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T17:04:12.546 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:05 smithi092 ceph-mon[35763]: pgmap v448: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 66 KiB/s wr, 29 op/s 2024-05-14T17:04:12.580 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:07 smithi092 ceph-mon[35763]: pgmap v449: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 87 KiB/s wr, 34 op/s 2024-05-14T17:04:12.614 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:09 smithi092 ceph-mon[35763]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T17:04:12.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:09 smithi092 ceph-mon[35763]: pgmap v451: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 43 op/s 2024-05-14T17:04:12.681 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:11 smithi092 ceph-mon[35763]: pgmap v452: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-05-14T17:04:12.714 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:04 smithi203 podman[73272]: 2024-05-14 17:04:04.52877723 +0000 UTC m=+0.763008095 container remove ac2cbd38a1eff678752634f37cf8f71f4b7260d169dc388245d516d599b96807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:04:12.747 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:04 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-05-14T17:04:12.781 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:04 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-14T17:04:12.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:03:59 smithi069 ceph-mon[31507]: pgmap v444: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:04:12.857 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:04 smithi203 ceph-mon[35787]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T17:04:12.889 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:05 smithi203 ceph-mon[35787]: pgmap v448: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 66 KiB/s wr, 29 op/s 2024-05-14T17:04:12.922 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:07 smithi203 ceph-mon[35787]: pgmap v449: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 87 KiB/s wr, 34 op/s 2024-05-14T17:04:12.956 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:09 smithi203 ceph-mon[35787]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T17:04:12.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:09 smithi203 ceph-mon[35787]: pgmap v451: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 43 op/s 2024-05-14T17:04:13.027 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:11 smithi203 ceph-mon[35787]: pgmap v452: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-05-14T17:04:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.200+0000 7ff95a8da640 1 -- 172.21.15.69:0/2799044717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff954102660 msgr2=0x7ff95410e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.200+0000 7ff95a8da640 1 --2- 172.21.15.69:0/2799044717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff954102660 0x7ff95410e770 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009ca0 tx=0x7ff94c02a8c0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.200+0000 7ff95a8da640 1 -- 172.21.15.69:0/2799044717 shutdown_connections 2024-05-14T17:04:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.200+0000 7ff95a8da640 1 --2- 172.21.15.69:0/2799044717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff954102660 0x7ff95410e770 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.200+0000 7ff95a8da640 1 --2- 172.21.15.69:0/2799044717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff954101ca0 0x7ff954102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.200+0000 7ff95a8da640 1 --2- 172.21.15.69:0/2799044717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9541068f0 0x7ff954106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.200+0000 7ff95a8da640 1 -- 172.21.15.69:0/2799044717 >> 172.21.15.69:0/2799044717 conn(0x7ff954077750 msgr2=0x7ff954077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.200+0000 7ff95a8da640 1 -- 172.21.15.69:0/2799044717 shutdown_connections 2024-05-14T17:04:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.200+0000 7ff95a8da640 1 -- 172.21.15.69:0/2799044717 wait complete. 2024-05-14T17:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.201+0000 7ff95a8da640 1 Processor -- start 2024-05-14T17:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.201+0000 7ff95a8da640 1 -- start start 2024-05-14T17:04:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.201+0000 7ff95a8da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff954101ca0 0x7ff9540ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.201+0000 7ff95a8da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff954102660 0x7ff9540ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.201+0000 7ff95a8da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9541068f0 0x7ff954100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.201+0000 7ff95a8da640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff954111ba0 con 0x7ff9541068f0 2024-05-14T17:04:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.201+0000 7ff95a8da640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff954111a20 con 0x7ff954102660 2024-05-14T17:04:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.201+0000 7ff95a8da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff954111d20 con 0x7ff954101ca0 2024-05-14T17:04:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.201+0000 7ff958e77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9541068f0 0x7ff954100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.201+0000 7ff958e77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9541068f0 0x7ff954100220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49312/0 (socket says 172.21.15.69:49312) 2024-05-14T17:04:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff958e77640 1 -- 172.21.15.69:0/1537340507 learned_addr learned my addr 172.21.15.69:0/1537340507 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff9537fe640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff954102660 0x7ff9540ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff953fff640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff954101ca0 0x7ff9540ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff958e77640 1 -- 172.21.15.69:0/1537340507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff954101ca0 msgr2=0x7ff9540ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff958e77640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff954101ca0 0x7ff9540ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff958e77640 1 -- 172.21.15.69:0/1537340507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff954102660 msgr2=0x7ff9540ffce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff958e77640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff954102660 0x7ff9540ffce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff958e77640 1 -- 172.21.15.69:0/1537340507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9541a75b0 con 0x7ff9541068f0 2024-05-14T17:04:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff958e77640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9541068f0 0x7ff954100220 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff94c005cc0 tx=0x7ff94c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff9517fa640 1 -- 172.21.15.69:0/1537340507 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c003bf0 con 0x7ff9541068f0 2024-05-14T17:04:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff9517fa640 1 -- 172.21.15.69:0/1537340507 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94c003d90 con 0x7ff9541068f0 2024-05-14T17:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff95a8da640 1 -- 172.21.15.69:0/1537340507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9541a7840 con 0x7ff9541068f0 2024-05-14T17:04:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff9517fa640 1 -- 172.21.15.69:0/1537340507 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c02fbc0 con 0x7ff9541068f0 2024-05-14T17:04:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.202+0000 7ff95a8da640 1 -- 172.21.15.69:0/1537340507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9541a7de0 con 0x7ff9541068f0 2024-05-14T17:04:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.205+0000 7ff95d5f8740 1 Processor -- start 2024-05-14T17:04:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.205+0000 7ff9517fa640 1 -- 172.21.15.69:0/1537340507 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff94c02fd60 con 0x7ff9541068f0 2024-05-14T17:04:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.205+0000 7ff95d5f8740 1 -- start start 2024-05-14T17:04:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.205+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cd8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.205+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.205+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76ce5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.205+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df76c4bc30 con 0x55df76cd9210 2024-05-14T17:04:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.205+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55df76c3b7d0 con 0x55df76ce3ad0 2024-05-14T17:04:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.205+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55df76c3f810 con 0x55df76cd88f0 2024-05-14T17:04:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff953fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff953fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49322/0 (socket says 172.21.15.69:49322) 2024-05-14T17:04:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff953fff640 1 -- 172.21.15.69:0/1298771617 learned_addr learned my addr 172.21.15.69:0/1298771617 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff9537fe640 1 --2- 172.21.15.69:0/1298771617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76ce5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff9537fe640 1 --2- 172.21.15.69:0/1298771617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cd8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff953fff640 1 -- 172.21.15.69:0/1298771617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 msgr2=0x55df76cd8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff953fff640 1 --2- 172.21.15.69:0/1298771617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cd8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff953fff640 1 -- 172.21.15.69:0/1298771617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 msgr2=0x55df76ce5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff9517fa640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff93c0b2a90 0x7ff93c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff953fff640 1 --2- 172.21.15.69:0/1298771617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76ce5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff953fff640 1 -- 172.21.15.69:0/1298771617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df76950210 con 0x55df76cd9210 2024-05-14T17:04:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff958e77640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff93c0b2a90 0x7ff93c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff953fff640 1 --2- 172.21.15.69:0/1298771617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce3590 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff94400bc60 tx=0x7ff94400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c21506a6e02323 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff9517fa640 1 -- 172.21.15.69:0/1537340507 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7ff94c103be0 con 0x7ff9541068f0 2024-05-14T17:04:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff93a7fc640 1 -- 172.21.15.69:0/1298771617 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944015070 con 0x55df76cd9210 2024-05-14T17:04:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff93a7fc640 1 -- 172.21.15.69:0/1298771617 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9440118f0 con 0x55df76cd9210 2024-05-14T17:04:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.206+0000 7ff93a7fc640 1 -- 172.21.15.69:0/1298771617 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9440107d0 con 0x55df76cd9210 2024-05-14T17:04:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff95d5f8740 1 -- 172.21.15.69:0/1298771617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 msgr2=0x55df76ce3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff958e77640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff93c0b2a90 0x7ff93c0b4f50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff94c1014e0 tx=0x7ff94c0978f0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/1298771617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce3590 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff94400bc60 tx=0x7ff94400f730 comp rx=0 tx=0).stop 2024-05-14T17:04:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff95d5f8740 1 -- 172.21.15.69:0/1298771617 shutdown_connections 2024-05-14T17:04:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/1298771617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76ce5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/1298771617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce3590 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/1298771617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cd8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff95d5f8740 1 -- 172.21.15.69:0/1298771617 >> 172.21.15.69:0/1298771617 conn(0x55df76cd4e00 msgr2=0x55df76cd51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff95d5f8740 1 -- 172.21.15.69:0/1298771617 shutdown_connections 2024-05-14T17:04:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff95d5f8740 1 -- 172.21.15.69:0/1298771617 wait complete. 2024-05-14T17:04:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.207+0000 7ff95d5f8740 1 -- start start 2024-05-14T17:04:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff95d5f8740 1 Processor -- start 2024-05-14T17:04:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76ce33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76ce0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df76921790 con 0x55df76cd9210 2024-05-14T17:04:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55df76bf7890 con 0x55df76ce3ad0 2024-05-14T17:04:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55df76ce67a0 con 0x55df76cd88f0 2024-05-14T17:04:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff95d5f8740 10 client.0 _ll_register_callbacks cb 0x55df76bfad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff953fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff953fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49334/0 (socket says 172.21.15.69:49334) 2024-05-14T17:04:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff953fff640 1 -- 172.21.15.69:0/3408952838 learned_addr learned my addr 172.21.15.69:0/3408952838 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff9537fe640 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76ce0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff9537fe640 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76ce33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff953fff640 1 -- 172.21.15.69:0/3408952838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 msgr2=0x55df76ce33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff953fff640 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76ce33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff953fff640 1 -- 172.21.15.69:0/3408952838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 msgr2=0x55df76ce0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff953fff640 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76ce0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff953fff640 1 -- 172.21.15.69:0/3408952838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df76a1c620 con 0x55df76cd9210 2024-05-14T17:04:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.208+0000 7ff9537fe640 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76ce33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff953fff640 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce06e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff94400bf80 tx=0x7ff94400bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b3505ac7f58c48 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff9337fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff9337fe640 1 -- 172.21.15.69:0/3408952838 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944015070 con 0x55df76cd9210 2024-05-14T17:04:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff9337fe640 1 -- 172.21.15.69:0/3408952838 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7ff944011aa0 con 0x55df76cd9210 2024-05-14T17:04:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff95d5f8740 1 -- 172.21.15.69:0/3408952838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9440118f0 con 0x55df76cd9210 2024-05-14T17:04:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff9337fe640 1 -- 172.21.15.69:0/3408952838 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff944011ca0 con 0x55df76cd9210 2024-05-14T17:04:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff95d5f8740 10 client.16866 fetch_fsmap learned FSMap version 53 2024-05-14T17:04:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff95d5f8740 1 -- 172.21.15.69:0/3408952838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55df76cf21b0 con 0x55df76cd9210 2024-05-14T17:04:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff9337fe640 1 -- 172.21.15.69:0/3408952838 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9440118f0 con 0x55df76cd9210 2024-05-14T17:04:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff9337fe640 1 -- 172.21.15.69:0/3408952838 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7ff944004990 con 0x55df76cd9210 2024-05-14T17:04:17.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff95d5f8740 10 client.16866 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T17:04:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff95d5f8740 10 client.16866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff95d5f8740 20 client.16866 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55df76cf35b0 0x55df76cf59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff95d5f8740 4 client.16866 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.209+0000 7ff95d5f8740 1 -- 172.21.15.69:0/3408952838 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df76c4db80 con 0x55df76cf35b0 2024-05-14T17:04:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.210+0000 7ff9537fe640 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55df76cf35b0 0x55df76cf59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.210+0000 7ff9337fe640 10 client.16866 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.210+0000 7ff9537fe640 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55df76cf35b0 0x55df76cf59a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5997c3608f157ab server_cookie=cbba3ae5affbd84d in_seq=0 out_seq=0 2024-05-14T17:04:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff9337fe640 1 -- 172.21.15.69:0/3408952838 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df76c4db80 con 0x55df76cf35b0 2024-05-14T17:04:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff9337fe640 10 client.16866 handle_command_reply: tid=0 2024-05-14T17:04:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 client.16866 shutdown 2024-05-14T17:04:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 -- 172.21.15.69:0/3408952838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 msgr2=0x55df76ce06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce06e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff94400bf80 tx=0x7ff94400bfb0 comp rx=0 tx=0).stop 2024-05-14T17:04:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 -- 172.21.15.69:0/3408952838 shutdown_connections 2024-05-14T17:04:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55df76cf35b0 0x55df76cf59a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76ce0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff9337fe640 0 client.16866 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce06e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/3408952838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76ce33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 -- 172.21.15.69:0/3408952838 >> 172.21.15.69:0/3408952838 conn(0x55df76cd4e00 msgr2=0x55df76cd4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 -- 172.21.15.69:0/3408952838 shutdown_connections 2024-05-14T17:04:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 1 -- 172.21.15.69:0/3408952838 wait complete. 2024-05-14T17:04:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.211+0000 7ff95d5f8740 20 client.16866 trim_cache size 0 max 16384 2024-05-14T17:04:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.226+0000 7ff95d5f8740 1 Processor -- start 2024-05-14T17:04:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.226+0000 7ff95d5f8740 1 -- start start 2024-05-14T17:04:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.226+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cd4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.226+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.226+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76cda9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.226+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541b3210 con 0x55df76cd9210 2024-05-14T17:04:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.226+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541b0a80 con 0x55df76ce3ad0 2024-05-14T17:04:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.226+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541ae8f0 con 0x55df76cd88f0 2024-05-14T17:04:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.226+0000 7ff953fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.226+0000 7ff953fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49342/0 (socket says 172.21.15.69:49342) 2024-05-14T17:04:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff953fff640 1 -- 172.21.15.69:0/2513808021 learned_addr learned my addr 172.21.15.69:0/2513808021 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff9537fe640 1 --2- 172.21.15.69:0/2513808021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76cda9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff953fff640 1 -- 172.21.15.69:0/2513808021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 msgr2=0x55df76cd4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff953fff640 1 --2- 172.21.15.69:0/2513808021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cd4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff953fff640 1 -- 172.21.15.69:0/2513808021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 msgr2=0x55df76cda9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff953fff640 1 --2- 172.21.15.69:0/2513808021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76cda9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff953fff640 1 -- 172.21.15.69:0/2513808021 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df76cdf690 con 0x55df76cd9210 2024-05-14T17:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff9537fe640 1 --2- 172.21.15.69:0/2513808021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76cda9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff953fff640 1 --2- 172.21.15.69:0/2513808021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce2f20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff944004f20 tx=0x7ff94400b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7cef2eaa85a33e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff939ffb640 1 -- 172.21.15.69:0/2513808021 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944015070 con 0x55df76cd9210 2024-05-14T17:04:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff939ffb640 1 -- 172.21.15.69:0/2513808021 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94400b460 con 0x55df76cd9210 2024-05-14T17:04:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.227+0000 7ff939ffb640 1 -- 172.21.15.69:0/2513808021 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944010990 con 0x55df76cd9210 2024-05-14T17:04:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2513808021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 msgr2=0x55df76ce2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/2513808021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce2f20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff944004f20 tx=0x7ff94400b9b0 comp rx=0 tx=0).stop 2024-05-14T17:04:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2513808021 shutdown_connections 2024-05-14T17:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/2513808021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76cda9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/2513808021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76ce2f20 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/2513808021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cd4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2513808021 >> 172.21.15.69:0/2513808021 conn(0x55df76cd4e00 msgr2=0x55df76cda2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2513808021 shutdown_connections 2024-05-14T17:04:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2513808021 wait complete. 2024-05-14T17:04:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 -- start start 2024-05-14T17:04:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 Processor -- start 2024-05-14T17:04:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.228+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76cdd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff95d5f8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76cddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541aef40 con 0x55df76cd9210 2024-05-14T17:04:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541b03b0 con 0x55df76ce3ad0 2024-05-14T17:04:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff95d5f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541af610 con 0x55df76cd88f0 2024-05-14T17:04:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff95d5f8740 10 client.0 _ll_register_callbacks cb 0x7ff94400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff953fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76cdd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff953fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76cdd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49348/0 (socket says 172.21.15.69:49348) 2024-05-14T17:04:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff953fff640 1 -- 172.21.15.69:0/2909682723 learned_addr learned my addr 172.21.15.69:0/2909682723 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff9537fe640 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff9537fe640 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76cddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff953fff640 1 -- 172.21.15.69:0/2909682723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 msgr2=0x55df76cec0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff953fff640 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cec0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff953fff640 1 -- 172.21.15.69:0/2909682723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 msgr2=0x55df76cddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff953fff640 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76cddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff953fff640 1 -- 172.21.15.69:0/2909682723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df76d5cda0 con 0x55df76cd9210 2024-05-14T17:04:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff953fff640 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76cdd440 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff944031000 tx=0x7ff944004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cc4ba905c70563 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff9337fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff9337fe640 1 -- 172.21.15.69:0/2909682723 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944015030 con 0x55df76cd9210 2024-05-14T17:04:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff9337fe640 1 -- 172.21.15.69:0/2909682723 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8542+0+0 (secure 0 0 0) 0x7ff9440119a0 con 0x55df76cd9210 2024-05-14T17:04:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2909682723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94400b460 con 0x55df76cd9210 2024-05-14T17:04:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff9337fe640 1 -- 172.21.15.69:0/2909682723 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff944004020 con 0x55df76cd9210 2024-05-14T17:04:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff95d5f8740 10 client.16872 fetch_fsmap learned FSMap version 53 2024-05-14T17:04:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.229+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2909682723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55df76d5de80 con 0x55df76cd9210 2024-05-14T17:04:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff9337fe640 1 -- 172.21.15.69:0/2909682723 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94400b460 con 0x55df76cd9210 2024-05-14T17:04:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff9337fe640 1 -- 172.21.15.69:0/2909682723 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7ff944034070 con 0x55df76cd9210 2024-05-14T17:04:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff95d5f8740 10 client.16872 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T17:04:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff95d5f8740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff95d5f8740 20 client.16872 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55df76d5e3e0 0x55df76d60800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff95d5f8740 4 client.16872 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2909682723 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df76c47360 con 0x55df76d5e3e0 2024-05-14T17:04:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff9537fe640 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55df76d5e3e0 0x55df76d60800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff9337fe640 10 client.16872 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:02.230+0000 7ff9537fe640 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55df76d5e3e0 0x55df76d60800 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ae52ace2b5ea1e server_cookie=521c1d3d9f184719 in_seq=0 out_seq=0 2024-05-14T17:04:20.711 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:01 smithi069 ceph-mon[31507]: pgmap v445: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:04:20.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:13 smithi203 ceph-mon[35787]: pgmap v453: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-14T17:04:20.777 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:14 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/476126762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff9337fe640 1 -- 172.21.15.69:0/2909682723 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55df76c47360 con 0x55df76d5e3e0 2024-05-14T17:04:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff9337fe640 10 client.16872 handle_command_reply: tid=0 2024-05-14T17:04:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 client.16872 shutdown 2024-05-14T17:04:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2909682723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 msgr2=0x55df76cdd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76cdd440 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff944031000 tx=0x7ff944004b40 comp rx=0 tx=0).stop 2024-05-14T17:04:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2909682723 shutdown_connections 2024-05-14T17:04:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55df76d5e3e0 0x55df76d60800 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df76ce3ad0 0x55df76cddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df76cd9210 0x55df76cdd440 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff9337fe640 0 client.16872 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 --2- 172.21.15.69:0/2909682723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df76cd88f0 0x55df76cec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2909682723 >> 172.21.15.69:0/2909682723 conn(0x55df76cd4e00 msgr2=0x55df76cdab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2909682723 shutdown_connections 2024-05-14T17:04:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 1 -- 172.21.15.69:0/2909682723 wait complete. 2024-05-14T17:04:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.637+0000 7ff95d5f8740 20 client.16872 trim_cache size 0 max 16384 2024-05-14T17:04:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:04:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:04:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:04:21.470 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:14 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 4. 2024-05-14T17:04:21.509 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:14 smithi203 systemd[1]: Stopped Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:04:21.545 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:14 smithi203 systemd[1]: Starting Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:04:21.578 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:15 smithi203 podman[73376]: 2024-05-14 17:04:15.095740471 +0000 UTC m=+0.023908635 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:04:21.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:15 smithi203 podman[73376]: 2024-05-14 17:04:15.230159191 +0000 UTC m=+0.158327344 container create e9116ee4de8e4dfa56d4ebd5e5b062d44a1a069e4b2516457f4b394132e42aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-14T17:04:21.653 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:15 smithi203 podman[73376]: 2024-05-14 17:04:15.508755907 +0000 UTC m=+0.436924051 container init e9116ee4de8e4dfa56d4ebd5e5b062d44a1a069e4b2516457f4b394132e42aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-14T17:04:21.686 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:15 smithi203 podman[73376]: 2024-05-14 17:04:15.511453833 +0000 UTC m=+0.439621978 container start e9116ee4de8e4dfa56d4ebd5e5b062d44a1a069e4b2516457f4b394132e42aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-14T17:04:21.736 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:15 smithi203 ceph-mds[73396]: -- 172.21.15.203:0/171517541 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e6cf389a40 con 0x55e6cf35cc00 2024-05-14T17:04:21.770 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:15 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73392]: starting mds.c at 2024-05-14T17:04:21.831 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:15 smithi203 bash[73376]: e9116ee4de8e4dfa56d4ebd5e5b062d44a1a069e4b2516457f4b394132e42aa6 2024-05-14T17:04:21.862 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:15 smithi203 systemd[1]: Started Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:04:21.896 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:13 smithi092 ceph-mon[35763]: pgmap v453: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-14T17:04:21.928 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:14 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/476126762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:21.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:15 smithi092 ceph-mon[35763]: pgmap v454: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-05-14T17:04:22.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:15 smithi092 ceph-mon[35763]: daemon mds.c restarted 2024-05-14T17:04:22.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:16 smithi092 ceph-mon[35763]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T17:04:22.070 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:16 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6836/2830800453,v1:172.21.15.203:6837/2830800453] up:boot 2024-05-14T17:04:22.103 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:16 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:04:22.143 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:16 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:04:22.178 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:16 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:22.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:17 smithi092 ceph-mon[35763]: pgmap v456: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:04:22.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:19 smithi092 ceph-mon[35763]: pgmap v457: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 KiB/s wr, 5 op/s 2024-05-14T17:04:22.305 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:15 smithi203 ceph-mon[35787]: pgmap v454: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-05-14T17:04:22.337 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:15 smithi203 ceph-mon[35787]: daemon mds.c restarted 2024-05-14T17:04:22.371 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:16 smithi203 ceph-mon[35787]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T17:04:22.404 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:16 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6836/2830800453,v1:172.21.15.203:6837/2830800453] up:boot 2024-05-14T17:04:22.452 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:16 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:04:22.487 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:16 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:04:22.520 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:16 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:22.554 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:17 smithi203 ceph-mon[35787]: pgmap v456: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:04:22.587 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:19 smithi203 ceph-mon[35787]: pgmap v457: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 KiB/s wr, 5 op/s 2024-05-14T17:04:22.621 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:21 smithi203 ceph-mon[35787]: pgmap v458: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 KiB/s wr, 5 op/s 2024-05-14T17:04:22.654 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:21 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:04:22.687 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:21 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:04:22.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:21 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:04:22.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:21 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:04:22.805 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:21 smithi092 ceph-mon[35763]: pgmap v458: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 KiB/s wr, 5 op/s 2024-05-14T17:04:22.838 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:21 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:04:22.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:21 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:04:22.904 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:21 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:04:23.004 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:21 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:04:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 -- 172.21.15.69:0/1537340507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff93c0b2a90 msgr2=0x7ff93c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff93c0b2a90 0x7ff93c0b4f50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff94c1014e0 tx=0x7ff94c0978f0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 -- 172.21.15.69:0/1537340507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9541068f0 msgr2=0x7ff954100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9541068f0 0x7ff954100220 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff94c005cc0 tx=0x7ff94c02f380 comp rx=0 tx=0).stop 2024-05-14T17:04:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 -- 172.21.15.69:0/1537340507 shutdown_connections 2024-05-14T17:04:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff93c0b2a90 0x7ff93c0b4f50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9541068f0 0x7ff954100220 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff954102660 0x7ff9540ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 --2- 172.21.15.69:0/1537340507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff954101ca0 0x7ff9540ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 -- 172.21.15.69:0/1537340507 >> 172.21.15.69:0/1537340507 conn(0x7ff954077750 msgr2=0x7ff95406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 -- 172.21.15.69:0/1537340507 shutdown_connections 2024-05-14T17:04:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:03.638+0000 7ff95a8da640 1 -- 172.21.15.69:0/1537340507 wait complete. 2024-05-14T17:04:23.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:03 smithi069 ceph-mon[31507]: pgmap v446: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 70 KiB/s wr, 30 op/s 2024-05-14T17:04:23.572 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 815.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:23.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:04 smithi069 ceph-mon[31507]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T17:04:23.638 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:05 smithi069 ceph-mon[31507]: pgmap v448: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 66 KiB/s wr, 29 op/s 2024-05-14T17:04:23.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:07 smithi069 ceph-mon[31507]: pgmap v449: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 87 KiB/s wr, 34 op/s 2024-05-14T17:04:23.721 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:09 smithi069 ceph-mon[31507]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T17:04:23.755 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:09 smithi069 ceph-mon[31507]: pgmap v451: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 43 op/s 2024-05-14T17:04:23.797 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 820.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:23.830 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:11 smithi069 ceph-mon[31507]: pgmap v452: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-05-14T17:04:23.864 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:23.897 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:13 smithi069 ceph-mon[31507]: pgmap v453: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-14T17:04:23.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.350+0000 7fa970de6700 1 -- 172.21.15.69:0/3259346282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa96c106ef0 msgr2=0x7fa96c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:23.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.350+0000 7fa970de6700 1 --2- 172.21.15.69:0/3259346282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa96c106ef0 0x7fa96c10eac0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c240 tx=0x7fa96400c550 comp rx=0 tx=0).stop 2024-05-14T17:04:23.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.350+0000 7fa970de6700 1 -- 172.21.15.69:0/3259346282 shutdown_connections 2024-05-14T17:04:24.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.350+0000 7fa970de6700 1 --2- 172.21.15.69:0/3259346282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa96c106ef0 0x7fa96c10eac0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.350+0000 7fa970de6700 1 --2- 172.21.15.69:0/3259346282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c101e60 0x7fa96c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.350+0000 7fa970de6700 1 --2- 172.21.15.69:0/3259346282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa96c102810 0x7fa96c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.350+0000 7fa970de6700 1 -- 172.21.15.69:0/3259346282 >> 172.21.15.69:0/3259346282 conn(0x7fa96c078ab0 msgr2=0x7fa96c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:24.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.350+0000 7fa970de6700 1 -- 172.21.15.69:0/3259346282 shutdown_connections 2024-05-14T17:04:24.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.350+0000 7fa970de6700 1 -- 172.21.15.69:0/3259346282 wait complete. 2024-05-14T17:04:24.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.351+0000 7fa970de6700 1 Processor -- start 2024-05-14T17:04:24.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.351+0000 7fa970de6700 1 -- start start 2024-05-14T17:04:24.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.351+0000 7fa970de6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa96c101e60 0x7fa96c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.351+0000 7fa970de6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c102810 0x7fa96c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.351+0000 7fa970de6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa96c106ef0 0x7fa96c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.351+0000 7fa970de6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c111570 con 0x7fa96c101e60 2024-05-14T17:04:24.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa970de6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c1113f0 con 0x7fa96c106ef0 2024-05-14T17:04:24.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa970de6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c1116f0 con 0x7fa96c102810 2024-05-14T17:04:24.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa96a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa96c101e60 0x7fa96c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa96a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa96c101e60 0x7fa96c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40298/0 (socket says 172.21.15.69:40298) 2024-05-14T17:04:24.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa96a59c700 1 -- 172.21.15.69:0/476126762 learned_addr learned my addr 172.21.15.69:0/476126762 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:24.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa969d9b700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c102810 0x7fa96c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa96ad9d700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa96c106ef0 0x7fa96c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa96a59c700 1 -- 172.21.15.69:0/476126762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c102810 msgr2=0x7fa96c19dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:24.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa96a59c700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c102810 0x7fa96c19dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa96a59c700 1 -- 172.21.15.69:0/476126762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa96c106ef0 msgr2=0x7fa96c197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa96a59c700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa96c106ef0 0x7fa96c197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa96a59c700 1 -- 172.21.15.69:0/476126762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c198070 con 0x7fa96c101e60 2024-05-14T17:04:24.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa96a59c700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa96c101e60 0x7fa96c19d6c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa960009970 tx=0x7fa960009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:24.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa95b7fe700 1 -- 172.21.15.69:0/476126762 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96001c070 con 0x7fa96c101e60 2024-05-14T17:04:25.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa970de6700 1 -- 172.21.15.69:0/476126762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96c198360 con 0x7fa96c101e60 2024-05-14T17:04:25.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa95b7fe700 1 -- 172.21.15.69:0/476126762 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa96000d260 con 0x7fa96c101e60 2024-05-14T17:04:25.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa95b7fe700 1 -- 172.21.15.69:0/476126762 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960017a60 con 0x7fa96c101e60 2024-05-14T17:04:25.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.352+0000 7fa970de6700 1 -- 172.21.15.69:0/476126762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96c1a4520 con 0x7fa96c101e60 2024-05-14T17:04:25.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.353+0000 7fa970de6700 1 -- 172.21.15.69:0/476126762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96c1032f0 con 0x7fa96c101e60 2024-05-14T17:04:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.355+0000 7fa95b7fe700 1 -- 172.21.15.69:0/476126762 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa96001f070 con 0x7fa96c101e60 2024-05-14T17:04:25.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.356+0000 7fa95b7fe700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9540b29f0 0x7fa9540b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.356+0000 7fa969d9b700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9540b29f0 0x7fa9540b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.356+0000 7fa95b7fe700 1 -- 172.21.15.69:0/476126762 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7fa9600e3b60 con 0x7fa96c101e60 2024-05-14T17:04:25.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.356+0000 7fa969d9b700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9540b29f0 0x7fa9540b4eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa96c104b30 tx=0x7fa95c009190 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:25.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.359+0000 7fa95b7fe700 1 -- 172.21.15.69:0/476126762 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9600ab530 con 0x7fa96c101e60 2024-05-14T17:04:25.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.624+0000 7fa970de6700 1 -- 172.21.15.69:0/476126762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa96c02e560 con 0x7fa96c101e60 2024-05-14T17:04:25.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.625+0000 7fa95b7fe700 1 -- 172.21.15.69:0/476126762 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11862 (secure 0 0 0) 0x7fa9600aae30 con 0x7fa96c101e60 2024-05-14T17:04:25.524 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:04:25.557 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:03:53.719188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25951":{"gid":25951,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3631522795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3631522795},{"type":"v1","addr":"172.21.15.203:6837","nonce":3631522795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24446,24422,24433]},"id":1}]} 2024-05-14T17:04:25.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.627+0000 7fa970de6700 1 -- 172.21.15.69:0/476126762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9540b29f0 msgr2=0x7fa9540b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:25.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.627+0000 7fa970de6700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9540b29f0 0x7fa9540b4eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa96c104b30 tx=0x7fa95c009190 comp rx=0 tx=0).stop 2024-05-14T17:04:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.627+0000 7fa970de6700 1 -- 172.21.15.69:0/476126762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa96c101e60 msgr2=0x7fa96c19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:25.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.627+0000 7fa970de6700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa96c101e60 0x7fa96c19d6c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa960009970 tx=0x7fa960009c80 comp rx=0 tx=0).stop 2024-05-14T17:04:25.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.628+0000 7fa970de6700 1 -- 172.21.15.69:0/476126762 shutdown_connections 2024-05-14T17:04:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.628+0000 7fa970de6700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9540b29f0 0x7fa9540b4eb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.628+0000 7fa970de6700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa96c101e60 0x7fa96c19d6c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.628+0000 7fa970de6700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa96c106ef0 0x7fa96c197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.628+0000 7fa970de6700 1 --2- 172.21.15.69:0/476126762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c102810 0x7fa96c19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.628+0000 7fa970de6700 1 -- 172.21.15.69:0/476126762 >> 172.21.15.69:0/476126762 conn(0x7fa96c078ab0 msgr2=0x7fa96c1056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:26.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.628+0000 7fa970de6700 1 -- 172.21.15.69:0/476126762 shutdown_connections 2024-05-14T17:04:26.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:14.628+0000 7fa970de6700 1 -- 172.21.15.69:0/476126762 wait complete. 2024-05-14T17:04:26.125 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 53 2024-05-14T17:04:26.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:14 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/476126762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:26.192 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id a8922838 --release --await' 2024-05-14T17:04:26.224 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id a8922838 --release --await 2024-05-14T17:04:26.257 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 825.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:26.291 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:15 smithi069 ceph-mon[31507]: pgmap v454: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-05-14T17:04:26.390 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:15 smithi069 ceph-mon[31507]: daemon mds.c restarted 2024-05-14T17:04:26.426 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:23 smithi092 ceph-mon[35763]: pgmap v459: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-14T17:04:26.469 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:25 smithi092 ceph-mon[35763]: pgmap v460: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-14T17:04:26.507 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:23 smithi203 ceph-mon[35787]: pgmap v459: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-14T17:04:26.541 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:25 smithi203 ceph-mon[35787]: pgmap v460: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-14T17:04:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.740+0000 7f4417b22640 1 -- 172.21.15.69:0/3695489011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4410101490 msgr2=0x7f44101018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.740+0000 7f4417b22640 1 --2- 172.21.15.69:0/3695489011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4410101490 0x7f44101018f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4400009900 tx=0x7f440002a410 comp rx=0 tx=0).stop 2024-05-14T17:04:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.740+0000 7f4417b22640 1 -- 172.21.15.69:0/3695489011 shutdown_connections 2024-05-14T17:04:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.740+0000 7f4417b22640 1 --2- 172.21.15.69:0/3695489011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f441010f6d0 0x7f4410111ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.740+0000 7f4417b22640 1 --2- 172.21.15.69:0/3695489011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4410101490 0x7f44101018f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.740+0000 7f4417b22640 1 --2- 172.21.15.69:0/3695489011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4410103b50 0x7f4410100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.740+0000 7f4417b22640 1 -- 172.21.15.69:0/3695489011 >> 172.21.15.69:0/3695489011 conn(0x7f4410077ed0 msgr2=0x7f44100ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.740+0000 7f4417b22640 1 -- 172.21.15.69:0/3695489011 shutdown_connections 2024-05-14T17:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.740+0000 7f4417b22640 1 -- 172.21.15.69:0/3695489011 wait complete. 2024-05-14T17:04:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f4417b22640 1 Processor -- start 2024-05-14T17:04:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f4417b22640 1 -- start start 2024-05-14T17:04:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f4417b22640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4410101490 0x7f441019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f4417b22640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4410103b50 0x7f441019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f4417b22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441010f6d0 0x7f441019fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f4417b22640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44101150d0 con 0x7f441010f6d0 2024-05-14T17:04:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f4417b22640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4410114f50 con 0x7f4410103b50 2024-05-14T17:04:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f4417b22640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4410115250 con 0x7f4410101490 2024-05-14T17:04:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f44160bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441010f6d0 0x7f441019fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f44160bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441010f6d0 0x7f441019fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35468/0 (socket says 172.21.15.69:35468) 2024-05-14T17:04:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f44160bf640 1 -- 172.21.15.69:0/455191290 learned_addr learned my addr 172.21.15.69:0/455191290 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f44158be640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4410101490 0x7f441019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f44150bd640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4410103b50 0x7f441019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f44160bf640 1 -- 172.21.15.69:0/455191290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4410101490 msgr2=0x7f441019b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f44160bf640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4410101490 0x7f441019b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f44160bf640 1 -- 172.21.15.69:0/455191290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4410103b50 msgr2=0x7f441019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f44160bf640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4410103b50 0x7f441019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.741+0000 7f44160bf640 1 -- 172.21.15.69:0/455191290 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44101a05c0 con 0x7f441010f6d0 2024-05-14T17:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.742+0000 7f44160bf640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441010f6d0 0x7f441019fe90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f440800d750 tx=0x7f440800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.742+0000 7f4406ffd640 1 -- 172.21.15.69:0/455191290 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408017070 con 0x7f441010f6d0 2024-05-14T17:04:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.742+0000 7f4406ffd640 1 -- 172.21.15.69:0/455191290 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44080054d0 con 0x7f441010f6d0 2024-05-14T17:04:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.742+0000 7f4417b22640 1 -- 172.21.15.69:0/455191290 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44101a08b0 con 0x7f441010f6d0 2024-05-14T17:04:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.742+0000 7f4406ffd640 1 -- 172.21.15.69:0/455191290 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408004c60 con 0x7f441010f6d0 2024-05-14T17:04:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.742+0000 7f4417b22640 1 -- 172.21.15.69:0/455191290 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44101a8140 con 0x7f441010f6d0 2024-05-14T17:04:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.744+0000 7f4406ffd640 1 -- 172.21.15.69:0/455191290 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4408009bd0 con 0x7f441010f6d0 2024-05-14T17:04:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f441a834740 1 Processor -- start 2024-05-14T17:04:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f441a834740 1 -- start start 2024-05-14T17:04:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f441a834740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f3a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f441a834740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f441a834740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f441a834740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55662f316c30 con 0x55662f3a4210 2024-05-14T17:04:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f441a834740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55662f3067d0 con 0x55662f3a38f0 2024-05-14T17:04:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f441a834740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55662f30a810 con 0x55662f3aead0 2024-05-14T17:04:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f44158be640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f44150bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f3a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f44158be640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35480/0 (socket says 172.21.15.69:35480) 2024-05-14T17:04:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f44158be640 1 -- 172.21.15.69:0/896192831 learned_addr learned my addr 172.21.15.69:0/896192831 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.745+0000 7f4406ffd640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f43fc0b2a90 0x7f43fc0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f44150bd640 1 --2- 172.21.15.69:0/896192831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f44160bf640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f43fc0b2a90 0x7f43fc0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f44158be640 1 -- 172.21.15.69:0/896192831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 msgr2=0x55662f3b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f44158be640 1 --2- 172.21.15.69:0/896192831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f44158be640 1 -- 172.21.15.69:0/896192831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 msgr2=0x55662f3a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f44158be640 1 --2- 172.21.15.69:0/896192831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f3a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f44158be640 1 -- 172.21.15.69:0/896192831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662f01b210 con 0x55662f3a4210 2024-05-14T17:04:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f4406ffd640 1 -- 172.21.15.69:0/455191290 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f44080e1e80 con 0x7f441010f6d0 2024-05-14T17:04:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f44150bd640 1 --2- 172.21.15.69:0/896192831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f3a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:04:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f44158be640 1 --2- 172.21.15.69:0/896192831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3ae590 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f440c0039f0 tx=0x7f440c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303234929f6ade95 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f44160bf640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f43fc0b2a90 0x7f43fc0b4f50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f44080df690 tx=0x7f4408076d40 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f43fbfff640 1 -- 172.21.15.69:0/896192831 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c004b20 con 0x55662f3a4210 2024-05-14T17:04:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f43fbfff640 1 -- 172.21.15.69:0/896192831 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440c004cc0 con 0x55662f3a4210 2024-05-14T17:04:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.746+0000 7f43fbfff640 1 -- 172.21.15.69:0/896192831 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c0126b0 con 0x55662f3a4210 2024-05-14T17:04:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 -- 172.21.15.69:0/896192831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 msgr2=0x55662f3ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 --2- 172.21.15.69:0/896192831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3ae590 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f440c0039f0 tx=0x7f440c003eb0 comp rx=0 tx=0).stop 2024-05-14T17:04:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 -- 172.21.15.69:0/896192831 shutdown_connections 2024-05-14T17:04:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 --2- 172.21.15.69:0/896192831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 --2- 172.21.15.69:0/896192831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3ae590 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 --2- 172.21.15.69:0/896192831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f3a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 -- 172.21.15.69:0/896192831 >> 172.21.15.69:0/896192831 conn(0x55662f39fe00 msgr2=0x55662f3a01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 -- 172.21.15.69:0/896192831 shutdown_connections 2024-05-14T17:04:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 -- 172.21.15.69:0/896192831 wait complete. 2024-05-14T17:04:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 -- start start 2024-05-14T17:04:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.747+0000 7f441a834740 1 Processor -- start 2024-05-14T17:04:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f441a834740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a38f0 0x55662f3ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f441a834740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3a4210 0x55662f3ab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f441a834740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3aead0 0x55662f3abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f441a834740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55662efec790 con 0x55662f3a38f0 2024-05-14T17:04:29.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f441a834740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55662f2c2890 con 0x55662f3aead0 2024-05-14T17:04:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f441a834740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55662f3b17a0 con 0x55662f3a4210 2024-05-14T17:04:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f441a834740 10 client.0 _ll_register_callbacks cb 0x55662f2c5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f44150bd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a38f0 0x55662f3ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f44150bd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a38f0 0x55662f3ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35482/0 (socket says 172.21.15.69:35482) 2024-05-14T17:04:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f44150bd640 1 -- 172.21.15.69:0/559592880 learned_addr learned my addr 172.21.15.69:0/559592880 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f44150bd640 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3aead0 0x55662f3abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f44150bd640 1 -- 172.21.15.69:0/559592880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3a4210 msgr2=0x55662f3ab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f44158be640 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3a4210 0x55662f3ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f44150bd640 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3a4210 0x55662f3ab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f44150bd640 1 -- 172.21.15.69:0/559592880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3aead0 msgr2=0x55662f3abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f44150bd640 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3aead0 0x55662f3abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.748+0000 7f44150bd640 1 -- 172.21.15.69:0/559592880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662f0e7620 con 0x55662f3a38f0 2024-05-14T17:04:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f44150bd640 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a38f0 0x55662f3ae3a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4400005c40 tx=0x7f440003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7690869c3c4071 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f43f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f43f97fa640 1 -- 172.21.15.69:0/559592880 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400035040 con 0x55662f3a38f0 2024-05-14T17:04:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f441a834740 1 -- 172.21.15.69:0/559592880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440c004cc0 con 0x55662f3a38f0 2024-05-14T17:04:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f43f97fa640 1 -- 172.21.15.69:0/559592880 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f4400004480 con 0x55662f3a38f0 2024-05-14T17:04:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f43f97fa640 1 -- 172.21.15.69:0/559592880 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f44000035f0 con 0x55662f3a38f0 2024-05-14T17:04:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f441a834740 10 client.16896 fetch_fsmap learned FSMap version 55 2024-05-14T17:04:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f441a834740 1 -- 172.21.15.69:0/559592880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55662f3bd190 con 0x55662f3a38f0 2024-05-14T17:04:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f43f97fa640 1 -- 172.21.15.69:0/559592880 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440c004cc0 con 0x55662f3a38f0 2024-05-14T17:04:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f43f97fa640 1 -- 172.21.15.69:0/559592880 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f440003a040 con 0x55662f3a38f0 2024-05-14T17:04:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f441a834740 10 client.16896 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:04:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f441a834740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f441a834740 20 client.16896 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f441a834740 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55662f3be6f0 0x55662f3c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f441a834740 4 client.16896 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.749+0000 7f441a834740 1 -- 172.21.15.69:0/559592880 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55662f318b80 con 0x55662f3be6f0 2024-05-14T17:04:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.750+0000 7f44158be640 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55662f3be6f0 0x55662f3c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.750+0000 7f44158be640 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55662f3be6f0 0x55662f3c0b60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfbcfb4be13bfdf5 server_cookie=b3eaddd0b168d728 in_seq=0 out_seq=0 2024-05-14T17:04:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.750+0000 7f43f97fa640 10 client.16896 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f43f97fa640 1 -- 172.21.15.69:0/559592880 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55662f318b80 con 0x55662f3be6f0 2024-05-14T17:04:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f43f97fa640 10 client.16896 handle_command_reply: tid=0 2024-05-14T17:04:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 client.16896 shutdown 2024-05-14T17:04:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 -- 172.21.15.69:0/559592880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a38f0 msgr2=0x55662f3ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a38f0 0x55662f3ae3a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4400005c40 tx=0x7f440003f040 comp rx=0 tx=0).stop 2024-05-14T17:04:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 -- 172.21.15.69:0/559592880 shutdown_connections 2024-05-14T17:04:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55662f3be6f0 0x55662f3c0b60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3aead0 0x55662f3abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3a4210 0x55662f3ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f43f97fa640 0 client.16896 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 --2- 172.21.15.69:0/559592880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a38f0 0x55662f3ae3a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 -- 172.21.15.69:0/559592880 >> 172.21.15.69:0/559592880 conn(0x55662f39fe00 msgr2=0x55662f39f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 -- 172.21.15.69:0/559592880 shutdown_connections 2024-05-14T17:04:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 1 -- 172.21.15.69:0/559592880 wait complete. 2024-05-14T17:04:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.751+0000 7f441a834740 20 client.16896 trim_cache size 0 max 16384 2024-05-14T17:04:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.766+0000 7f441a834740 1 Processor -- start 2024-05-14T17:04:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.766+0000 7f441a834740 1 -- start start 2024-05-14T17:04:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f441a834740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f3ba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f441a834740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3aded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f441a834740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3c6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f441a834740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44101b3160 con 0x55662f3a4210 2024-05-14T17:04:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f441a834740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44101b09d0 con 0x55662f3a38f0 2024-05-14T17:04:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f441a834740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44101ae820 con 0x55662f3aead0 2024-05-14T17:04:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44158be640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44158be640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35490/0 (socket says 172.21.15.69:35490) 2024-05-14T17:04:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44158be640 1 -- 172.21.15.69:0/4035628415 learned_addr learned my addr 172.21.15.69:0/4035628415 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44150bd640 1 --2- 172.21.15.69:0/4035628415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44158be640 1 -- 172.21.15.69:0/4035628415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 msgr2=0x55662f3c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44158be640 1 --2- 172.21.15.69:0/4035628415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44158be640 1 -- 172.21.15.69:0/4035628415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 msgr2=0x55662f3ba670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44158be640 1 --2- 172.21.15.69:0/4035628415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f3ba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44158be640 1 -- 172.21.15.69:0/4035628415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662f3a4950 con 0x55662f3a4210 2024-05-14T17:04:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44150bd640 1 --2- 172.21.15.69:0/4035628415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f44158be640 1 --2- 172.21.15.69:0/4035628415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3aded0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f440c004410 tx=0x7f440c009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c93ea51b3d5467 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f43fb7fe640 1 -- 172.21.15.69:0/4035628415 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c028110 con 0x55662f3a4210 2024-05-14T17:04:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f43fb7fe640 1 -- 172.21.15.69:0/4035628415 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440c02d070 con 0x55662f3a4210 2024-05-14T17:04:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.767+0000 7f43fb7fe640 1 -- 172.21.15.69:0/4035628415 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c012870 con 0x55662f3a4210 2024-05-14T17:04:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 -- 172.21.15.69:0/4035628415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 msgr2=0x55662f3aded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 --2- 172.21.15.69:0/4035628415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3aded0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f440c004410 tx=0x7f440c009aa0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 -- 172.21.15.69:0/4035628415 shutdown_connections 2024-05-14T17:04:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 --2- 172.21.15.69:0/4035628415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 --2- 172.21.15.69:0/4035628415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f3aded0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 --2- 172.21.15.69:0/4035628415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f3ba670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 -- 172.21.15.69:0/4035628415 >> 172.21.15.69:0/4035628415 conn(0x55662f39fe00 msgr2=0x55662f3a6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 -- 172.21.15.69:0/4035628415 shutdown_connections 2024-05-14T17:04:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 -- 172.21.15.69:0/4035628415 wait complete. 2024-05-14T17:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 -- start start 2024-05-14T17:04:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.768+0000 7f441a834740 1 Processor -- start 2024-05-14T17:04:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f441a834740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f421350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f441a834740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f421890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f441a834740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f441a834740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44101aee90 con 0x55662f3a4210 2024-05-14T17:04:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f441a834740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44101b0300 con 0x55662f3a38f0 2024-05-14T17:04:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f441a834740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44101af560 con 0x55662f3aead0 2024-05-14T17:04:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f441a834740 10 client.0 _ll_register_callbacks cb 0x7f44000317f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44158be640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f421890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44158be640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f421890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35494/0 (socket says 172.21.15.69:35494) 2024-05-14T17:04:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44158be640 1 -- 172.21.15.69:0/2666904818 learned_addr learned my addr 172.21.15.69:0/2666904818 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44150bd640 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f421350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44150bd640 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44158be640 1 -- 172.21.15.69:0/2666904818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 msgr2=0x55662f3b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44158be640 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44158be640 1 -- 172.21.15.69:0/2666904818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 msgr2=0x55662f421350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44158be640 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f421350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44158be640 1 -- 172.21.15.69:0/2666904818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662f3b26a0 con 0x55662f3a4210 2024-05-14T17:04:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44150bd640 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f44158be640 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f421890 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f440c0047c0 tx=0x7f440c0047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd12f239f9ae1df server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f43f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f43f97fa640 1 -- 172.21.15.69:0/2666904818 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c012a00 con 0x55662f3a4210 2024-05-14T17:04:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f43f97fa640 1 -- 172.21.15.69:0/2666904818 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f440c012ba0 con 0x55662f3a4210 2024-05-14T17:04:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.769+0000 7f441a834740 1 -- 172.21.15.69:0/2666904818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440c02d070 con 0x55662f3a4210 2024-05-14T17:04:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f43f97fa640 1 -- 172.21.15.69:0/2666904818 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f440c012da0 con 0x55662f3a4210 2024-05-14T17:04:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f441a834740 10 client.16902 fetch_fsmap learned FSMap version 55 2024-05-14T17:04:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f441a834740 1 -- 172.21.15.69:0/2666904818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55662f428b20 con 0x55662f3a4210 2024-05-14T17:04:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f43f97fa640 1 -- 172.21.15.69:0/2666904818 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440c02d070 con 0x55662f3a4210 2024-05-14T17:04:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f43f97fa640 1 -- 172.21.15.69:0/2666904818 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f440c00f040 con 0x55662f3a4210 2024-05-14T17:04:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f441a834740 10 client.16902 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:04:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f441a834740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f441a834740 20 client.16902 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f441a834740 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55662f429120 0x55662f42b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f441a834740 4 client.16902 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f441a834740 1 -- 172.21.15.69:0/2666904818 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55662f312360 con 0x55662f429120 2024-05-14T17:04:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.770+0000 7f44150bd640 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55662f429120 0x55662f42b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.771+0000 7f44150bd640 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55662f429120 0x55662f42b540 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def26a82fe97480b server_cookie=ca5678a53cbeaebb in_seq=0 out_seq=0 2024-05-14T17:04:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.771+0000 7f43f97fa640 10 client.16902 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.782+0000 7f43f97fa640 1 -- 172.21.15.69:0/2666904818 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55662f312360 con 0x55662f429120 2024-05-14T17:04:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.782+0000 7f43f97fa640 10 client.16902 handle_command_reply: tid=0 2024-05-14T17:04:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.782+0000 7f441a834740 1 client.16902 shutdown 2024-05-14T17:04:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.782+0000 7f441a834740 1 -- 172.21.15.69:0/2666904818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 msgr2=0x55662f421890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.782+0000 7f441a834740 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f421890 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f440c0047c0 tx=0x7f440c0047f0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f441a834740 1 -- 172.21.15.69:0/2666904818 shutdown_connections 2024-05-14T17:04:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f441a834740 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55662f429120 0x55662f42b540 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f441a834740 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662f3aead0 0x55662f3b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f441a834740 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55662f3a4210 0x55662f421890 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f43f97fa640 0 client.16902 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f441a834740 1 --2- 172.21.15.69:0/2666904818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55662f3a38f0 0x55662f421350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f441a834740 1 -- 172.21.15.69:0/2666904818 >> 172.21.15.69:0/2666904818 conn(0x55662f39fe00 msgr2=0x55662f3a5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f441a834740 1 -- 172.21.15.69:0/2666904818 shutdown_connections 2024-05-14T17:04:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f441a834740 1 -- 172.21.15.69:0/2666904818 wait complete. 2024-05-14T17:04:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f441a834740 20 client.16902 trim_cache size 0 max 16384 2024-05-14T17:04:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:04:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:04:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:04:36.352 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:27 smithi203 ceph-mon[35787]: pgmap v461: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-05-14T17:04:36.392 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:29 smithi203 ceph-mon[35787]: pgmap v462: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-14T17:04:36.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:31 smithi203 ceph-mon[35787]: pgmap v463: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-14T17:04:36.475 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:33 smithi203 ceph-mon[35787]: pgmap v464: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-05-14T17:04:36.517 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:27 smithi092 ceph-mon[35763]: pgmap v461: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-05-14T17:04:36.597 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:29 smithi092 ceph-mon[35763]: pgmap v462: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-14T17:04:36.634 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:31 smithi092 ceph-mon[35763]: pgmap v463: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-14T17:04:36.717 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:33 smithi092 ceph-mon[35763]: pgmap v464: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-05-14T17:04:36.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:35 smithi092 ceph-mon[35763]: pgmap v465: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 KiB/s wr, 9 op/s 2024-05-14T17:04:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f4417b22640 1 -- 172.21.15.69:0/455191290 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f43fc0b2a90 msgr2=0x7f43fc0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f4417b22640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f43fc0b2a90 0x7f43fc0b4f50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f44080df690 tx=0x7f4408076d40 comp rx=0 tx=0).stop 2024-05-14T17:04:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f4417b22640 1 -- 172.21.15.69:0/455191290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441010f6d0 msgr2=0x7f441019fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f4417b22640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441010f6d0 0x7f441019fe90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f440800d750 tx=0x7f440800dc10 comp rx=0 tx=0).stop 2024-05-14T17:04:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f4417b22640 1 -- 172.21.15.69:0/455191290 shutdown_connections 2024-05-14T17:04:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f4417b22640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f43fc0b2a90 0x7f43fc0b4f50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f4417b22640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441010f6d0 0x7f441019fe90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f4417b22640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4410103b50 0x7f441019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f4417b22640 1 --2- 172.21.15.69:0/455191290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4410101490 0x7f441019b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.783+0000 7f4417b22640 1 -- 172.21.15.69:0/455191290 >> 172.21.15.69:0/455191290 conn(0x7f4410077ed0 msgr2=0x7f4410110af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.784+0000 7f4417b22640 1 -- 172.21.15.69:0/455191290 shutdown_connections 2024-05-14T17:04:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:16.784+0000 7f4417b22640 1 -- 172.21.15.69:0/455191290 wait complete. 2024-05-14T17:04:37.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:16 smithi069 ceph-mon[31507]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T17:04:37.368 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:16 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6836/2830800453,v1:172.21.15.203:6837/2830800453] up:boot 2024-05-14T17:04:37.409 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:16 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:04:37.491 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:16 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:04:37.526 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:16 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:37.568 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:17 smithi069 ceph-mon[31507]: pgmap v456: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:04:37.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:19 smithi069 ceph-mon[31507]: pgmap v457: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 KiB/s wr, 5 op/s 2024-05-14T17:04:37.652 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 830.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:37.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:21 smithi069 ceph-mon[31507]: pgmap v458: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 KiB/s wr, 5 op/s 2024-05-14T17:04:37.735 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:21 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:04:37.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:21 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:04:37.835 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:21 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:04:37.876 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:21 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:04:37.918 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:23 smithi069 ceph-mon[31507]: pgmap v459: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-14T17:04:37.960 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 835.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:38.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:35 smithi203 ceph-mon[35787]: pgmap v465: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 KiB/s wr, 9 op/s 2024-05-14T17:04:38.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:25 smithi069 ceph-mon[31507]: pgmap v460: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-14T17:04:38.068 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:27 smithi069 ceph-mon[31507]: pgmap v461: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-05-14T17:04:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.882+0000 7f8bf27c1640 1 -- 172.21.15.69:0/1124962628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bec101950 msgr2=0x7f8bec101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.882+0000 7f8bf27c1640 1 --2- 172.21.15.69:0/1124962628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bec101950 0x7f8bec101dd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009930 tx=0x7f8be002a420 comp rx=0 tx=0).stop 2024-05-14T17:04:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.882+0000 7f8bf27c1640 1 -- 172.21.15.69:0/1124962628 shutdown_connections 2024-05-14T17:04:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.882+0000 7f8bf27c1640 1 --2- 172.21.15.69:0/1124962628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bec102310 0x7f8bec10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.882+0000 7f8bf27c1640 1 --2- 172.21.15.69:0/1124962628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bec101950 0x7f8bec101dd0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.882+0000 7f8bf27c1640 1 --2- 172.21.15.69:0/1124962628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec107960 0x7f8bec107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.882+0000 7f8bf27c1640 1 -- 172.21.15.69:0/1124962628 >> 172.21.15.69:0/1124962628 conn(0x7f8bec078ad0 msgr2=0x7f8bec078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.882+0000 7f8bf27c1640 1 -- 172.21.15.69:0/1124962628 shutdown_connections 2024-05-14T17:04:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.882+0000 7f8bf27c1640 1 -- 172.21.15.69:0/1124962628 wait complete. 2024-05-14T17:04:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf27c1640 1 Processor -- start 2024-05-14T17:04:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf27c1640 1 -- start start 2024-05-14T17:04:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf27c1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec101950 0x7f8bec199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf27c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bec102310 0x7f8bec199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf27c1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bec107960 0x7f8bec19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf27c1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec112d20 con 0x7f8bec107960 2024-05-14T17:04:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf27c1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec112ba0 con 0x7f8bec102310 2024-05-14T17:04:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf27c1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec112ea0 con 0x7f8bec101950 2024-05-14T17:04:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf0d5e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bec107960 0x7f8bec19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf0d5e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bec107960 0x7f8bec19dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40904/0 (socket says 172.21.15.69:40904) 2024-05-14T17:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8bf0d5e640 1 -- 172.21.15.69:0/3073603342 learned_addr learned my addr 172.21.15.69:0/3073603342 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.883+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bec102310 0x7f8bec199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8bebfff640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec101950 0x7f8bec199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8bf0d5e640 1 -- 172.21.15.69:0/3073603342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec101950 msgr2=0x7f8bec199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8bf0d5e640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec101950 0x7f8bec199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8bf0d5e640 1 -- 172.21.15.69:0/3073603342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bec102310 msgr2=0x7f8bec199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8bf0d5e640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bec102310 0x7f8bec199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8bf0d5e640 1 -- 172.21.15.69:0/3073603342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec19e1f0 con 0x7f8bec107960 2024-05-14T17:04:39.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8bebfff640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec101950 0x7f8bec199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8bf0d5e640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bec107960 0x7f8bec19dac0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8be400eaa0 tx=0x7f8be400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8be97fa640 1 -- 172.21.15.69:0/3073603342 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be400cdb0 con 0x7f8bec107960 2024-05-14T17:04:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8bf27c1640 1 -- 172.21.15.69:0/3073603342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bec19e4e0 con 0x7f8bec107960 2024-05-14T17:04:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8be97fa640 1 -- 172.21.15.69:0/3073603342 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be4005020 con 0x7f8bec107960 2024-05-14T17:04:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8be97fa640 1 -- 172.21.15.69:0/3073603342 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40116d0 con 0x7f8bec107960 2024-05-14T17:04:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.884+0000 7f8bf27c1640 1 -- 172.21.15.69:0/3073603342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bec1a5d70 con 0x7f8bec107960 2024-05-14T17:04:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.886+0000 7f8be97fa640 1 -- 172.21.15.69:0/3073603342 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8be40051d0 con 0x7f8bec107960 2024-05-14T17:04:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bf55ba740 1 Processor -- start 2024-05-14T17:04:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bf55ba740 1 -- start start 2024-05-14T17:04:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe78f0 0x558f8afe7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe8210 0x558f8aff2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8aff2ad0 0x558f8aff4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558f8af5ac30 con 0x558f8afe8210 2024-05-14T17:04:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558f8af4a7d0 con 0x558f8aff2ad0 2024-05-14T17:04:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558f8af4e810 con 0x558f8afe78f0 2024-05-14T17:04:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bebfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe8210 0x558f8aff2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bebfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe8210 0x558f8aff2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40916/0 (socket says 172.21.15.69:40916) 2024-05-14T17:04:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bebfff640 1 -- 172.21.15.69:0/3187005921 learned_addr learned my addr 172.21.15.69:0/3187005921 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8be97fa640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8bd40b2a10 0x7f8bd40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/3187005921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8aff2ad0 0x558f8aff4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/3187005921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe78f0 0x558f8afe7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bf0d5e640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8bd40b2a10 0x7f8bd40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bebfff640 1 -- 172.21.15.69:0/3187005921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe78f0 msgr2=0x558f8afe7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bebfff640 1 --2- 172.21.15.69:0/3187005921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe78f0 0x558f8afe7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bebfff640 1 -- 172.21.15.69:0/3187005921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8aff2ad0 msgr2=0x558f8aff4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bebfff640 1 --2- 172.21.15.69:0/3187005921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8aff2ad0 0x558f8aff4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8bebfff640 1 -- 172.21.15.69:0/3187005921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8ac5f210 con 0x558f8afe8210 2024-05-14T17:04:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.887+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/3187005921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8aff2ad0 0x558f8aff4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.888+0000 7f8be97fa640 1 -- 172.21.15.69:0/3073603342 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f8be40e2810 con 0x7f8bec107960 2024-05-14T17:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.888+0000 7f8bebfff640 1 --2- 172.21.15.69:0/3187005921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe8210 0x558f8aff2590 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009bd0 tx=0x7f8bdc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939c088c037c7f6c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.888+0000 7f8bd27fc640 1 -- 172.21.15.69:0/3187005921 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc016070 con 0x558f8afe8210 2024-05-14T17:04:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.888+0000 7f8bf0d5e640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8bd40b2a10 0x7f8bd40b4ed0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8be400efd0 tx=0x7f8be40cc040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.888+0000 7f8bd27fc640 1 -- 172.21.15.69:0/3187005921 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc011920 con 0x558f8afe8210 2024-05-14T17:04:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.888+0000 7f8bd27fc640 1 -- 172.21.15.69:0/3187005921 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc010890 con 0x558f8afe8210 2024-05-14T17:04:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3187005921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe8210 msgr2=0x558f8aff2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/3187005921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe8210 0x558f8aff2590 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009bd0 tx=0x7f8bdc00e6c0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3187005921 shutdown_connections 2024-05-14T17:04:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/3187005921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8aff2ad0 0x558f8aff4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/3187005921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe8210 0x558f8aff2590 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/3187005921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe78f0 0x558f8afe7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3187005921 >> 172.21.15.69:0/3187005921 conn(0x558f8afe3e00 msgr2=0x558f8afe41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3187005921 shutdown_connections 2024-05-14T17:04:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3187005921 wait complete. 2024-05-14T17:04:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 -- start start 2024-05-14T17:04:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 Processor -- start 2024-05-14T17:04:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe8210 0x558f8afef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8aff2ad0 0x558f8afefc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558f8ac30790 con 0x558f8afe78f0 2024-05-14T17:04:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558f8af06890 con 0x558f8afe8210 2024-05-14T17:04:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.889+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558f8aff57a0 con 0x558f8aff2ad0 2024-05-14T17:04:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8bf55ba740 10 client.0 _ll_register_callbacks cb 0x558f8af09d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8beb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8beb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40922/0 (socket says 172.21.15.69:40922) 2024-05-14T17:04:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8beb7fe640 1 -- 172.21.15.69:0/3141822423 learned_addr learned my addr 172.21.15.69:0/3141822423 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8bebfff640 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe8210 0x558f8afef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8aff2ad0 0x558f8afefc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8beb7fe640 1 -- 172.21.15.69:0/3141822423 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8aff2ad0 msgr2=0x558f8afefc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8aff2ad0 0x558f8afefc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8beb7fe640 1 -- 172.21.15.69:0/3141822423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe8210 msgr2=0x558f8afef6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe8210 0x558f8afef6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8beb7fe640 1 -- 172.21.15.69:0/3141822423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f8ad2b620 con 0x558f8afe78f0 2024-05-14T17:04:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff23d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8be002a940 tx=0x7f8be0041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a140a60631a7c254 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8bcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/3141822423 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0035040 con 0x558f8afe78f0 2024-05-14T17:04:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/3141822423 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f8be0004250 con 0x558f8afe78f0 2024-05-14T17:04:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3141822423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bdc011920 con 0x558f8afe78f0 2024-05-14T17:04:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/3141822423 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8be0052050 con 0x558f8afe78f0 2024-05-14T17:04:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8bf55ba740 10 client.16914 fetch_fsmap learned FSMap version 55 2024-05-14T17:04:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.890+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3141822423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558f8b001160 con 0x558f8afe78f0 2024-05-14T17:04:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/3141822423 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc011920 con 0x558f8afe78f0 2024-05-14T17:04:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/3141822423 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f8be003d450 con 0x558f8afe78f0 2024-05-14T17:04:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bf55ba740 10 client.16914 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:04:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bf55ba740 10 client.16914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bf55ba740 20 client.16914 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f8b002670 0x558f8b004ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bf55ba740 4 client.16914 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3141822423 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f8af5cb80 con 0x558f8b002670 2024-05-14T17:04:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bebfff640 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f8b002670 0x558f8b004ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bebfff640 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f8b002670 0x558f8b004ac0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76dfec94f2cc6440 server_cookie=f6c7b36f068f7a5b in_seq=0 out_seq=0 2024-05-14T17:04:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.891+0000 7f8bcb7fe640 10 client.16914 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/3141822423 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f8af5cb80 con 0x558f8b002670 2024-05-14T17:04:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bcb7fe640 10 client.16914 handle_command_reply: tid=0 2024-05-14T17:04:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 client.16914 shutdown 2024-05-14T17:04:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3141822423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 msgr2=0x558f8aff23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff23d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8be002a940 tx=0x7f8be0041040 comp rx=0 tx=0).stop 2024-05-14T17:04:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3141822423 shutdown_connections 2024-05-14T17:04:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f8b002670 0x558f8b004ac0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8aff2ad0 0x558f8afefc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe8210 0x558f8afef6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bcb7fe640 0 client.16914 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/3141822423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff23d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3141822423 >> 172.21.15.69:0/3141822423 conn(0x558f8afe3e00 msgr2=0x558f8afe3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3141822423 shutdown_connections 2024-05-14T17:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.892+0000 7f8bf55ba740 1 -- 172.21.15.69:0/3141822423 wait complete. 2024-05-14T17:04:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.893+0000 7f8bf55ba740 20 client.16914 trim_cache size 0 max 16384 2024-05-14T17:04:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.907+0000 7f8bf55ba740 1 Processor -- start 2024-05-14T17:04:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.907+0000 7f8bf55ba740 1 -- start start 2024-05-14T17:04:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe78f0 0x558f8affe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe8210 0x558f8aff1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8aff2ad0 0x558f8b00a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1b0f10 con 0x558f8aff2ad0 2024-05-14T17:04:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1ae780 con 0x558f8afe78f0 2024-05-14T17:04:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1ac5d0 con 0x558f8afe8210 2024-05-14T17:04:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8beb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8aff2ad0 0x558f8b00a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8beb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8aff2ad0 0x558f8b00a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40932/0 (socket says 172.21.15.69:40932) 2024-05-14T17:04:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8beb7fe640 1 -- 172.21.15.69:0/4029570831 learned_addr learned my addr 172.21.15.69:0/4029570831 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bebfff640 1 --2- 172.21.15.69:0/4029570831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe8210 0x558f8aff1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8beb7fe640 1 -- 172.21.15.69:0/4029570831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe8210 msgr2=0x558f8aff1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/4029570831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe8210 0x558f8aff1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8beb7fe640 1 -- 172.21.15.69:0/4029570831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe78f0 msgr2=0x558f8affe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/4029570831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe78f0 0x558f8affe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8beb7fe640 1 -- 172.21.15.69:0/4029570831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8afe3c30 con 0x558f8aff2ad0 2024-05-14T17:04:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bebfff640 1 --2- 172.21.15.69:0/4029570831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe8210 0x558f8aff1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/4029570831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8aff2ad0 0x558f8b00a390 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8be00096b0 tx=0x7f8be0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46511b49d49e607 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bd1ffb640 1 -- 172.21.15.69:0/4029570831 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0035070 con 0x558f8aff2ad0 2024-05-14T17:04:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bd1ffb640 1 -- 172.21.15.69:0/4029570831 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be00054e0 con 0x558f8aff2ad0 2024-05-14T17:04:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.908+0000 7f8bd1ffb640 1 -- 172.21.15.69:0/4029570831 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0004250 con 0x558f8aff2ad0 2024-05-14T17:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.909+0000 7f8bf55ba740 1 -- 172.21.15.69:0/4029570831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8aff2ad0 msgr2=0x558f8b00a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.909+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/4029570831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8aff2ad0 0x558f8b00a390 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8be00096b0 tx=0x7f8be0000bc0 comp rx=0 tx=0).stop 2024-05-14T17:04:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.909+0000 7f8bf55ba740 1 -- 172.21.15.69:0/4029570831 shutdown_connections 2024-05-14T17:04:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.909+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/4029570831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8aff2ad0 0x558f8b00a390 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.909+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/4029570831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8afe8210 0x558f8aff1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.909+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/4029570831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe78f0 0x558f8affe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.909+0000 7f8bf55ba740 1 -- 172.21.15.69:0/4029570831 >> 172.21.15.69:0/4029570831 conn(0x558f8afe3e00 msgr2=0x558f8afe9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.909+0000 7f8bf55ba740 1 -- 172.21.15.69:0/4029570831 shutdown_connections 2024-05-14T17:04:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.909+0000 7f8bf55ba740 1 -- 172.21.15.69:0/4029570831 wait complete. 2024-05-14T17:04:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.909+0000 7f8bf55ba740 1 -- start start 2024-05-14T17:04:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8bf55ba740 1 Processor -- start 2024-05-14T17:04:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe8210 0x558f8aff6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8bf55ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8aff2ad0 0x558f8aff6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1acc40 con 0x558f8afe78f0 2024-05-14T17:04:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1ae0b0 con 0x558f8afe8210 2024-05-14T17:04:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8bf55ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1ad310 con 0x558f8aff2ad0 2024-05-14T17:04:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8bf55ba740 10 client.0 _ll_register_callbacks cb 0x7f8be0031f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8beb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8beb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40934/0 (socket says 172.21.15.69:40934) 2024-05-14T17:04:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8beb7fe640 1 -- 172.21.15.69:0/92430912 learned_addr learned my addr 172.21.15.69:0/92430912 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8bebfff640 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe8210 0x558f8aff6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8beb7fe640 1 -- 172.21.15.69:0/92430912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8aff2ad0 msgr2=0x558f8aff6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8aff2ad0 0x558f8aff6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8beb7fe640 1 -- 172.21.15.69:0/92430912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe8210 msgr2=0x558f8aff6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe8210 0x558f8aff6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8beb7fe640 1 -- 172.21.15.69:0/92430912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f8b06bf40 con 0x558f8afe78f0 2024-05-14T17:04:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8beb7fe640 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff6570 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8be0001eb0 tx=0x7f8be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60690dc9a06fd12 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.910+0000 7f8bcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/92430912 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0035070 con 0x558f8afe78f0 2024-05-14T17:04:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/92430912 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f8be0004250 con 0x558f8afe78f0 2024-05-14T17:04:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bf55ba740 1 -- 172.21.15.69:0/92430912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be00054e0 con 0x558f8afe78f0 2024-05-14T17:04:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/92430912 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8be0041070 con 0x558f8afe78f0 2024-05-14T17:04:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bf55ba740 10 client.16920 fetch_fsmap learned FSMap version 55 2024-05-14T17:04:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bf55ba740 1 -- 172.21.15.69:0/92430912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558f8b06d190 con 0x558f8afe78f0 2024-05-14T17:04:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/92430912 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be00054e0 con 0x558f8afe78f0 2024-05-14T17:04:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/92430912 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f8be0067470 con 0x558f8afe78f0 2024-05-14T17:04:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bf55ba740 10 client.16920 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:04:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bf55ba740 10 client.16920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bf55ba740 20 client.16920 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f8b06d800 0x558f8b06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bf55ba740 4 client.16920 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bf55ba740 1 -- 172.21.15.69:0/92430912 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f8afeff90 con 0x558f8b06d800 2024-05-14T17:04:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.911+0000 7f8bebfff640 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f8b06d800 0x558f8b06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:45.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.912+0000 7f8bcb7fe640 10 client.16920 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.912+0000 7f8bebfff640 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f8b06d800 0x558f8b06fc20 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db7086b603823c4 server_cookie=6177da3b815333ec in_seq=0 out_seq=0 2024-05-14T17:04:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bcb7fe640 1 -- 172.21.15.69:0/92430912 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558f8afeff90 con 0x558f8b06d800 2024-05-14T17:04:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bcb7fe640 10 client.16920 handle_command_reply: tid=0 2024-05-14T17:04:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 client.16920 shutdown 2024-05-14T17:04:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 -- 172.21.15.69:0/92430912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 msgr2=0x558f8aff6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff6570 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8be0001eb0 tx=0x7f8be00046c0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 -- 172.21.15.69:0/92430912 shutdown_connections 2024-05-14T17:04:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558f8b06d800 0x558f8b06fc20 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f8aff2ad0 0x558f8aff6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558f8afe8210 0x558f8aff6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 --2- 172.21.15.69:0/92430912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f8afe78f0 0x558f8aff6570 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bcb7fe640 0 client.16920 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 -- 172.21.15.69:0/92430912 >> 172.21.15.69:0/92430912 conn(0x558f8afe3e00 msgr2=0x558f8afe92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 -- 172.21.15.69:0/92430912 shutdown_connections 2024-05-14T17:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.923+0000 7f8bf55ba740 1 -- 172.21.15.69:0/92430912 wait complete. 2024-05-14T17:04:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf55ba740 20 client.16920 trim_cache size 0 max 16384 2024-05-14T17:04:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:04:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:04:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:04:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 -- 172.21.15.69:0/3073603342 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8bd40b2a10 msgr2=0x7f8bd40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8bd40b2a10 0x7f8bd40b4ed0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8be400efd0 tx=0x7f8be40cc040 comp rx=0 tx=0).stop 2024-05-14T17:04:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 -- 172.21.15.69:0/3073603342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bec107960 msgr2=0x7f8bec19dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bec107960 0x7f8bec19dac0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8be400eaa0 tx=0x7f8be400ef60 comp rx=0 tx=0).stop 2024-05-14T17:04:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 -- 172.21.15.69:0/3073603342 shutdown_connections 2024-05-14T17:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8bd40b2a10 0x7f8bd40b4ed0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bec107960 0x7f8bec19dac0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bec102310 0x7f8bec199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 --2- 172.21.15.69:0/3073603342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bec101950 0x7f8bec199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 -- 172.21.15.69:0/3073603342 >> 172.21.15.69:0/3073603342 conn(0x7f8bec078ad0 msgr2=0x7f8bec104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 -- 172.21.15.69:0/3073603342 shutdown_connections 2024-05-14T17:04:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:29.924+0000 7f8bf27c1640 1 -- 172.21.15.69:0/3073603342 wait complete. 2024-05-14T17:04:46.935 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:29 smithi069 ceph-mon[31507]: pgmap v462: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-14T17:04:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 840.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:47.018 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:31 smithi069 ceph-mon[31507]: pgmap v463: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-14T17:04:47.051 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:33 smithi069 ceph-mon[31507]: pgmap v464: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-05-14T17:04:47.084 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 845.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:47.118 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:35 smithi069 ceph-mon[31507]: pgmap v465: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 KiB/s wr, 9 op/s 2024-05-14T17:04:47.152 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:47.185 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:37 smithi069 ceph-mon[31507]: pgmap v466: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-05-14T17:04:47.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:39 smithi069 ceph-mon[31507]: pgmap v467: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-05-14T17:04:47.253 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:37 smithi203 ceph-mon[35787]: pgmap v466: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-05-14T17:04:47.301 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:39 smithi203 ceph-mon[35787]: pgmap v467: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-05-14T17:04:47.335 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:41 smithi203 ceph-mon[35787]: pgmap v468: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-05-14T17:04:47.368 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:43 smithi203 ceph-mon[35787]: pgmap v469: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-05-14T17:04:47.401 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:44 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:47.456 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:46 smithi203 ceph-mon[35787]: pgmap v470: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-05-14T17:04:47.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:37 smithi092 ceph-mon[35763]: pgmap v466: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-05-14T17:04:47.529 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:39 smithi092 ceph-mon[35763]: pgmap v467: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-05-14T17:04:47.568 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:41 smithi092 ceph-mon[35763]: pgmap v468: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-05-14T17:04:47.601 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:43 smithi092 ceph-mon[35763]: pgmap v469: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-05-14T17:04:47.643 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:44 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:47.676 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:46 smithi092 ceph-mon[35763]: pgmap v470: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-05-14T17:04:47.710 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73392]: *** Caught signal (Segmentation fault) ** 2024-05-14T17:04:47.743 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73392]: in thread 7fe49bc0b700 thread_name:MR_Finisher 2024-05-14T17:04:47.820 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:46 smithi203 podman[73576]: 2024-05-14 17:04:46.234465406 +0000 UTC m=+0.044384870 container died e9116ee4de8e4dfa56d4ebd5e5b062d44a1a069e4b2516457f4b394132e42aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:04:47.861 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:47 smithi203 podman[73576]: 2024-05-14 17:04:47.476730785 +0000 UTC m=+1.286650244 container remove e9116ee4de8e4dfa56d4ebd5e5b062d44a1a069e4b2516457f4b394132e42aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-14T17:04:47.893 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:47 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-05-14T17:04:47.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.129+0000 7f2741a0c700 1 -- 172.21.15.69:0/1362742638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f273c1095b0 msgr2=0x7f273c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:47.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.129+0000 7f2741a0c700 1 --2- 172.21.15.69:0/1362742638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f273c1095b0 0x7f273c109990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f272c009990 tx=0x7f272c009ca0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.129+0000 7f2741a0c700 1 -- 172.21.15.69:0/1362742638 shutdown_connections 2024-05-14T17:04:48.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.129+0000 7f2741a0c700 1 --2- 172.21.15.69:0/1362742638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f273c1095b0 0x7f273c109990 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.129+0000 7f2741a0c700 1 --2- 172.21.15.69:0/1362742638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f273c06c650 0x7f273c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.129+0000 7f2741a0c700 1 --2- 172.21.15.69:0/1362742638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c06bc90 0x7f273c06c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.129+0000 7f2741a0c700 1 -- 172.21.15.69:0/1362742638 >> 172.21.15.69:0/1362742638 conn(0x7f273c079410 msgr2=0x7f273c079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:48.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.129+0000 7f2741a0c700 1 -- 172.21.15.69:0/1362742638 shutdown_connections 2024-05-14T17:04:48.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.129+0000 7f2741a0c700 1 -- 172.21.15.69:0/1362742638 wait complete. 2024-05-14T17:04:48.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.129+0000 7f2741a0c700 1 Processor -- start 2024-05-14T17:04:48.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f2741a0c700 1 -- start start 2024-05-14T17:04:48.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f2741a0c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c06bc90 0x7f273c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:48.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f2741a0c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f273c06c650 0x7f273c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:48.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f2741a0c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f273c1095b0 0x7f273c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:48.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f2741a0c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1137c0 con 0x7f273c06c650 2024-05-14T17:04:48.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f2741a0c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f273c113640 con 0x7f273c1095b0 2024-05-14T17:04:48.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f2741a0c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f273c113940 con 0x7f273c06bc90 2024-05-14T17:04:48.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f273c06c650 0x7f273c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:48.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f273c06c650 0x7f273c19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44408/0 (socket says 172.21.15.69:44408) 2024-05-14T17:04:48.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273a7fc700 1 -- 172.21.15.69:0/3839880216 learned_addr learned my addr 172.21.15.69:0/3839880216 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:48.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273affd700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c06bc90 0x7f273c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:48.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273a7fc700 1 -- 172.21.15.69:0/3839880216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c06bc90 msgr2=0x7f273c199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:48.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273a7fc700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c06bc90 0x7f273c199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273b7fe700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f273c1095b0 0x7f273c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:48.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273a7fc700 1 -- 172.21.15.69:0/3839880216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f273c1095b0 msgr2=0x7f273c19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273a7fc700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f273c1095b0 0x7f273c19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:49.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273a7fc700 1 -- 172.21.15.69:0/3839880216 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c19ed40 con 0x7f273c06c650 2024-05-14T17:04:49.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273affd700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c06bc90 0x7f273c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:49.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273b7fe700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f273c1095b0 0x7f273c19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:49.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f273a7fc700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f273c06c650 0x7f273c19a270 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f273000b530 tx=0x7f273000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:49.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.130+0000 7f2740a0a700 1 -- 172.21.15.69:0/3839880216 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730015020 con 0x7f273c06c650 2024-05-14T17:04:49.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.131+0000 7f2740a0a700 1 -- 172.21.15.69:0/3839880216 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2730011ce0 con 0x7f273c06c650 2024-05-14T17:04:49.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.131+0000 7f2740a0a700 1 -- 172.21.15.69:0/3839880216 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730010d20 con 0x7f273c06c650 2024-05-14T17:04:49.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.131+0000 7f2741a0c700 1 -- 172.21.15.69:0/3839880216 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273c19f030 con 0x7f273c06c650 2024-05-14T17:04:49.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.131+0000 7f2741a0c700 1 -- 172.21.15.69:0/3839880216 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273c1a68b0 con 0x7f273c06c650 2024-05-14T17:04:49.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.133+0000 7f2740a0a700 1 -- 172.21.15.69:0/3839880216 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2730018020 con 0x7f273c06c650 2024-05-14T17:04:49.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.134+0000 7f2740a0a700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f27240b2a70 0x7f27240b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:49.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.134+0000 7f273affd700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f27240b2a70 0x7f27240b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.134+0000 7f2740a0a700 1 -- 172.21.15.69:0/3839880216 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f27300e1df0 con 0x7f273c06c650 2024-05-14T17:04:49.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.134+0000 7f274c2a2b80 1 Processor -- start 2024-05-14T17:04:49.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.134+0000 7f274c2a2b80 1 -- start start 2024-05-14T17:04:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273affd700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f27240b2a70 0x7f27240b4f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f272c006010 tx=0x7f272c00b540 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:49.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 0x557579480800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:49.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x55757947bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:49.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x55757947c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:49.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5575793d9e70 con 0x55757947c2c0 2024-05-14T17:04:49.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5575793c9a10 con 0x557579480d40 2024-05-14T17:04:49.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5575793cda50 con 0x557579480420 2024-05-14T17:04:49.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x55757947c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:49.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x55757947c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44422/0 (socket says 172.21.15.69:44422) 2024-05-14T17:04:49.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273b7fe700 1 -- 172.21.15.69:0/4262714714 learned_addr learned my addr 172.21.15.69:0/4262714714 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:50.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273affd700 1 --2- 172.21.15.69:0/4262714714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 0x557579480800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:50.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273a7fc700 1 --2- 172.21.15.69:0/4262714714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x55757947bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:50.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273b7fe700 1 -- 172.21.15.69:0/4262714714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 msgr2=0x557579480800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:50.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273b7fe700 1 --2- 172.21.15.69:0/4262714714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 0x557579480800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273b7fe700 1 -- 172.21.15.69:0/4262714714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 msgr2=0x55757947bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:50.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273b7fe700 1 --2- 172.21.15.69:0/4262714714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x55757947bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273b7fe700 1 -- 172.21.15.69:0/4262714714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575793730b0 con 0x55757947c2c0 2024-05-14T17:04:50.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273a7fc700 1 --2- 172.21.15.69:0/4262714714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x55757947bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:50.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f273b7fe700 1 --2- 172.21.15.69:0/4262714714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x55757947c720 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2734007f30 tx=0x7f273400c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dff905712bcf0b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:50.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f2721ffb700 1 -- 172.21.15.69:0/4262714714 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734014070 con 0x55757947c2c0 2024-05-14T17:04:50.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f2721ffb700 1 -- 172.21.15.69:0/4262714714 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273400ce10 con 0x55757947c2c0 2024-05-14T17:04:50.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.135+0000 7f2721ffb700 1 -- 172.21.15.69:0/4262714714 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273400e990 con 0x55757947c2c0 2024-05-14T17:04:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.136+0000 7f274c2a2b80 1 -- 172.21.15.69:0/4262714714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 msgr2=0x55757947c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:50.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.136+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/4262714714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x55757947c720 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2734007f30 tx=0x7f273400c5a0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.136+0000 7f274c2a2b80 1 -- 172.21.15.69:0/4262714714 shutdown_connections 2024-05-14T17:04:50.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.136+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/4262714714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x55757947c720 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.136+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/4262714714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 0x557579480800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.136+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/4262714714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x55757947bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.136+0000 7f274c2a2b80 1 -- 172.21.15.69:0/4262714714 >> 172.21.15.69:0/4262714714 conn(0x5575793a8eb0 msgr2=0x5575794764a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:50.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.136+0000 7f274c2a2b80 1 -- 172.21.15.69:0/4262714714 shutdown_connections 2024-05-14T17:04:50.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.136+0000 7f274c2a2b80 1 -- 172.21.15.69:0/4262714714 wait complete. 2024-05-14T17:04:50.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f274c2a2b80 1 -- start start 2024-05-14T17:04:50.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f274c2a2b80 1 Processor -- start 2024-05-14T17:04:50.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x557579487b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:50.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 0x557579484f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x557579485470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5575792d00a0 con 0x55757947c2c0 2024-05-14T17:04:51.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557579331ea0 con 0x557579480d40 2024-05-14T17:04:51.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557579488880 con 0x557579480420 2024-05-14T17:04:51.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f274c2a2b80 10 client.0 _ll_register_callbacks cb 0x557579364a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:51.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f273affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x557579487b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:51.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f273affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x557579487b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44438/0 (socket says 172.21.15.69:44438) 2024-05-14T17:04:51.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f273affd700 1 -- 172.21.15.69:0/1734723904 learned_addr learned my addr 172.21.15.69:0/1734723904 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f273a7fc700 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 0x557579484f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:51.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.137+0000 7f273b7fe700 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x557579485470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:51.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f273affd700 1 -- 172.21.15.69:0/1734723904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 msgr2=0x557579484f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:51.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f273affd700 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 0x557579484f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f273affd700 1 -- 172.21.15.69:0/1734723904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 msgr2=0x557579485470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f273affd700 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x557579485470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f273affd700 1 -- 172.21.15.69:0/1734723904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575793199d0 con 0x55757947c2c0 2024-05-14T17:04:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f273a7fc700 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 0x557579484f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:04:51.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f273affd700 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x557579487b90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f272c000c00 tx=0x7f272c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685fd05fb354cd69 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:51.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f271affd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:51.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f271affd700 1 -- 172.21.15.69:0/1734723904 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c035060 con 0x55757947c2c0 2024-05-14T17:04:51.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f271affd700 1 -- 172.21.15.69:0/1734723904 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f272c003860 con 0x55757947c2c0 2024-05-14T17:04:51.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f274c2a2b80 1 -- 172.21.15.69:0/1734723904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f273400ce10 con 0x55757947c2c0 2024-05-14T17:04:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f271affd700 1 -- 172.21.15.69:0/1734723904 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f272c003a60 con 0x55757947c2c0 2024-05-14T17:04:51.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f274c2a2b80 10 client.16932 fetch_fsmap learned FSMap version 55 2024-05-14T17:04:51.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f274c2a2b80 1 -- 172.21.15.69:0/1734723904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557579383860 con 0x55757947c2c0 2024-05-14T17:04:51.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.138+0000 7f271affd700 1 -- 172.21.15.69:0/1734723904 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273400ce10 con 0x55757947c2c0 2024-05-14T17:04:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.139+0000 7f271affd700 1 -- 172.21.15.69:0/1734723904 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f272c0304f0 con 0x55757947c2c0 2024-05-14T17:04:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.139+0000 7f274c2a2b80 10 client.16932 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:04:52.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.139+0000 7f274c2a2b80 10 client.16932 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:04:52.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.139+0000 7f274c2a2b80 20 client.16932 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:52.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.139+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5575794954c0 0x5575794978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:52.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.139+0000 7f274c2a2b80 4 client.16932 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:52.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.139+0000 7f274c2a2b80 1 -- 172.21.15.69:0/1734723904 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575793dbdc0 con 0x5575794954c0 2024-05-14T17:04:52.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.139+0000 7f273a7fc700 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5575794954c0 0x5575794978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:52.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.139+0000 7f271affd700 10 client.16932 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:52.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.139+0000 7f273a7fc700 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5575794954c0 0x5575794978b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ef032f09b80d086 server_cookie=31316734001826ca in_seq=0 out_seq=0 2024-05-14T17:04:52.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f271affd700 1 -- 172.21.15.69:0/1734723904 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5575793dbdc0 con 0x5575794954c0 2024-05-14T17:04:52.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f271affd700 10 client.16932 handle_command_reply: tid=0 2024-05-14T17:04:52.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 client.16932 shutdown 2024-05-14T17:04:52.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 -- 172.21.15.69:0/1734723904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 msgr2=0x557579487b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:52.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x557579487b90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f272c000c00 tx=0x7f272c01a050 comp rx=0 tx=0).stop 2024-05-14T17:04:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 -- 172.21.15.69:0/1734723904 shutdown_connections 2024-05-14T17:04:52.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5575794954c0 0x5575794978b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55757947c2c0 0x557579487b90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x557579485470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/1734723904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557579480420 0x557579484f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 -- 172.21.15.69:0/1734723904 >> 172.21.15.69:0/1734723904 conn(0x5575793a8eb0 msgr2=0x557579470fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f271affd700 0 client.16932 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:52.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 -- 172.21.15.69:0/1734723904 shutdown_connections 2024-05-14T17:04:52.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 1 -- 172.21.15.69:0/1734723904 wait complete. 2024-05-14T17:04:52.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.140+0000 7f274c2a2b80 20 client.16932 trim_cache size 0 max 16384 2024-05-14T17:04:52.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f274c2a2b80 1 Processor -- start 2024-05-14T17:04:52.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f274c2a2b80 1 -- start start 2024-05-14T17:04:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757947c2c0 0x55757947f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:53.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480420 0x5575794710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x55757948def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:53.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1b0780 con 0x557579480420 2024-05-14T17:04:53.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1adfb0 con 0x557579480d40 2024-05-14T17:04:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1abe00 con 0x55757947c2c0 2024-05-14T17:04:53.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f273a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480420 0x5575794710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f273a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480420 0x5575794710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44440/0 (socket says 172.21.15.69:44440) 2024-05-14T17:04:53.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.160+0000 7f273a7fc700 1 -- 172.21.15.69:0/3324716416 learned_addr learned my addr 172.21.15.69:0/3324716416 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:53.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.161+0000 7f273affd700 1 --2- 172.21.15.69:0/3324716416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757947c2c0 0x55757947f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:53.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.161+0000 7f273a7fc700 1 -- 172.21.15.69:0/3324716416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757947c2c0 msgr2=0x55757947f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.161+0000 7f273a7fc700 1 --2- 172.21.15.69:0/3324716416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757947c2c0 0x55757947f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.161+0000 7f273a7fc700 1 -- 172.21.15.69:0/3324716416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 msgr2=0x55757948def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:53.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.161+0000 7f273a7fc700 1 --2- 172.21.15.69:0/3324716416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x55757948def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.161+0000 7f273a7fc700 1 -- 172.21.15.69:0/3324716416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557579485e60 con 0x557579480420 2024-05-14T17:04:53.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.161+0000 7f273a7fc700 1 --2- 172.21.15.69:0/3324716416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480420 0x5575794710d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f27300a8e40 tx=0x7f273001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7dcb074f434040 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:53.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.161+0000 7f27217fa700 1 -- 172.21.15.69:0/3324716416 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27300b1ff0 con 0x557579480420 2024-05-14T17:04:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.161+0000 7f27217fa700 1 -- 172.21.15.69:0/3324716416 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27300ad160 con 0x557579480420 2024-05-14T17:04:53.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.161+0000 7f27217fa700 1 -- 172.21.15.69:0/3324716416 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27300b1b60 con 0x557579480420 2024-05-14T17:04:53.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 -- 172.21.15.69:0/3324716416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480420 msgr2=0x5575794710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:53.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/3324716416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480420 0x5575794710d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f27300a8e40 tx=0x7f273001c040 comp rx=0 tx=0).stop 2024-05-14T17:04:53.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 -- 172.21.15.69:0/3324716416 shutdown_connections 2024-05-14T17:04:53.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/3324716416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480d40 0x55757948def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/3324716416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757947c2c0 0x55757947f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/3324716416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480420 0x5575794710d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 -- 172.21.15.69:0/3324716416 >> 172.21.15.69:0/3324716416 conn(0x5575793a8eb0 msgr2=0x55757947b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:53.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 -- 172.21.15.69:0/3324716416 shutdown_connections 2024-05-14T17:04:53.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 -- 172.21.15.69:0/3324716416 wait complete. 2024-05-14T17:04:54.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 -- start start 2024-05-14T17:04:54.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.162+0000 7f274c2a2b80 1 Processor -- start 2024-05-14T17:04:54.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757947c2c0 0x55757947a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:54.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480420 0x55757947aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:54.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f274c2a2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480d40 0x5575794fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1ac470 con 0x557579480d40 2024-05-14T17:04:54.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1ad8e0 con 0x557579480420 2024-05-14T17:04:54.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f274c2a2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1acb40 con 0x55757947c2c0 2024-05-14T17:04:54.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f274c2a2b80 10 client.0 _ll_register_callbacks cb 0x557579364a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:54.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480d40 0x5575794fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:54.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480d40 0x5575794fdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44442/0 (socket says 172.21.15.69:44442) 2024-05-14T17:04:54.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273b7fe700 1 -- 172.21.15.69:0/922470652 learned_addr learned my addr 172.21.15.69:0/922470652 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:54.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273affd700 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757947c2c0 0x55757947a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:54.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273a7fc700 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480420 0x55757947aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:54.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273b7fe700 1 -- 172.21.15.69:0/922470652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757947c2c0 msgr2=0x55757947a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:54.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273b7fe700 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757947c2c0 0x55757947a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:54.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273b7fe700 1 -- 172.21.15.69:0/922470652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480420 msgr2=0x55757947aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:54.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273b7fe700 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480420 0x55757947aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:54.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273b7fe700 1 -- 172.21.15.69:0/922470652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575794fe520 con 0x557579480d40 2024-05-14T17:04:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273a7fc700 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480420 0x55757947aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:54.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f273b7fe700 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480d40 0x5575794fdeb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2734007f30 tx=0x7f273400eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199857f85b118a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:54.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f271affd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:04:54.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f271affd700 1 -- 172.21.15.69:0/922470652 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734014040 con 0x557579480d40 2024-05-14T17:04:54.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f271affd700 1 -- 172.21.15.69:0/922470652 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f273400f440 con 0x557579480d40 2024-05-14T17:04:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.163+0000 7f274c2a2b80 1 -- 172.21.15.69:0/922470652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27300ad160 con 0x557579480d40 2024-05-14T17:04:54.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f271affd700 1 -- 172.21.15.69:0/922470652 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2734031050 con 0x557579480d40 2024-05-14T17:04:55.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f274c2a2b80 10 client.16938 fetch_fsmap learned FSMap version 55 2024-05-14T17:04:55.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f274c2a2b80 1 -- 172.21.15.69:0/922470652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5575795058e0 con 0x557579480d40 2024-05-14T17:04:55.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f271affd700 1 -- 172.21.15.69:0/922470652 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27300ad160 con 0x557579480d40 2024-05-14T17:04:55.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f271affd700 1 -- 172.21.15.69:0/922470652 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f2734023450 con 0x557579480d40 2024-05-14T17:04:55.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f274c2a2b80 10 client.16938 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:04:55.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f274c2a2b80 10 client.16938 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:04:55.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f274c2a2b80 20 client.16938 populate_metadata read hostname 'smithi069' 2024-05-14T17:04:55.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x557579505db0 0x5575795081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:55.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f274c2a2b80 4 client.16938 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a8922838", "release": true, "await": true}] 2024-05-14T17:04:55.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f274c2a2b80 1 -- 172.21.15.69:0/922470652 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a8922838", "release": true, "await": true}) -- 0x5575793d55a0 con 0x557579505db0 2024-05-14T17:04:55.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.164+0000 7f273affd700 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x557579505db0 0x5575795081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:55.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.165+0000 7f271affd700 10 client.16938 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:55.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.165+0000 7f273affd700 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x557579505db0 0x5575795081d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9374cd04e4af353e server_cookie=b046bc5f26d9d277 in_seq=0 out_seq=0 2024-05-14T17:04:55.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f271affd700 1 -- 172.21.15.69:0/922470652 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5575793d55a0 con 0x557579505db0 2024-05-14T17:04:55.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f271affd700 10 client.16938 handle_command_reply: tid=0 2024-05-14T17:04:55.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 client.16938 shutdown 2024-05-14T17:04:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 -- 172.21.15.69:0/922470652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480d40 msgr2=0x5575794fdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:55.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480d40 0x5575794fdeb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2734007f30 tx=0x7f273400eaf0 comp rx=0 tx=0).stop 2024-05-14T17:04:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 -- 172.21.15.69:0/922470652 shutdown_connections 2024-05-14T17:04:55.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x557579505db0 0x5575795081d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557579480d40 0x5575794fdeb0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:55.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757947c2c0 0x55757947a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 --2- 172.21.15.69:0/922470652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557579480420 0x55757947aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:55.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f271affd700 0 client.16938 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:04:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 -- 172.21.15.69:0/922470652 >> 172.21.15.69:0/922470652 conn(0x5575793a8eb0 msgr2=0x5575794700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:55.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 -- 172.21.15.69:0/922470652 shutdown_connections 2024-05-14T17:04:55.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 1 -- 172.21.15.69:0/922470652 wait complete. 2024-05-14T17:04:55.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.169+0000 7f274c2a2b80 20 client.16938 trim_cache size 0 max 16384 2024-05-14T17:04:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:04:55.659 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 55, 2024-05-14T17:04:55.681 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T17:04:55.740 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 40, 2024-05-14T17:04:55.756 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T17:04:55.772 INFO:teuthology.orchestra.run.smithi069.stdout: "a8922838": { 2024-05-14T17:04:55.789 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 40, 2024-05-14T17:04:55.806 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T17:04:55.839 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:04:55.856 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T17:04:55.873 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:04:55.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:04:55.906 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T17:04:55.923 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 149.7, 2024-05-14T17:04:55.939 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T17:04:55.973 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T17:04:55.989 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T17:04:56.006 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:04:56.035 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T17:04:56.056 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:04:56.115 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:04:56.139 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:04:56.156 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:04:56.173 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:04:56.189 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:04:56.206 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:04:56.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 -- 172.21.15.69:0/3839880216 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f27240b2a70 msgr2=0x7f27240b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:56.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f27240b2a70 0x7f27240b4f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f272c006010 tx=0x7f272c00b540 comp rx=0 tx=0).stop 2024-05-14T17:04:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 -- 172.21.15.69:0/3839880216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f273c06c650 msgr2=0x7f273c19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:56.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f273c06c650 0x7f273c19a270 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f273000b530 tx=0x7f273000b8f0 comp rx=0 tx=0).stop 2024-05-14T17:04:56.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 -- 172.21.15.69:0/3839880216 shutdown_connections 2024-05-14T17:04:56.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f273c1095b0 0x7f273c19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:56.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c06bc90 0x7f273c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:56.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f27240b2a70 0x7f27240b4f30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:56.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 --2- 172.21.15.69:0/3839880216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f273c06c650 0x7f273c19a270 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:56.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 -- 172.21.15.69:0/3839880216 >> 172.21.15.69:0/3839880216 conn(0x7f273c079410 msgr2=0x7f273c101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 -- 172.21.15.69:0/3839880216 shutdown_connections 2024-05-14T17:04:56.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:04:40.170+0000 7f2741a0c700 1 -- 172.21.15.69:0/3839880216 wait complete. 2024-05-14T17:04:56.499 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 850.0 s, 30.0 tps, lat 44471.487 ms stddev 36275.357 2024-05-14T17:04:56.516 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:48 smithi203 ceph-mon[35787]: pgmap v471: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 8.6 MiB/s wr, 447 op/s 2024-05-14T17:04:56.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:41 smithi069 ceph-mon[31507]: pgmap v468: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-05-14T17:04:56.574 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:47 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-14T17:04:56.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:48 smithi092 ceph-mon[35763]: pgmap v471: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 8.6 MiB/s wr, 447 op/s 2024-05-14T17:04:56.673 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:50 smithi092 ceph-mon[35763]: pgmap v472: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.6 MiB/s wr, 503 op/s 2024-05-14T17:04:56.724 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:52 smithi092 ceph-mon[35763]: pgmap v473: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.6 MiB/s wr, 503 op/s 2024-05-14T17:04:56.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:54 smithi092 ceph-mon[35763]: pgmap v474: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 786 op/s 2024-05-14T17:04:56.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:56 smithi092 ceph-mon[35763]: pgmap v475: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 629 op/s 2024-05-14T17:04:56.891 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:50 smithi203 ceph-mon[35787]: pgmap v472: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.6 MiB/s wr, 503 op/s 2024-05-14T17:04:56.940 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:52 smithi203 ceph-mon[35787]: pgmap v473: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.6 MiB/s wr, 503 op/s 2024-05-14T17:04:56.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:54 smithi203 ceph-mon[35787]: pgmap v474: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 786 op/s 2024-05-14T17:04:57.040 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:56 smithi203 ceph-mon[35787]: pgmap v475: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 629 op/s 2024-05-14T17:04:57.091 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a8922838', total seconds elapsed: 136.8 2024-05-14T17:04:57.149 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.65 2024-05-14T17:04:57.199 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:04:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.071+0000 7f1811247640 1 -- 172.21.15.69:0/2674739111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c111620 msgr2=0x7f180c109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.071+0000 7f1811247640 1 --2- 172.21.15.69:0/2674739111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c111620 0x7f180c109f90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1804009cd0 tx=0x7f180402a9c0 comp rx=0 tx=0).stop 2024-05-14T17:04:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.072+0000 7f1811247640 1 -- 172.21.15.69:0/2674739111 shutdown_connections 2024-05-14T17:04:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.072+0000 7f1811247640 1 --2- 172.21.15.69:0/2674739111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f180c11c230 0x7f180c11e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.072+0000 7f1811247640 1 --2- 172.21.15.69:0/2674739111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c111620 0x7f180c109f90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.072+0000 7f1811247640 1 --2- 172.21.15.69:0/2674739111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f180c111240 0x7f180c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.072+0000 7f1811247640 1 -- 172.21.15.69:0/2674739111 >> 172.21.15.69:0/2674739111 conn(0x7f180c06db20 msgr2=0x7f180c06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.076+0000 7f1811247640 1 -- 172.21.15.69:0/2674739111 shutdown_connections 2024-05-14T17:04:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.079+0000 7f1811247640 1 -- 172.21.15.69:0/2674739111 wait complete. 2024-05-14T17:04:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.080+0000 7f1811247640 1 Processor -- start 2024-05-14T17:04:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.082+0000 7f1811247640 1 -- start start 2024-05-14T17:04:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.082+0000 7f1811247640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f180c111240 0x7f180c11b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.082+0000 7f1811247640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f180c11c230 0x7f180c115980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.082+0000 7f1811247640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c115ec0 0x7f180c116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.082+0000 7f1811247640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f180c112310 con 0x7f180c111240 2024-05-14T17:04:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.082+0000 7f1811247640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f180c112190 con 0x7f180c115ec0 2024-05-14T17:04:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.082+0000 7f1811247640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f180c112490 con 0x7f180c11c230 2024-05-14T17:04:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.082+0000 7f180a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f180c11c230 0x7f180c115980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.083+0000 7f180a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f180c11c230 0x7f180c115980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:35020/0 (socket says 172.21.15.69:35020) 2024-05-14T17:04:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.083+0000 7f180a59c640 1 -- 172.21.15.69:0/1047962951 learned_addr learned my addr 172.21.15.69:0/1047962951 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.084+0000 7f180b59e640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c115ec0 0x7f180c116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.084+0000 7f180ad9d640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f180c111240 0x7f180c11b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.086+0000 7f180b59e640 1 -- 172.21.15.69:0/1047962951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f180c11c230 msgr2=0x7f180c115980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.086+0000 7f180b59e640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f180c11c230 0x7f180c115980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.086+0000 7f180b59e640 1 -- 172.21.15.69:0/1047962951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f180c111240 msgr2=0x7f180c11b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.086+0000 7f180b59e640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f180c111240 0x7f180c11b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.086+0000 7f180b59e640 1 -- 172.21.15.69:0/1047962951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c1bd020 con 0x7f180c115ec0 2024-05-14T17:04:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.087+0000 7f180b59e640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c115ec0 0x7f180c116370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00d680 tx=0x7f17fc00db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.089+0000 7f17f3fff640 1 -- 172.21.15.69:0/1047962951 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc017070 con 0x7f180c115ec0 2024-05-14T17:04:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.089+0000 7f17f3fff640 1 -- 172.21.15.69:0/1047962951 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17fc009820 con 0x7f180c115ec0 2024-05-14T17:04:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.089+0000 7f17f3fff640 1 -- 172.21.15.69:0/1047962951 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc004c40 con 0x7f180c115ec0 2024-05-14T17:04:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.090+0000 7f1811247640 1 -- 172.21.15.69:0/1047962951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c1bd310 con 0x7f180c115ec0 2024-05-14T17:04:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.090+0000 7f1811247640 1 -- 172.21.15.69:0/1047962951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c1bd810 con 0x7f180c115ec0 2024-05-14T17:04:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.093+0000 7f1813f65740 1 Processor -- start 2024-05-14T17:04:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.096+0000 7f1813f65740 1 -- start start 2024-05-14T17:04:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.096+0000 7f1813f65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435decd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.096+0000 7f1813f65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.096+0000 7f1813f65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a435ebec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.096+0000 7f1813f65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558a43551c30 con 0x558a435de8f0 2024-05-14T17:04:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.096+0000 7f1813f65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558a435417d0 con 0x558a435e9ad0 2024-05-14T17:04:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.096+0000 7f1813f65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558a43545810 con 0x558a435df210 2024-05-14T17:04:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.096+0000 7f17f3fff640 1 -- 172.21.15.69:0/1047962951 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f17fc005220 con 0x7f180c115ec0 2024-05-14T17:04:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.097+0000 7f180ad9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.097+0000 7f180a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a435ebec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.097+0000 7f180ad9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:35030/0 (socket says 172.21.15.69:35030) 2024-05-14T17:04:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.097+0000 7f180a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a435ebec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:48296/0 (socket says 172.21.15.69:48296) 2024-05-14T17:04:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.097+0000 7f180ad9d640 1 -- 172.21.15.69:0/3518104052 learned_addr learned my addr 172.21.15.69:0/3518104052 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:04:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.097+0000 7f17f3fff640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17f40b2c20 0x7f17f40b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.097+0000 7f180b59e640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17f40b2c20 0x7f17f40b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.098+0000 7f17f3fff640 1 -- 172.21.15.69:0/1047962951 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f17fc0e2120 con 0x7f180c115ec0 2024-05-14T17:04:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.098+0000 7f180ad9d640 1 -- 172.21.15.69:0/3518104052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 msgr2=0x558a435ebec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.098+0000 7f180ad9d640 1 --2- 172.21.15.69:0/3518104052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a435ebec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.098+0000 7f180ad9d640 1 -- 172.21.15.69:0/3518104052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 msgr2=0x558a435decd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.098+0000 7f180ad9d640 1 --2- 172.21.15.69:0/3518104052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435decd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.098+0000 7f180ad9d640 1 -- 172.21.15.69:0/3518104052 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a43256210 con 0x558a435df210 2024-05-14T17:05:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.098+0000 7f180a59c640 1 --2- 172.21.15.69:0/3518104052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a435ebec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-14T17:05:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.098+0000 7f180ad9d640 1 --2- 172.21.15.69:0/3518104052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e9590 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f180000b820 tx=0x7f180000bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e36ed10673027ace server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.098+0000 7f17f17fa640 1 -- 172.21.15.69:0/3518104052 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180000d950 con 0x558a435df210 2024-05-14T17:05:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.099+0000 7f17f17fa640 1 -- 172.21.15.69:0/3518104052 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1800019070 con 0x558a435df210 2024-05-14T17:05:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.099+0000 7f17f17fa640 1 -- 172.21.15.69:0/3518104052 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800014e30 con 0x558a435df210 2024-05-14T17:05:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.100+0000 7f1813f65740 1 -- 172.21.15.69:0/3518104052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 msgr2=0x558a435e9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.100+0000 7f1813f65740 1 --2- 172.21.15.69:0/3518104052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e9590 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f180000b820 tx=0x7f180000bce0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.100+0000 7f180b59e640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17f40b2c20 0x7f17f40b50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0a54b0 tx=0x7f17fc069660 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.100+0000 7f1813f65740 1 -- 172.21.15.69:0/3518104052 shutdown_connections 2024-05-14T17:05:00.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.100+0000 7f1813f65740 1 --2- 172.21.15.69:0/3518104052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a435ebec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.100+0000 7f1813f65740 1 --2- 172.21.15.69:0/3518104052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e9590 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.100+0000 7f1813f65740 1 --2- 172.21.15.69:0/3518104052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435decd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.100+0000 7f1813f65740 1 -- 172.21.15.69:0/3518104052 >> 172.21.15.69:0/3518104052 conn(0x558a435dae00 msgr2=0x558a435db1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.103+0000 7f1813f65740 1 -- 172.21.15.69:0/3518104052 shutdown_connections 2024-05-14T17:05:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.103+0000 7f1813f65740 1 -- 172.21.15.69:0/3518104052 wait complete. 2024-05-14T17:05:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f1813f65740 1 -- start start 2024-05-14T17:05:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f1813f65740 1 Processor -- start 2024-05-14T17:05:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f1813f65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435e9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f1813f65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f1813f65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a435e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f1813f65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558a43227790 con 0x558a435de8f0 2024-05-14T17:05:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f1813f65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558a434fd890 con 0x558a435e9ad0 2024-05-14T17:05:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f1813f65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558a435ec7a0 con 0x558a435df210 2024-05-14T17:05:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f1813f65740 10 client.0 _ll_register_callbacks cb 0x558a43500d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f180a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.104+0000 7f180a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435e9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44480/0 (socket says 172.21.15.69:44480) 2024-05-14T17:05:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.105+0000 7f180a59c640 1 -- 172.21.15.69:0/940328715 learned_addr learned my addr 172.21.15.69:0/940328715 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.105+0000 7f180ad9d640 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.105+0000 7f180a59c640 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a435e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.105+0000 7f180a59c640 1 -- 172.21.15.69:0/940328715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 msgr2=0x558a435e66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.105+0000 7f180a59c640 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.105+0000 7f180a59c640 1 -- 172.21.15.69:0/940328715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 msgr2=0x558a435e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.105+0000 7f180a59c640 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a435e6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.105+0000 7f180a59c640 1 -- 172.21.15.69:0/940328715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a43322620 con 0x558a435de8f0 2024-05-14T17:05:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.105+0000 7f180a59c640 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435e9400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1804009ca0 tx=0x7f18040097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0f5c6e62c108be server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.105+0000 7f17eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:05:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.110+0000 7f17eeff5640 1 -- 172.21.15.69:0/940328715 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804042040 con 0x558a435de8f0 2024-05-14T17:05:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.110+0000 7f17eeff5640 1 -- 172.21.15.69:0/940328715 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f1804030ae0 con 0x558a435de8f0 2024-05-14T17:05:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.110+0000 7f1813f65740 1 -- 172.21.15.69:0/940328715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1800019070 con 0x558a435de8f0 2024-05-14T17:05:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.110+0000 7f17eeff5640 1 -- 172.21.15.69:0/940328715 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1804034040 con 0x558a435de8f0 2024-05-14T17:05:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.110+0000 7f1813f65740 10 client.16947 fetch_fsmap learned FSMap version 55 2024-05-14T17:05:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.110+0000 7f1813f65740 1 -- 172.21.15.69:0/940328715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558a435f81a0 con 0x558a435de8f0 2024-05-14T17:05:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.112+0000 7f17eeff5640 1 -- 172.21.15.69:0/940328715 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1800019070 con 0x558a435de8f0 2024-05-14T17:05:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.112+0000 7f17eeff5640 1 -- 172.21.15.69:0/940328715 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f1804004370 con 0x558a435de8f0 2024-05-14T17:05:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.112+0000 7f1813f65740 10 client.16947 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:05:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.112+0000 7f1813f65740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.112+0000 7f1813f65740 20 client.16947 populate_metadata read hostname 'smithi069' 2024-05-14T17:05:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.112+0000 7f1813f65740 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558a435f98b0 0x558a435fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.112+0000 7f1813f65740 4 client.16947 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.112+0000 7f1813f65740 1 -- 172.21.15.69:0/940328715 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a43553b80 con 0x558a435f98b0 2024-05-14T17:05:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.114+0000 7f180ad9d640 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558a435f98b0 0x558a435fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.114+0000 7f180ad9d640 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558a435f98b0 0x558a435fbca0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4abf50338186407 server_cookie=cd737b20fb66ddc5 in_seq=0 out_seq=0 2024-05-14T17:05:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.114+0000 7f17eeff5640 10 client.16947 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.115+0000 7f17eeff5640 1 -- 172.21.15.69:0/940328715 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a43553b80 con 0x558a435f98b0 2024-05-14T17:05:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.115+0000 7f17eeff5640 10 client.16947 handle_command_reply: tid=0 2024-05-14T17:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.116+0000 7f1813f65740 1 client.16947 shutdown 2024-05-14T17:05:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.116+0000 7f1813f65740 1 -- 172.21.15.69:0/940328715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 msgr2=0x558a435e9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.116+0000 7f1813f65740 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435e9400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1804009ca0 tx=0x7f18040097e0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.117+0000 7f1813f65740 1 -- 172.21.15.69:0/940328715 shutdown_connections 2024-05-14T17:05:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.117+0000 7f1813f65740 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558a435f98b0 0x558a435fbca0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.117+0000 7f1813f65740 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a435e6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.117+0000 7f1813f65740 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.117+0000 7f1813f65740 1 --2- 172.21.15.69:0/940328715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435e9400 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.117+0000 7f1813f65740 1 -- 172.21.15.69:0/940328715 >> 172.21.15.69:0/940328715 conn(0x558a435dae00 msgr2=0x558a435da100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.118+0000 7f17eeff5640 0 client.16947 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.118+0000 7f1813f65740 1 -- 172.21.15.69:0/940328715 shutdown_connections 2024-05-14T17:05:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.118+0000 7f1813f65740 1 -- 172.21.15.69:0/940328715 wait complete. 2024-05-14T17:05:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.119+0000 7f1813f65740 20 client.16947 trim_cache size 0 max 16384 2024-05-14T17:05:03.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f1813f65740 1 Processor -- start 2024-05-14T17:05:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f1813f65740 1 -- start start 2024-05-14T17:05:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f1813f65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435de8f0 0x558a435f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f1813f65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:03.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f1813f65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435e9ad0 0x558a436012a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f1813f65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1c8800 con 0x558a435e9ad0 2024-05-14T17:05:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f1813f65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1c6070 con 0x558a435de8f0 2024-05-14T17:05:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f1813f65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1c3ec0 con 0x558a435df210 2024-05-14T17:05:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f180a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435e9ad0 0x558a436012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f180a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435e9ad0 0x558a436012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44496/0 (socket says 172.21.15.69:44496) 2024-05-14T17:05:04.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f180a59c640 1 -- 172.21.15.69:0/2385346145 learned_addr learned my addr 172.21.15.69:0/2385346145 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f180a59c640 1 --2- 172.21.15.69:0/2385346145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435de8f0 0x558a435f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f180a59c640 1 -- 172.21.15.69:0/2385346145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 msgr2=0x558a435e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f180a59c640 1 --2- 172.21.15.69:0/2385346145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f180a59c640 1 -- 172.21.15.69:0/2385346145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435de8f0 msgr2=0x558a435f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f180a59c640 1 --2- 172.21.15.69:0/2385346145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435de8f0 0x558a435f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.141+0000 7f180a59c640 1 -- 172.21.15.69:0/2385346145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a435d6830 con 0x558a435e9ad0 2024-05-14T17:05:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.142+0000 7f180a59c640 1 --2- 172.21.15.69:0/2385346145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435e9ad0 0x558a436012a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f180402fbc0 tx=0x7f180402faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3823332abd5c8850 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.142+0000 7f17f0ff9640 1 -- 172.21.15.69:0/2385346145 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804058070 con 0x558a435e9ad0 2024-05-14T17:05:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.142+0000 7f17f0ff9640 1 -- 172.21.15.69:0/2385346145 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1804059040 con 0x558a435e9ad0 2024-05-14T17:05:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.142+0000 7f17f0ff9640 1 -- 172.21.15.69:0/2385346145 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804052a20 con 0x558a435e9ad0 2024-05-14T17:05:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 -- 172.21.15.69:0/2385346145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435e9ad0 msgr2=0x558a436012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 --2- 172.21.15.69:0/2385346145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435e9ad0 0x558a436012a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f180402fbc0 tx=0x7f180402faf0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 -- 172.21.15.69:0/2385346145 shutdown_connections 2024-05-14T17:05:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 --2- 172.21.15.69:0/2385346145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435e9ad0 0x558a436012a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 --2- 172.21.15.69:0/2385346145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 --2- 172.21.15.69:0/2385346145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435de8f0 0x558a435f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 -- 172.21.15.69:0/2385346145 >> 172.21.15.69:0/2385346145 conn(0x558a435dae00 msgr2=0x558a435e7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 -- 172.21.15.69:0/2385346145 shutdown_connections 2024-05-14T17:05:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 -- 172.21.15.69:0/2385346145 wait complete. 2024-05-14T17:05:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 -- start start 2024-05-14T17:05:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 Processor -- start 2024-05-14T17:05:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435ed520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f180a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f180a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435ecfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44498/0 (socket says 172.21.15.69:44498) 2024-05-14T17:05:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f180a59c640 1 -- 172.21.15.69:0/274334389 learned_addr learned my addr 172.21.15.69:0/274334389 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a43661e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1c4530 con 0x558a435de8f0 2024-05-14T17:05:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1c59a0 con 0x558a435e9ad0 2024-05-14T17:05:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.143+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1c4c00 con 0x558a435df210 2024-05-14T17:05:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f180ad9d640 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435ed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f1813f65740 10 client.0 _ll_register_callbacks cb 0x7f180402f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f180a59c640 1 -- 172.21.15.69:0/274334389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 msgr2=0x558a435ed520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f180a59c640 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435ed520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f180a59c640 1 -- 172.21.15.69:0/274334389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 msgr2=0x558a43661e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f180a59c640 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a43661e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f180a59c640 1 -- 172.21.15.69:0/274334389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a43662480 con 0x558a435de8f0 2024-05-14T17:05:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f180ad9d640 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435ed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f180a59c640 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435ecfe0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1804004860 tx=0x7f1804009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2466272acb92ba6b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f17eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:05:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f17eeff5640 1 -- 172.21.15.69:0/274334389 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804063060 con 0x558a435de8f0 2024-05-14T17:05:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f17eeff5640 1 -- 172.21.15.69:0/274334389 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7f1804030890 con 0x558a435de8f0 2024-05-14T17:05:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1804059040 con 0x558a435de8f0 2024-05-14T17:05:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f17eeff5640 1 -- 172.21.15.69:0/274334389 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1804058050 con 0x558a435de8f0 2024-05-14T17:05:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f1813f65740 10 client.16953 fetch_fsmap learned FSMap version 55 2024-05-14T17:05:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558a43663770 con 0x558a435de8f0 2024-05-14T17:05:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.144+0000 7f17eeff5640 1 -- 172.21.15.69:0/274334389 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180403d070 con 0x558a435de8f0 2024-05-14T17:05:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.145+0000 7f17eeff5640 1 -- 172.21.15.69:0/274334389 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f18040527c0 con 0x558a435de8f0 2024-05-14T17:05:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.145+0000 7f1813f65740 10 client.16953 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:05:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.145+0000 7f1813f65740 10 client.16953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.145+0000 7f1813f65740 20 client.16953 populate_metadata read hostname 'smithi069' 2024-05-14T17:05:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.145+0000 7f1813f65740 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558a43663db0 0x558a436661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.145+0000 7f1813f65740 4 client.16953 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.145+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1804059040 con 0x558a43663db0 2024-05-14T17:05:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.145+0000 7f180ad9d640 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558a43663db0 0x558a436661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.146+0000 7f180ad9d640 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558a43663db0 0x558a436661d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef4476ecb2a92d6d server_cookie=2108581c51807146 in_seq=0 out_seq=0 2024-05-14T17:05:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.146+0000 7f17eeff5640 10 client.16953 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f17eeff5640 1 -- 172.21.15.69:0/274334389 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f1804059040 con 0x558a43663db0 2024-05-14T17:05:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f17eeff5640 10 client.16953 handle_command_reply: tid=0 2024-05-14T17:05:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f1813f65740 1 client.16953 shutdown 2024-05-14T17:05:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 msgr2=0x558a435ecfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f1813f65740 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435ecfe0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1804004860 tx=0x7f1804009540 comp rx=0 tx=0).stop 2024-05-14T17:05:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 shutdown_connections 2024-05-14T17:05:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f1813f65740 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x558a43663db0 0x558a436661d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f1813f65740 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a435e9ad0 0x558a43661e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f1813f65740 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a435df210 0x558a435ed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f1813f65740 1 --2- 172.21.15.69:0/274334389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a435de8f0 0x558a435ecfe0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 >> 172.21.15.69:0/274334389 conn(0x558a435dae00 msgr2=0x558a435e0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.682+0000 7f17eeff5640 0 client.16953 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.683+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 shutdown_connections 2024-05-14T17:05:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.683+0000 7f1813f65740 1 -- 172.21.15.69:0/274334389 wait complete. 2024-05-14T17:05:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.683+0000 7f1813f65740 20 client.16953 trim_cache size 0 max 16384 2024-05-14T17:05:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:05:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:05:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.683+0000 7f1811247640 1 -- 172.21.15.69:0/1047962951 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17f40b2c20 msgr2=0x7f17f40b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.683+0000 7f1811247640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17f40b2c20 0x7f17f40b50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0a54b0 tx=0x7f17fc069660 comp rx=0 tx=0).stop 2024-05-14T17:05:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.684+0000 7f1811247640 1 -- 172.21.15.69:0/1047962951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c115ec0 msgr2=0x7f180c116370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.684+0000 7f1811247640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c115ec0 0x7f180c116370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00d680 tx=0x7f17fc00db40 comp rx=0 tx=0).stop 2024-05-14T17:05:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.684+0000 7f1811247640 1 -- 172.21.15.69:0/1047962951 shutdown_connections 2024-05-14T17:05:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.684+0000 7f1811247640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f17f40b2c20 0x7f17f40b50e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.684+0000 7f1811247640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c115ec0 0x7f180c116370 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.684+0000 7f1811247640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f180c11c230 0x7f180c115980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.684+0000 7f1811247640 1 --2- 172.21.15.69:0/1047962951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f180c111240 0x7f180c11b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.684+0000 7f1811247640 1 -- 172.21.15.69:0/1047962951 >> 172.21.15.69:0/1047962951 conn(0x7f180c06db20 msgr2=0x7f180c11c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.684+0000 7f1811247640 1 -- 172.21.15.69:0/1047962951 shutdown_connections 2024-05-14T17:05:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:43.684+0000 7f1811247640 1 -- 172.21.15.69:0/1047962951 wait complete. 2024-05-14T17:05:08.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:58 smithi203 ceph-mon[35787]: pgmap v476: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 850 op/s 2024-05-14T17:05:08.889 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:43 smithi069 ceph-mon[31507]: pgmap v469: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-05-14T17:05:08.939 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 855.0 s, 1197.2 tps, lat 206.312 ms stddev 3076.771 2024-05-14T17:05:08.987 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:57 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 5. 2024-05-14T17:05:09.030 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:57 smithi203 systemd[1]: Stopped Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:05:09.080 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:57 smithi203 systemd[1]: Starting Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:05:09.131 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:58 smithi203 podman[73672]: 2024-05-14 17:04:58.107052987 +0000 UTC m=+0.020660019 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:05:09.180 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:58 smithi203 podman[73672]: 2024-05-14 17:04:58.248970034 +0000 UTC m=+0.162577053 container create 92f0d2387ee8312c8f2eb29edd4c5ee7d4bfd480ce290e7b630dacd4eb105b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-14T17:05:09.231 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:58 smithi203 podman[73672]: 2024-05-14 17:04:58.504652715 +0000 UTC m=+0.418259736 container init 92f0d2387ee8312c8f2eb29edd4c5ee7d4bfd480ce290e7b630dacd4eb105b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS) 2024-05-14T17:05:09.297 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:58 smithi203 podman[73672]: 2024-05-14 17:04:58.507264032 +0000 UTC m=+0.420871060 container start 92f0d2387ee8312c8f2eb29edd4c5ee7d4bfd480ce290e7b630dacd4eb105b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-14T17:05:09.339 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:58 smithi203 ceph-mds[73691]: -- 172.21.15.203:0/4234332909 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a344de9a40 con 0x55a344dbcc00 2024-05-14T17:05:09.389 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:58 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: starting mds.c at 2024-05-14T17:05:09.447 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:58 smithi203 bash[73672]: 92f0d2387ee8312c8f2eb29edd4c5ee7d4bfd480ce290e7b630dacd4eb105b62 2024-05-14T17:05:09.497 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:04:58 smithi203 systemd[1]: Started Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:05:09.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:44 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:05:09.581 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:46 smithi069 ceph-mon[31507]: pgmap v470: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-05-14T17:05:09.631 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:48 smithi069 ceph-mon[31507]: pgmap v471: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 8.6 MiB/s wr, 447 op/s 2024-05-14T17:05:09.674 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:58 smithi092 ceph-mon[35763]: pgmap v476: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 850 op/s 2024-05-14T17:05:09.714 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:59 smithi092 ceph-mon[35763]: daemon mds.c restarted 2024-05-14T17:05:09.756 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:59 smithi092 ceph-mon[35763]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T17:05:09.806 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:59 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6836/3691870879,v1:172.21.15.203:6837/3691870879] up:boot 2024-05-14T17:05:09.856 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:59 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:05:09.897 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:04:59 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:05:09.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:00 smithi092 ceph-mon[35763]: pgmap v478: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 646 op/s 2024-05-14T17:05:09.998 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:02 smithi092 ceph-mon[35763]: pgmap v479: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 679 op/s 2024-05-14T17:05:10.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:04 smithi092 ceph-mon[35763]: pgmap v480: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 631 op/s 2024-05-14T17:05:10.098 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:05 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:05:10.148 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:06 smithi092 ceph-mon[35763]: pgmap v481: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 631 op/s 2024-05-14T17:05:10.223 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:08 smithi092 ceph-mon[35763]: pgmap v482: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 1.61k op/s 2024-05-14T17:05:10.273 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 860.0 s, 2300.2 tps, lat 44.212 ms stddev 54.017 2024-05-14T17:05:10.315 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:59 smithi203 ceph-mon[35787]: daemon mds.c restarted 2024-05-14T17:05:10.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:59 smithi203 ceph-mon[35787]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T17:05:10.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:59 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6836/3691870879,v1:172.21.15.203:6837/3691870879] up:boot 2024-05-14T17:05:10.431 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:59 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:05:10.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:04:59 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:05:10.531 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:00 smithi203 ceph-mon[35787]: pgmap v478: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 646 op/s 2024-05-14T17:05:10.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:02 smithi203 ceph-mon[35787]: pgmap v479: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 679 op/s 2024-05-14T17:05:10.627 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:04 smithi203 ceph-mon[35787]: pgmap v480: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 631 op/s 2024-05-14T17:05:10.673 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:05 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:05:10.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:06 smithi203 ceph-mon[35787]: pgmap v481: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 631 op/s 2024-05-14T17:05:10.773 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:08 smithi203 ceph-mon[35787]: pgmap v482: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 1.61k op/s 2024-05-14T17:05:10.824 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:50 smithi069 ceph-mon[31507]: pgmap v472: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.6 MiB/s wr, 503 op/s 2024-05-14T17:05:10.873 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:52 smithi069 ceph-mon[31507]: pgmap v473: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.6 MiB/s wr, 503 op/s 2024-05-14T17:05:10.924 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:54 smithi069 ceph-mon[31507]: pgmap v474: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 786 op/s 2024-05-14T17:05:10.974 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 865.0 s, 2066.0 tps, lat 48.184 ms stddev 52.445 2024-05-14T17:05:11.049 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:56 smithi069 ceph-mon[31507]: pgmap v475: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 629 op/s 2024-05-14T17:05:11.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:10 smithi203 ceph-mon[35787]: pgmap v483: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 MiB/s wr, 1.82k op/s 2024-05-14T17:05:11.149 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:10 smithi092 ceph-mon[35763]: pgmap v483: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 MiB/s wr, 1.82k op/s 2024-05-14T17:05:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.802+0000 7fb72edaf640 1 -- 172.21.15.69:0/4241478854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb728107940 msgr2=0x7fb728107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.802+0000 7fb72edaf640 1 --2- 172.21.15.69:0/4241478854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb728107940 0x7fb728107d20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009930 tx=0x7fb71c02a420 comp rx=0 tx=0).stop 2024-05-14T17:05:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.803+0000 7fb72edaf640 1 -- 172.21.15.69:0/4241478854 shutdown_connections 2024-05-14T17:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.803+0000 7fb72edaf640 1 --2- 172.21.15.69:0/4241478854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb728102340 0x7fb72810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.803+0000 7fb72edaf640 1 --2- 172.21.15.69:0/4241478854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb728101980 0x7fb728101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.803+0000 7fb72edaf640 1 --2- 172.21.15.69:0/4241478854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb728107940 0x7fb728107d20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.803+0000 7fb72edaf640 1 -- 172.21.15.69:0/4241478854 >> 172.21.15.69:0/4241478854 conn(0x7fb728078ad0 msgr2=0x7fb728078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.803+0000 7fb72edaf640 1 -- 172.21.15.69:0/4241478854 shutdown_connections 2024-05-14T17:05:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.803+0000 7fb72edaf640 1 -- 172.21.15.69:0/4241478854 wait complete. 2024-05-14T17:05:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.804+0000 7fb72edaf640 1 Processor -- start 2024-05-14T17:05:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.804+0000 7fb72edaf640 1 -- start start 2024-05-14T17:05:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.804+0000 7fb72edaf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb728101980 0x7fb7281991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.804+0000 7fb72edaf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb728102340 0x7fb7281996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.804+0000 7fb72edaf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb728107940 0x7fb72819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.804+0000 7fb72edaf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb728111c00 con 0x7fb728102340 2024-05-14T17:05:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.804+0000 7fb72edaf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb728111a80 con 0x7fb728101980 2024-05-14T17:05:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.804+0000 7fb72edaf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb728111d80 con 0x7fb728107940 2024-05-14T17:05:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb72cb4b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb728101980 0x7fb7281991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb72d34c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb728107940 0x7fb72819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb72cb4b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb728101980 0x7fb7281991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:54344/0 (socket says 172.21.15.69:54344) 2024-05-14T17:05:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb72d34c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb728107940 0x7fb72819da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:51476/0 (socket says 172.21.15.69:51476) 2024-05-14T17:05:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb72cb4b640 1 -- 172.21.15.69:0/2228377237 learned_addr learned my addr 172.21.15.69:0/2228377237 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb727fff640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb728102340 0x7fb7281996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb72cb4b640 1 -- 172.21.15.69:0/2228377237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb728107940 msgr2=0x7fb72819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb728107940 0x7fb72819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb72cb4b640 1 -- 172.21.15.69:0/2228377237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb728102340 msgr2=0x7fb7281996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb728102340 0x7fb7281996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.805+0000 7fb72cb4b640 1 -- 172.21.15.69:0/2228377237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72819e1a0 con 0x7fb728101980 2024-05-14T17:05:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.806+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb728101980 0x7fb7281991b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb71c005940 tx=0x7fb71c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.806+0000 7fb725ffb640 1 -- 172.21.15.69:0/2228377237 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c03f030 con 0x7fb728101980 2024-05-14T17:05:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.806+0000 7fb725ffb640 1 -- 172.21.15.69:0/2228377237 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c0044c0 con 0x7fb728101980 2024-05-14T17:05:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.806+0000 7fb72edaf640 1 -- 172.21.15.69:0/2228377237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72819e430 con 0x7fb728101980 2024-05-14T17:05:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.806+0000 7fb725ffb640 1 -- 172.21.15.69:0/2228377237 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c035070 con 0x7fb728101980 2024-05-14T17:05:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.806+0000 7fb72edaf640 1 -- 172.21.15.69:0/2228377237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7280fe840 con 0x7fb728101980 2024-05-14T17:05:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.809+0000 7fb731ba0740 1 Processor -- start 2024-05-14T17:05:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.809+0000 7fb731ba0740 1 -- start start 2024-05-14T17:05:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.809+0000 7fb725ffb640 1 -- 172.21.15.69:0/2228377237 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb71c042030 con 0x7fb728101980 2024-05-14T17:05:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4b7d0 0x562d2ed4bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed67ae0 0x562d2ed69ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb731ba0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562d2ecbec30 con 0x562d2ed4c0f0 2024-05-14T17:05:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb731ba0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562d2ecae7d0 con 0x562d2ed67ae0 2024-05-14T17:05:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb731ba0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562d2ecb2810 con 0x562d2ed4b7d0 2024-05-14T17:05:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb72cb4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb72cb4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43512/0 (socket says 172.21.15.69:43512) 2024-05-14T17:05:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb72cb4b640 1 -- 172.21.15.69:0/2554012814 learned_addr learned my addr 172.21.15.69:0/2554012814 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb72d34c640 1 --2- 172.21.15.69:0/2554012814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4b7d0 0x562d2ed4bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.810+0000 7fb727fff640 1 --2- 172.21.15.69:0/2554012814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed67ae0 0x562d2ed69ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.811+0000 7fb72cb4b640 1 -- 172.21.15.69:0/2554012814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4b7d0 msgr2=0x562d2ed4bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.811+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/2554012814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4b7d0 0x562d2ed4bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.811+0000 7fb72cb4b640 1 -- 172.21.15.69:0/2554012814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed67ae0 msgr2=0x562d2ed69ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.811+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/2554012814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed67ae0 0x562d2ed69ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.811+0000 7fb72cb4b640 1 -- 172.21.15.69:0/2554012814 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d2e9c3210 con 0x562d2ed4c0f0 2024-05-14T17:05:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.811+0000 7fb72d34c640 1 --2- 172.21.15.69:0/2554012814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4b7d0 0x562d2ed4bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.811+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/2554012814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed675a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb71c12ac00 tx=0x7fb71c12acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c3771fba7845da server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.811+0000 7fb712ffd640 1 -- 172.21.15.69:0/2554012814 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c133020 con 0x562d2ed4c0f0 2024-05-14T17:05:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.811+0000 7fb712ffd640 1 -- 172.21.15.69:0/2554012814 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c102e50 con 0x562d2ed4c0f0 2024-05-14T17:05:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.811+0000 7fb712ffd640 1 -- 172.21.15.69:0/2554012814 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c12e390 con 0x562d2ed4c0f0 2024-05-14T17:05:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.812+0000 7fb725ffb640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb7180b2a90 0x7fb7180b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.812+0000 7fb72d34c640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb7180b2a90 0x7fb7180b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.812+0000 7fb725ffb640 1 -- 172.21.15.69:0/2228377237 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7fb71c102aa0 con 0x7fb728101980 2024-05-14T17:05:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.812+0000 7fb731ba0740 1 -- 172.21.15.69:0/2554012814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 msgr2=0x562d2ed675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.812+0000 7fb731ba0740 1 --2- 172.21.15.69:0/2554012814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed675a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb71c12ac00 tx=0x7fb71c12acf0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.812+0000 7fb731ba0740 1 -- 172.21.15.69:0/2554012814 shutdown_connections 2024-05-14T17:05:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.812+0000 7fb731ba0740 1 --2- 172.21.15.69:0/2554012814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed67ae0 0x562d2ed69ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.813+0000 7fb731ba0740 1 --2- 172.21.15.69:0/2554012814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed675a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.813+0000 7fb731ba0740 1 --2- 172.21.15.69:0/2554012814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4b7d0 0x562d2ed4bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.813+0000 7fb731ba0740 1 -- 172.21.15.69:0/2554012814 >> 172.21.15.69:0/2554012814 conn(0x562d2ed47e00 msgr2=0x562d2ed481e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.813+0000 7fb731ba0740 1 -- 172.21.15.69:0/2554012814 shutdown_connections 2024-05-14T17:05:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.813+0000 7fb72d34c640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb7180b2a90 0x7fb7180b4f50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb720009fd0 tx=0x7fb720009500 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:14.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.813+0000 7fb731ba0740 1 -- 172.21.15.69:0/2554012814 wait complete. 2024-05-14T17:05:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.813+0000 7fb731ba0740 1 -- start start 2024-05-14T17:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.813+0000 7fb731ba0740 1 Processor -- start 2024-05-14T17:05:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed58660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed58ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed67ae0 0x562d2ed539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb731ba0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562d2e994790 con 0x562d2ed4c0f0 2024-05-14T17:05:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb731ba0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562d2ec6a890 con 0x562d2ed4b7d0 2024-05-14T17:05:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb731ba0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562d2ed4e5d0 con 0x562d2ed67ae0 2024-05-14T17:05:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb727fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed58ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb72cb4b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed58660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb72cb4b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed58660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:54350/0 (socket says 172.21.15.69:54350) 2024-05-14T17:05:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb72d34c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed67ae0 0x562d2ed539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb727fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed58ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43516/0 (socket says 172.21.15.69:43516) 2024-05-14T17:05:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb72cb4b640 1 -- 172.21.15.69:0/408649359 learned_addr learned my addr 172.21.15.69:0/408649359 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb731ba0740 10 client.0 _ll_register_callbacks cb 0x562d2ec6dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb72cb4b640 1 -- 172.21.15.69:0/408649359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed67ae0 msgr2=0x562d2ed539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed67ae0 0x562d2ed539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb72cb4b640 1 -- 172.21.15.69:0/408649359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 msgr2=0x562d2ed58ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed58ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb72cb4b640 1 -- 172.21.15.69:0/408649359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d2ea8f620 con 0x562d2ed4b7d0 2024-05-14T17:05:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb727fff640 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed58ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed58660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0c5db0 tx=0x7fb71c03b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4617af7b61a5f90c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb707fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:05:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb707fff640 1 -- 172.21.15.69:0/408649359 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c049070 con 0x562d2ed4b7d0 2024-05-14T17:05:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb707fff640 1 -- 172.21.15.69:0/408649359 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7fb71c003cc0 con 0x562d2ed4b7d0 2024-05-14T17:05:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.814+0000 7fb731ba0740 1 -- 172.21.15.69:0/408649359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb71c102e50 con 0x562d2ed4b7d0 2024-05-14T17:05:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.816+0000 7fb707fff640 1 -- 172.21.15.69:0/408649359 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb71c133050 con 0x562d2ed4b7d0 2024-05-14T17:05:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.816+0000 7fb731ba0740 10 client.26339 fetch_fsmap learned FSMap version 56 2024-05-14T17:05:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.816+0000 7fb731ba0740 1 -- 172.21.15.69:0/408649359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x562d2ed62570 con 0x562d2ed4b7d0 2024-05-14T17:05:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.816+0000 7fb707fff640 1 -- 172.21.15.69:0/408649359 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c102e50 con 0x562d2ed4b7d0 2024-05-14T17:05:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.816+0000 7fb707fff640 1 -- 172.21.15.69:0/408649359 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 56) ==== 8342+0+0 (secure 0 0 0) 0x7fb71c100420 con 0x562d2ed4b7d0 2024-05-14T17:05:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.817+0000 7fb731ba0740 10 client.26339 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:05:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.817+0000 7fb731ba0740 10 client.26339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.817+0000 7fb731ba0740 20 client.26339 populate_metadata read hostname 'smithi069' 2024-05-14T17:05:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.817+0000 7fb731ba0740 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562d2ed63980 0x562d2ed65e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.817+0000 7fb731ba0740 4 client.26339 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.817+0000 7fb731ba0740 1 -- 172.21.15.69:0/408649359 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d2ecc0b80 con 0x562d2ed63980 2024-05-14T17:05:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.817+0000 7fb727fff640 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562d2ed63980 0x562d2ed65e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.817+0000 7fb707fff640 10 client.26339 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.817+0000 7fb727fff640 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562d2ed63980 0x562d2ed65e40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbe38f1e82076c5b server_cookie=e80941c2cf5749de in_seq=0 out_seq=0 2024-05-14T17:05:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.818+0000 7fb707fff640 1 -- 172.21.15.69:0/408649359 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562d2ecc0b80 con 0x562d2ed63980 2024-05-14T17:05:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.818+0000 7fb707fff640 10 client.26339 handle_command_reply: tid=0 2024-05-14T17:05:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.818+0000 7fb731ba0740 1 client.26339 shutdown 2024-05-14T17:05:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.818+0000 7fb731ba0740 1 -- 172.21.15.69:0/408649359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 msgr2=0x562d2ed58660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.818+0000 7fb731ba0740 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed58660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0c5db0 tx=0x7fb71c03b4f0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.819+0000 7fb731ba0740 1 -- 172.21.15.69:0/408649359 shutdown_connections 2024-05-14T17:05:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.819+0000 7fb731ba0740 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562d2ed63980 0x562d2ed65e40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.819+0000 7fb731ba0740 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed67ae0 0x562d2ed539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.819+0000 7fb731ba0740 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed4c0f0 0x562d2ed58ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.819+0000 7fb731ba0740 1 --2- 172.21.15.69:0/408649359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed58660 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.819+0000 7fb731ba0740 1 -- 172.21.15.69:0/408649359 >> 172.21.15.69:0/408649359 conn(0x562d2ed47e00 msgr2=0x562d2ed47100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.819+0000 7fb707fff640 0 client.26339 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.819+0000 7fb731ba0740 1 -- 172.21.15.69:0/408649359 shutdown_connections 2024-05-14T17:05:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.819+0000 7fb731ba0740 1 -- 172.21.15.69:0/408649359 wait complete. 2024-05-14T17:05:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.819+0000 7fb731ba0740 20 client.26339 trim_cache size 0 max 16384 2024-05-14T17:05:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.847+0000 7fb731ba0740 1 Processor -- start 2024-05-14T17:05:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.847+0000 7fb731ba0740 1 -- start start 2024-05-14T17:05:17.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.848+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed4f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.848+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed4d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.848+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 0x562d2ed6f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.848+0000 7fb731ba0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb7281a6560 con 0x562d2ed67ae0 2024-05-14T17:05:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.848+0000 7fb731ba0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb728072d50 con 0x562d2ed4b7d0 2024-05-14T17:05:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.848+0000 7fb731ba0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb728070ba0 con 0x562d2ed4c0f0 2024-05-14T17:05:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.849+0000 7fb72cb4b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed4f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.850+0000 7fb72d34c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 0x562d2ed6f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.850+0000 7fb72d34c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 0x562d2ed6f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43522/0 (socket says 172.21.15.69:43522) 2024-05-14T17:05:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.850+0000 7fb72d34c640 1 -- 172.21.15.69:0/1608722128 learned_addr learned my addr 172.21.15.69:0/1608722128 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.853+0000 7fb727fff640 1 --2- 172.21.15.69:0/1608722128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed4d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.855+0000 7fb727fff640 1 -- 172.21.15.69:0/1608722128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 msgr2=0x562d2ed4f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.855+0000 7fb727fff640 1 --2- 172.21.15.69:0/1608722128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed4f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.855+0000 7fb727fff640 1 -- 172.21.15.69:0/1608722128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 msgr2=0x562d2ed6f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.855+0000 7fb727fff640 1 --2- 172.21.15.69:0/1608722128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 0x562d2ed6f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.855+0000 7fb727fff640 1 -- 172.21.15.69:0/1608722128 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d2ed47c30 con 0x562d2ed4c0f0 2024-05-14T17:05:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.857+0000 7fb727fff640 1 --2- 172.21.15.69:0/1608722128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed4d910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x562d2ed66570 tx=0x7fb714005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bea69c885e45e9b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.858+0000 7fb7127fc640 1 -- 172.21.15.69:0/1608722128 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714016030 con 0x562d2ed4c0f0 2024-05-14T17:05:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.858+0000 7fb7127fc640 1 -- 172.21.15.69:0/1608722128 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb714029e50 con 0x562d2ed4c0f0 2024-05-14T17:05:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.858+0000 7fb7127fc640 1 -- 172.21.15.69:0/1608722128 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7140047a0 con 0x562d2ed4c0f0 2024-05-14T17:05:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.860+0000 7fb731ba0740 1 -- 172.21.15.69:0/1608722128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 msgr2=0x562d2ed4d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.860+0000 7fb731ba0740 1 --2- 172.21.15.69:0/1608722128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed4d910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x562d2ed66570 tx=0x7fb714005f90 comp rx=0 tx=0).stop 2024-05-14T17:05:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.860+0000 7fb731ba0740 1 -- 172.21.15.69:0/1608722128 shutdown_connections 2024-05-14T17:05:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.860+0000 7fb731ba0740 1 --2- 172.21.15.69:0/1608722128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 0x562d2ed6f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.860+0000 7fb731ba0740 1 --2- 172.21.15.69:0/1608722128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed4d910 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.860+0000 7fb731ba0740 1 --2- 172.21.15.69:0/1608722128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed4f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.860+0000 7fb731ba0740 1 -- 172.21.15.69:0/1608722128 >> 172.21.15.69:0/1608722128 conn(0x562d2ed47e00 msgr2=0x562d2ed4d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.861+0000 7fb731ba0740 1 -- 172.21.15.69:0/1608722128 shutdown_connections 2024-05-14T17:05:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.865+0000 7fb731ba0740 1 -- 172.21.15.69:0/1608722128 wait complete. 2024-05-14T17:05:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.865+0000 7fb731ba0740 1 -- start start 2024-05-14T17:05:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.865+0000 7fb731ba0740 1 Processor -- start 2024-05-14T17:05:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed60320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb731ba0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 0x562d2ed598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb731ba0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb728071210 con 0x562d2ed67ae0 2024-05-14T17:05:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb731ba0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb728072680 con 0x562d2ed4b7d0 2024-05-14T17:05:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb731ba0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb7280718e0 con 0x562d2ed4c0f0 2024-05-14T17:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb731ba0740 10 client.0 _ll_register_callbacks cb 0x7fb71c12aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb727fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb727fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:51504/0 (socket says 172.21.15.69:51504) 2024-05-14T17:05:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb727fff640 1 -- 172.21.15.69:0/630643497 learned_addr learned my addr 172.21.15.69:0/630643497 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed60320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.867+0000 7fb72d34c640 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 0x562d2ed598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb727fff640 1 -- 172.21.15.69:0/630643497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 msgr2=0x562d2ed60320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb727fff640 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed60320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb727fff640 1 -- 172.21.15.69:0/630643497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 msgr2=0x562d2ed598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb727fff640 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 0x562d2ed598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb727fff640 1 -- 172.21.15.69:0/630643497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d2ed5a100 con 0x562d2ed4c0f0 2024-05-14T17:05:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb727fff640 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed592c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb714029000 tx=0x7fb714010e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ebda0608c1702a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb707fff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T17:05:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb707fff640 1 -- 172.21.15.69:0/630643497 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714016070 con 0x562d2ed4c0f0 2024-05-14T17:05:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb731ba0740 1 -- 172.21.15.69:0/630643497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb714029e50 con 0x562d2ed4c0f0 2024-05-14T17:05:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb707fff640 1 -- 172.21.15.69:0/630643497 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8498+0+0 (secure 0 0 0) 0x7fb714005680 con 0x562d2ed4c0f0 2024-05-14T17:05:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb707fff640 1 -- 172.21.15.69:0/630643497 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb71401c450 con 0x562d2ed4c0f0 2024-05-14T17:05:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb731ba0740 10 client.26239 fetch_fsmap learned FSMap version 56 2024-05-14T17:05:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.868+0000 7fb731ba0740 1 -- 172.21.15.69:0/630643497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x562d2edd11e0 con 0x562d2ed4c0f0 2024-05-14T17:05:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.869+0000 7fb707fff640 1 -- 172.21.15.69:0/630643497 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb714029e50 con 0x562d2ed4c0f0 2024-05-14T17:05:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.869+0000 7fb707fff640 1 -- 172.21.15.69:0/630643497 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 56) ==== 8342+0+0 (secure 0 0 0) 0x7fb714004cd0 con 0x562d2ed4c0f0 2024-05-14T17:05:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.869+0000 7fb731ba0740 10 client.26239 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:05:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.869+0000 7fb731ba0740 10 client.26239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.869+0000 7fb731ba0740 20 client.26239 populate_metadata read hostname 'smithi069' 2024-05-14T17:05:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.869+0000 7fb731ba0740 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562d2edd18b0 0x562d2edd3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.869+0000 7fb731ba0740 4 client.26239 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.869+0000 7fb731ba0740 1 -- 172.21.15.69:0/630643497 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d2ecba360 con 0x562d2edd18b0 2024-05-14T17:05:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.869+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562d2edd18b0 0x562d2edd3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.872+0000 7fb72cb4b640 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562d2edd18b0 0x562d2edd3cd0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4d1511d612b80c server_cookie=9d570a6c8580f5cc in_seq=0 out_seq=0 2024-05-14T17:05:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.873+0000 7fb707fff640 10 client.26239 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb707fff640 1 -- 172.21.15.69:0/630643497 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562d2ecba360 con 0x562d2edd18b0 2024-05-14T17:05:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb707fff640 10 client.26239 handle_command_reply: tid=0 2024-05-14T17:05:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb731ba0740 1 client.26239 shutdown 2024-05-14T17:05:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb731ba0740 1 -- 172.21.15.69:0/630643497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 msgr2=0x562d2ed592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb731ba0740 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed592c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb714029000 tx=0x7fb714010e00 comp rx=0 tx=0).stop 2024-05-14T17:05:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb731ba0740 1 -- 172.21.15.69:0/630643497 shutdown_connections 2024-05-14T17:05:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb731ba0740 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562d2edd18b0 0x562d2edd3cd0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb731ba0740 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d2ed67ae0 0x562d2ed598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb731ba0740 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d2ed4c0f0 0x562d2ed592c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb731ba0740 1 --2- 172.21.15.69:0/630643497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d2ed4b7d0 0x562d2ed60320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb731ba0740 1 -- 172.21.15.69:0/630643497 >> 172.21.15.69:0/630643497 conn(0x562d2ed47e00 msgr2=0x562d2ed4da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.891+0000 7fb707fff640 0 client.26239 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.892+0000 7fb731ba0740 1 -- 172.21.15.69:0/630643497 shutdown_connections 2024-05-14T17:05:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.892+0000 7fb731ba0740 1 -- 172.21.15.69:0/630643497 wait complete. 2024-05-14T17:05:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.892+0000 7fb731ba0740 20 client.26239 trim_cache size 0 max 16384 2024-05-14T17:05:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:05:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:05:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:05:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 -- 172.21.15.69:0/2228377237 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb7180b2a90 msgr2=0x7fb7180b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb7180b2a90 0x7fb7180b4f50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb720009fd0 tx=0x7fb720009500 comp rx=0 tx=0).stop 2024-05-14T17:05:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 -- 172.21.15.69:0/2228377237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb728101980 msgr2=0x7fb7281991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb728101980 0x7fb7281991b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb71c005940 tx=0x7fb71c0040c0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 -- 172.21.15.69:0/2228377237 shutdown_connections 2024-05-14T17:05:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fb7180b2a90 0x7fb7180b4f50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb728107940 0x7fb72819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb728102340 0x7fb7281996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 --2- 172.21.15.69:0/2228377237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb728101980 0x7fb7281991b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 -- 172.21.15.69:0/2228377237 >> 172.21.15.69:0/2228377237 conn(0x7fb728078ad0 msgr2=0x7fb72806a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 -- 172.21.15.69:0/2228377237 shutdown_connections 2024-05-14T17:05:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:04:56.893+0000 7fb72edaf640 1 -- 172.21.15.69:0/2228377237 wait complete. 2024-05-14T17:05:21.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:58 smithi069 ceph-mon[31507]: pgmap v476: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 850 op/s 2024-05-14T17:05:21.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:59 smithi069 ceph-mon[31507]: daemon mds.c restarted 2024-05-14T17:05:21.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:59 smithi069 ceph-mon[31507]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T17:05:21.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:59 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6836/3691870879,v1:172.21.15.203:6837/3691870879] up:boot 2024-05-14T17:05:21.230 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:59 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:05:21.264 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:04:59 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:05:21.298 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 870.0 s, 2193.2 tps, lat 45.670 ms stddev 52.913 2024-05-14T17:05:21.331 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:00 smithi069 ceph-mon[31507]: pgmap v478: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 646 op/s 2024-05-14T17:05:21.364 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:02 smithi069 ceph-mon[31507]: pgmap v479: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 679 op/s 2024-05-14T17:05:21.435 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:04 smithi069 ceph-mon[31507]: pgmap v480: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 631 op/s 2024-05-14T17:05:21.481 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 875.0 s, 995.2 tps, lat 99.442 ms stddev 257.140 2024-05-14T17:05:21.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:05 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:05:21.573 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:06 smithi069 ceph-mon[31507]: pgmap v481: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 631 op/s 2024-05-14T17:05:21.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:08 smithi069 ceph-mon[31507]: pgmap v482: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 1.61k op/s 2024-05-14T17:05:21.639 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:12 smithi092 ceph-mon[35763]: pgmap v484: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-05-14T17:05:21.672 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:14 smithi092 ceph-mon[35763]: pgmap v485: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 27 MiB/s wr, 1.85k op/s 2024-05-14T17:05:21.725 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:16 smithi092 ceph-mon[35763]: pgmap v486: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 1.60k op/s 2024-05-14T17:05:21.764 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:18 smithi092 ceph-mon[35763]: pgmap v487: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2024-05-14T17:05:21.797 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:19 smithi092 ceph-mon[35763]: pgmap v488: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 888 op/s 2024-05-14T17:05:21.831 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:12 smithi203 ceph-mon[35787]: pgmap v484: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-05-14T17:05:21.864 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:14 smithi203 ceph-mon[35787]: pgmap v485: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 27 MiB/s wr, 1.85k op/s 2024-05-14T17:05:21.898 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:16 smithi203 ceph-mon[35787]: pgmap v486: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 1.60k op/s 2024-05-14T17:05:21.931 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:18 smithi203 ceph-mon[35787]: pgmap v487: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2024-05-14T17:05:21.964 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:19 smithi203 ceph-mon[35787]: pgmap v488: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 888 op/s 2024-05-14T17:05:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.032+0000 7f2010f19640 1 -- 172.21.15.69:0/3043672419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f200c111240 msgr2=0x7f200c109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.032+0000 7f2010f19640 1 --2- 172.21.15.69:0/3043672419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f200c111240 0x7f200c109a50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f200401ace0 tx=0x7f200403ba20 comp rx=0 tx=0).stop 2024-05-14T17:05:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.035+0000 7f2010f19640 1 -- 172.21.15.69:0/3043672419 shutdown_connections 2024-05-14T17:05:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.035+0000 7f2010f19640 1 --2- 172.21.15.69:0/3043672419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c11c230 0x7f200c11e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.035+0000 7f2010f19640 1 --2- 172.21.15.69:0/3043672419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f200c111620 0x7f200c109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.035+0000 7f2010f19640 1 --2- 172.21.15.69:0/3043672419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f200c111240 0x7f200c109a50 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.035+0000 7f2010f19640 1 -- 172.21.15.69:0/3043672419 >> 172.21.15.69:0/3043672419 conn(0x7f200c06db20 msgr2=0x7f200c06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.037+0000 7f2010f19640 1 -- 172.21.15.69:0/3043672419 shutdown_connections 2024-05-14T17:05:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.040+0000 7f2010f19640 1 -- 172.21.15.69:0/3043672419 wait complete. 2024-05-14T17:05:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.042+0000 7f2010f19640 1 Processor -- start 2024-05-14T17:05:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.042+0000 7f2010f19640 1 -- start start 2024-05-14T17:05:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.042+0000 7f2010f19640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f200c111240 0x7f200c11b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.042+0000 7f2010f19640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f200c11c230 0x7f200c117980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.042+0000 7f2010f19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c117ec0 0x7f200c118370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.042+0000 7f2010f19640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1142a0 con 0x7f200c11c230 2024-05-14T17:05:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.042+0000 7f2010f19640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f200c114120 con 0x7f200c117ec0 2024-05-14T17:05:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.042+0000 7f2010f19640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f200c114420 con 0x7f200c111240 2024-05-14T17:05:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.043+0000 7f2009d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f200c11c230 0x7f200c117980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.043+0000 7f2009d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f200c11c230 0x7f200c117980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60804/0 (socket says 172.21.15.69:60804) 2024-05-14T17:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.043+0000 7f2009d9b640 1 -- 172.21.15.69:0/3564699762 learned_addr learned my addr 172.21.15.69:0/3564699762 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f200ad9d640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c117ec0 0x7f200c118370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f200a59c640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f200c111240 0x7f200c11b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f2009d9b640 1 -- 172.21.15.69:0/3564699762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f200c111240 msgr2=0x7f200c11b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f2009d9b640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f200c111240 0x7f200c11b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f2009d9b640 1 -- 172.21.15.69:0/3564699762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c117ec0 msgr2=0x7f200c118370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f2009d9b640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c117ec0 0x7f200c118370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f2009d9b640 1 -- 172.21.15.69:0/3564699762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c1ac020 con 0x7f200c11c230 2024-05-14T17:05:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f200ad9d640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c117ec0 0x7f200c118370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f2009d9b640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f200c11c230 0x7f200c117980 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c970 tx=0x7f1ffc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f1ffb7fe640 1 -- 172.21.15.69:0/3564699762 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc007bb0 con 0x7f200c11c230 2024-05-14T17:05:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f1ffb7fe640 1 -- 172.21.15.69:0/3564699762 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ffc007d50 con 0x7f200c11c230 2024-05-14T17:05:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f1ffb7fe640 1 -- 172.21.15.69:0/3564699762 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00f450 con 0x7f200c11c230 2024-05-14T17:05:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f2010f19640 1 -- 172.21.15.69:0/3564699762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200c1ac2b0 con 0x7f200c11c230 2024-05-14T17:05:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.044+0000 7f2010f19640 1 -- 172.21.15.69:0/3564699762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200c1ac780 con 0x7f200c11c230 2024-05-14T17:05:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.050+0000 7f2013c38740 1 Processor -- start 2024-05-14T17:05:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.050+0000 7f2013c38740 1 -- start start 2024-05-14T17:05:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f2013c38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f2013c38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c5210 0x55611a8cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f2013c38740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8cfad0 0x55611a8d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f2013c38740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55611a837c30 con 0x55611a8cfad0 2024-05-14T17:05:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f2013c38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55611a8277d0 con 0x55611a8c48f0 2024-05-14T17:05:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f2013c38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55611a82b810 con 0x55611a8c5210 2024-05-14T17:05:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f200ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f200ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:36678/0 (socket says 172.21.15.69:36678) 2024-05-14T17:05:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f200ad9d640 1 -- 172.21.15.69:0/3964199297 learned_addr learned my addr 172.21.15.69:0/3964199297 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f200ad9d640 1 -- 172.21.15.69:0/3964199297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c5210 msgr2=0x55611a8cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f200ad9d640 1 --2- 172.21.15.69:0/3964199297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c5210 0x55611a8cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f200ad9d640 1 -- 172.21.15.69:0/3964199297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8cfad0 msgr2=0x55611a8d1ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:05:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f200ad9d640 1 --2- 172.21.15.69:0/3964199297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8cfad0 0x55611a8d1ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:23.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f200ad9d640 1 -- 172.21.15.69:0/3964199297 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611a53c210 con 0x55611a8c48f0 2024-05-14T17:05:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f1ffb7fe640 1 -- 172.21.15.69:0/3564699762 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1ffc0046c0 con 0x7f200c11c230 2024-05-14T17:05:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.051+0000 7f200ad9d640 1 --2- 172.21.15.69:0/3964199297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8c4cd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2000005940 tx=0x7f200000e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceec33cdaecc86a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.052+0000 7f1ff8ff9640 1 -- 172.21.15.69:0/3964199297 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000004020 con 0x55611a8c48f0 2024-05-14T17:05:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.052+0000 7f1ff8ff9640 1 -- 172.21.15.69:0/3964199297 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20000041c0 con 0x55611a8c48f0 2024-05-14T17:05:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.052+0000 7f1ff8ff9640 1 -- 172.21.15.69:0/3964199297 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000106b0 con 0x55611a8c48f0 2024-05-14T17:05:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.052+0000 7f2013c38740 1 -- 172.21.15.69:0/3964199297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 msgr2=0x55611a8c4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:24.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.052+0000 7f2013c38740 1 --2- 172.21.15.69:0/3964199297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8c4cd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2000005940 tx=0x7f200000e350 comp rx=0 tx=0).stop 2024-05-14T17:05:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f2013c38740 1 -- 172.21.15.69:0/3964199297 shutdown_connections 2024-05-14T17:05:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f2013c38740 1 --2- 172.21.15.69:0/3964199297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8cfad0 0x55611a8d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f2013c38740 1 --2- 172.21.15.69:0/3964199297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c5210 0x55611a8cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f2013c38740 1 --2- 172.21.15.69:0/3964199297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8c4cd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f2013c38740 1 -- 172.21.15.69:0/3964199297 >> 172.21.15.69:0/3964199297 conn(0x55611a8c0e00 msgr2=0x55611a8c11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f1ffb7fe640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ff40b2ae0 0x7f1ff40b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f2013c38740 1 -- 172.21.15.69:0/3964199297 shutdown_connections 2024-05-14T17:05:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f2013c38740 1 -- 172.21.15.69:0/3964199297 wait complete. 2024-05-14T17:05:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f1ffb7fe640 1 -- 172.21.15.69:0/3564699762 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f1ffc0e2190 con 0x7f200c11c230 2024-05-14T17:05:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f2013c38740 1 -- start start 2024-05-14T17:05:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f200ad9d640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ff40b2ae0 0x7f1ff40b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.053+0000 7f2013c38740 1 Processor -- start 2024-05-14T17:05:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f2013c38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f2013c38740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f2013c38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 0x55611a8cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f2013c38740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55611a50d790 con 0x55611a8c5210 2024-05-14T17:05:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f2013c38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55611a7e3890 con 0x55611a8cfad0 2024-05-14T17:05:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f2013c38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55611a8d27a0 con 0x55611a8c48f0 2024-05-14T17:05:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f2013c38740 10 client.0 _ll_register_callbacks cb 0x55611a7e6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f200a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f200a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:58968/0 (socket says 172.21.15.69:58968) 2024-05-14T17:05:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f200a59c640 1 -- 172.21.15.69:0/1548184068 learned_addr learned my addr 172.21.15.69:0/1548184068 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f2009d9b640 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f200ad9d640 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 0x55611a8cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f200ad9d640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ff40b2ae0 0x7f1ff40b4fa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2000003d40 tx=0x7f2000003e40 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.054+0000 7f200ad9d640 1 -- 172.21.15.69:0/1548184068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 msgr2=0x55611a8cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f200ad9d640 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f200ad9d640 1 -- 172.21.15.69:0/1548184068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 msgr2=0x55611a8cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f200ad9d640 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f200ad9d640 1 -- 172.21.15.69:0/1548184068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611a608620 con 0x55611a8cfad0 2024-05-14T17:05:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f200a59c640 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f200ad9d640 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 0x55611a8cccd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2000000c00 tx=0x7f200000e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e77d82d4282044 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f1ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:05:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/1548184068 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000003fe0 con 0x55611a8cfad0 2024-05-14T17:05:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/1548184068 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f2000004370 con 0x55611a8cfad0 2024-05-14T17:05:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f2013c38740 1 -- 172.21.15.69:0/1548184068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20000041c0 con 0x55611a8cfad0 2024-05-14T17:05:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/1548184068 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2000011990 con 0x55611a8cfad0 2024-05-14T17:05:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f2013c38740 10 client.26360 fetch_fsmap learned FSMap version 58 2024-05-14T17:05:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f2013c38740 1 -- 172.21.15.69:0/1548184068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55611a8ddff0 con 0x55611a8cfad0 2024-05-14T17:05:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.055+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/1548184068 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20000041c0 con 0x55611a8cfad0 2024-05-14T17:05:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.056+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/1548184068 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 58) ==== 8338+0+0 (secure 0 0 0) 0x7f200002f070 con 0x55611a8cfad0 2024-05-14T17:05:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.056+0000 7f2013c38740 10 client.26360 fetch_fsmap finished waiting for FSMap version 58 2024-05-14T17:05:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.056+0000 7f2013c38740 10 client.26360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.056+0000 7f2013c38740 20 client.26360 populate_metadata read hostname 'smithi069' 2024-05-14T17:05:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.056+0000 7f2013c38740 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55611a8df400 0x55611a8e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.056+0000 7f2013c38740 4 client.26360 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.056+0000 7f2013c38740 1 -- 172.21.15.69:0/1548184068 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55611a839b80 con 0x55611a8df400 2024-05-14T17:05:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.056+0000 7f200a59c640 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55611a8df400 0x55611a8e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.056+0000 7f1ff1ffb640 10 client.26360 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.057+0000 7f200a59c640 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55611a8df400 0x55611a8e17f0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aba46a41bfe8f8af server_cookie=86be27a94145479b in_seq=0 out_seq=0 2024-05-14T17:05:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/1548184068 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55611a839b80 con 0x55611a8df400 2024-05-14T17:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f1ff1ffb640 10 client.26360 handle_command_reply: tid=0 2024-05-14T17:05:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f2013c38740 1 client.26360 shutdown 2024-05-14T17:05:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f2013c38740 1 -- 172.21.15.69:0/1548184068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 msgr2=0x55611a8cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f2013c38740 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 0x55611a8cccd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2000000c00 tx=0x7f200000e350 comp rx=0 tx=0).stop 2024-05-14T17:05:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f2013c38740 1 -- 172.21.15.69:0/1548184068 shutdown_connections 2024-05-14T17:05:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f2013c38740 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55611a8df400 0x55611a8e17f0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f2013c38740 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 0x55611a8cccd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f2013c38740 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f2013c38740 1 --2- 172.21.15.69:0/1548184068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f2013c38740 1 -- 172.21.15.69:0/1548184068 >> 172.21.15.69:0/1548184068 conn(0x55611a8c0e00 msgr2=0x55611a8c0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f1ff1ffb640 0 client.26360 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.058+0000 7f2013c38740 1 -- 172.21.15.69:0/1548184068 shutdown_connections 2024-05-14T17:05:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.059+0000 7f2013c38740 1 -- 172.21.15.69:0/1548184068 wait complete. 2024-05-14T17:05:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.059+0000 7f2013c38740 20 client.26360 trim_cache size 0 max 16384 2024-05-14T17:05:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.077+0000 7f2013c38740 1 Processor -- start 2024-05-14T17:05:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f2013c38740 1 -- start start 2024-05-14T17:05:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f2013c38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f2013c38740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f2013c38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 0x55611a8e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f2013c38740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec011010 con 0x55611a8c5210 2024-05-14T17:05:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f2013c38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec00e8e0 con 0x55611a8cfad0 2024-05-14T17:05:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f200a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f200a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8db670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:58978/0 (socket says 172.21.15.69:58978) 2024-05-14T17:05:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f200a59c640 1 -- 172.21.15.69:0/2639515050 learned_addr learned my addr 172.21.15.69:0/2639515050 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f2009d9b640 1 --2- 172.21.15.69:0/2639515050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f2013c38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec00c730 con 0x55611a8c48f0 2024-05-14T17:05:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.078+0000 7f200ad9d640 1 --2- 172.21.15.69:0/2639515050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 0x55611a8e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.079+0000 7f200ad9d640 1 -- 172.21.15.69:0/2639515050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 msgr2=0x55611a8db670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.079+0000 7f200ad9d640 1 --2- 172.21.15.69:0/2639515050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8db670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.079+0000 7f200ad9d640 1 -- 172.21.15.69:0/2639515050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 msgr2=0x55611a8cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.079+0000 7f200ad9d640 1 --2- 172.21.15.69:0/2639515050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.079+0000 7f200ad9d640 1 -- 172.21.15.69:0/2639515050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611a8e1db0 con 0x55611a8cfad0 2024-05-14T17:05:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.079+0000 7f200ad9d640 1 --2- 172.21.15.69:0/2639515050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 0x55611a8e6fe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2000003eb0 tx=0x7f2000011d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27507ba88c8ba3d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.079+0000 7f1ff3fff640 1 -- 172.21.15.69:0/2639515050 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000031e60 con 0x55611a8cfad0 2024-05-14T17:05:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.079+0000 7f1ff3fff640 1 -- 172.21.15.69:0/2639515050 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200003e020 con 0x55611a8cfad0 2024-05-14T17:05:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.079+0000 7f1ff3fff640 1 -- 172.21.15.69:0/2639515050 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000316a0 con 0x55611a8cfad0 2024-05-14T17:05:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.080+0000 7f2013c38740 1 -- 172.21.15.69:0/2639515050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 msgr2=0x55611a8e6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.080+0000 7f2013c38740 1 --2- 172.21.15.69:0/2639515050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 0x55611a8e6fe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2000003eb0 tx=0x7f2000011d80 comp rx=0 tx=0).stop 2024-05-14T17:05:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.080+0000 7f2013c38740 1 -- 172.21.15.69:0/2639515050 shutdown_connections 2024-05-14T17:05:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.080+0000 7f2013c38740 1 --2- 172.21.15.69:0/2639515050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8cfad0 0x55611a8e6fe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.080+0000 7f2013c38740 1 --2- 172.21.15.69:0/2639515050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.080+0000 7f2013c38740 1 --2- 172.21.15.69:0/2639515050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8c48f0 0x55611a8db670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.080+0000 7f2013c38740 1 -- 172.21.15.69:0/2639515050 >> 172.21.15.69:0/2639515050 conn(0x55611a8c0e00 msgr2=0x55611a8c6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.080+0000 7f2013c38740 1 -- 172.21.15.69:0/2639515050 shutdown_connections 2024-05-14T17:05:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.080+0000 7f2013c38740 1 -- 172.21.15.69:0/2639515050 wait complete. 2024-05-14T17:05:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.081+0000 7f2013c38740 1 -- start start 2024-05-14T17:05:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.081+0000 7f2013c38740 1 Processor -- start 2024-05-14T17:05:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f2013c38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8d7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f2013c38740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f2013c38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8cfad0 0x55611a8d34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f200a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8d7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f200a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8d7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:36696/0 (socket says 172.21.15.69:36696) 2024-05-14T17:05:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f200a59c640 1 -- 172.21.15.69:0/2754531312 learned_addr learned my addr 172.21.15.69:0/2754531312 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec00cda0 con 0x55611a8c5210 2024-05-14T17:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec00e210 con 0x55611a8c48f0 2024-05-14T17:05:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec00d470 con 0x55611a8cfad0 2024-05-14T17:05:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f2013c38740 10 client.0 _ll_register_callbacks cb 0x7f2000028750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f2009d9b640 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f200ad9d640 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8cfad0 0x55611a8d34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f200a59c640 1 -- 172.21.15.69:0/2754531312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8cfad0 msgr2=0x55611a8d34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f200a59c640 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8cfad0 0x55611a8d34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f200a59c640 1 -- 172.21.15.69:0/2754531312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 msgr2=0x55611a8d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f200a59c640 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8d2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f200a59c640 1 -- 172.21.15.69:0/2754531312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611a8c0c30 con 0x55611a8c48f0 2024-05-14T17:05:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f200a59c640 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8d7c40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f200401a630 tx=0x7f200403fa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c91ba393c6b521e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.082+0000 7f1ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:05:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/2754531312 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004044040 con 0x55611a8c48f0 2024-05-14T17:05:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/2754531312 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f200403fd90 con 0x55611a8c48f0 2024-05-14T17:05:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f200003e020 con 0x55611a8c48f0 2024-05-14T17:05:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/2754531312 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2004068050 con 0x55611a8c48f0 2024-05-14T17:05:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f2013c38740 10 client.26366 fetch_fsmap learned FSMap version 58 2024-05-14T17:05:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55611a9496c0 con 0x55611a8c48f0 2024-05-14T17:05:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/2754531312 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200405c070 con 0x55611a8c48f0 2024-05-14T17:05:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/2754531312 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 58) ==== 8338+0+0 (secure 0 0 0) 0x7f2004058520 con 0x55611a8c48f0 2024-05-14T17:05:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f2013c38740 10 client.26366 fetch_fsmap finished waiting for FSMap version 58 2024-05-14T17:05:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f2013c38740 10 client.26366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f2013c38740 20 client.26366 populate_metadata read hostname 'smithi069' 2024-05-14T17:05:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f2013c38740 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55611a949e40 0x55611a94c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f2013c38740 4 client.26366 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f200003e020 con 0x55611a949e40 2024-05-14T17:05:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.083+0000 7f2009d9b640 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55611a949e40 0x55611a94c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.084+0000 7f2009d9b640 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55611a949e40 0x55611a94c260 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=208d5b8c3eb5bf28 server_cookie=ebf6fec419c7b4d5 in_seq=0 out_seq=0 2024-05-14T17:05:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.084+0000 7f1ff1ffb640 10 client.26366 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.096+0000 7f1ff1ffb640 1 -- 172.21.15.69:0/2754531312 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f200003e020 con 0x55611a949e40 2024-05-14T17:05:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.096+0000 7f1ff1ffb640 10 client.26366 handle_command_reply: tid=0 2024-05-14T17:05:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.096+0000 7f2013c38740 1 client.26366 shutdown 2024-05-14T17:05:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.096+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 msgr2=0x55611a8d7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.096+0000 7f2013c38740 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8d7c40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f200401a630 tx=0x7f200403fa50 comp rx=0 tx=0).stop 2024-05-14T17:05:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.096+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 shutdown_connections 2024-05-14T17:05:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2013c38740 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55611a949e40 0x55611a94c260 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f1ff1ffb640 0 client.26366 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2013c38740 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55611a8cfad0 0x55611a8d34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2013c38740 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55611a8c5210 0x55611a8d2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2013c38740 1 --2- 172.21.15.69:0/2754531312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55611a8c48f0 0x55611a8d7c40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 >> 172.21.15.69:0/2754531312 conn(0x55611a8c0e00 msgr2=0x55611a8cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 shutdown_connections 2024-05-14T17:05:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2013c38740 1 -- 172.21.15.69:0/2754531312 wait complete. 2024-05-14T17:05:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2013c38740 20 client.26366 trim_cache size 0 max 16384 2024-05-14T17:05:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:05:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:05:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:05:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 -- 172.21.15.69:0/3564699762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ff40b2ae0 msgr2=0x7f1ff40b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ff40b2ae0 0x7f1ff40b4fa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2000003d40 tx=0x7f2000003e40 comp rx=0 tx=0).stop 2024-05-14T17:05:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 -- 172.21.15.69:0/3564699762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f200c11c230 msgr2=0x7f200c117980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f200c11c230 0x7f200c117980 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c970 tx=0x7f1ffc00ce30 comp rx=0 tx=0).stop 2024-05-14T17:05:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 -- 172.21.15.69:0/3564699762 shutdown_connections 2024-05-14T17:05:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ff40b2ae0 0x7f1ff40b4fa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f200c117ec0 0x7f200c118370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f200c11c230 0x7f200c117980 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 --2- 172.21.15.69:0/3564699762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f200c111240 0x7f200c11b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 -- 172.21.15.69:0/3564699762 >> 172.21.15.69:0/3564699762 conn(0x7f200c06db20 msgr2=0x7f200c11c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.097+0000 7f2010f19640 1 -- 172.21.15.69:0/3564699762 shutdown_connections 2024-05-14T17:05:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:10.098+0000 7f2010f19640 1 -- 172.21.15.69:0/3564699762 wait complete. 2024-05-14T17:05:31.689 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 880.0 s, 1905.2 tps, lat 52.828 ms stddev 77.347 2024-05-14T17:05:31.739 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:10 smithi069 ceph-mon[31507]: pgmap v483: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 MiB/s wr, 1.82k op/s 2024-05-14T17:05:31.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:12 smithi069 ceph-mon[31507]: pgmap v484: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-05-14T17:05:31.854 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:14 smithi069 ceph-mon[31507]: pgmap v485: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 27 MiB/s wr, 1.85k op/s 2024-05-14T17:05:31.905 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 885.0 s, 1730.6 tps, lat 57.804 ms stddev 61.145 2024-05-14T17:05:31.955 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:21 smithi092 ceph-mon[35763]: pgmap v489: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 792 op/s 2024-05-14T17:05:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:22 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:05:32.071 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:23 smithi092 ceph-mon[35763]: pgmap v490: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 24 MiB/s wr, 1.81k op/s 2024-05-14T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:23 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:05:32.171 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:23 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:05:32.221 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:23 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:05:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:24 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3267096311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:32.321 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:25 smithi092 ceph-mon[35763]: pgmap v491: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-05-14T17:05:32.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:27 smithi092 ceph-mon[35763]: pgmap v492: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 22 MiB/s wr, 1.72k op/s 2024-05-14T17:05:32.421 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:29 smithi092 ceph-mon[35763]: pgmap v493: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-14T17:05:32.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:21 smithi203 ceph-mon[35787]: pgmap v489: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 792 op/s 2024-05-14T17:05:32.521 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:22 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:05:32.571 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:23 smithi203 ceph-mon[35787]: pgmap v490: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 24 MiB/s wr, 1.81k op/s 2024-05-14T17:05:32.621 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:23 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:05:32.705 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:23 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:05:32.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:23 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:05:32.805 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:24 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3267096311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:32.863 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:25 smithi203 ceph-mon[35787]: pgmap v491: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-05-14T17:05:32.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:27 smithi203 ceph-mon[35787]: pgmap v492: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 22 MiB/s wr, 1.72k op/s 2024-05-14T17:05:32.978 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:29 smithi203 ceph-mon[35787]: pgmap v493: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-14T17:05:33.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:31 smithi092 ceph-mon[35763]: pgmap v494: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-14T17:05:33.080 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:31 smithi203 ceph-mon[35787]: pgmap v494: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-14T17:05:33.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:16 smithi069 ceph-mon[31507]: pgmap v486: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 1.60k op/s 2024-05-14T17:05:33.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:18 smithi069 ceph-mon[31507]: pgmap v487: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2024-05-14T17:05:33.230 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:19 smithi069 ceph-mon[31507]: pgmap v488: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 888 op/s 2024-05-14T17:05:33.280 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 890.0 s, 1543.9 tps, lat 57.920 ms stddev 63.350 2024-05-14T17:05:33.331 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:05:33.381 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:21 smithi069 ceph-mon[31507]: pgmap v489: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 792 op/s 2024-05-14T17:05:33.430 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:22 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:05:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.268+0000 7f511b7fe640 1 -- 172.21.15.69:0/1553010486 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511402fe40 con 0x7f511c11c230 2024-05-14T17:05:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.269+0000 7f5122c2a640 1 -- 172.21.15.69:0/1553010486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f511c11c230 msgr2=0x7f511c11e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.269+0000 7f5122c2a640 1 --2- 172.21.15.69:0/1553010486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f511c11c230 0x7f511c11e620 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5114009cd0 tx=0x7f511402a970 comp rx=0 tx=0).stop 2024-05-14T17:05:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.269+0000 7f5122c2a640 1 -- 172.21.15.69:0/1553010486 shutdown_connections 2024-05-14T17:05:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.269+0000 7f5122c2a640 1 --2- 172.21.15.69:0/1553010486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f511c11c230 0x7f511c11e620 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.269+0000 7f5122c2a640 1 --2- 172.21.15.69:0/1553010486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f511c111620 0x7f511c109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.269+0000 7f5122c2a640 1 --2- 172.21.15.69:0/1553010486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c111240 0x7f511c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.269+0000 7f5122c2a640 1 -- 172.21.15.69:0/1553010486 >> 172.21.15.69:0/1553010486 conn(0x7f511c06db20 msgr2=0x7f511c06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.269+0000 7f5122c2a640 1 -- 172.21.15.69:0/1553010486 shutdown_connections 2024-05-14T17:05:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.269+0000 7f5122c2a640 1 -- 172.21.15.69:0/1553010486 wait complete. 2024-05-14T17:05:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.270+0000 7f5122c2a640 1 Processor -- start 2024-05-14T17:05:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.270+0000 7f5122c2a640 1 -- start start 2024-05-14T17:05:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.270+0000 7f5122c2a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c111240 0x7f511c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.270+0000 7f5122c2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f511c11c230 0x7f511c119c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.270+0000 7f5122c2a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f511c1128c0 0x7f511c112d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.270+0000 7f5122c2a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f511c121b50 con 0x7f511c11c230 2024-05-14T17:05:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.270+0000 7f5122c2a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f511c1219d0 con 0x7f511c1128c0 2024-05-14T17:05:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.270+0000 7f5122c2a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f511c121cd0 con 0x7f511c111240 2024-05-14T17:05:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f51211c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f511c1128c0 0x7f511c112d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f51209c6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c111240 0x7f511c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f51211c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f511c1128c0 0x7f511c112d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:53886/0 (socket says 172.21.15.69:53886) 2024-05-14T17:05:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f51211c7640 1 -- 172.21.15.69:0/1768293769 learned_addr learned my addr 172.21.15.69:0/1768293769 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f511bfff640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f511c11c230 0x7f511c119c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f51209c6640 1 -- 172.21.15.69:0/1768293769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f511c1128c0 msgr2=0x7f511c112d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f51209c6640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f511c1128c0 0x7f511c112d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f51209c6640 1 -- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f511c11c230 msgr2=0x7f511c119c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f51209c6640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f511c11c230 0x7f511c119c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f51209c6640 1 -- 172.21.15.69:0/1768293769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511c113630 con 0x7f511c111240 2024-05-14T17:05:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f511bfff640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f511c11c230 0x7f511c119c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.271+0000 7f51209c6640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c111240 0x7f511c119750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f511000d6d0 tx=0x7f511000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.272+0000 7f5119ffb640 1 -- 172.21.15.69:0/1768293769 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110017070 con 0x7f511c111240 2024-05-14T17:05:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.272+0000 7f5122c2a640 1 -- 172.21.15.69:0/1768293769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511c117390 con 0x7f511c111240 2024-05-14T17:05:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.272+0000 7f5122c2a640 1 -- 172.21.15.69:0/1768293769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511c117930 con 0x7f511c111240 2024-05-14T17:05:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.272+0000 7f5119ffb640 1 -- 172.21.15.69:0/1768293769 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5110009820 con 0x7f511c111240 2024-05-14T17:05:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.272+0000 7f5119ffb640 1 -- 172.21.15.69:0/1768293769 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110004c40 con 0x7f511c111240 2024-05-14T17:05:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.275+0000 7f5125a23740 1 Processor -- start 2024-05-14T17:05:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.275+0000 7f5125a23740 1 -- start start 2024-05-14T17:05:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f5125a23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f5125a23740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f5125a23740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059cbad0 0x5578059cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f5125a23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557805933c30 con 0x5578059c08f0 2024-05-14T17:05:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f5125a23740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5578059237d0 con 0x5578059cbad0 2024-05-14T17:05:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f5125a23740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557805927810 con 0x5578059c1210 2024-05-14T17:05:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f51211c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f51211c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52768/0 (socket says 172.21.15.69:52768) 2024-05-14T17:05:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f51211c7640 1 -- 172.21.15.69:0/1021006562 learned_addr learned my addr 172.21.15.69:0/1021006562 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f511bfff640 1 --2- 172.21.15.69:0/1021006562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059cbad0 0x5578059cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f51211c7640 1 -- 172.21.15.69:0/1021006562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 msgr2=0x5578059cb590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:05:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f51211c7640 1 --2- 172.21.15.69:0/1021006562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059cb590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f51211c7640 1 -- 172.21.15.69:0/1021006562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059cbad0 msgr2=0x5578059cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f51211c7640 1 --2- 172.21.15.69:0/1021006562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059cbad0 0x5578059cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.276+0000 7f51211c7640 1 -- 172.21.15.69:0/1021006562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557805638210 con 0x5578059c08f0 2024-05-14T17:05:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.277+0000 7f51211c7640 1 --2- 172.21.15.69:0/1021006562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059c0cd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f511402fd10 tx=0x7f5114005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0d82d445dd5d03 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.277+0000 7f5119ffb640 1 -- 172.21.15.69:0/1768293769 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5110010460 con 0x7f511c111240 2024-05-14T17:05:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.277+0000 7f5102ffd640 1 -- 172.21.15.69:0/1021006562 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114048070 con 0x5578059c08f0 2024-05-14T17:05:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.277+0000 7f5102ffd640 1 -- 172.21.15.69:0/1021006562 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51140098e0 con 0x5578059c08f0 2024-05-14T17:05:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.277+0000 7f5102ffd640 1 -- 172.21.15.69:0/1021006562 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114034070 con 0x5578059c08f0 2024-05-14T17:05:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5119ffb640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f51040b2a90 0x7f51040b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5119ffb640 1 -- 172.21.15.69:0/1768293769 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f51100e1dd0 con 0x7f511c111240 2024-05-14T17:05:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f51209c6640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f51040b2a90 0x7f51040b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5125a23740 1 -- 172.21.15.69:0/1021006562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 msgr2=0x5578059c0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5125a23740 1 --2- 172.21.15.69:0/1021006562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059c0cd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f511402fd10 tx=0x7f5114005d80 comp rx=0 tx=0).stop 2024-05-14T17:05:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5125a23740 1 -- 172.21.15.69:0/1021006562 shutdown_connections 2024-05-14T17:05:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5125a23740 1 --2- 172.21.15.69:0/1021006562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059cbad0 0x5578059cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5125a23740 1 --2- 172.21.15.69:0/1021006562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5125a23740 1 --2- 172.21.15.69:0/1021006562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059c0cd0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5125a23740 1 -- 172.21.15.69:0/1021006562 >> 172.21.15.69:0/1021006562 conn(0x5578059bce00 msgr2=0x5578059bd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5125a23740 1 -- 172.21.15.69:0/1021006562 shutdown_connections 2024-05-14T17:05:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f5125a23740 1 -- 172.21.15.69:0/1021006562 wait complete. 2024-05-14T17:05:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.278+0000 7f51209c6640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f51040b2a90 0x7f51040b4f50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5578059c1e90 tx=0x7f51100046c0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f5125a23740 1 -- start start 2024-05-14T17:05:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f5125a23740 1 Processor -- start 2024-05-14T17:05:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f5125a23740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c08f0 0x5578059cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:37.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f5125a23740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f5125a23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059cbad0 0x5578059c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f5125a23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557805609790 con 0x5578059cbad0 2024-05-14T17:05:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f5125a23740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5578058df890 con 0x5578059c08f0 2024-05-14T17:05:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f5125a23740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5578059ce740 con 0x5578059c1210 2024-05-14T17:05:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f5125a23740 10 client.0 _ll_register_callbacks cb 0x5578058e2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f51211c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c08f0 0x5578059cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f51211c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c08f0 0x5578059cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:53906/0 (socket says 172.21.15.69:53906) 2024-05-14T17:05:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f51211c7640 1 -- 172.21.15.69:0/2048733580 learned_addr learned my addr 172.21.15.69:0/2048733580 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f511bfff640 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.279+0000 7f51211c7640 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059cbad0 0x5578059c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.280+0000 7f51211c7640 1 -- 172.21.15.69:0/2048733580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 msgr2=0x5578059c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.280+0000 7f51211c7640 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.280+0000 7f51211c7640 1 -- 172.21.15.69:0/2048733580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059cbad0 msgr2=0x5578059c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.280+0000 7f51211c7640 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059cbad0 0x5578059c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.280+0000 7f51211c7640 1 -- 172.21.15.69:0/2048733580 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557805704620 con 0x5578059c08f0 2024-05-14T17:05:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.281+0000 7f51211c7640 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c08f0 0x5578059cb430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5114009620 tx=0x7f51140040e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60ced926d4eb242c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.281+0000 7f50fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:05:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f50fbfff640 1 -- 172.21.15.69:0/2048733580 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114034030 con 0x5578059c08f0 2024-05-14T17:05:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f50fbfff640 1 -- 172.21.15.69:0/2048733580 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f511402f380 con 0x5578059c08f0 2024-05-14T17:05:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f5125a23740 1 -- 172.21.15.69:0/2048733580 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51140098e0 con 0x5578059c08f0 2024-05-14T17:05:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f50fbfff640 1 -- 172.21.15.69:0/2048733580 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5114048040 con 0x5578059c08f0 2024-05-14T17:05:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f5125a23740 10 client.26375 fetch_fsmap learned FSMap version 58 2024-05-14T17:05:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f5125a23740 1 -- 172.21.15.69:0/2048733580 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5578059da200 con 0x5578059c08f0 2024-05-14T17:05:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f50fbfff640 1 -- 172.21.15.69:0/2048733580 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51140098e0 con 0x5578059c08f0 2024-05-14T17:05:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f50fbfff640 1 -- 172.21.15.69:0/2048733580 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 58) ==== 8338+0+0 (secure 0 0 0) 0x7f511404e070 con 0x5578059c08f0 2024-05-14T17:05:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f5125a23740 10 client.26375 fetch_fsmap finished waiting for FSMap version 58 2024-05-14T17:05:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f5125a23740 10 client.26375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f5125a23740 20 client.26375 populate_metadata read hostname 'smithi069' 2024-05-14T17:05:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f5125a23740 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5578059db550 0x5578059dd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.282+0000 7f5125a23740 4 client.26375 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.283+0000 7f5125a23740 1 -- 172.21.15.69:0/2048733580 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557805935b80 con 0x5578059db550 2024-05-14T17:05:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.283+0000 7f511bfff640 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5578059db550 0x5578059dd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.283+0000 7f50fbfff640 10 client.26375 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.283+0000 7f511bfff640 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5578059db550 0x5578059dd940 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5943c1cf79c496a4 server_cookie=1be3b3a914c209c2 in_seq=0 out_seq=0 2024-05-14T17:05:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f50fbfff640 1 -- 172.21.15.69:0/2048733580 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557805935b80 con 0x5578059db550 2024-05-14T17:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f50fbfff640 10 client.26375 handle_command_reply: tid=0 2024-05-14T17:05:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 client.26375 shutdown 2024-05-14T17:05:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 -- 172.21.15.69:0/2048733580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c08f0 msgr2=0x5578059cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c08f0 0x5578059cb430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5114009620 tx=0x7f51140040e0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 -- 172.21.15.69:0/2048733580 shutdown_connections 2024-05-14T17:05:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5578059db550 0x5578059dd940 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059cbad0 0x5578059c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 --2- 172.21.15.69:0/2048733580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c08f0 0x5578059cb430 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 -- 172.21.15.69:0/2048733580 >> 172.21.15.69:0/2048733580 conn(0x5578059bce00 msgr2=0x5578059bc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f50fbfff640 0 client.26375 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 -- 172.21.15.69:0/2048733580 shutdown_connections 2024-05-14T17:05:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.284+0000 7f5125a23740 1 -- 172.21.15.69:0/2048733580 wait complete. 2024-05-14T17:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.285+0000 7f5125a23740 20 client.26375 trim_cache size 0 max 16384 2024-05-14T17:05:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.304+0000 7f5125a23740 1 Processor -- start 2024-05-14T17:05:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.304+0000 7f5125a23740 1 -- start start 2024-05-14T17:05:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.304+0000 7f5125a23740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c08f0 0x5578059caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f5125a23740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c1210 0x5578059e0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f5125a23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059cbad0 0x5578059e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f5125a23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5108011010 con 0x5578059cbad0 2024-05-14T17:05:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f5125a23740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f510800e8e0 con 0x5578059c1210 2024-05-14T17:05:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f5125a23740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f510800c730 con 0x5578059c08f0 2024-05-14T17:05:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f511bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c1210 0x5578059e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f511bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c1210 0x5578059e0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:53910/0 (socket says 172.21.15.69:53910) 2024-05-14T17:05:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f51211c7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c08f0 0x5578059caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f511bfff640 1 -- 172.21.15.69:0/1113355662 learned_addr learned my addr 172.21.15.69:0/1113355662 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f51211c7640 1 --2- 172.21.15.69:0/1113355662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059cbad0 0x5578059e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f511bfff640 1 -- 172.21.15.69:0/1113355662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c08f0 msgr2=0x5578059caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f511bfff640 1 --2- 172.21.15.69:0/1113355662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c08f0 0x5578059caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f511bfff640 1 -- 172.21.15.69:0/1113355662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059cbad0 msgr2=0x5578059e0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f511bfff640 1 --2- 172.21.15.69:0/1113355662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059cbad0 0x5578059e0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f511bfff640 1 -- 172.21.15.69:0/1113355662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578059c8ff0 con 0x5578059c1210 2024-05-14T17:05:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f51211c7640 1 --2- 172.21.15.69:0/1113355662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c08f0 0x5578059caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f511bfff640 1 --2- 172.21.15.69:0/1113355662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c1210 0x5578059e0960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f511c06c120 tx=0x7f510c0108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8706cca582ee302 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.305+0000 7f51027fc640 1 -- 172.21.15.69:0/1113355662 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c01b6d0 con 0x5578059c1210 2024-05-14T17:05:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.306+0000 7f51027fc640 1 -- 172.21.15.69:0/1113355662 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f510c01bd20 con 0x5578059c1210 2024-05-14T17:05:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.306+0000 7f51027fc640 1 -- 172.21.15.69:0/1113355662 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c01c8f0 con 0x5578059c1210 2024-05-14T17:05:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.307+0000 7f5125a23740 1 -- 172.21.15.69:0/1113355662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c1210 msgr2=0x5578059e0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.307+0000 7f5125a23740 1 --2- 172.21.15.69:0/1113355662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c1210 0x5578059e0960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f511c06c120 tx=0x7f510c0108c0 comp rx=0 tx=0).stop 2024-05-14T17:05:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.307+0000 7f5125a23740 1 -- 172.21.15.69:0/1113355662 shutdown_connections 2024-05-14T17:05:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.307+0000 7f5125a23740 1 --2- 172.21.15.69:0/1113355662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059cbad0 0x5578059e0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.307+0000 7f5125a23740 1 --2- 172.21.15.69:0/1113355662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059c1210 0x5578059e0960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.307+0000 7f5125a23740 1 --2- 172.21.15.69:0/1113355662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c08f0 0x5578059caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.307+0000 7f5125a23740 1 -- 172.21.15.69:0/1113355662 >> 172.21.15.69:0/1113355662 conn(0x5578059bce00 msgr2=0x5578059c3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.307+0000 7f5125a23740 1 -- 172.21.15.69:0/1113355662 shutdown_connections 2024-05-14T17:05:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.307+0000 7f5125a23740 1 -- 172.21.15.69:0/1113355662 wait complete. 2024-05-14T17:05:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.307+0000 7f5125a23740 1 -- start start 2024-05-14T17:05:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.308+0000 7f5125a23740 1 Processor -- start 2024-05-14T17:05:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.308+0000 7f5125a23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.308+0000 7f5125a23740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059deb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.308+0000 7f5125a23740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059cbad0 0x5578059df050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.308+0000 7f5125a23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f510800cda0 con 0x5578059c08f0 2024-05-14T17:05:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.308+0000 7f5125a23740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f510800e210 con 0x5578059cbad0 2024-05-14T17:05:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.308+0000 7f5125a23740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f510800d470 con 0x5578059c1210 2024-05-14T17:05:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.308+0000 7f5125a23740 10 client.0 _ll_register_callbacks cb 0x7f5114004330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f51211c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f51211c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059de5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52796/0 (socket says 172.21.15.69:52796) 2024-05-14T17:05:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f51211c7640 1 -- 172.21.15.69:0/2463424213 learned_addr learned my addr 172.21.15.69:0/2463424213 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f511bfff640 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f51211c7640 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059cbad0 0x5578059df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f511bfff640 1 -- 172.21.15.69:0/2463424213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059cbad0 msgr2=0x5578059df050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f511bfff640 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059cbad0 0x5578059df050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f511bfff640 1 -- 172.21.15.69:0/2463424213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 msgr2=0x5578059de5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f511bfff640 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059de5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f511bfff640 1 -- 172.21.15.69:0/2463424213 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557805a44b20 con 0x5578059c1210 2024-05-14T17:05:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f511bfff640 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059deb10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5578059bcdd0 tx=0x7f510c01c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97c40831aede5bac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f50fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-05-14T17:05:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f50fbfff640 1 -- 172.21.15.69:0/2463424213 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c01c980 con 0x5578059c1210 2024-05-14T17:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f50fbfff640 1 -- 172.21.15.69:0/2463424213 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f510c01b4d0 con 0x5578059c1210 2024-05-14T17:05:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.309+0000 7f5125a23740 1 -- 172.21.15.69:0/2463424213 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f510c01bd20 con 0x5578059c1210 2024-05-14T17:05:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f50fbfff640 1 -- 172.21.15.69:0/2463424213 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f510c00dd50 con 0x5578059c1210 2024-05-14T17:05:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f5125a23740 10 client.26272 fetch_fsmap learned FSMap version 58 2024-05-14T17:05:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f5125a23740 1 -- 172.21.15.69:0/2463424213 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557805a45df0 con 0x5578059c1210 2024-05-14T17:05:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f50fbfff640 1 -- 172.21.15.69:0/2463424213 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f510c01bd20 con 0x5578059c1210 2024-05-14T17:05:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f50fbfff640 1 -- 172.21.15.69:0/2463424213 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 58) ==== 8338+0+0 (secure 0 0 0) 0x7f510c02c070 con 0x5578059c1210 2024-05-14T17:05:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f5125a23740 10 client.26272 fetch_fsmap finished waiting for FSMap version 58 2024-05-14T17:05:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f5125a23740 10 client.26272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f5125a23740 20 client.26272 populate_metadata read hostname 'smithi069' 2024-05-14T17:05:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f5125a23740 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x557805a464c0 0x557805a488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f5125a23740 4 client.26272 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f5125a23740 1 -- 172.21.15.69:0/2463424213 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55780592f360 con 0x557805a464c0 2024-05-14T17:05:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.310+0000 7f51211c7640 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x557805a464c0 0x557805a488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.311+0000 7f50fbfff640 10 client.26272 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.311+0000 7f51211c7640 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x557805a464c0 0x557805a488e0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac10a1f6bfb2b270 server_cookie=e4524f069e3ee7b7 in_seq=0 out_seq=0 2024-05-14T17:05:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.329+0000 7f50fbfff640 1 -- 172.21.15.69:0/2463424213 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55780592f360 con 0x557805a464c0 2024-05-14T17:05:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.329+0000 7f50fbfff640 10 client.26272 handle_command_reply: tid=0 2024-05-14T17:05:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 client.26272 shutdown 2024-05-14T17:05:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 -- 172.21.15.69:0/2463424213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 msgr2=0x5578059deb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059deb10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5578059bcdd0 tx=0x7f510c01c280 comp rx=0 tx=0).stop 2024-05-14T17:05:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 -- 172.21.15.69:0/2463424213 shutdown_connections 2024-05-14T17:05:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x557805a464c0 0x557805a488e0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f50fbfff640 0 client.26272 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578059cbad0 0x5578059df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578059c1210 0x5578059deb10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 --2- 172.21.15.69:0/2463424213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578059c08f0 0x5578059de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 -- 172.21.15.69:0/2463424213 >> 172.21.15.69:0/2463424213 conn(0x5578059bce00 msgr2=0x5578059c33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 -- 172.21.15.69:0/2463424213 shutdown_connections 2024-05-14T17:05:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 1 -- 172.21.15.69:0/2463424213 wait complete. 2024-05-14T17:05:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.330+0000 7f5125a23740 20 client.26272 trim_cache size 0 max 16384 2024-05-14T17:05:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:05:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:05:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:05:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 -- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f51040b2a90 msgr2=0x7f51040b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f51040b2a90 0x7f51040b4f50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5578059c1e90 tx=0x7f51100046c0 comp rx=0 tx=0).stop 2024-05-14T17:05:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 -- 172.21.15.69:0/1768293769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c111240 msgr2=0x7f511c119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c111240 0x7f511c119750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f511000d6d0 tx=0x7f511000db90 comp rx=0 tx=0).stop 2024-05-14T17:05:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 -- 172.21.15.69:0/1768293769 shutdown_connections 2024-05-14T17:05:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f51040b2a90 0x7f51040b4f50 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5578059c1e90 tx=0x7f51100046c0 comp rx=0 tx=0).stop 2024-05-14T17:05:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f511c1128c0 0x7f511c112d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f511c11c230 0x7f511c119c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 --2- 172.21.15.69:0/1768293769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c111240 0x7f511c119750 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f511000d6d0 tx=0x7f511000db90 comp rx=0 tx=0).stop 2024-05-14T17:05:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 -- 172.21.15.69:0/1768293769 >> 172.21.15.69:0/1768293769 conn(0x7f511c06db20 msgr2=0x7f511c10ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 -- 172.21.15.69:0/1768293769 shutdown_connections 2024-05-14T17:05:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:23.331+0000 7f5122c2a640 1 -- 172.21.15.69:0/1768293769 wait complete. 2024-05-14T17:05:45.583 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:33 smithi203 ceph-mon[35787]: pgmap v495: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-05-14T17:05:45.637 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:35 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:05:45.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:36 smithi203 ceph-mon[35787]: pgmap v496: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 7.1 MiB/s wr, 529 op/s 2024-05-14T17:05:45.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:38 smithi203 ceph-mon[35787]: pgmap v497: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 7.5 MiB/s wr, 585 op/s 2024-05-14T17:05:45.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:40 smithi203 ceph-mon[35787]: pgmap v498: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.1 MiB/s wr, 353 op/s 2024-05-14T17:05:45.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:42 smithi203 ceph-mon[35787]: pgmap v499: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-05-14T17:05:45.904 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:44 smithi203 ceph-mon[35787]: pgmap v500: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-05-14T17:05:45.955 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:33 smithi092 ceph-mon[35763]: pgmap v495: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-05-14T17:05:46.005 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:35 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:05:46.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:36 smithi092 ceph-mon[35763]: pgmap v496: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 7.1 MiB/s wr, 529 op/s 2024-05-14T17:05:46.104 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:38 smithi092 ceph-mon[35763]: pgmap v497: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 7.5 MiB/s wr, 585 op/s 2024-05-14T17:05:46.155 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:40 smithi092 ceph-mon[35763]: pgmap v498: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.1 MiB/s wr, 353 op/s 2024-05-14T17:05:46.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:42 smithi092 ceph-mon[35763]: pgmap v499: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-05-14T17:05:46.255 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:44 smithi092 ceph-mon[35763]: pgmap v500: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-05-14T17:05:46.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:46 smithi092 ceph-mon[35763]: pgmap v501: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 955 KiB/s wr, 126 op/s 2024-05-14T17:05:46.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.828+0000 7fda0f4de700 1 -- 172.21.15.69:0/1582428401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda0811cd20 msgr2=0x7fda0811f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:46.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.828+0000 7fda0f4de700 1 --2- 172.21.15.69:0/1582428401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda0811cd20 0x7fda0811f110 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80098f0 tx=0x7fd9f8009c00 comp rx=0 tx=0).stop 2024-05-14T17:05:46.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.830+0000 7fda0f4de700 1 -- 172.21.15.69:0/1582428401 shutdown_connections 2024-05-14T17:05:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.830+0000 7fda0f4de700 1 --2- 172.21.15.69:0/1582428401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda0811cd20 0x7fda0811f110 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:46.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.830+0000 7fda0f4de700 1 --2- 172.21.15.69:0/1582428401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda08111d20 0x7fda0810a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:46.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.830+0000 7fda0f4de700 1 --2- 172.21.15.69:0/1582428401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda0810aa80 0x7fda0810af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:46.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.830+0000 7fda0f4de700 1 -- 172.21.15.69:0/1582428401 >> 172.21.15.69:0/1582428401 conn(0x7fda0806e6b0 msgr2=0x7fda0806eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:46.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.835+0000 7fda0f4de700 1 -- 172.21.15.69:0/1582428401 shutdown_connections 2024-05-14T17:05:46.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.836+0000 7fda0f4de700 1 -- 172.21.15.69:0/1582428401 wait complete. 2024-05-14T17:05:46.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.837+0000 7fda0f4de700 1 Processor -- start 2024-05-14T17:05:46.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.838+0000 7fda0f4de700 1 -- start start 2024-05-14T17:05:46.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.839+0000 7fda0f4de700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda0810aa80 0x7fda081ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:46.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.839+0000 7fda0f4de700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda08111d20 0x7fda081aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:47.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.839+0000 7fda0f4de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda0811cd20 0x7fda081a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:47.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.839+0000 7fda0f4de700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fda081225b0 con 0x7fda0810aa80 2024-05-14T17:05:47.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.839+0000 7fda0f4de700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fda08122430 con 0x7fda0811cd20 2024-05-14T17:05:47.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.839+0000 7fda0f4de700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fda08122730 con 0x7fda08111d20 2024-05-14T17:05:47.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.840+0000 7fda0da7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda0811cd20 0x7fda081a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:47.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.840+0000 7fda0da7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda0811cd20 0x7fda081a88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:53940/0 (socket says 172.21.15.69:53940) 2024-05-14T17:05:47.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.840+0000 7fda0da7b700 1 -- 172.21.15.69:0/3267096311 learned_addr learned my addr 172.21.15.69:0/3267096311 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:47.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.840+0000 7fda0d27a700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda0810aa80 0x7fda081ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:47.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.841+0000 7fda0d27a700 1 -- 172.21.15.69:0/3267096311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda08111d20 msgr2=0x7fda081aec50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T17:05:47.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.841+0000 7fda0d27a700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda08111d20 0x7fda081aec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:47.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.841+0000 7fda0d27a700 1 -- 172.21.15.69:0/3267096311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda0811cd20 msgr2=0x7fda081a88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:47.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.841+0000 7fda0d27a700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda0811cd20 0x7fda081a88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:47.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.841+0000 7fda0d27a700 1 -- 172.21.15.69:0/3267096311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda081a9030 con 0x7fda0810aa80 2024-05-14T17:05:47.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.842+0000 7fda0d27a700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda0810aa80 0x7fda081ae710 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fda0000e9c0 tx=0x7fda0000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:47.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.843+0000 7fda067fc700 1 -- 172.21.15.69:0/3267096311 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0000c9c0 con 0x7fda0810aa80 2024-05-14T17:05:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.843+0000 7fda067fc700 1 -- 172.21.15.69:0/3267096311 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda0000cb60 con 0x7fda0810aa80 2024-05-14T17:05:47.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.843+0000 7fda067fc700 1 -- 172.21.15.69:0/3267096311 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00018420 con 0x7fda0810aa80 2024-05-14T17:05:47.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.843+0000 7fda0f4de700 1 -- 172.21.15.69:0/3267096311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda081a9320 con 0x7fda0810aa80 2024-05-14T17:05:48.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.843+0000 7fda0f4de700 1 -- 172.21.15.69:0/3267096311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda08112500 con 0x7fda0810aa80 2024-05-14T17:05:48.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.844+0000 7fda0f4de700 1 -- 172.21.15.69:0/3267096311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0806fc40 con 0x7fda0810aa80 2024-05-14T17:05:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.849+0000 7fda067fc700 1 -- 172.21.15.69:0/3267096311 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fda0001f020 con 0x7fda0810aa80 2024-05-14T17:05:48.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.850+0000 7fda067fc700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd9f00b2b80 0x7fd9f00b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:48.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.850+0000 7fda067fc700 1 -- 172.21.15.69:0/3267096311 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7fda000a62a0 con 0x7fda0810aa80 2024-05-14T17:05:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.852+0000 7fda0ca79700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd9f00b2b80 0x7fd9f00b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:48.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.853+0000 7fda0ca79700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd9f00b2b80 0x7fd9f00b5040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda0811cba0 tx=0x7fd9fc00b040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:48.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:23.856+0000 7fda067fc700 1 -- 172.21.15.69:0/3267096311 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda00026020 con 0x7fda0810aa80 2024-05-14T17:05:48.523 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:23 smithi069 ceph-mon[31507]: pgmap v490: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 24 MiB/s wr, 1.81k op/s 2024-05-14T17:05:48.573 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:23 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:05:48.623 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:23 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:05:48.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:23 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:05:48.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.393+0000 7fda0f4de700 1 -- 172.21.15.69:0/3267096311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda0806a230 con 0x7fda0810aa80 2024-05-14T17:05:48.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.394+0000 7fda067fc700 1 -- 172.21.15.69:0/3267096311 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11859 (secure 0 0 0) 0x7fda00026020 con 0x7fda0810aa80 2024-05-14T17:05:48.857 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:05:48.906 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:05:03.624613+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26245":{"gid":26245,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3691870879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3691870879},{"type":"v1","addr":"172.21.15.203:6837","nonce":3691870879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T17:05:48.958 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:48 smithi092 ceph-mon[35763]: pgmap v502: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.4 MiB/s wr, 214 op/s 2024-05-14T17:05:49.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:46 smithi203 ceph-mon[35787]: pgmap v501: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 955 KiB/s wr, 126 op/s 2024-05-14T17:05:49.056 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:48 smithi203 ceph-mon[35787]: pgmap v502: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.4 MiB/s wr, 214 op/s 2024-05-14T17:05:49.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.397+0000 7fda0f4de700 1 -- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd9f00b2b80 msgr2=0x7fd9f00b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:49.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.397+0000 7fda0f4de700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd9f00b2b80 0x7fd9f00b5040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda0811cba0 tx=0x7fd9fc00b040 comp rx=0 tx=0).stop 2024-05-14T17:05:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.397+0000 7fda0f4de700 1 -- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda0810aa80 msgr2=0x7fda081ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:49.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.397+0000 7fda0f4de700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda0810aa80 0x7fda081ae710 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fda0000e9c0 tx=0x7fda0000ecd0 comp rx=0 tx=0).stop 2024-05-14T17:05:49.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.398+0000 7fda0f4de700 1 -- 172.21.15.69:0/3267096311 shutdown_connections 2024-05-14T17:05:49.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.398+0000 7fda0f4de700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd9f00b2b80 0x7fd9f00b5040 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:49.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.398+0000 7fda0f4de700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda0810aa80 0x7fda081ae710 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.398+0000 7fda0f4de700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda0811cd20 0x7fda081a88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:49.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.398+0000 7fda0f4de700 1 --2- 172.21.15.69:0/3267096311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda08111d20 0x7fda081aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:49.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.398+0000 7fda0f4de700 1 -- 172.21.15.69:0/3267096311 >> 172.21.15.69:0/3267096311 conn(0x7fda0806e6b0 msgr2=0x7fda0811ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:49.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.398+0000 7fda0f4de700 1 -- 172.21.15.69:0/3267096311 shutdown_connections 2024-05-14T17:05:49.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:05:24.398+0000 7fda0f4de700 1 -- 172.21.15.69:0/3267096311 wait complete. 2024-05-14T17:05:49.748 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 58 2024-05-14T17:05:49.799 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 149.65 --await-for 45.0' 2024-05-14T17:05:49.848 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 149.65 --await-for 45.0 2024-05-14T17:05:49.899 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 895.0 s, 1514.7 tps, lat 72.742 ms stddev 108.686 2024-05-14T17:05:49.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:24 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3267096311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:49.998 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:25 smithi069 ceph-mon[31507]: pgmap v491: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-05-14T17:05:50.048 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:27 smithi069 ceph-mon[31507]: pgmap v492: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 22 MiB/s wr, 1.72k op/s 2024-05-14T17:05:50.099 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 900.0 s, 2027.2 tps, lat 49.402 ms stddev 55.951 2024-05-14T17:05:50.165 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:29 smithi069 ceph-mon[31507]: pgmap v493: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-14T17:05:50.216 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:49 smithi203 ceph-mon[35787]: pgmap v503: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-14T17:05:50.266 INFO:teuthology.orchestra.run.smithi069.stdout:transaction type: 2024-05-14T17:05:50.315 INFO:teuthology.orchestra.run.smithi069.stdout:scaling factor: 32 2024-05-14T17:05:50.365 INFO:teuthology.orchestra.run.smithi069.stdout:query mode: simple 2024-05-14T17:05:50.415 INFO:teuthology.orchestra.run.smithi069.stdout:number of clients: 100 2024-05-14T17:05:50.465 INFO:teuthology.orchestra.run.smithi069.stdout:number of threads: 4 2024-05-14T17:05:50.537 INFO:teuthology.orchestra.run.smithi069.stdout:duration: 900 s 2024-05-14T17:05:50.591 INFO:teuthology.orchestra.run.smithi069.stdout:number of transactions actually processed: 1529501 2024-05-14T17:05:50.641 INFO:teuthology.orchestra.run.smithi069.stdout:latency average = 58.839 ms 2024-05-14T17:05:50.691 INFO:teuthology.orchestra.run.smithi069.stdout:latency stddev = 966.895 ms 2024-05-14T17:05:50.741 INFO:teuthology.orchestra.run.smithi069.stdout:tps = 1699.181292 (including connections establishing) 2024-05-14T17:05:50.790 INFO:teuthology.orchestra.run.smithi069.stdout:tps = 1699.185691 (excluding connections establishing) 2024-05-14T17:05:50.841 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-14T17:05:50.891 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:31 smithi069 ceph-mon[31507]: pgmap v494: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-14T17:05:50.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:33 smithi069 ceph-mon[31507]: pgmap v495: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-05-14T17:05:50.999 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:34 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:05:51.049 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:36 smithi069 ceph-mon[31507]: pgmap v496: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 7.1 MiB/s wr, 529 op/s 2024-05-14T17:05:51.101 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:49 smithi092 ceph-mon[35763]: pgmap v503: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-14T17:05:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.456+0000 7f102dffb640 1 -- 172.21.15.69:0/860429555 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10240318a0 con 0x7f102809ac40 2024-05-14T17:05:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.456+0000 7f102ffff640 1 -- 172.21.15.69:0/860429555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102809ac40 msgr2=0x7f1028006540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.456+0000 7f102ffff640 1 --2- 172.21.15.69:0/860429555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102809ac40 0x7f1028006540 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1024009900 tx=0x7f102402a3f0 comp rx=0 tx=0).stop 2024-05-14T17:05:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.456+0000 7f102ffff640 1 -- 172.21.15.69:0/860429555 shutdown_connections 2024-05-14T17:05:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.456+0000 7f102ffff640 1 --2- 172.21.15.69:0/860429555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1028007160 0x7f10280987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.456+0000 7f102ffff640 1 --2- 172.21.15.69:0/860429555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f102809b0b0 0x7f1028006a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.456+0000 7f102ffff640 1 --2- 172.21.15.69:0/860429555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102809ac40 0x7f1028006540 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.456+0000 7f102ffff640 1 -- 172.21.15.69:0/860429555 >> 172.21.15.69:0/860429555 conn(0x7f102800a750 msgr2=0x7f102800ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.456+0000 7f102ffff640 1 -- 172.21.15.69:0/860429555 shutdown_connections 2024-05-14T17:05:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.456+0000 7f102ffff640 1 -- 172.21.15.69:0/860429555 wait complete. 2024-05-14T17:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.457+0000 7f102ffff640 1 Processor -- start 2024-05-14T17:05:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102ffff640 1 -- start start 2024-05-14T17:05:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102809ac40 0x7f1028138720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f102809b0b0 0x7f1028138c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10281328d0 0x7f1028132d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102ffff640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f10280ac5d0 con 0x7f102809b0b0 2024-05-14T17:05:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102ffff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10280ac450 con 0x7f102809ac40 2024-05-14T17:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102ffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f10280ac750 con 0x7f10281328d0 2024-05-14T17:05:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102809ac40 0x7f1028138720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10281328d0 0x7f1028132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10281328d0 0x7f1028132d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:55476/0 (socket says 172.21.15.69:55476) 2024-05-14T17:05:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102f7fe640 1 -- 172.21.15.69:0/4165254224 learned_addr learned my addr 172.21.15.69:0/4165254224 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102809ac40 0x7f1028138720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:37284/0 (socket says 172.21.15.69:37284) 2024-05-14T17:05:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.458+0000 7f102e7fc640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f102809b0b0 0x7f1028138c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f102f7fe640 1 -- 172.21.15.69:0/4165254224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102809ac40 msgr2=0x7f1028138720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f102f7fe640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102809ac40 0x7f1028138720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f102f7fe640 1 -- 172.21.15.69:0/4165254224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f102809b0b0 msgr2=0x7f1028138c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f102f7fe640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f102809b0b0 0x7f1028138c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f102f7fe640 1 -- 172.21.15.69:0/4165254224 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10281333f0 con 0x7f10281328d0 2024-05-14T17:05:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f102f7fe640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10281328d0 0x7f1028132d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1030000b60 tx=0x7f103007c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f1017fff640 1 -- 172.21.15.69:0/4165254224 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103007e420 con 0x7f10281328d0 2024-05-14T17:05:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f102ffff640 1 -- 172.21.15.69:0/4165254224 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10281336e0 con 0x7f10281328d0 2024-05-14T17:05:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f1017fff640 1 -- 172.21.15.69:0/4165254224 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f103007ea70 con 0x7f10281328d0 2024-05-14T17:05:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f1017fff640 1 -- 172.21.15.69:0/4165254224 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030086a10 con 0x7f10281328d0 2024-05-14T17:05:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.459+0000 7f102ffff640 1 -- 172.21.15.69:0/4165254224 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102813f8e0 con 0x7f10281328d0 2024-05-14T17:05:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f1037faa740 1 Processor -- start 2024-05-14T17:05:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f1017fff640 1 -- 172.21.15.69:0/4165254224 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f103007b040 con 0x7f10281328d0 2024-05-14T17:05:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f1037faa740 1 -- start start 2024-05-14T17:05:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f1037faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab5641eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f1037faa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56429780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f1037faa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 0x55ab5642c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f1037faa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ab56391c30 con 0x55ab5641f400 2024-05-14T17:05:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f1037faa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ab563817d0 con 0x55ab56429cc0 2024-05-14T17:05:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f1037faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ab56385810 con 0x55ab5641eae0 2024-05-14T17:05:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f102effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56429780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f102effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56429780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48476/0 (socket says 172.21.15.69:48476) 2024-05-14T17:05:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f102effd640 1 -- 172.21.15.69:0/3335862122 learned_addr learned my addr 172.21.15.69:0/3335862122 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f102e7fc640 1 --2- 172.21.15.69:0/3335862122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab5641eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f102e7fc640 1 --2- 172.21.15.69:0/3335862122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 0x55ab5642c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f102effd640 1 -- 172.21.15.69:0/3335862122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 msgr2=0x55ab5641eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f102effd640 1 --2- 172.21.15.69:0/3335862122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab5641eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.462+0000 7f102effd640 1 -- 172.21.15.69:0/3335862122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 msgr2=0x55ab5642c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.463+0000 7f102effd640 1 --2- 172.21.15.69:0/3335862122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 0x55ab5642c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.463+0000 7f102effd640 1 -- 172.21.15.69:0/3335862122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab56096210 con 0x55ab5641f400 2024-05-14T17:05:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.463+0000 7f1017fff640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10180b29e0 0x7f10180b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.463+0000 7f102effd640 1 --2- 172.21.15.69:0/3335862122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56429780 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f102402a910 tx=0x7f1024005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b25d77cf63707b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.463+0000 7f102e7fc640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10180b29e0 0x7f10180b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.463+0000 7f1017fff640 1 -- 172.21.15.69:0/4165254224 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f103014f6f0 con 0x7f10281328d0 2024-05-14T17:05:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.463+0000 7f10157fa640 1 -- 172.21.15.69:0/3335862122 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024040070 con 0x55ab5641f400 2024-05-14T17:05:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.463+0000 7f10157fa640 1 -- 172.21.15.69:0/3335862122 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10240316a0 con 0x55ab5641f400 2024-05-14T17:05:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.463+0000 7f10157fa640 1 -- 172.21.15.69:0/3335862122 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024035040 con 0x55ab5641f400 2024-05-14T17:05:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f1037faa740 1 -- 172.21.15.69:0/3335862122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 msgr2=0x55ab56429780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f102e7fc640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10180b29e0 0x7f10180b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55ab5641c410 tx=0x7f10200046c0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f1037faa740 1 --2- 172.21.15.69:0/3335862122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56429780 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f102402a910 tx=0x7f1024005fd0 comp rx=0 tx=0).stop 2024-05-14T17:05:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f1037faa740 1 -- 172.21.15.69:0/3335862122 shutdown_connections 2024-05-14T17:05:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f1037faa740 1 --2- 172.21.15.69:0/3335862122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 0x55ab5642c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f1037faa740 1 --2- 172.21.15.69:0/3335862122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56429780 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f1037faa740 1 --2- 172.21.15.69:0/3335862122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab5641eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f1037faa740 1 -- 172.21.15.69:0/3335862122 >> 172.21.15.69:0/3335862122 conn(0x55ab5641aff0 msgr2=0x55ab5641b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f1037faa740 1 -- 172.21.15.69:0/3335862122 shutdown_connections 2024-05-14T17:05:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f1037faa740 1 -- 172.21.15.69:0/3335862122 wait complete. 2024-05-14T17:05:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.464+0000 7f1037faa740 1 -- start start 2024-05-14T17:05:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f1037faa740 1 Processor -- start 2024-05-14T17:05:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f1037faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab56429510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f1037faa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56426800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f1037faa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 0x55ab56426d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f1037faa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ab56067790 con 0x55ab5641f400 2024-05-14T17:05:55.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f1037faa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ab5633d890 con 0x55ab56429cc0 2024-05-14T17:05:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f1037faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ab5642c930 con 0x55ab5641eae0 2024-05-14T17:05:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f1037faa740 10 client.0 _ll_register_callbacks cb 0x55ab56340d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56426800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56426800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48486/0 (socket says 172.21.15.69:48486) 2024-05-14T17:05:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 0x55ab56426d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102e7fc640 1 -- 172.21.15.69:0/2058116774 learned_addr learned my addr 172.21.15.69:0/2058116774 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102effd640 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab56429510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102e7fc640 1 -- 172.21.15.69:0/2058116774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 msgr2=0x55ab56429510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102e7fc640 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab56429510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102e7fc640 1 -- 172.21.15.69:0/2058116774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 msgr2=0x55ab56426d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102e7fc640 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 0x55ab56426d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102e7fc640 1 -- 172.21.15.69:0/2058116774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab56162620 con 0x55ab5641f400 2024-05-14T17:05:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.465+0000 7f102f7fe640 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 0x55ab56426d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f102e7fc640 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56426800 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1020010f50 tx=0x7f102000e3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac05aa059fa6b3bb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1012ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:05:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1012ff5640 1 -- 172.21.15.69:0/2058116774 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020025070 con 0x55ab5641f400 2024-05-14T17:05:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1037faa740 1 -- 172.21.15.69:0/2058116774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10240316a0 con 0x55ab5641f400 2024-05-14T17:05:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1012ff5640 1 -- 172.21.15.69:0/2058116774 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f102001f640 con 0x55ab5641f400 2024-05-14T17:05:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1012ff5640 1 -- 172.21.15.69:0/2058116774 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f102001f840 con 0x55ab5641f400 2024-05-14T17:05:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1037faa740 10 client.17022 fetch_fsmap learned FSMap version 59 2024-05-14T17:05:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1037faa740 1 -- 172.21.15.69:0/2058116774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ab564383c0 con 0x55ab5641f400 2024-05-14T17:05:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1012ff5640 1 -- 172.21.15.69:0/2058116774 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10240316a0 con 0x55ab5641f400 2024-05-14T17:05:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1012ff5640 1 -- 172.21.15.69:0/2058116774 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f1020029a40 con 0x55ab5641f400 2024-05-14T17:05:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1037faa740 10 client.17022 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:05:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1037faa740 10 client.17022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1037faa740 20 client.17022 populate_metadata read hostname 'smithi069' 2024-05-14T17:05:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1037faa740 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ab56439990 0x55ab5643bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1037faa740 4 client.17022 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.466+0000 7f1037faa740 1 -- 172.21.15.69:0/2058116774 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab56393b80 con 0x55ab56439990 2024-05-14T17:05:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.467+0000 7f102effd640 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ab56439990 0x55ab5643bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.467+0000 7f102effd640 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ab56439990 0x55ab5643bdd0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c694168131158c server_cookie=4fa6e7496e9d1ef in_seq=0 out_seq=0 2024-05-14T17:05:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.467+0000 7f1012ff5640 10 client.17022 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1012ff5640 1 -- 172.21.15.69:0/2058116774 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ab56393b80 con 0x55ab56439990 2024-05-14T17:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1012ff5640 10 client.17022 handle_command_reply: tid=0 2024-05-14T17:05:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 client.17022 shutdown 2024-05-14T17:05:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 -- 172.21.15.69:0/2058116774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 msgr2=0x55ab56426800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56426800 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1020010f50 tx=0x7f102000e3f0 comp rx=0 tx=0).stop 2024-05-14T17:05:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 -- 172.21.15.69:0/2058116774 shutdown_connections 2024-05-14T17:05:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ab56439990 0x55ab5643bdd0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab56429cc0 0x55ab56426d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641f400 0x55ab56426800 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 --2- 172.21.15.69:0/2058116774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab56429510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 -- 172.21.15.69:0/2058116774 >> 172.21.15.69:0/2058116774 conn(0x55ab5641aff0 msgr2=0x55ab5641a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1012ff5640 0 client.17022 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 -- 172.21.15.69:0/2058116774 shutdown_connections 2024-05-14T17:05:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 1 -- 172.21.15.69:0/2058116774 wait complete. 2024-05-14T17:05:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.468+0000 7f1037faa740 20 client.17022 trim_cache size 0 max 16384 2024-05-14T17:05:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f1037faa740 1 Processor -- start 2024-05-14T17:05:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f1037faa740 1 -- start start 2024-05-14T17:05:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f1037faa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641eae0 0x55ab56421fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f1037faa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 0x55ab56429040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f1037faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab56429cc0 0x55ab5641a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f1037faa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f102814a790 con 0x55ab5641eae0 2024-05-14T17:05:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f1037faa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1028148030 con 0x55ab5641f400 2024-05-14T17:05:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f1037faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1028145e80 con 0x55ab56429cc0 2024-05-14T17:05:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641eae0 0x55ab56421fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641eae0 0x55ab56421fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48498/0 (socket says 172.21.15.69:48498) 2024-05-14T17:05:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102effd640 1 -- 172.21.15.69:0/1335867420 learned_addr learned my addr 172.21.15.69:0/1335867420 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:05:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102f7fe640 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab56429cc0 0x55ab5641a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102e7fc640 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 0x55ab56429040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102effd640 1 -- 172.21.15.69:0/1335867420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab56429cc0 msgr2=0x55ab5641a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102effd640 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab56429cc0 0x55ab5641a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102effd640 1 -- 172.21.15.69:0/1335867420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 msgr2=0x55ab56429040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102effd640 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 0x55ab56429040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102effd640 1 -- 172.21.15.69:0/1335867420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab5641ae20 con 0x55ab5641eae0 2024-05-14T17:05:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102f7fe640 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab56429cc0 0x55ab5641a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102e7fc640 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 0x55ab56429040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.483+0000 7f102effd640 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641eae0 0x55ab56421fb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1024005230 tx=0x7f1024047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99fe76bfba8fb68 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1014ff9640 1 -- 172.21.15.69:0/1335867420 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024040040 con 0x55ab5641eae0 2024-05-14T17:05:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1014ff9640 1 -- 172.21.15.69:0/1335867420 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1024004040 con 0x55ab5641eae0 2024-05-14T17:05:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1014ff9640 1 -- 172.21.15.69:0/1335867420 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024035040 con 0x55ab5641eae0 2024-05-14T17:05:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1037faa740 1 -- 172.21.15.69:0/1335867420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641eae0 msgr2=0x55ab56421fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1037faa740 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641eae0 0x55ab56421fb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1024005230 tx=0x7f1024047040 comp rx=0 tx=0).stop 2024-05-14T17:05:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1037faa740 1 -- 172.21.15.69:0/1335867420 shutdown_connections 2024-05-14T17:05:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1037faa740 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab56429cc0 0x55ab5641a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1037faa740 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 0x55ab56429040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1037faa740 1 --2- 172.21.15.69:0/1335867420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab5641eae0 0x55ab56421fb0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1037faa740 1 -- 172.21.15.69:0/1335867420 >> 172.21.15.69:0/1335867420 conn(0x55ab5641aff0 msgr2=0x55ab56419ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1037faa740 1 -- 172.21.15.69:0/1335867420 shutdown_connections 2024-05-14T17:05:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.484+0000 7f1037faa740 1 -- 172.21.15.69:0/1335867420 wait complete. 2024-05-14T17:05:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1037faa740 1 -- start start 2024-05-14T17:05:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1037faa740 1 Processor -- start 2024-05-14T17:05:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1037faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab56431a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1037faa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 0x55ab56431f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1037faa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab56429cc0 0x55ab5642cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1037faa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f10281464f0 con 0x55ab56429cc0 2024-05-14T17:06:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1037faa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1028147960 con 0x55ab5641f400 2024-05-14T17:06:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1037faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1028146bc0 con 0x55ab5641eae0 2024-05-14T17:06:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1037faa740 10 client.0 _ll_register_callbacks cb 0x7f1020010c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab56429cc0 0x55ab5642cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab56429cc0 0x55ab5642cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48502/0 (socket says 172.21.15.69:48502) 2024-05-14T17:06:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102f7fe640 1 -- 172.21.15.69:0/1515519053 learned_addr learned my addr 172.21.15.69:0/1515519053 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102e7fc640 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 0x55ab56431f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102effd640 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab56431a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102f7fe640 1 -- 172.21.15.69:0/1515519053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 msgr2=0x55ab56431a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102f7fe640 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab56431a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102f7fe640 1 -- 172.21.15.69:0/1515519053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 msgr2=0x55ab56431f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102f7fe640 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 0x55ab56431f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102f7fe640 1 -- 172.21.15.69:0/1515519053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab5642d7e0 con 0x55ab56429cc0 2024-05-14T17:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102e7fc640 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 0x55ab56431f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f102f7fe640 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab56429cc0 0x55ab5642cfb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f103014ceb0 tx=0x7f1030068d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24c320867a44892 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1012ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:06:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.485+0000 7f1012ff5640 1 -- 172.21.15.69:0/1515519053 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030112170 con 0x55ab56429cc0 2024-05-14T17:06:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1012ff5640 1 -- 172.21.15.69:0/1515519053 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f1030139280 con 0x55ab56429cc0 2024-05-14T17:06:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1037faa740 1 -- 172.21.15.69:0/1515519053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1024004040 con 0x55ab56429cc0 2024-05-14T17:06:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1012ff5640 1 -- 172.21.15.69:0/1515519053 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f103011f180 con 0x55ab56429cc0 2024-05-14T17:06:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1037faa740 10 client.17028 fetch_fsmap learned FSMap version 59 2024-05-14T17:06:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1037faa740 1 -- 172.21.15.69:0/1515519053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ab564a3bf0 con 0x55ab56429cc0 2024-05-14T17:06:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1012ff5640 1 -- 172.21.15.69:0/1515519053 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1024004040 con 0x55ab56429cc0 2024-05-14T17:06:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1012ff5640 1 -- 172.21.15.69:0/1515519053 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f103008e030 con 0x55ab56429cc0 2024-05-14T17:06:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1037faa740 10 client.17028 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:06:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1037faa740 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1037faa740 20 client.17028 populate_metadata read hostname 'smithi069' 2024-05-14T17:06:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1037faa740 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ab564a4340 0x55ab564a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1037faa740 4 client.17028 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f1037faa740 1 -- 172.21.15.69:0/1515519053 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab5638d360 con 0x55ab564a4340 2024-05-14T17:06:01.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.486+0000 7f102effd640 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ab564a4340 0x55ab564a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.487+0000 7f1012ff5640 10 client.17028 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.487+0000 7f102effd640 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ab564a4340 0x55ab564a6760 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6cc1278e4745cbc server_cookie=53e434c506f96c98 in_seq=0 out_seq=0 2024-05-14T17:06:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1012ff5640 1 -- 172.21.15.69:0/1515519053 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ab5638d360 con 0x55ab564a4340 2024-05-14T17:06:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1012ff5640 10 client.17028 handle_command_reply: tid=0 2024-05-14T17:06:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 client.17028 shutdown 2024-05-14T17:06:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 -- 172.21.15.69:0/1515519053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab56429cc0 msgr2=0x55ab5642cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab56429cc0 0x55ab5642cfb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f103014ceb0 tx=0x7f1030068d20 comp rx=0 tx=0).stop 2024-05-14T17:06:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 -- 172.21.15.69:0/1515519053 shutdown_connections 2024-05-14T17:06:02.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55ab564a4340 0x55ab564a6760 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab56429cc0 0x55ab5642cfb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1012ff5640 0 client.17028 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ab5641f400 0x55ab56431f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 --2- 172.21.15.69:0/1515519053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab5641eae0 0x55ab56431a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 -- 172.21.15.69:0/1515519053 >> 172.21.15.69:0/1515519053 conn(0x55ab5641aff0 msgr2=0x55ab5641a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 -- 172.21.15.69:0/1515519053 shutdown_connections 2024-05-14T17:06:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 1 -- 172.21.15.69:0/1515519053 wait complete. 2024-05-14T17:06:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.513+0000 7f1037faa740 20 client.17028 trim_cache size 0 max 16384 2024-05-14T17:06:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:06:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:06:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:06:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 -- 172.21.15.69:0/4165254224 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10180b29e0 msgr2=0x7f10180b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10180b29e0 0x7f10180b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55ab5641c410 tx=0x7f10200046c0 comp rx=0 tx=0).stop 2024-05-14T17:06:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 -- 172.21.15.69:0/4165254224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10281328d0 msgr2=0x7f1028132d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10281328d0 0x7f1028132d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1030000b60 tx=0x7f103007c710 comp rx=0 tx=0).stop 2024-05-14T17:06:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 -- 172.21.15.69:0/4165254224 shutdown_connections 2024-05-14T17:06:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f10180b29e0 0x7f10180b4ea0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10281328d0 0x7f1028132d80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f102809b0b0 0x7f1028138c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 --2- 172.21.15.69:0/4165254224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102809ac40 0x7f1028138720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 -- 172.21.15.69:0/4165254224 >> 172.21.15.69:0/4165254224 conn(0x7f102800a750 msgr2=0x7f1028095c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 -- 172.21.15.69:0/4165254224 shutdown_connections 2024-05-14T17:06:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:36.514+0000 7f102ffff640 1 -- 172.21.15.69:0/4165254224 wait complete. 2024-05-14T17:06:03.500 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:38 smithi069 ceph-mon[31507]: pgmap v497: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 7.5 MiB/s wr, 585 op/s 2024-05-14T17:06:03.548 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:40 smithi069 ceph-mon[31507]: pgmap v498: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.1 MiB/s wr, 353 op/s 2024-05-14T17:06:03.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:42 smithi069 ceph-mon[31507]: pgmap v499: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-05-14T17:06:03.648 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:44 smithi069 ceph-mon[31507]: pgmap v500: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-05-14T17:06:03.698 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:46 smithi069 ceph-mon[31507]: pgmap v501: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 955 KiB/s wr, 126 op/s 2024-05-14T17:06:03.757 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:48 smithi069 ceph-mon[31507]: pgmap v502: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.4 MiB/s wr, 214 op/s 2024-05-14T17:06:03.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:51 smithi092 ceph-mon[35763]: pgmap v504: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.1 MiB/s wr, 164 op/s 2024-05-14T17:06:03.857 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:53 smithi092 ceph-mon[35763]: pgmap v505: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.3 MiB/s wr, 200 op/s 2024-05-14T17:06:03.907 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:55 smithi092 ceph-mon[35763]: pgmap v506: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-05-14T17:06:03.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:57 smithi092 ceph-mon[35763]: pgmap v507: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-14T17:06:04.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:05:59 smithi092 ceph-mon[35763]: pgmap v508: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 637 KiB/s wr, 83 op/s 2024-05-14T17:06:04.090 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:01 smithi092 ceph-mon[35763]: pgmap v509: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 556 KiB/s wr, 74 op/s 2024-05-14T17:06:04.140 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:51 smithi203 ceph-mon[35787]: pgmap v504: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.1 MiB/s wr, 164 op/s 2024-05-14T17:06:04.190 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:53 smithi203 ceph-mon[35787]: pgmap v505: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.3 MiB/s wr, 200 op/s 2024-05-14T17:06:04.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:55 smithi203 ceph-mon[35787]: pgmap v506: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-05-14T17:06:04.299 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:57 smithi203 ceph-mon[35787]: pgmap v507: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-14T17:06:04.373 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:05:59 smithi203 ceph-mon[35787]: pgmap v508: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 637 KiB/s wr, 83 op/s 2024-05-14T17:06:04.424 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:01 smithi203 ceph-mon[35787]: pgmap v509: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 556 KiB/s wr, 74 op/s 2024-05-14T17:06:04.475 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:03 smithi092 ceph-mon[35763]: pgmap v510: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 575 KiB/s wr, 90 op/s 2024-05-14T17:06:04.524 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:03 smithi203 ceph-mon[35787]: pgmap v510: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 575 KiB/s wr, 90 op/s 2024-05-14T17:06:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.605+0000 7ff2f32b0640 1 -- 172.21.15.69:0/799430058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2ec106400 msgr2=0x7ff2ec10dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.605+0000 7ff2f32b0640 1 --2- 172.21.15.69:0/799430058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2ec106400 0x7ff2ec10dfd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009ca0 tx=0x7ff2e402a8c0 comp rx=0 tx=0).stop 2024-05-14T17:06:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.605+0000 7ff2f32b0640 1 -- 172.21.15.69:0/799430058 shutdown_connections 2024-05-14T17:06:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.605+0000 7ff2f32b0640 1 --2- 172.21.15.69:0/799430058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2ec106400 0x7ff2ec10dfd0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.605+0000 7ff2f32b0640 1 --2- 172.21.15.69:0/799430058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2ec101ce0 0x7ff2ec105d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.605+0000 7ff2f32b0640 1 --2- 172.21.15.69:0/799430058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2ec101330 0x7ff2ec101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.605+0000 7ff2f32b0640 1 -- 172.21.15.69:0/799430058 >> 172.21.15.69:0/799430058 conn(0x7ff2ec077ee0 msgr2=0x7ff2ec0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.605+0000 7ff2f32b0640 1 -- 172.21.15.69:0/799430058 shutdown_connections 2024-05-14T17:06:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.605+0000 7ff2f32b0640 1 -- 172.21.15.69:0/799430058 wait complete. 2024-05-14T17:06:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f32b0640 1 Processor -- start 2024-05-14T17:06:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f32b0640 1 -- start start 2024-05-14T17:06:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f32b0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2ec101330 0x7ff2ec1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f32b0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2ec106400 0x7ff2ec1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f32b0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2ec19da60 0x7ff2ec19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f32b0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec111370 con 0x7ff2ec106400 2024-05-14T17:06:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f32b0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1111f0 con 0x7ff2ec19da60 2024-05-14T17:06:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f32b0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1114f0 con 0x7ff2ec101330 2024-05-14T17:06:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f084b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2ec106400 0x7ff2ec1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f084b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2ec106400 0x7ff2ec1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41184/0 (socket says 172.21.15.69:41184) 2024-05-14T17:06:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f084b640 1 -- 172.21.15.69:0/29747307 learned_addr learned my addr 172.21.15.69:0/29747307 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f104c640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2ec101330 0x7ff2ec1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.606+0000 7ff2f184d640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2ec19da60 0x7ff2ec19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2f084b640 1 -- 172.21.15.69:0/29747307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2ec101330 msgr2=0x7ff2ec1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2f084b640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2ec101330 0x7ff2ec1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2f084b640 1 -- 172.21.15.69:0/29747307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2ec19da60 msgr2=0x7ff2ec19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2f084b640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2ec19da60 0x7ff2ec19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2f084b640 1 -- 172.21.15.69:0/29747307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec19e550 con 0x7ff2ec106400 2024-05-14T17:06:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2f084b640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2ec106400 0x7ff2ec1996e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00d740 tx=0x7ff2dc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2e27fc640 1 -- 172.21.15.69:0/29747307 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc017070 con 0x7ff2ec106400 2024-05-14T17:06:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2f32b0640 1 -- 172.21.15.69:0/29747307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ec19e840 con 0x7ff2ec106400 2024-05-14T17:06:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2e27fc640 1 -- 172.21.15.69:0/29747307 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2dc00de50 con 0x7ff2ec106400 2024-05-14T17:06:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2e27fc640 1 -- 172.21.15.69:0/29747307 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc004ce0 con 0x7ff2ec106400 2024-05-14T17:06:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.607+0000 7ff2f32b0640 1 -- 172.21.15.69:0/29747307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ec1a5fe0 con 0x7ff2ec106400 2024-05-14T17:06:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.609+0000 7ff2f5fcc740 1 Processor -- start 2024-05-14T17:06:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.609+0000 7ff2f5fcc740 1 -- start start 2024-05-14T17:06:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.609+0000 7ff2e27fc640 1 -- 172.21.15.69:0/29747307 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff2dc004e80 con 0x7ff2ec106400 2024-05-14T17:06:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f14cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f15210 0x564307f1f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f1fad0 0x564307f21ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564307e87c30 con 0x564307f148f0 2024-05-14T17:06:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564307e777d0 con 0x564307f15210 2024-05-14T17:06:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564307e7b810 con 0x564307f1fad0 2024-05-14T17:06:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f184d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f14cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f184d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f14cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41186/0 (socket says 172.21.15.69:41186) 2024-05-14T17:06:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f184d640 1 -- 172.21.15.69:0/4265444807 learned_addr learned my addr 172.21.15.69:0/4265444807 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f104c640 1 --2- 172.21.15.69:0/4265444807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f15210 0x564307f1f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f084b640 1 --2- 172.21.15.69:0/4265444807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f1fad0 0x564307f21ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f184d640 1 -- 172.21.15.69:0/4265444807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f1fad0 msgr2=0x564307f21ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f184d640 1 --2- 172.21.15.69:0/4265444807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f1fad0 0x564307f21ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f184d640 1 -- 172.21.15.69:0/4265444807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f15210 msgr2=0x564307f1f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f184d640 1 --2- 172.21.15.69:0/4265444807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f15210 0x564307f1f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f184d640 1 -- 172.21.15.69:0/4265444807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564307b8c210 con 0x564307f148f0 2024-05-14T17:06:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f104c640 1 --2- 172.21.15.69:0/4265444807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f15210 0x564307f1f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2e27fc640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff2d80b2a10 0x7ff2d80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.610+0000 7ff2f184d640 1 --2- 172.21.15.69:0/4265444807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f14cd0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005cc0 tx=0x7ff2e40099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79891eb8cc5d5163 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f104c640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff2d80b2a10 0x7ff2d80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2e27fc640 1 -- 172.21.15.69:0/29747307 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7ff2dc0e2040 con 0x7ff2ec106400 2024-05-14T17:06:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2d77fe640 1 -- 172.21.15.69:0/4265444807 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4031450 con 0x564307f148f0 2024-05-14T17:06:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2d77fe640 1 -- 172.21.15.69:0/4265444807 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e4036040 con 0x564307f148f0 2024-05-14T17:06:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2d77fe640 1 -- 172.21.15.69:0/4265444807 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4004440 con 0x564307f148f0 2024-05-14T17:06:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/4265444807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 msgr2=0x564307f14cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f104c640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff2d80b2a10 0x7ff2d80b4ed0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005fd0 tx=0x7ff2e8005f40 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/4265444807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f14cd0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005cc0 tx=0x7ff2e40099c0 comp rx=0 tx=0).stop 2024-05-14T17:06:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/4265444807 shutdown_connections 2024-05-14T17:06:08.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/4265444807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f1fad0 0x564307f21ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/4265444807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f15210 0x564307f1f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/4265444807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f14cd0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/4265444807 >> 172.21.15.69:0/4265444807 conn(0x564307f10e00 msgr2=0x564307f111e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/4265444807 shutdown_connections 2024-05-14T17:06:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.611+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/4265444807 wait complete. 2024-05-14T17:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f5fcc740 1 -- start start 2024-05-14T17:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f5fcc740 1 Processor -- start 2024-05-14T17:06:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f1f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f15210 0x564307f1c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f1fad0 0x564307f1cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564307b5d790 con 0x564307f148f0 2024-05-14T17:06:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564307e33890 con 0x564307f1fad0 2024-05-14T17:06:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564307f22740 con 0x564307f15210 2024-05-14T17:06:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f5fcc740 10 client.0 _ll_register_callbacks cb 0x564307e36d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f104c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f1f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f104c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f1f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41196/0 (socket says 172.21.15.69:41196) 2024-05-14T17:06:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.612+0000 7ff2f104c640 1 -- 172.21.15.69:0/360701579 learned_addr learned my addr 172.21.15.69:0/360701579 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f084b640 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f15210 0x564307f1c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f184d640 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f1fad0 0x564307f1cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f104c640 1 -- 172.21.15.69:0/360701579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f15210 msgr2=0x564307f1c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f104c640 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f15210 0x564307f1c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f104c640 1 -- 172.21.15.69:0/360701579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f1fad0 msgr2=0x564307f1cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f104c640 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f1fad0 0x564307f1cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f104c640 1 -- 172.21.15.69:0/360701579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564307c58620 con 0x564307f148f0 2024-05-14T17:06:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f104c640 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f1f380 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8004360 tx=0x7ff2e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c969ed81ed2b8cab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:06:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/360701579 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e801f660 con 0x564307f148f0 2024-05-14T17:06:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/360701579 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7ff2e801f800 con 0x564307f148f0 2024-05-14T17:06:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/360701579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e4036040 con 0x564307f148f0 2024-05-14T17:06:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/360701579 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff2e802d050 con 0x564307f148f0 2024-05-14T17:06:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f5fcc740 10 client.17040 fetch_fsmap learned FSMap version 59 2024-05-14T17:06:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/360701579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564307f2e080 con 0x564307f148f0 2024-05-14T17:06:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.613+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/360701579 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e4036040 con 0x564307f148f0 2024-05-14T17:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.614+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/360701579 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7ff2e8011b40 con 0x564307f148f0 2024-05-14T17:06:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.614+0000 7ff2f5fcc740 10 client.17040 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:06:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.614+0000 7ff2f5fcc740 10 client.17040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.614+0000 7ff2f5fcc740 20 client.17040 populate_metadata read hostname 'smithi069' 2024-05-14T17:06:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.614+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564307f2f590 0x564307f31980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.614+0000 7ff2f5fcc740 4 client.17040 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.614+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/360701579 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564307e89b80 con 0x564307f2f590 2024-05-14T17:06:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.614+0000 7ff2f084b640 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564307f2f590 0x564307f31980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.614+0000 7ff2f084b640 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564307f2f590 0x564307f31980 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dede283e1af73c server_cookie=98af9d060023336 in_seq=0 out_seq=0 2024-05-14T17:06:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.614+0000 7ff2d4ff9640 10 client.17040 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/360701579 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564307e89b80 con 0x564307f2f590 2024-05-14T17:06:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2d4ff9640 10 client.17040 handle_command_reply: tid=0 2024-05-14T17:06:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2f5fcc740 1 client.17040 shutdown 2024-05-14T17:06:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/360701579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 msgr2=0x564307f1f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f1f380 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8004360 tx=0x7ff2e8004730 comp rx=0 tx=0).stop 2024-05-14T17:06:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/360701579 shutdown_connections 2024-05-14T17:06:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564307f2f590 0x564307f31980 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f1fad0 0x564307f1cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2d4ff9640 0 client.17040 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f15210 0x564307f1c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/360701579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f148f0 0x564307f1f380 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/360701579 >> 172.21.15.69:0/360701579 conn(0x564307f10e00 msgr2=0x564307f10100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.615+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/360701579 shutdown_connections 2024-05-14T17:06:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.616+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/360701579 wait complete. 2024-05-14T17:06:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.616+0000 7ff2f5fcc740 20 client.17040 trim_cache size 0 max 16384 2024-05-14T17:06:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f5fcc740 1 Processor -- start 2024-05-14T17:06:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f5fcc740 1 -- start start 2024-05-14T17:06:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 0x564307f2b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f15210 0x564307f1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f1fad0 0x564307f37270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1b0e50 con 0x564307f15210 2024-05-14T17:06:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1ae6f0 con 0x564307f148f0 2024-05-14T17:06:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1ac560 con 0x564307f1fad0 2024-05-14T17:06:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f084b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f15210 0x564307f1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f084b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f15210 0x564307f1eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41200/0 (socket says 172.21.15.69:41200) 2024-05-14T17:06:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f084b640 1 -- 172.21.15.69:0/82733153 learned_addr learned my addr 172.21.15.69:0/82733153 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f084b640 1 -- 172.21.15.69:0/82733153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f1fad0 msgr2=0x564307f37270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f084b640 1 --2- 172.21.15.69:0/82733153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f1fad0 0x564307f37270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f104c640 1 --2- 172.21.15.69:0/82733153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 0x564307f2b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f084b640 1 -- 172.21.15.69:0/82733153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 msgr2=0x564307f2b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f084b640 1 --2- 172.21.15.69:0/82733153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 0x564307f2b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f084b640 1 -- 172.21.15.69:0/82733153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564307f10c30 con 0x564307f15210 2024-05-14T17:06:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.631+0000 7ff2f084b640 1 --2- 172.21.15.69:0/82733153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f15210 0x564307f1eeb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0a53b0 tx=0x7ff2dc0a5290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43881db5ac8dc2d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2d6ffd640 1 -- 172.21.15.69:0/82733153 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc020040 con 0x564307f15210 2024-05-14T17:06:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2d6ffd640 1 -- 172.21.15.69:0/82733153 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2dc0a9a30 con 0x564307f15210 2024-05-14T17:06:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2d6ffd640 1 -- 172.21.15.69:0/82733153 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0a55c0 con 0x564307f15210 2024-05-14T17:06:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/82733153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f15210 msgr2=0x564307f1eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/82733153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f15210 0x564307f1eeb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0a53b0 tx=0x7ff2dc0a5290 comp rx=0 tx=0).stop 2024-05-14T17:06:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/82733153 shutdown_connections 2024-05-14T17:06:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/82733153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f1fad0 0x564307f37270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/82733153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f15210 0x564307f1eeb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/82733153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 0x564307f2b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/82733153 >> 172.21.15.69:0/82733153 conn(0x564307f10e00 msgr2=0x564307f16b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/82733153 shutdown_connections 2024-05-14T17:06:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/82733153 wait complete. 2024-05-14T17:06:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 -- start start 2024-05-14T17:06:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.632+0000 7ff2f5fcc740 1 Processor -- start 2024-05-14T17:06:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 0x564307f280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f15210 0x564307f193e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f5fcc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f1fad0 0x564307f19b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1acbb0 con 0x564307f1fad0 2024-05-14T17:06:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1ae020 con 0x564307f148f0 2024-05-14T17:06:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f5fcc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1ad280 con 0x564307f15210 2024-05-14T17:06:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f5fcc740 10 client.0 _ll_register_callbacks cb 0x7ff2e8004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f184d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f1fad0 0x564307f19b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f184d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f1fad0 0x564307f19b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41216/0 (socket says 172.21.15.69:41216) 2024-05-14T17:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f184d640 1 -- 172.21.15.69:0/2470253604 learned_addr learned my addr 172.21.15.69:0/2470253604 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f104c640 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 0x564307f280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f084b640 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f15210 0x564307f193e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f184d640 1 -- 172.21.15.69:0/2470253604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f15210 msgr2=0x564307f193e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f184d640 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f15210 0x564307f193e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f184d640 1 -- 172.21.15.69:0/2470253604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 msgr2=0x564307f280b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f184d640 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 0x564307f280b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f184d640 1 -- 172.21.15.69:0/2470253604 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564307f98e20 con 0x564307f1fad0 2024-05-14T17:06:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f104c640 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 0x564307f280b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2f184d640 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f1fad0 0x564307f19b20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005cc0 tx=0x7ff2e4009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55c2ef818b142e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.633+0000 7ff2d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:06:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/2470253604 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40317a0 con 0x564307f1fad0 2024-05-14T17:06:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/2470253604 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7ff2e4003970 con 0x564307f1fad0 2024-05-14T17:06:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/2470253604 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2dc0a9a30 con 0x564307f1fad0 2024-05-14T17:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/2470253604 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff2e405a050 con 0x564307f1fad0 2024-05-14T17:06:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2f5fcc740 10 client.17046 fetch_fsmap learned FSMap version 59 2024-05-14T17:06:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/2470253604 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564307f9a030 con 0x564307f1fad0 2024-05-14T17:06:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/2470253604 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2dc0a9a30 con 0x564307f1fad0 2024-05-14T17:06:14.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/2470253604 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7ff2e4055030 con 0x564307f1fad0 2024-05-14T17:06:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2f5fcc740 10 client.17046 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:06:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2f5fcc740 10 client.17046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2f5fcc740 20 client.17046 populate_metadata read hostname 'smithi069' 2024-05-14T17:06:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564307f2f590 0x564307f9c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2f5fcc740 4 client.17046 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/2470253604 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564307e83360 con 0x564307f2f590 2024-05-14T17:06:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.634+0000 7ff2f104c640 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564307f2f590 0x564307f9c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.635+0000 7ff2d4ff9640 10 client.17046 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.635+0000 7ff2f104c640 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564307f2f590 0x564307f9c3d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bebba107a5a67526 server_cookie=b2431b145e441578 in_seq=0 out_seq=0 2024-05-14T17:06:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2d4ff9640 1 -- 172.21.15.69:0/2470253604 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564307e83360 con 0x564307f2f590 2024-05-14T17:06:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2d4ff9640 10 client.17046 handle_command_reply: tid=0 2024-05-14T17:06:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 client.17046 shutdown 2024-05-14T17:06:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/2470253604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f1fad0 msgr2=0x564307f19b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f1fad0 0x564307f19b20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005cc0 tx=0x7ff2e4009540 comp rx=0 tx=0).stop 2024-05-14T17:06:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/2470253604 shutdown_connections 2024-05-14T17:06:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x564307f2f590 0x564307f9c3d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564307f1fad0 0x564307f19b20 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564307f15210 0x564307f193e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 --2- 172.21.15.69:0/2470253604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564307f148f0 0x564307f280b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2d4ff9640 0 client.17046 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/2470253604 >> 172.21.15.69:0/2470253604 conn(0x564307f10e00 msgr2=0x564307f162a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/2470253604 shutdown_connections 2024-05-14T17:06:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 1 -- 172.21.15.69:0/2470253604 wait complete. 2024-05-14T17:06:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.649+0000 7ff2f5fcc740 20 client.17046 trim_cache size 0 max 16384 2024-05-14T17:06:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:06:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:06:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:06:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 -- 172.21.15.69:0/29747307 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff2d80b2a10 msgr2=0x7ff2d80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff2d80b2a10 0x7ff2d80b4ed0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005fd0 tx=0x7ff2e8005f40 comp rx=0 tx=0).stop 2024-05-14T17:06:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 -- 172.21.15.69:0/29747307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2ec106400 msgr2=0x7ff2ec1996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2ec106400 0x7ff2ec1996e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00d740 tx=0x7ff2dc00dc00 comp rx=0 tx=0).stop 2024-05-14T17:06:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 -- 172.21.15.69:0/29747307 shutdown_connections 2024-05-14T17:06:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7ff2d80b2a10 0x7ff2d80b4ed0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2ec19da60 0x7ff2ec19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2ec106400 0x7ff2ec1996e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 --2- 172.21.15.69:0/29747307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2ec101330 0x7ff2ec1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 -- 172.21.15.69:0/29747307 >> 172.21.15.69:0/29747307 conn(0x7ff2ec077ee0 msgr2=0x7ff2ec107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 -- 172.21.15.69:0/29747307 shutdown_connections 2024-05-14T17:06:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:05:49.650+0000 7ff2f32b0640 1 -- 172.21.15.69:0/29747307 wait complete. 2024-05-14T17:06:16.729 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:49 smithi069 ceph-mon[31507]: pgmap v503: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-14T17:06:16.776 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:51 smithi069 ceph-mon[31507]: pgmap v504: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.1 MiB/s wr, 164 op/s 2024-05-14T17:06:16.826 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:53 smithi069 ceph-mon[31507]: pgmap v505: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.3 MiB/s wr, 200 op/s 2024-05-14T17:06:16.876 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:55 smithi069 ceph-mon[31507]: pgmap v506: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-05-14T17:06:16.926 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:57 smithi069 ceph-mon[31507]: pgmap v507: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-14T17:06:16.976 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:05:59 smithi069 ceph-mon[31507]: pgmap v508: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 637 KiB/s wr, 83 op/s 2024-05-14T17:06:17.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:01 smithi069 ceph-mon[31507]: pgmap v509: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 556 KiB/s wr, 74 op/s 2024-05-14T17:06:17.077 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:04 smithi203 ceph-mon[35787]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T17:06:17.126 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:05 smithi203 ceph-mon[35787]: pgmap v512: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 310 KiB/s wr, 57 op/s 2024-05-14T17:06:17.201 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:07 smithi203 ceph-mon[35787]: pgmap v513: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 137 KiB/s wr, 40 op/s 2024-05-14T17:06:17.251 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:09 smithi203 ceph-mon[35787]: pgmap v514: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 107 KiB/s wr, 37 op/s 2024-05-14T17:06:17.301 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:11 smithi203 ceph-mon[35787]: pgmap v515: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 107 KiB/s wr, 37 op/s 2024-05-14T17:06:17.354 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:13 smithi203 ceph-mon[35787]: pgmap v516: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-05-14T17:06:17.401 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:13 smithi203 ceph-mon[35787]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T17:06:17.451 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:16 smithi203 ceph-mon[35787]: pgmap v518: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-05-14T17:06:17.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:04 smithi092 ceph-mon[35763]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T17:06:17.551 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:05 smithi092 ceph-mon[35763]: pgmap v512: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 310 KiB/s wr, 57 op/s 2024-05-14T17:06:17.601 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:07 smithi092 ceph-mon[35763]: pgmap v513: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 137 KiB/s wr, 40 op/s 2024-05-14T17:06:17.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:09 smithi092 ceph-mon[35763]: pgmap v514: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 107 KiB/s wr, 37 op/s 2024-05-14T17:06:17.701 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:11 smithi092 ceph-mon[35763]: pgmap v515: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 107 KiB/s wr, 37 op/s 2024-05-14T17:06:17.751 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:13 smithi092 ceph-mon[35763]: pgmap v516: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-05-14T17:06:17.801 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:13 smithi092 ceph-mon[35763]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T17:06:17.851 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:16 smithi092 ceph-mon[35763]: pgmap v518: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-05-14T17:06:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.747+0000 7f4b35346640 1 -- 172.21.15.69:0/141568066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b30107940 msgr2=0x7f4b30107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.747+0000 7f4b35346640 1 --2- 172.21.15.69:0/141568066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b30107940 0x7f4b30107d20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009930 tx=0x7f4b2002a420 comp rx=0 tx=0).stop 2024-05-14T17:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.747+0000 7f4b35346640 1 -- 172.21.15.69:0/141568066 shutdown_connections 2024-05-14T17:06:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.747+0000 7f4b35346640 1 --2- 172.21.15.69:0/141568066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b30102340 0x7f4b3010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.747+0000 7f4b35346640 1 --2- 172.21.15.69:0/141568066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b30101980 0x7f4b30101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.747+0000 7f4b35346640 1 --2- 172.21.15.69:0/141568066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b30107940 0x7f4b30107d20 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.747+0000 7f4b35346640 1 -- 172.21.15.69:0/141568066 >> 172.21.15.69:0/141568066 conn(0x7f4b30078ad0 msgr2=0x7f4b30078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.747+0000 7f4b35346640 1 -- 172.21.15.69:0/141568066 shutdown_connections 2024-05-14T17:06:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.747+0000 7f4b35346640 1 -- 172.21.15.69:0/141568066 wait complete. 2024-05-14T17:06:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b35346640 1 Processor -- start 2024-05-14T17:06:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b35346640 1 -- start start 2024-05-14T17:06:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b35346640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b30101980 0x7f4b301991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b35346640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b30102340 0x7f4b30199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b35346640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b30107940 0x7f4b3019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b35346640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30111bd0 con 0x7f4b30107940 2024-05-14T17:06:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b35346640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30111a50 con 0x7f4b30102340 2024-05-14T17:06:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b35346640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30111d50 con 0x7f4b30101980 2024-05-14T17:06:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b30107940 0x7f4b3019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b30107940 0x7f4b3019daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57486/0 (socket says 172.21.15.69:57486) 2024-05-14T17:06:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b2f7fe640 1 -- 172.21.15.69:0/691510851 learned_addr learned my addr 172.21.15.69:0/691510851 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b2effd640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b30101980 0x7f4b301991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.748+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b30102340 0x7f4b30199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b2f7fe640 1 -- 172.21.15.69:0/691510851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b30101980 msgr2=0x7f4b301991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b2f7fe640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b30101980 0x7f4b301991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b2f7fe640 1 -- 172.21.15.69:0/691510851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b30102340 msgr2=0x7f4b30199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b2f7fe640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b30102340 0x7f4b30199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b2f7fe640 1 -- 172.21.15.69:0/691510851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3019e1d0 con 0x7f4b30107940 2024-05-14T17:06:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b2f7fe640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b30107940 0x7f4b3019daa0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800eb10 tx=0x7f4b2800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b27fff640 1 -- 172.21.15.69:0/691510851 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28004b20 con 0x7f4b30107940 2024-05-14T17:06:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b27fff640 1 -- 172.21.15.69:0/691510851 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b28004cc0 con 0x7f4b30107940 2024-05-14T17:06:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b35346640 1 -- 172.21.15.69:0/691510851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3019e4c0 con 0x7f4b30107940 2024-05-14T17:06:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b27fff640 1 -- 172.21.15.69:0/691510851 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28010670 con 0x7f4b30107940 2024-05-14T17:06:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.749+0000 7f4b35346640 1 -- 172.21.15.69:0/691510851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b301a5d50 con 0x7f4b30107940 2024-05-14T17:06:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b38069740 1 Processor -- start 2024-05-14T17:06:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b27fff640 1 -- 172.21.15.69:0/691510851 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4b28005020 con 0x7f4b30107940 2024-05-14T17:06:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b38069740 1 -- start start 2024-05-14T17:06:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b38069740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d6534cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b38069740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d6535210 0x5630d653f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b38069740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 0x5630d6541ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b38069740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5630d64a7c30 con 0x5630d65348f0 2024-05-14T17:06:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b38069740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5630d64977d0 con 0x5630d6535210 2024-05-14T17:06:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b38069740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5630d649b810 con 0x5630d653fad0 2024-05-14T17:06:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b2e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d6534cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b2e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d6534cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57502/0 (socket says 172.21.15.69:57502) 2024-05-14T17:06:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/793115092 learned_addr learned my addr 172.21.15.69:0/793115092 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.752+0000 7f4b2effd640 1 --2- 172.21.15.69:0/793115092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d6535210 0x5630d653f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/793115092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 msgr2=0x5630d6541ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/793115092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 0x5630d6541ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/793115092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d6535210 msgr2=0x5630d653f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/793115092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d6535210 0x5630d653f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/793115092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d61ac210 con 0x5630d65348f0 2024-05-14T17:06:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b2effd640 1 --2- 172.21.15.69:0/793115092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d6535210 0x5630d653f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b27fff640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4b1c0b2a10 0x7f4b1c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/793115092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d6534cd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4b18005bc0 tx=0x7f4b18011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552a99f26b18f58e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b2effd640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4b1c0b2a10 0x7f4b1c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b257fa640 1 -- 172.21.15.69:0/793115092 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1801e070 con 0x5630d65348f0 2024-05-14T17:06:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b27fff640 1 -- 172.21.15.69:0/691510851 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f4b280e2b10 con 0x7f4b30107940 2024-05-14T17:06:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b257fa640 1 -- 172.21.15.69:0/793115092 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b18009e20 con 0x5630d65348f0 2024-05-14T17:06:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.753+0000 7f4b257fa640 1 -- 172.21.15.69:0/793115092 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800f740 con 0x5630d65348f0 2024-05-14T17:06:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 -- 172.21.15.69:0/793115092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 msgr2=0x5630d6534cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 --2- 172.21.15.69:0/793115092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d6534cd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4b18005bc0 tx=0x7f4b18011810 comp rx=0 tx=0).stop 2024-05-14T17:06:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b2effd640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4b1c0b2a10 0x7f4b1c0b4ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009900 tx=0x7f4b20000bc0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 -- 172.21.15.69:0/793115092 shutdown_connections 2024-05-14T17:06:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 --2- 172.21.15.69:0/793115092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 0x5630d6541ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 --2- 172.21.15.69:0/793115092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d6535210 0x5630d653f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 --2- 172.21.15.69:0/793115092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d6534cd0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 -- 172.21.15.69:0/793115092 >> 172.21.15.69:0/793115092 conn(0x5630d6530e00 msgr2=0x5630d65311e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 -- 172.21.15.69:0/793115092 shutdown_connections 2024-05-14T17:06:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 -- 172.21.15.69:0/793115092 wait complete. 2024-05-14T17:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 -- start start 2024-05-14T17:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.754+0000 7f4b38069740 1 Processor -- start 2024-05-14T17:06:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b38069740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d653f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b38069740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d6535210 0x5630d653c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b38069740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d653fad0 0x5630d653cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b38069740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5630d617d790 con 0x5630d65348f0 2024-05-14T17:06:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b38069740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5630d6453890 con 0x5630d653fad0 2024-05-14T17:06:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b38069740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5630d65427a0 con 0x5630d6535210 2024-05-14T17:06:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b38069740 10 client.0 _ll_register_callbacks cb 0x5630d6456d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d653f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d653f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57504/0 (socket says 172.21.15.69:57504) 2024-05-14T17:06:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2effd640 1 -- 172.21.15.69:0/2399153227 learned_addr learned my addr 172.21.15.69:0/2399153227 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2f7fe640 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d653fad0 0x5630d653cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d6535210 0x5630d653c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2effd640 1 -- 172.21.15.69:0/2399153227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d6535210 msgr2=0x5630d653c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2effd640 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d6535210 0x5630d653c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2effd640 1 -- 172.21.15.69:0/2399153227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d653fad0 msgr2=0x5630d653cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2effd640 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d653fad0 0x5630d653cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2effd640 1 -- 172.21.15.69:0/2399153227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d6278620 con 0x5630d65348f0 2024-05-14T17:06:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2f7fe640 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d653fad0 0x5630d653cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b2effd640 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d653f450 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009930 tx=0x7f4b20009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69038c642bdf74de server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b167fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:06:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b167fc640 1 -- 172.21.15.69:0/2399153227 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2004c030 con 0x5630d65348f0 2024-05-14T17:06:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b167fc640 1 -- 172.21.15.69:0/2399153227 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f4b200035f0 con 0x5630d65348f0 2024-05-14T17:06:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.755+0000 7f4b38069740 1 -- 172.21.15.69:0/2399153227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b18009e20 con 0x5630d65348f0 2024-05-14T17:06:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b167fc640 1 -- 172.21.15.69:0/2399153227 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f4b20035040 con 0x5630d65348f0 2024-05-14T17:06:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b38069740 10 client.17058 fetch_fsmap learned FSMap version 59 2024-05-14T17:06:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b38069740 1 -- 172.21.15.69:0/2399153227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5630d654e230 con 0x5630d65348f0 2024-05-14T17:06:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b167fc640 1 -- 172.21.15.69:0/2399153227 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b18009e20 con 0x5630d65348f0 2024-05-14T17:06:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b167fc640 1 -- 172.21.15.69:0/2399153227 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f4b2005aa00 con 0x5630d65348f0 2024-05-14T17:06:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b38069740 10 client.17058 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:06:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b38069740 10 client.17058 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b38069740 20 client.17058 populate_metadata read hostname 'smithi069' 2024-05-14T17:06:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b38069740 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5630d654f760 0x5630d6551b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b38069740 4 client.17058 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b38069740 1 -- 172.21.15.69:0/2399153227 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630d64a9b80 con 0x5630d654f760 2024-05-14T17:06:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.756+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5630d654f760 0x5630d6551b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b167fc640 10 client.17058 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5630d654f760 0x5630d6551b50 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eae6f9f818c39e9 server_cookie=64c0ec0bd890d41 in_seq=0 out_seq=0 2024-05-14T17:06:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b167fc640 1 -- 172.21.15.69:0/2399153227 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630d64a9b80 con 0x5630d654f760 2024-05-14T17:06:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b167fc640 10 client.17058 handle_command_reply: tid=0 2024-05-14T17:06:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b38069740 1 client.17058 shutdown 2024-05-14T17:06:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b38069740 1 -- 172.21.15.69:0/2399153227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 msgr2=0x5630d653f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b38069740 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d653f450 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009930 tx=0x7f4b20009260 comp rx=0 tx=0).stop 2024-05-14T17:06:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b38069740 1 -- 172.21.15.69:0/2399153227 shutdown_connections 2024-05-14T17:06:24.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b38069740 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5630d654f760 0x5630d6551b50 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b38069740 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d653fad0 0x5630d653cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b167fc640 0 client.17058 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b38069740 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d6535210 0x5630d653c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b38069740 1 --2- 172.21.15.69:0/2399153227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d65348f0 0x5630d653f450 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.757+0000 7f4b38069740 1 -- 172.21.15.69:0/2399153227 >> 172.21.15.69:0/2399153227 conn(0x5630d6530e00 msgr2=0x5630d6530100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.758+0000 7f4b38069740 1 -- 172.21.15.69:0/2399153227 shutdown_connections 2024-05-14T17:06:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.758+0000 7f4b38069740 1 -- 172.21.15.69:0/2399153227 wait complete. 2024-05-14T17:06:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.758+0000 7f4b38069740 20 client.17058 trim_cache size 0 max 16384 2024-05-14T17:06:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b38069740 1 Processor -- start 2024-05-14T17:06:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b38069740 1 -- start start 2024-05-14T17:06:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b38069740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 0x5630d654b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b38069740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d653eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b38069740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 0x5630d65573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b38069740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b0fe0 con 0x5630d6535210 2024-05-14T17:06:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b38069740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ae880 con 0x5630d65348f0 2024-05-14T17:06:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b38069740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ac6d0 con 0x5630d653fad0 2024-05-14T17:06:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b2e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d653eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b2e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d653eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57518/0 (socket says 172.21.15.69:57518) 2024-05-14T17:06:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/1655559885 learned_addr learned my addr 172.21.15.69:0/1655559885 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b2effd640 1 --2- 172.21.15.69:0/1655559885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 0x5630d654b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:25.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/1655559885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 msgr2=0x5630d65573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/1655559885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 0x5630d65573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/1655559885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 msgr2=0x5630d654b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/1655559885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 0x5630d654b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/1655559885 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d653d150 con 0x5630d6535210 2024-05-14T17:06:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.773+0000 7f4b2effd640 1 --2- 172.21.15.69:0/1655559885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 0x5630d654b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.774+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/1655559885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d653eff0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4b180058c0 tx=0x7f4b18005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5a1c3dce814940 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.774+0000 7f4b24ff9640 1 -- 172.21.15.69:0/1655559885 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1802d070 con 0x5630d6535210 2024-05-14T17:06:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.774+0000 7f4b24ff9640 1 -- 172.21.15.69:0/1655559885 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b18009e20 con 0x5630d6535210 2024-05-14T17:06:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.774+0000 7f4b24ff9640 1 -- 172.21.15.69:0/1655559885 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1801e040 con 0x5630d6535210 2024-05-14T17:06:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.774+0000 7f4b38069740 1 -- 172.21.15.69:0/1655559885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 msgr2=0x5630d653eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.774+0000 7f4b38069740 1 --2- 172.21.15.69:0/1655559885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d653eff0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4b180058c0 tx=0x7f4b18005f90 comp rx=0 tx=0).stop 2024-05-14T17:06:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 -- 172.21.15.69:0/1655559885 shutdown_connections 2024-05-14T17:06:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 --2- 172.21.15.69:0/1655559885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 0x5630d65573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 --2- 172.21.15.69:0/1655559885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d653eff0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 --2- 172.21.15.69:0/1655559885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 0x5630d654b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 -- 172.21.15.69:0/1655559885 >> 172.21.15.69:0/1655559885 conn(0x5630d6530e00 msgr2=0x5630d6536b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 -- 172.21.15.69:0/1655559885 shutdown_connections 2024-05-14T17:06:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 -- 172.21.15.69:0/1655559885 wait complete. 2024-05-14T17:06:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 -- start start 2024-05-14T17:06:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 Processor -- start 2024-05-14T17:06:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 0x5630d6543500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d6543a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:26.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 0x5630d6543f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301acd40 con 0x5630d6535210 2024-05-14T17:06:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ae1b0 con 0x5630d65348f0 2024-05-14T17:06:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ad410 con 0x5630d653fad0 2024-05-14T17:06:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b38069740 10 client.0 _ll_register_callbacks cb 0x7f4b20031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b2e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d6543a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b2e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d6543a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57532/0 (socket says 172.21.15.69:57532) 2024-05-14T17:06:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/2258682914 learned_addr learned my addr 172.21.15.69:0/2258682914 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b2effd640 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 0x5630d6543500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b2f7fe640 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 0x5630d6543f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/2258682914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 msgr2=0x5630d6543f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 0x5630d6543f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/2258682914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 msgr2=0x5630d6543500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.775+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 0x5630d6543500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b2e7fc640 1 -- 172.21.15.69:0/2258682914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d65b8fa0 con 0x5630d6535210 2024-05-14T17:06:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b2e7fc640 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d6543a40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4b18013010 tx=0x7f4b18005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02d3a3a71e23ad9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b167fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:06:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b167fc640 1 -- 172.21.15.69:0/2258682914 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1801e070 con 0x5630d6535210 2024-05-14T17:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b167fc640 1 -- 172.21.15.69:0/2258682914 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8498+0+0 (secure 0 0 0) 0x7f4b18016040 con 0x5630d6535210 2024-05-14T17:06:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b38069740 1 -- 172.21.15.69:0/2258682914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b18009e20 con 0x5630d6535210 2024-05-14T17:06:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b167fc640 1 -- 172.21.15.69:0/2258682914 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f4b1803e050 con 0x5630d6535210 2024-05-14T17:06:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b38069740 10 client.17064 fetch_fsmap learned FSMap version 59 2024-05-14T17:06:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b38069740 1 -- 172.21.15.69:0/2258682914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5630d65ba160 con 0x5630d6535210 2024-05-14T17:06:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b167fc640 1 -- 172.21.15.69:0/2258682914 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b18009e20 con 0x5630d6535210 2024-05-14T17:06:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b167fc640 1 -- 172.21.15.69:0/2258682914 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f4b18029270 con 0x5630d6535210 2024-05-14T17:06:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b38069740 10 client.17064 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:06:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b38069740 10 client.17064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b38069740 20 client.17064 populate_metadata read hostname 'smithi069' 2024-05-14T17:06:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b38069740 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5630d65ba600 0x5630d65bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b38069740 4 client.17064 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.776+0000 7f4b38069740 1 -- 172.21.15.69:0/2258682914 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630d64a3360 con 0x5630d65ba600 2024-05-14T17:06:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.777+0000 7f4b2effd640 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5630d65ba600 0x5630d65bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.777+0000 7f4b2effd640 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5630d65ba600 0x5630d65bca20 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d94b16aa3c7771d server_cookie=611b41e4849aa567 in_seq=0 out_seq=0 2024-05-14T17:06:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.777+0000 7f4b167fc640 10 client.17064 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b167fc640 1 -- 172.21.15.69:0/2258682914 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5630d64a3360 con 0x5630d65ba600 2024-05-14T17:06:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b167fc640 10 client.17064 handle_command_reply: tid=0 2024-05-14T17:06:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 client.17064 shutdown 2024-05-14T17:06:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 -- 172.21.15.69:0/2258682914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 msgr2=0x5630d6543a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d6543a40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4b18013010 tx=0x7f4b18005600 comp rx=0 tx=0).stop 2024-05-14T17:06:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 -- 172.21.15.69:0/2258682914 shutdown_connections 2024-05-14T17:06:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5630d65ba600 0x5630d65bca20 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5630d653fad0 0x5630d6543f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630d6535210 0x5630d6543a40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b167fc640 0 client.17064 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 --2- 172.21.15.69:0/2258682914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630d65348f0 0x5630d6543500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 -- 172.21.15.69:0/2258682914 >> 172.21.15.69:0/2258682914 conn(0x5630d6530e00 msgr2=0x5630d65362a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 -- 172.21.15.69:0/2258682914 shutdown_connections 2024-05-14T17:06:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 1 -- 172.21.15.69:0/2258682914 wait complete. 2024-05-14T17:06:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.793+0000 7f4b38069740 20 client.17064 trim_cache size 0 max 16384 2024-05-14T17:06:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:06:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:06:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:06:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 -- 172.21.15.69:0/691510851 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4b1c0b2a10 msgr2=0x7f4b1c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4b1c0b2a10 0x7f4b1c0b4ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009900 tx=0x7f4b20000bc0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 -- 172.21.15.69:0/691510851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b30107940 msgr2=0x7f4b3019daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b30107940 0x7f4b3019daa0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800eb10 tx=0x7f4b2800c2d0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 -- 172.21.15.69:0/691510851 shutdown_connections 2024-05-14T17:06:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4b1c0b2a10 0x7f4b1c0b4ed0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b30107940 0x7f4b3019daa0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b30102340 0x7f4b30199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 --2- 172.21.15.69:0/691510851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b30101980 0x7f4b301991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 -- 172.21.15.69:0/691510851 >> 172.21.15.69:0/691510851 conn(0x7f4b30078ad0 msgr2=0x7f4b3006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 -- 172.21.15.69:0/691510851 shutdown_connections 2024-05-14T17:06:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:02.794+0000 7f4b35346640 1 -- 172.21.15.69:0/691510851 wait complete. 2024-05-14T17:06:29.860 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:18 smithi092 ceph-mon[35763]: pgmap v519: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-05-14T17:06:29.909 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:20 smithi092 ceph-mon[35763]: pgmap v520: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 787 op/s 2024-05-14T17:06:29.959 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:22 smithi092 ceph-mon[35763]: pgmap v521: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 844 op/s 2024-05-14T17:06:30.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:24 smithi092 ceph-mon[35763]: pgmap v522: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-14T17:06:30.092 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:24 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:06:30.142 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:25 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:06:30.192 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:25 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:06:30.242 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:25 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:06:30.293 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:26 smithi092 ceph-mon[35763]: pgmap v523: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-05-14T17:06:30.343 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:28 smithi092 ceph-mon[35763]: pgmap v524: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 11 MiB/s wr, 1.20k op/s 2024-05-14T17:06:30.393 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:18 smithi203 ceph-mon[35787]: pgmap v519: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-05-14T17:06:30.443 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:20 smithi203 ceph-mon[35787]: pgmap v520: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 787 op/s 2024-05-14T17:06:30.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:22 smithi203 ceph-mon[35787]: pgmap v521: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 844 op/s 2024-05-14T17:06:30.543 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:24 smithi203 ceph-mon[35787]: pgmap v522: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-14T17:06:30.593 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:24 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:06:30.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:25 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:06:30.693 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:25 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:06:30.743 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:25 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:06:30.793 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:26 smithi203 ceph-mon[35787]: pgmap v523: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-05-14T17:06:30.893 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:28 smithi203 ceph-mon[35787]: pgmap v524: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 11 MiB/s wr, 1.20k op/s 2024-05-14T17:06:30.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:03 smithi069 ceph-mon[31507]: pgmap v510: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 575 KiB/s wr, 90 op/s 2024-05-14T17:06:30.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:04 smithi069 ceph-mon[31507]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T17:06:31.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:05 smithi069 ceph-mon[31507]: pgmap v512: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 310 KiB/s wr, 57 op/s 2024-05-14T17:06:31.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:07 smithi069 ceph-mon[31507]: pgmap v513: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 137 KiB/s wr, 40 op/s 2024-05-14T17:06:31.143 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:09 smithi069 ceph-mon[31507]: pgmap v514: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 107 KiB/s wr, 37 op/s 2024-05-14T17:06:31.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:11 smithi069 ceph-mon[31507]: pgmap v515: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 107 KiB/s wr, 37 op/s 2024-05-14T17:06:31.243 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:13 smithi069 ceph-mon[31507]: pgmap v516: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-05-14T17:06:31.293 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:13 smithi069 ceph-mon[31507]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T17:06:31.343 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:30 smithi092 ceph-mon[35763]: pgmap v525: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.1 MiB/s wr, 668 op/s 2024-05-14T17:06:31.393 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:30 smithi203 ceph-mon[35787]: pgmap v525: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.1 MiB/s wr, 668 op/s 2024-05-14T17:06:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.893+0000 7f0ec0c83640 1 -- 172.21.15.69:0/3729338556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ebc106900 msgr2=0x7f0ebc106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.893+0000 7f0ec0c83640 1 --2- 172.21.15.69:0/3729338556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ebc106900 0x7f0ebc106ce0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009ca0 tx=0x7f0eac02a8c0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.893+0000 7f0ec0c83640 1 -- 172.21.15.69:0/3729338556 shutdown_connections 2024-05-14T17:06:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.893+0000 7f0ec0c83640 1 --2- 172.21.15.69:0/3729338556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ebc102660 0x7f0ebc10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.893+0000 7f0ec0c83640 1 --2- 172.21.15.69:0/3729338556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ebc101ca0 0x7f0ebc102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.893+0000 7f0ec0c83640 1 --2- 172.21.15.69:0/3729338556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ebc106900 0x7f0ebc106ce0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.893+0000 7f0ec0c83640 1 -- 172.21.15.69:0/3729338556 >> 172.21.15.69:0/3729338556 conn(0x7f0ebc077750 msgr2=0x7f0ebc077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.893+0000 7f0ec0c83640 1 -- 172.21.15.69:0/3729338556 shutdown_connections 2024-05-14T17:06:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.893+0000 7f0ec0c83640 1 -- 172.21.15.69:0/3729338556 wait complete. 2024-05-14T17:06:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ec0c83640 1 Processor -- start 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ec0c83640 1 -- start start 2024-05-14T17:06:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ec0c83640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ebc101ca0 0x7f0ebc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ec0c83640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ebc102660 0x7f0ebc198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ec0c83640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ebc106900 0x7f0ebc19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ec0c83640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc111bf0 con 0x7f0ebc106900 2024-05-14T17:06:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ec0c83640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc111a70 con 0x7f0ebc101ca0 2024-05-14T17:06:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ec0c83640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc111d70 con 0x7f0ebc102660 2024-05-14T17:06:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ebad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ebc106900 0x7f0ebc19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ebad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ebc106900 0x7f0ebc19c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50154/0 (socket says 172.21.15.69:50154) 2024-05-14T17:06:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ebad9d640 1 -- 172.21.15.69:0/1962636846 learned_addr learned my addr 172.21.15.69:0/1962636846 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0eba59c640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ebc101ca0 0x7f0ebc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ebad9d640 1 -- 172.21.15.69:0/1962636846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ebc102660 msgr2=0x7f0ebc198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ebc102660 0x7f0ebc198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ebad9d640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ebc102660 0x7f0ebc198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ebad9d640 1 -- 172.21.15.69:0/1962636846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ebc101ca0 msgr2=0x7f0ebc198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ebad9d640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ebc101ca0 0x7f0ebc198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0ebad9d640 1 -- 172.21.15.69:0/1962636846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc19d040 con 0x7f0ebc106900 2024-05-14T17:06:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.894+0000 7f0eba59c640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ebc101ca0 0x7f0ebc198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.895+0000 7f0ebad9d640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ebc106900 0x7f0ebc19c910 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400e970 tx=0x7f0eb400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.895+0000 7f0eab7fe640 1 -- 172.21.15.69:0/1962636846 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400ccc0 con 0x7f0ebc106900 2024-05-14T17:06:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.895+0000 7f0ec0c83640 1 -- 172.21.15.69:0/1962636846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc19d330 con 0x7f0ebc106900 2024-05-14T17:06:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.895+0000 7f0eab7fe640 1 -- 172.21.15.69:0/1962636846 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb400ce60 con 0x7f0ebc106900 2024-05-14T17:06:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.895+0000 7f0eab7fe640 1 -- 172.21.15.69:0/1962636846 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4010430 con 0x7f0ebc106900 2024-05-14T17:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.895+0000 7f0ec0c83640 1 -- 172.21.15.69:0/1962636846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc1a4bc0 con 0x7f0ebc106900 2024-05-14T17:06:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.898+0000 7f0ec39a6740 1 Processor -- start 2024-05-14T17:06:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.898+0000 7f0eab7fe640 1 -- 172.21.15.69:0/1962636846 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0eb4004f00 con 0x7f0ebc106900 2024-05-14T17:06:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.898+0000 7f0ec39a6740 1 -- start start 2024-05-14T17:06:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278aa0b0 0x55bf278b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278b6ce0 0x55bf278b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bf2781cc30 con 0x55bf278a9790 2024-05-14T17:06:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf2780c7d0 con 0x55bf278aa0b0 2024-05-14T17:06:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bf27810810 con 0x55bf278b6ce0 2024-05-14T17:06:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eb9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eb9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278a9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50168/0 (socket says 172.21.15.69:50168) 2024-05-14T17:06:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/3122622238 learned_addr learned my addr 172.21.15.69:0/3122622238 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eba59c640 1 --2- 172.21.15.69:0/3122622238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278aa0b0 0x55bf278b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3122622238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278b6ce0 0x55bf278b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/3122622238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278b6ce0 msgr2=0x55bf278b90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eab7fe640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0ea40b2a10 0x7f0ea40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3122622238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278b6ce0 0x55bf278b90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/3122622238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278aa0b0 msgr2=0x55bf278b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3122622238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278aa0b0 0x55bf278b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/3122622238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf27521210 con 0x55bf278a9790 2024-05-14T17:06:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0ebad9d640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0ea40b2a10 0x7f0ea40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3122622238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278a9b70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009fc0 tx=0x7f0eb00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6708318af9e267f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.899+0000 7f0eab7fe640 1 -- 172.21.15.69:0/1962636846 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f0eb40e2a30 con 0x7f0ebc106900 2024-05-14T17:06:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ea8ff9640 1 -- 172.21.15.69:0/3122622238 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0004030 con 0x55bf278a9790 2024-05-14T17:06:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ea8ff9640 1 -- 172.21.15.69:0/3122622238 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb001b070 con 0x55bf278a9790 2024-05-14T17:06:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ea8ff9640 1 -- 172.21.15.69:0/3122622238 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0013400 con 0x55bf278a9790 2024-05-14T17:06:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3122622238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 msgr2=0x55bf278a9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3122622238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278a9b70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009fc0 tx=0x7f0eb00096c0 comp rx=0 tx=0).stop 2024-05-14T17:06:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ebad9d640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0ea40b2a10 0x7f0ea40b4ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40e0220 tx=0x7f0eb4004980 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3122622238 shutdown_connections 2024-05-14T17:06:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3122622238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278b6ce0 0x55bf278b90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3122622238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278aa0b0 0x55bf278b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3122622238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278a9b70 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3122622238 >> 172.21.15.69:0/3122622238 conn(0x55bf278a5e00 msgr2=0x55bf278a61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.900+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3122622238 shutdown_connections 2024-05-14T17:06:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3122622238 wait complete. 2024-05-14T17:06:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0ec39a6740 1 -- start start 2024-05-14T17:06:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0ec39a6740 1 Processor -- start 2024-05-14T17:06:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278aa0b0 0x55bf278b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278b6ce0 0x55bf278b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bf274f2790 con 0x55bf278a9790 2024-05-14T17:06:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf277c8890 con 0x55bf278b6ce0 2024-05-14T17:06:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bf278b9950 con 0x55bf278aa0b0 2024-05-14T17:06:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0ec39a6740 10 client.0 _ll_register_callbacks cb 0x55bf277cbd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0eb9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0eb9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50176/0 (socket says 172.21.15.69:50176) 2024-05-14T17:06:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/2560923971 learned_addr learned my addr 172.21.15.69:0/2560923971 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/2560923971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278aa0b0 msgr2=0x55bf278b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278aa0b0 0x55bf278b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/2560923971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278b6ce0 msgr2=0x55bf278b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278b6ce0 0x55bf278b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.901+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/2560923971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf275ed620 con 0x55bf278a9790 2024-05-14T17:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278b6640 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00045f0 tx=0x7f0eb0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ba0122cc5bb1a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:06:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/2560923971 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00040d0 con 0x55bf278a9790 2024-05-14T17:06:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/2560923971 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f0eb0023430 con 0x55bf278a9790 2024-05-14T17:06:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ec39a6740 1 -- 172.21.15.69:0/2560923971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb001b070 con 0x55bf278a9790 2024-05-14T17:06:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/2560923971 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f0eb0023630 con 0x55bf278a9790 2024-05-14T17:06:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ec39a6740 10 client.17076 fetch_fsmap learned FSMap version 59 2024-05-14T17:06:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ec39a6740 1 -- 172.21.15.69:0/2560923971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55bf278c53b0 con 0x55bf278a9790 2024-05-14T17:06:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/2560923971 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb001b070 con 0x55bf278a9790 2024-05-14T17:06:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/2560923971 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f0eb000f040 con 0x55bf278a9790 2024-05-14T17:06:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ec39a6740 10 client.17076 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:06:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ec39a6740 10 client.17076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ec39a6740 20 client.17076 populate_metadata read hostname 'smithi069' 2024-05-14T17:06:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bf278c6810 0x55bf278c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ec39a6740 4 client.17076 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.902+0000 7f0ec39a6740 1 -- 172.21.15.69:0/2560923971 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf2781eb80 con 0x55bf278c6810 2024-05-14T17:06:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.903+0000 7f0eba59c640 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bf278c6810 0x55bf278c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:36.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.903+0000 7f0eba59c640 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bf278c6810 0x55bf278c8c00 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83b2e2089b8c10f server_cookie=e5e2da4f094d45c7 in_seq=0 out_seq=0 2024-05-14T17:06:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.903+0000 7f0ea1ffb640 10 client.17076 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/2560923971 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf2781eb80 con 0x55bf278c6810 2024-05-14T17:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ea1ffb640 10 client.17076 handle_command_reply: tid=0 2024-05-14T17:06:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 client.17076 shutdown 2024-05-14T17:06:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 -- 172.21.15.69:0/2560923971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 msgr2=0x55bf278b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278b6640 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00045f0 tx=0x7f0eb0005850 comp rx=0 tx=0).stop 2024-05-14T17:06:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 -- 172.21.15.69:0/2560923971 shutdown_connections 2024-05-14T17:06:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bf278c6810 0x55bf278c8c00 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278b6ce0 0x55bf278b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278aa0b0 0x55bf278b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ea1ffb640 0 client.17076 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/2560923971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278a9790 0x55bf278b6640 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 -- 172.21.15.69:0/2560923971 >> 172.21.15.69:0/2560923971 conn(0x55bf278a5e00 msgr2=0x55bf278a5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 -- 172.21.15.69:0/2560923971 shutdown_connections 2024-05-14T17:06:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 1 -- 172.21.15.69:0/2560923971 wait complete. 2024-05-14T17:06:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.904+0000 7f0ec39a6740 20 client.17076 trim_cache size 0 max 16384 2024-05-14T17:06:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.919+0000 7f0ec39a6740 1 Processor -- start 2024-05-14T17:06:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.919+0000 7f0ec39a6740 1 -- start start 2024-05-14T17:06:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278a9790 0x55bf278b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278aa0b0 0x55bf278ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278b6ce0 0x55bf278abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1aedb0 con 0x55bf278b6ce0 2024-05-14T17:06:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1ac650 con 0x55bf278a9790 2024-05-14T17:06:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1aa4c0 con 0x55bf278aa0b0 2024-05-14T17:06:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eb9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278b6ce0 0x55bf278abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eb9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278b6ce0 0x55bf278abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50178/0 (socket says 172.21.15.69:50178) 2024-05-14T17:06:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/3630980380 learned_addr learned my addr 172.21.15.69:0/3630980380 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3630980380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278a9790 0x55bf278b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eba59c640 1 --2- 172.21.15.69:0/3630980380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278aa0b0 0x55bf278ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/3630980380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278aa0b0 msgr2=0x55bf278ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3630980380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278aa0b0 0x55bf278ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/3630980380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278a9790 msgr2=0x55bf278b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3630980380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278a9790 0x55bf278b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eb9d9b640 1 -- 172.21.15.69:0/3630980380 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf278a1830 con 0x55bf278b6ce0 2024-05-14T17:06:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3630980380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278b6ce0 0x55bf278abf20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55bf278a6b70 tx=0x7f0eb0003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b955cd457adab1e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0ea3fff640 1 -- 172.21.15.69:0/3630980380 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00096c0 con 0x55bf278b6ce0 2024-05-14T17:06:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0ea3fff640 1 -- 172.21.15.69:0/3630980380 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb001b070 con 0x55bf278b6ce0 2024-05-14T17:06:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.920+0000 7f0ea3fff640 1 -- 172.21.15.69:0/3630980380 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00052d0 con 0x55bf278b6ce0 2024-05-14T17:06:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3630980380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278b6ce0 msgr2=0x55bf278abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3630980380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278b6ce0 0x55bf278abf20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55bf278a6b70 tx=0x7f0eb0003a20 comp rx=0 tx=0).stop 2024-05-14T17:06:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3630980380 shutdown_connections 2024-05-14T17:06:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3630980380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278b6ce0 0x55bf278abf20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3630980380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278aa0b0 0x55bf278ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3630980380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278a9790 0x55bf278b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3630980380 >> 172.21.15.69:0/3630980380 conn(0x55bf278a5e00 msgr2=0x55bf278ab2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3630980380 shutdown_connections 2024-05-14T17:06:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3630980380 wait complete. 2024-05-14T17:06:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 -- start start 2024-05-14T17:06:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 Processor -- start 2024-05-14T17:06:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278a9790 0x55bf278ace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278aa0b0 0x55bf278ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278b6ce0 0x55bf278bdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1aab10 con 0x55bf278aa0b0 2024-05-14T17:06:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1abf80 con 0x55bf278a9790 2024-05-14T17:06:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.921+0000 7f0ec39a6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1ab1e0 con 0x55bf278b6ce0 2024-05-14T17:06:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0ec39a6740 10 client.0 _ll_register_callbacks cb 0x7f0eb0003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eba59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278aa0b0 0x55bf278ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eba59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278aa0b0 0x55bf278ad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50186/0 (socket says 172.21.15.69:50186) 2024-05-14T17:06:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eba59c640 1 -- 172.21.15.69:0/3488602066 learned_addr learned my addr 172.21.15.69:0/3488602066 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278a9790 0x55bf278ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278b6ce0 0x55bf278bdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eba59c640 1 -- 172.21.15.69:0/3488602066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278b6ce0 msgr2=0x55bf278bdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eba59c640 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278b6ce0 0x55bf278bdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eba59c640 1 -- 172.21.15.69:0/3488602066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278a9790 msgr2=0x55bf278ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eba59c640 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278a9790 0x55bf278ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eba59c640 1 -- 172.21.15.69:0/3488602066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf278be160 con 0x55bf278aa0b0 2024-05-14T17:06:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0eba59c640 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278aa0b0 0x55bf278ad3d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0eac02ade0 tx=0x7f0eac031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8f14664ed22a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0ea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:06:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/3488602066 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac031650 con 0x55bf278aa0b0 2024-05-14T17:06:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/3488602066 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f0eac0317f0 con 0x55bf278aa0b0 2024-05-14T17:06:39.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3488602066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb001b070 con 0x55bf278aa0b0 2024-05-14T17:06:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/3488602066 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f0eac003970 con 0x55bf278aa0b0 2024-05-14T17:06:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0ec39a6740 10 client.17082 fetch_fsmap learned FSMap version 59 2024-05-14T17:06:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3488602066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55bf278bf500 con 0x55bf278aa0b0 2024-05-14T17:06:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.922+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/3488602066 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb001b070 con 0x55bf278aa0b0 2024-05-14T17:06:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.923+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/3488602066 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f0eac004040 con 0x55bf278aa0b0 2024-05-14T17:06:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.923+0000 7f0ec39a6740 10 client.17082 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:06:39.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.923+0000 7f0ec39a6740 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.923+0000 7f0ec39a6740 20 client.17082 populate_metadata read hostname 'smithi069' 2024-05-14T17:06:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.923+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bf27933780 0x55bf27935ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.923+0000 7f0ec39a6740 4 client.17082 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.923+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3488602066 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf27818360 con 0x55bf27933780 2024-05-14T17:06:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.923+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bf27933780 0x55bf27935ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.923+0000 7f0ea1ffb640 10 client.17082 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.923+0000 7f0eb9d9b640 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bf27933780 0x55bf27935ba0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d5261d301f975d server_cookie=1deb94d133409967 in_seq=0 out_seq=0 2024-05-14T17:06:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.936+0000 7f0ea1ffb640 1 -- 172.21.15.69:0/3488602066 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bf27818360 con 0x55bf27933780 2024-05-14T17:06:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.936+0000 7f0ea1ffb640 10 client.17082 handle_command_reply: tid=0 2024-05-14T17:06:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.936+0000 7f0ec39a6740 1 client.17082 shutdown 2024-05-14T17:06:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3488602066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278aa0b0 msgr2=0x55bf278ad3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278aa0b0 0x55bf278ad3d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0eac02ade0 tx=0x7f0eac031b70 comp rx=0 tx=0).stop 2024-05-14T17:06:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3488602066 shutdown_connections 2024-05-14T17:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55bf27933780 0x55bf27935ba0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf278b6ce0 0x55bf278bdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf278aa0b0 0x55bf278ad3d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 1 --2- 172.21.15.69:0/3488602066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf278a9790 0x55bf278ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ea1ffb640 0 client.17082 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3488602066 >> 172.21.15.69:0/3488602066 conn(0x55bf278a5e00 msgr2=0x55bf278ab2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3488602066 shutdown_connections 2024-05-14T17:06:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 1 -- 172.21.15.69:0/3488602066 wait complete. 2024-05-14T17:06:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec39a6740 20 client.17082 trim_cache size 0 max 16384 2024-05-14T17:06:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:06:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:06:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec0c83640 1 -- 172.21.15.69:0/1962636846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0ea40b2a10 msgr2=0x7f0ea40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec0c83640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0ea40b2a10 0x7f0ea40b4ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40e0220 tx=0x7f0eb4004980 comp rx=0 tx=0).stop 2024-05-14T17:06:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec0c83640 1 -- 172.21.15.69:0/1962636846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ebc106900 msgr2=0x7f0ebc19c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec0c83640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ebc106900 0x7f0ebc19c910 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400e970 tx=0x7f0eb400ee30 comp rx=0 tx=0).stop 2024-05-14T17:06:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec0c83640 1 -- 172.21.15.69:0/1962636846 shutdown_connections 2024-05-14T17:06:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec0c83640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0ea40b2a10 0x7f0ea40b4ed0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec0c83640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ebc106900 0x7f0ebc19c910 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec0c83640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ebc102660 0x7f0ebc198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec0c83640 1 --2- 172.21.15.69:0/1962636846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ebc101ca0 0x7f0ebc198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.937+0000 7f0ec0c83640 1 -- 172.21.15.69:0/1962636846 >> 172.21.15.69:0/1962636846 conn(0x7f0ebc077750 msgr2=0x7f0ebc0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.938+0000 7f0ec0c83640 1 -- 172.21.15.69:0/1962636846 shutdown_connections 2024-05-14T17:06:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:15.938+0000 7f0ec0c83640 1 -- 172.21.15.69:0/1962636846 wait complete. 2024-05-14T17:06:41.960 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:16 smithi069 ceph-mon[31507]: pgmap v518: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-05-14T17:06:42.011 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:06:42.060 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:18 smithi069 ceph-mon[31507]: pgmap v519: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-05-14T17:06:42.112 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:32 smithi203 ceph-mon[35787]: pgmap v526: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-05-14T17:06:42.160 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:34 smithi203 ceph-mon[35787]: pgmap v527: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 4.6 MiB/s wr, 510 op/s 2024-05-14T17:06:42.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:36 smithi203 ceph-mon[35787]: pgmap v528: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 222 KiB/s wr, 32 op/s 2024-05-14T17:06:42.260 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:38 smithi203 ceph-mon[35787]: pgmap v529: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 226 KiB/s wr, 35 op/s 2024-05-14T17:06:42.310 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:39 smithi203 ceph-mon[35787]: pgmap v530: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-05-14T17:06:42.360 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:32 smithi092 ceph-mon[35763]: pgmap v526: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-05-14T17:06:42.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:34 smithi092 ceph-mon[35763]: pgmap v527: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 4.6 MiB/s wr, 510 op/s 2024-05-14T17:06:42.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:36 smithi092 ceph-mon[35763]: pgmap v528: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 222 KiB/s wr, 32 op/s 2024-05-14T17:06:42.527 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:38 smithi092 ceph-mon[35763]: pgmap v529: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 226 KiB/s wr, 35 op/s 2024-05-14T17:06:42.577 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:39 smithi092 ceph-mon[35763]: pgmap v530: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-05-14T17:06:42.612 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:41 smithi203 ceph-mon[35787]: pgmap v531: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:06:42.644 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:41 smithi092 ceph-mon[35763]: pgmap v531: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:06:42.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.255+0000 7fbc9a00a700 1 -- 172.21.15.69:0/1785455659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc9410b0c0 msgr2=0x7fbc9410e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:42.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.255+0000 7fbc9a00a700 1 --2- 172.21.15.69:0/1785455659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc9410b0c0 0x7fbc9410e7d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800b970 tx=0x7fbc88009260 comp rx=0 tx=0).stop 2024-05-14T17:06:42.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.258+0000 7fbc9a00a700 1 -- 172.21.15.69:0/1785455659 shutdown_connections 2024-05-14T17:06:42.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.258+0000 7fbc9a00a700 1 --2- 172.21.15.69:0/1785455659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc9410a710 0x7fbc9410aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.258+0000 7fbc9a00a700 1 --2- 172.21.15.69:0/1785455659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc9410ef00 0x7fbc9407c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:42.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.258+0000 7fbc9a00a700 1 --2- 172.21.15.69:0/1785455659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc9410b0c0 0x7fbc9410e7d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:42.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.258+0000 7fbc9a00a700 1 -- 172.21.15.69:0/1785455659 >> 172.21.15.69:0/1785455659 conn(0x7fbc9406e6b0 msgr2=0x7fbc9406eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:42.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.258+0000 7fbc9a00a700 1 -- 172.21.15.69:0/1785455659 shutdown_connections 2024-05-14T17:06:43.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.258+0000 7fbc9a00a700 1 -- 172.21.15.69:0/1785455659 wait complete. 2024-05-14T17:06:43.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.259+0000 7fbc9a00a700 1 Processor -- start 2024-05-14T17:06:43.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.259+0000 7fbc9a00a700 1 -- start start 2024-05-14T17:06:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.259+0000 7fbc9a00a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc9410a710 0x7fbc94085e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:43.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.259+0000 7fbc9a00a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc9410b0c0 0x7fbc9407fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:43.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.259+0000 7fbc9a00a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc9410ef00 0x7fbc94080410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.259+0000 7fbc9a00a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9407eb80 con 0x7fbc9410a710 2024-05-14T17:06:43.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.259+0000 7fbc9a00a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9407ea00 con 0x7fbc9410ef00 2024-05-14T17:06:43.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.259+0000 7fbc9a00a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9407ed00 con 0x7fbc9410b0c0 2024-05-14T17:06:43.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.260+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc9410ef00 0x7fbc94080410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:43.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.260+0000 7fbc92ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc9410b0c0 0x7fbc9407fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:43.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.261+0000 7fbc937fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc9410a710 0x7fbc94085e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:43.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.261+0000 7fbc937fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc9410a710 0x7fbc94085e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50202/0 (socket says 172.21.15.69:50202) 2024-05-14T17:06:43.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.261+0000 7fbc937fe700 1 -- 172.21.15.69:0/321178829 learned_addr learned my addr 172.21.15.69:0/321178829 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:43.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.268+0000 7fbc937fe700 1 -- 172.21.15.69:0/321178829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc9410b0c0 msgr2=0x7fbc9407fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:43.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.268+0000 7fbc937fe700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc9410b0c0 0x7fbc9407fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:43.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.268+0000 7fbc937fe700 1 -- 172.21.15.69:0/321178829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc9410ef00 msgr2=0x7fbc94080410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:43.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.268+0000 7fbc937fe700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc9410ef00 0x7fbc94080410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:43.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.268+0000 7fbc937fe700 1 -- 172.21.15.69:0/321178829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc94080b90 con 0x7fbc9410a710 2024-05-14T17:06:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.274+0000 7fbc937fe700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc9410a710 0x7fbc94085e50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009810 tx=0x7fbc84009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:43.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.279+0000 7fbc90ff9700 1 -- 172.21.15.69:0/321178829 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8401b020 con 0x7fbc9410a710 2024-05-14T17:06:43.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.279+0000 7fbc90ff9700 1 -- 172.21.15.69:0/321178829 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc84005020 con 0x7fbc9410a710 2024-05-14T17:06:43.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.279+0000 7fbc90ff9700 1 -- 172.21.15.69:0/321178829 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400ca80 con 0x7fbc9410a710 2024-05-14T17:06:44.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.280+0000 7fbc9a00a700 1 -- 172.21.15.69:0/321178829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc941bd750 con 0x7fbc9410a710 2024-05-14T17:06:44.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.280+0000 7fbc9a00a700 1 -- 172.21.15.69:0/321178829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc941bdbb0 con 0x7fbc9410a710 2024-05-14T17:06:44.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.285+0000 7fbca48a0b80 1 Processor -- start 2024-05-14T17:06:44.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.285+0000 7fbca48a0b80 1 -- start start 2024-05-14T17:06:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.286+0000 7fbc90ff9700 1 -- 172.21.15.69:0/321178829 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbc8400cc20 con 0x7fbc9410a710 2024-05-14T17:06:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.287+0000 7fbc90ff9700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbc7c0b2cb0 0x7fbc7c0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:44.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.288+0000 7fbc90ff9700 1 -- 172.21.15.69:0/321178829 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7fbc840e22d0 con 0x7fbc9410a710 2024-05-14T17:06:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.288+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1dca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:44.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.288+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dcade0 0x5606d1dc5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:44.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.289+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dc6360 0x5606d1dc67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:44.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.289+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5606d1d23f10 con 0x5606d1dc6360 2024-05-14T17:06:44.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.289+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5606d1d13ab0 con 0x5606d1dcade0 2024-05-14T17:06:44.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.289+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5606d1d17af0 con 0x5606d1dca4c0 2024-05-14T17:06:44.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.289+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dc6360 0x5606d1dc67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:44.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.289+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dc6360 0x5606d1dc67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50204/0 (socket says 172.21.15.69:50204) 2024-05-14T17:06:44.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.289+0000 7fbc93fff700 1 -- 172.21.15.69:0/2092237517 learned_addr learned my addr 172.21.15.69:0/2092237517 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:44.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.289+0000 7fbc93fff700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbc7c0b2cb0 0x7fbc7c0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:44.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.290+0000 7fbc92ffd700 1 --2- 172.21.15.69:0/2092237517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dcade0 0x5606d1dc5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:44.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.291+0000 7fbc937fe700 1 --2- 172.21.15.69:0/2092237517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1dca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:44.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.295+0000 7fbc92ffd700 1 -- 172.21.15.69:0/2092237517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 msgr2=0x5606d1dca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:44.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.295+0000 7fbc92ffd700 1 --2- 172.21.15.69:0/2092237517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1dca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.295+0000 7fbc92ffd700 1 -- 172.21.15.69:0/2092237517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dc6360 msgr2=0x5606d1dc67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:44.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.295+0000 7fbc92ffd700 1 --2- 172.21.15.69:0/2092237517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dc6360 0x5606d1dc67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:44.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.295+0000 7fbc92ffd700 1 -- 172.21.15.69:0/2092237517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d1cbd150 con 0x5606d1dcade0 2024-05-14T17:06:44.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.296+0000 7fbc92ffd700 1 --2- 172.21.15.69:0/2092237517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dcade0 0x5606d1dc5d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbc88000c00 tx=0x7fbc88005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d746caa5f42cf34b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:44.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.299+0000 7fbc79ffb700 1 -- 172.21.15.69:0/2092237517 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88013d80 con 0x5606d1dcade0 2024-05-14T17:06:44.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.299+0000 7fbc79ffb700 1 -- 172.21.15.69:0/2092237517 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc880138b0 con 0x5606d1dcade0 2024-05-14T17:06:44.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.299+0000 7fbc79ffb700 1 -- 172.21.15.69:0/2092237517 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880038e0 con 0x5606d1dcade0 2024-05-14T17:06:45.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.299+0000 7fbc93fff700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbc7c0b2cb0 0x7fbc7c0b5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00f9f0 tx=0x7fbc8c008900 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:45.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.300+0000 7fbca48a0b80 1 -- 172.21.15.69:0/2092237517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dcade0 msgr2=0x5606d1dc5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.300+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/2092237517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dcade0 0x5606d1dc5d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbc88000c00 tx=0x7fbc88005dc0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.305+0000 7fbca48a0b80 1 -- 172.21.15.69:0/2092237517 shutdown_connections 2024-05-14T17:06:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.305+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/2092237517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dc6360 0x5606d1dc67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.305+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/2092237517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1dca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.305+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/2092237517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dcade0 0x5606d1dc5d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.305+0000 7fbca48a0b80 1 -- 172.21.15.69:0/2092237517 >> 172.21.15.69:0/2092237517 conn(0x5606d1cf2f50 msgr2=0x5606d1dc0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:45.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.311+0000 7fbca48a0b80 1 -- 172.21.15.69:0/2092237517 shutdown_connections 2024-05-14T17:06:45.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.312+0000 7fbca48a0b80 1 -- 172.21.15.69:0/2092237517 wait complete. 2024-05-14T17:06:45.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.312+0000 7fbca48a0b80 1 -- start start 2024-05-14T17:06:45.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.312+0000 7fbca48a0b80 1 Processor -- start 2024-05-14T17:06:45.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.313+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dc6360 0x5606d1dd1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.313+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dca4c0 0x5606d1dcef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.313+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1dcf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.313+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5606d1c1a140 con 0x5606d1dcade0 2024-05-14T17:06:45.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.313+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5606d1c7bf40 con 0x5606d1dca4c0 2024-05-14T17:06:45.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.313+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5606d1dd28c0 con 0x5606d1dc6360 2024-05-14T17:06:45.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.314+0000 7fbca48a0b80 10 client.0 _ll_register_callbacks cb 0x5606d1caeaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:45.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.314+0000 7fbc92ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dca4c0 0x5606d1dcef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.314+0000 7fbc92ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dca4c0 0x5606d1dcef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:37902/0 (socket says 172.21.15.69:37902) 2024-05-14T17:06:45.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.314+0000 7fbc92ffd700 1 -- 172.21.15.69:0/668179129 learned_addr learned my addr 172.21.15.69:0/668179129 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:45.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.314+0000 7fbc92ffd700 1 -- 172.21.15.69:0/668179129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dc6360 msgr2=0x5606d1dd1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.315+0000 7fbc92ffd700 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dc6360 0x5606d1dd1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.315+0000 7fbc92ffd700 1 -- 172.21.15.69:0/668179129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 msgr2=0x5606d1dcf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.315+0000 7fbc92ffd700 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1dcf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:46.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.315+0000 7fbc92ffd700 1 -- 172.21.15.69:0/668179129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d1c63a70 con 0x5606d1dca4c0 2024-05-14T17:06:46.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.316+0000 7fbc92ffd700 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dca4c0 0x5606d1dcef00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800c1d0 tx=0x7fbc880136e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b08f73167172cc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:46.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.316+0000 7fbc72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:06:46.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.318+0000 7fbc72ffd700 1 -- 172.21.15.69:0/668179129 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88003680 con 0x5606d1dca4c0 2024-05-14T17:06:46.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.318+0000 7fbc72ffd700 1 -- 172.21.15.69:0/668179129 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7fbc88003820 con 0x5606d1dca4c0 2024-05-14T17:06:46.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.319+0000 7fbca48a0b80 1 -- 172.21.15.69:0/668179129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc880138b0 con 0x5606d1dca4c0 2024-05-14T17:06:46.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.319+0000 7fbc72ffd700 1 -- 172.21.15.69:0/668179129 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fbc88039050 con 0x5606d1dca4c0 2024-05-14T17:06:46.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.319+0000 7fbca48a0b80 10 client.26453 fetch_fsmap learned FSMap version 59 2024-05-14T17:06:46.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.319+0000 7fbca48a0b80 1 -- 172.21.15.69:0/668179129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5606d1ccd900 con 0x5606d1dca4c0 2024-05-14T17:06:46.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.322+0000 7fbc72ffd700 1 -- 172.21.15.69:0/668179129 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc88012600 con 0x5606d1dca4c0 2024-05-14T17:06:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.322+0000 7fbc72ffd700 1 -- 172.21.15.69:0/668179129 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7fbc8803c020 con 0x5606d1dca4c0 2024-05-14T17:06:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.322+0000 7fbca48a0b80 10 client.26453 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:06:46.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.322+0000 7fbca48a0b80 10 client.26453 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:06:46.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.322+0000 7fbca48a0b80 20 client.26453 populate_metadata read hostname 'smithi069' 2024-05-14T17:06:46.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.322+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5606d1ddf0d0 0x5606d1de14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.322+0000 7fbca48a0b80 4 client.26453 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:46.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.322+0000 7fbca48a0b80 1 -- 172.21.15.69:0/668179129 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbc880138b0 con 0x5606d1ddf0d0 2024-05-14T17:06:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.323+0000 7fbc937fe700 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5606d1ddf0d0 0x5606d1de14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:46.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.323+0000 7fbc937fe700 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5606d1ddf0d0 0x5606d1de14c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5854805ac7f569 server_cookie=5d67dcd2a6073467 in_seq=0 out_seq=0 2024-05-14T17:06:46.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.323+0000 7fbc72ffd700 10 client.26453 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:46.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.324+0000 7fbc72ffd700 1 -- 172.21.15.69:0/668179129 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fbc880138b0 con 0x5606d1ddf0d0 2024-05-14T17:06:46.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.324+0000 7fbc72ffd700 10 client.26453 handle_command_reply: tid=0 2024-05-14T17:06:46.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.324+0000 7fbca48a0b80 1 client.26453 shutdown 2024-05-14T17:06:46.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.327+0000 7fbca48a0b80 1 -- 172.21.15.69:0/668179129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dca4c0 msgr2=0x5606d1dcef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:47.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.327+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dca4c0 0x5606d1dcef00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800c1d0 tx=0x7fbc880136e0 comp rx=0 tx=0).stop 2024-05-14T17:06:47.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.328+0000 7fbca48a0b80 1 -- 172.21.15.69:0/668179129 shutdown_connections 2024-05-14T17:06:47.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.328+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5606d1ddf0d0 0x5606d1de14c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:47.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.328+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1dcf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:47.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.328+0000 7fbc72ffd700 0 client.26453 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:47.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.328+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dc6360 0x5606d1dd1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:47.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.328+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/668179129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dca4c0 0x5606d1dcef00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:47.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.328+0000 7fbca48a0b80 1 -- 172.21.15.69:0/668179129 >> 172.21.15.69:0/668179129 conn(0x5606d1cf2f50 msgr2=0x5606d1dbb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.329+0000 7fbca48a0b80 1 -- 172.21.15.69:0/668179129 shutdown_connections 2024-05-14T17:06:47.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.329+0000 7fbca48a0b80 1 -- 172.21.15.69:0/668179129 wait complete. 2024-05-14T17:06:47.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.329+0000 7fbca48a0b80 20 client.26453 trim_cache size 0 max 16384 2024-05-14T17:06:47.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.371+0000 7fbca48a0b80 1 Processor -- start 2024-05-14T17:06:47.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.372+0000 7fbca48a0b80 1 -- start start 2024-05-14T17:06:47.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.372+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 0x5606d1dbb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:47.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.372+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1dc53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:47.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.372+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1dcc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:47.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.372+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbc80010030 con 0x5606d1dcade0 2024-05-14T17:06:47.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.372+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8000d890 con 0x5606d1dc6360 2024-05-14T17:06:47.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.372+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8000b700 con 0x5606d1dca4c0 2024-05-14T17:06:47.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.377+0000 7fbc937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 0x5606d1dbb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.377+0000 7fbc937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 0x5606d1dbb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:37908/0 (socket says 172.21.15.69:37908) 2024-05-14T17:06:47.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.377+0000 7fbc937fe700 1 -- 172.21.15.69:0/3851210927 learned_addr learned my addr 172.21.15.69:0/3851210927 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:47.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.377+0000 7fbc93fff700 1 --2- 172.21.15.69:0/3851210927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1dcc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.377+0000 7fbc92ffd700 1 --2- 172.21.15.69:0/3851210927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1dc53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.379+0000 7fbc937fe700 1 -- 172.21.15.69:0/3851210927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 msgr2=0x5606d1dc53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:47.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.379+0000 7fbc937fe700 1 --2- 172.21.15.69:0/3851210927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1dc53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.379+0000 7fbc937fe700 1 -- 172.21.15.69:0/3851210927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 msgr2=0x5606d1dcc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:47.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.379+0000 7fbc937fe700 1 --2- 172.21.15.69:0/3851210927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1dcc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:48.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.379+0000 7fbc937fe700 1 -- 172.21.15.69:0/3851210927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d1dcd470 con 0x5606d1dc6360 2024-05-14T17:06:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.384+0000 7fbc937fe700 1 --2- 172.21.15.69:0/3851210927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 0x5606d1dbb170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbc840a6840 tx=0x7fbc8400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f0d81ca2352e803 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:48.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.384+0000 7fbc797fa700 1 -- 172.21.15.69:0/3851210927 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc840c8070 con 0x5606d1dc6360 2024-05-14T17:06:48.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.384+0000 7fbc797fa700 1 -- 172.21.15.69:0/3851210927 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc840ad980 con 0x5606d1dc6360 2024-05-14T17:06:48.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.384+0000 7fbc797fa700 1 -- 172.21.15.69:0/3851210927 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc840e2040 con 0x5606d1dc6360 2024-05-14T17:06:48.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.387+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3851210927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 msgr2=0x5606d1dbb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:48.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.387+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/3851210927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 0x5606d1dbb170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbc840a6840 tx=0x7fbc8400cf90 comp rx=0 tx=0).stop 2024-05-14T17:06:48.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.387+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3851210927 shutdown_connections 2024-05-14T17:06:48.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.387+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/3851210927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1dcc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:48.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.387+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/3851210927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 0x5606d1dbb170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:48.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.387+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/3851210927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1dc53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:48.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.387+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3851210927 >> 172.21.15.69:0/3851210927 conn(0x5606d1cf2f50 msgr2=0x5606d1dc9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:48.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.391+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3851210927 shutdown_connections 2024-05-14T17:06:48.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.391+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3851210927 wait complete. 2024-05-14T17:06:48.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.391+0000 7fbca48a0b80 1 -- start start 2024-05-14T17:06:48.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.392+0000 7fbca48a0b80 1 Processor -- start 2024-05-14T17:06:48.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.392+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 0x5606d1de3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:48.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.392+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1de3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:48.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.393+0000 7fbca48a0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1de3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:48.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.393+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8000bd50 con 0x5606d1dcade0 2024-05-14T17:06:48.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.393+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8000d1c0 con 0x5606d1dc6360 2024-05-14T17:06:48.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.393+0000 7fbca48a0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8000c420 con 0x5606d1dca4c0 2024-05-14T17:06:48.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.393+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1de3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:48.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.393+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1de3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50234/0 (socket says 172.21.15.69:50234) 2024-05-14T17:06:48.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.393+0000 7fbc93fff700 1 -- 172.21.15.69:0/3776120397 learned_addr learned my addr 172.21.15.69:0/3776120397 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.393+0000 7fbca48a0b80 10 client.0 _ll_register_callbacks cb 0x5606d1caeaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:49.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.394+0000 7fbc937fe700 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 0x5606d1de3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:49.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.394+0000 7fbc92ffd700 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1de3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:49.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.395+0000 7fbc93fff700 1 -- 172.21.15.69:0/3776120397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 msgr2=0x5606d1de3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:49.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.395+0000 7fbc93fff700 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1de3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:49.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.395+0000 7fbc93fff700 1 -- 172.21.15.69:0/3776120397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 msgr2=0x5606d1de3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:49.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.395+0000 7fbc93fff700 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 0x5606d1de3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:49.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.395+0000 7fbc93fff700 1 -- 172.21.15.69:0/3776120397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d1e49600 con 0x5606d1dcade0 2024-05-14T17:06:49.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.398+0000 7fbc93fff700 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1de3ad0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c011c10 tx=0x7fbc8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da675bc485d984d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:49.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.399+0000 7fbc72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:06:49.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.403+0000 7fbc72ffd700 1 -- 172.21.15.69:0/3776120397 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c028030 con 0x5606d1dcade0 2024-05-14T17:06:49.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.403+0000 7fbc72ffd700 1 -- 172.21.15.69:0/3776120397 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7fbc8c01abe0 con 0x5606d1dcade0 2024-05-14T17:06:49.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.404+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3776120397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc840ad980 con 0x5606d1dcade0 2024-05-14T17:06:49.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.411+0000 7fbc72ffd700 1 -- 172.21.15.69:0/3776120397 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fbc8c020040 con 0x5606d1dcade0 2024-05-14T17:06:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.412+0000 7fbca48a0b80 10 client.17097 fetch_fsmap learned FSMap version 59 2024-05-14T17:06:49.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.412+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3776120397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5606d1e4a810 con 0x5606d1dcade0 2024-05-14T17:06:49.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.415+0000 7fbc72ffd700 1 -- 172.21.15.69:0/3776120397 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc840ad980 con 0x5606d1dcade0 2024-05-14T17:06:49.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.415+0000 7fbc72ffd700 1 -- 172.21.15.69:0/3776120397 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7fbc8c023030 con 0x5606d1dcade0 2024-05-14T17:06:49.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.415+0000 7fbca48a0b80 10 client.17097 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:06:49.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.415+0000 7fbca48a0b80 10 client.17097 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:06:49.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.415+0000 7fbca48a0b80 20 client.17097 populate_metadata read hostname 'smithi069' 2024-05-14T17:06:49.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.415+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5606d1ddf0d0 0x5606d1e4d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:49.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.415+0000 7fbca48a0b80 4 client.17097 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.65, "await_for": 45.0}] 2024-05-14T17:06:49.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.415+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3776120397 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.65, "await_for": 45.0}) -- 0x5606d1d25e60 con 0x5606d1ddf0d0 2024-05-14T17:06:49.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.417+0000 7fbc937fe700 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5606d1ddf0d0 0x5606d1e4d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:49.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.429+0000 7fbc72ffd700 10 client.17097 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:49.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:19.429+0000 7fbc937fe700 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5606d1ddf0d0 0x5606d1e4d000 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bce55b33f800ef9 server_cookie=f8403ac520a7de5f in_seq=0 out_seq=0 2024-05-14T17:06:50.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:20 smithi069 ceph-mon[31507]: pgmap v520: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 787 op/s 2024-05-14T17:06:50.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.597+0000 7fbc72ffd700 1 -- 172.21.15.69:0/3776120397 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5606d1d25e60 con 0x5606d1ddf0d0 2024-05-14T17:06:50.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.597+0000 7fbc72ffd700 10 client.17097 handle_command_reply: tid=0 2024-05-14T17:06:50.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.598+0000 7fbca48a0b80 1 client.17097 shutdown 2024-05-14T17:06:50.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.598+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3776120397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 msgr2=0x5606d1de3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.598+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1de3ad0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c011c10 tx=0x7fbc8c004ad0 comp rx=0 tx=0).stop 2024-05-14T17:06:50.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.598+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3776120397 shutdown_connections 2024-05-14T17:06:50.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.598+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5606d1ddf0d0 0x5606d1e4d000 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:50.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.599+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606d1dcade0 0x5606d1de3ad0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:50.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.599+0000 7fbc72ffd700 0 client.17097 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:50.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.599+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606d1dc6360 0x5606d1de3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:50.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.599+0000 7fbca48a0b80 1 --2- 172.21.15.69:0/3776120397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5606d1dca4c0 0x5606d1de3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:50.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.599+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3776120397 >> 172.21.15.69:0/3776120397 conn(0x5606d1cf2f50 msgr2=0x5606d1dc9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:50.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.599+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3776120397 shutdown_connections 2024-05-14T17:06:50.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.599+0000 7fbca48a0b80 1 -- 172.21.15.69:0/3776120397 wait complete. 2024-05-14T17:06:50.663 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:06:50.692 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 59, 2024-05-14T17:06:50.725 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T17:06:50.759 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 42, 2024-05-14T17:06:50.792 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T17:06:50.825 INFO:teuthology.orchestra.run.smithi069.stdout: "471a1ec6": { 2024-05-14T17:06:50.859 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 42, 2024-05-14T17:06:50.892 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T17:06:50.925 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:06:50.959 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T17:06:50.992 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:06:51.026 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:06:51.109 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T17:06:51.166 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 149.7, 2024-05-14T17:06:51.225 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T17:06:51.259 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T17:06:51.292 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T17:06:51.332 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:06:51.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T17:06:51.401 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:06:51.434 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:06:51.477 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:06:51.509 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:06:51.543 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:06:51.576 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:06:51.609 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:06:51.645 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:43 smithi092 ceph-mon[35763]: pgmap v532: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-05-14T17:06:51.676 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:45 smithi092 ceph-mon[35763]: pgmap v533: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-05-14T17:06:51.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:47 smithi092 ceph-mon[35763]: pgmap v534: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-14T17:06:51.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:49 smithi092 ceph-mon[35763]: pgmap v535: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-05-14T17:06:51.826 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:43 smithi203 ceph-mon[35787]: pgmap v532: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-05-14T17:06:51.881 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:45 smithi203 ceph-mon[35787]: pgmap v533: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-05-14T17:06:51.918 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:47 smithi203 ceph-mon[35787]: pgmap v534: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-14T17:06:51.951 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:49 smithi203 ceph-mon[35787]: pgmap v535: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-05-14T17:06:52.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.599+0000 7fbca48a0b80 20 client.17097 trim_cache size 0 max 16384 2024-05-14T17:06:52.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.601+0000 7fbc9a00a700 1 -- 172.21.15.69:0/321178829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbc7c0b2cb0 msgr2=0x7fbc7c0b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:52.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.601+0000 7fbc9a00a700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbc7c0b2cb0 0x7fbc7c0b5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00f9f0 tx=0x7fbc8c008900 comp rx=0 tx=0).stop 2024-05-14T17:06:52.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.601+0000 7fbc9a00a700 1 -- 172.21.15.69:0/321178829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc9410a710 msgr2=0x7fbc94085e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:52.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.601+0000 7fbc9a00a700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc9410a710 0x7fbc94085e50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009810 tx=0x7fbc84009b20 comp rx=0 tx=0).stop 2024-05-14T17:06:52.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.601+0000 7fbc9a00a700 1 -- 172.21.15.69:0/321178829 shutdown_connections 2024-05-14T17:06:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.601+0000 7fbc9a00a700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbc7c0b2cb0 0x7fbc7c0b5170 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:52.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.601+0000 7fbc9a00a700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc9410a710 0x7fbc94085e50 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.601+0000 7fbc9a00a700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc9410ef00 0x7fbc94080410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:52.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.601+0000 7fbc9a00a700 1 --2- 172.21.15.69:0/321178829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc9410b0c0 0x7fbc9407fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:52.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.601+0000 7fbc9a00a700 1 -- 172.21.15.69:0/321178829 >> 172.21.15.69:0/321178829 conn(0x7fbc9406e6b0 msgr2=0x7fbc94072f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:52.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.602+0000 7fbc9a00a700 1 -- 172.21.15.69:0/321178829 shutdown_connections 2024-05-14T17:06:52.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:06:20.602+0000 7fbc9a00a700 1 -- 172.21.15.69:0/321178829 wait complete. 2024-05-14T17:06:52.478 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '471a1ec6', quiesce took 115.3 seconds. Will release after: -85.65 2024-05-14T17:06:52.519 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:22 smithi069 ceph-mon[31507]: pgmap v521: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 844 op/s 2024-05-14T17:06:52.552 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:24 smithi069 ceph-mon[31507]: pgmap v522: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-14T17:06:52.585 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:24 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:06:52.618 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:25 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:06:52.652 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:25 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:06:52.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:25 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:06:52.719 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:26 smithi069 ceph-mon[31507]: pgmap v523: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-05-14T17:06:52.772 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:28 smithi069 ceph-mon[31507]: pgmap v524: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 11 MiB/s wr, 1.20k op/s 2024-05-14T17:06:52.802 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:51 smithi092 ceph-mon[35763]: pgmap v536: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-05-14T17:06:52.835 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:51 smithi203 ceph-mon[35787]: pgmap v536: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-05-14T17:06:52.871 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:06:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.035+0000 7f76eb714640 1 -- 172.21.15.69:0/3808963095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76e4101a90 msgr2=0x7f76e4101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.035+0000 7f76eb714640 1 --2- 172.21.15.69:0/3808963095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76e4101a90 0x7f76e4101e70 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009cd0 tx=0x7f76d402a970 comp rx=0 tx=0).stop 2024-05-14T17:06:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.035+0000 7f76eb714640 1 -- 172.21.15.69:0/3808963095 shutdown_connections 2024-05-14T17:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.035+0000 7f76eb714640 1 --2- 172.21.15.69:0/3808963095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e4106e90 0x7f76e4107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.035+0000 7f76eb714640 1 --2- 172.21.15.69:0/3808963095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e41023b0 0x7f76e4106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.035+0000 7f76eb714640 1 --2- 172.21.15.69:0/3808963095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76e4101a90 0x7f76e4101e70 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.035+0000 7f76eb714640 1 -- 172.21.15.69:0/3808963095 >> 172.21.15.69:0/3808963095 conn(0x7f76e4078b60 msgr2=0x7f76e4078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.035+0000 7f76eb714640 1 -- 172.21.15.69:0/3808963095 shutdown_connections 2024-05-14T17:06:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.035+0000 7f76eb714640 1 -- 172.21.15.69:0/3808963095 wait complete. 2024-05-14T17:06:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76eb714640 1 Processor -- start 2024-05-14T17:06:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76eb714640 1 -- start start 2024-05-14T17:06:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76eb714640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e4101a90 0x7f76e41a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76eb714640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e41023b0 0x7f76e41a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76eb714640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76e4106e90 0x7f76e419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76eb714640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f76e41192e0 con 0x7f76e4106e90 2024-05-14T17:06:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76eb714640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f76e4119160 con 0x7f76e4101a90 2024-05-14T17:06:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76eb714640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76e4119460 con 0x7f76e41023b0 2024-05-14T17:06:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76e9cb1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76e4106e90 0x7f76e419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76e94b0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e4101a90 0x7f76e41a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76e9cb1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76e4106e90 0x7f76e419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42580/0 (socket says 172.21.15.69:42580) 2024-05-14T17:06:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76e94b0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e4101a90 0x7f76e41a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.69:53132/0 (socket says 172.21.15.69:53132) 2024-05-14T17:06:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76e9cb1640 1 -- 172.21.15.69:0/724193127 learned_addr learned my addr 172.21.15.69:0/724193127 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:53.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.036+0000 7f76e8caf640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e41023b0 0x7f76e41a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76e9cb1640 1 -- 172.21.15.69:0/724193127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e41023b0 msgr2=0x7f76e41a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76e9cb1640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e41023b0 0x7f76e41a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76e9cb1640 1 -- 172.21.15.69:0/724193127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e4101a90 msgr2=0x7f76e41a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76e9cb1640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e4101a90 0x7f76e41a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76e9cb1640 1 -- 172.21.15.69:0/724193127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e419fdc0 con 0x7f76e4106e90 2024-05-14T17:06:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76e9cb1640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76e4106e90 0x7f76e419f640 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00d680 tx=0x7f76dc00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76e27fc640 1 -- 172.21.15.69:0/724193127 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc017070 con 0x7f76e4106e90 2024-05-14T17:06:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76e27fc640 1 -- 172.21.15.69:0/724193127 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76dc009820 con 0x7f76e4106e90 2024-05-14T17:06:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76eb714640 1 -- 172.21.15.69:0/724193127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e41a00b0 con 0x7f76e4106e90 2024-05-14T17:06:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76e27fc640 1 -- 172.21.15.69:0/724193127 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc004c40 con 0x7f76e4106e90 2024-05-14T17:06:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.037+0000 7f76eb714640 1 -- 172.21.15.69:0/724193127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e41ac300 con 0x7f76e4106e90 2024-05-14T17:06:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e27fc640 1 -- 172.21.15.69:0/724193127 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f76dc0099d0 con 0x7f76e4106e90 2024-05-14T17:06:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76ee42e740 1 Processor -- start 2024-05-14T17:06:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76ee42e740 1 -- start start 2024-05-14T17:06:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de48f0 0x560452de4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de5210 0x560452def590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76ee42e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560452d57c30 con 0x560452defad0 2024-05-14T17:06:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76ee42e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560452d477d0 con 0x560452de48f0 2024-05-14T17:06:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76ee42e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560452d4b810 con 0x560452de5210 2024-05-14T17:06:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e8caf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e8caf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42592/0 (socket says 172.21.15.69:42592) 2024-05-14T17:06:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e8caf640 1 -- 172.21.15.69:0/632511050 learned_addr learned my addr 172.21.15.69:0/632511050 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e8caf640 1 --2- 172.21.15.69:0/632511050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de48f0 0x560452de4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e94b0640 1 --2- 172.21.15.69:0/632511050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de5210 0x560452def590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e8caf640 1 -- 172.21.15.69:0/632511050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de5210 msgr2=0x560452def590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e8caf640 1 --2- 172.21.15.69:0/632511050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de5210 0x560452def590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e8caf640 1 -- 172.21.15.69:0/632511050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de48f0 msgr2=0x560452de4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e8caf640 1 --2- 172.21.15.69:0/632511050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de48f0 0x560452de4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e27fc640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76cc0b2a90 0x7f76cc0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e8caf640 1 -- 172.21.15.69:0/632511050 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560452a5c210 con 0x560452defad0 2024-05-14T17:06:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.040+0000 7f76e94b0640 1 --2- 172.21.15.69:0/632511050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de5210 0x560452def590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.041+0000 7f76e9cb1640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76cc0b2a90 0x7f76cc0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.041+0000 7f76e8caf640 1 --2- 172.21.15.69:0/632511050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df1ec0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f76d800cfd0 tx=0x7f76d800c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa931b16957e776 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.041+0000 7f76e27fc640 1 -- 172.21.15.69:0/724193127 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f76dc0e1f20 con 0x7f76e4106e90 2024-05-14T17:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.041+0000 7f76cb7fe640 1 -- 172.21.15.69:0/632511050 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8019030 con 0x560452defad0 2024-05-14T17:06:55.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.041+0000 7f76cb7fe640 1 -- 172.21.15.69:0/632511050 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76d80054d0 con 0x560452defad0 2024-05-14T17:06:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.041+0000 7f76cb7fe640 1 -- 172.21.15.69:0/632511050 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8011720 con 0x560452defad0 2024-05-14T17:06:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 -- 172.21.15.69:0/632511050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 msgr2=0x560452df1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76e9cb1640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76cc0b2a90 0x7f76cc0b4f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0a6460 tx=0x7f76dc0046c0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 --2- 172.21.15.69:0/632511050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df1ec0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f76d800cfd0 tx=0x7f76d800c670 comp rx=0 tx=0).stop 2024-05-14T17:06:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 -- 172.21.15.69:0/632511050 shutdown_connections 2024-05-14T17:06:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 --2- 172.21.15.69:0/632511050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df1ec0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 --2- 172.21.15.69:0/632511050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de5210 0x560452def590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 --2- 172.21.15.69:0/632511050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de48f0 0x560452de4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 -- 172.21.15.69:0/632511050 >> 172.21.15.69:0/632511050 conn(0x560452de0e00 msgr2=0x560452de11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 -- 172.21.15.69:0/632511050 shutdown_connections 2024-05-14T17:06:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 -- 172.21.15.69:0/632511050 wait complete. 2024-05-14T17:06:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 -- start start 2024-05-14T17:06:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.042+0000 7f76ee42e740 1 Processor -- start 2024-05-14T17:06:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452def430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452de5210 0x560452dec720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452defad0 0x560452decc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76ee42e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560452a2d790 con 0x560452de5210 2024-05-14T17:06:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76ee42e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560452d03890 con 0x560452defad0 2024-05-14T17:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76ee42e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560452df27a0 con 0x560452de48f0 2024-05-14T17:06:55.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76ee42e740 10 client.0 _ll_register_callbacks cb 0x560452d06d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e94b0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452de5210 0x560452dec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e94b0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452de5210 0x560452dec720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42604/0 (socket says 172.21.15.69:42604) 2024-05-14T17:06:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e94b0640 1 -- 172.21.15.69:0/3087934430 learned_addr learned my addr 172.21.15.69:0/3087934430 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e8caf640 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452def430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e8caf640 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452defad0 0x560452decc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e94b0640 1 -- 172.21.15.69:0/3087934430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 msgr2=0x560452def430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e94b0640 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452def430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e94b0640 1 -- 172.21.15.69:0/3087934430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452defad0 msgr2=0x560452decc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e94b0640 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452defad0 0x560452decc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e94b0640 1 -- 172.21.15.69:0/3087934430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560452b28620 con 0x560452de5210 2024-05-14T17:06:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76e94b0640 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452de5210 0x560452dec720 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f76d402ae90 tx=0x7f76d402fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f627188b28eb0eb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:06:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76c8ff9640 1 -- 172.21.15.69:0/3087934430 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4034040 con 0x560452de5210 2024-05-14T17:06:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76c8ff9640 1 -- 172.21.15.69:0/3087934430 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f76d4003b80 con 0x560452de5210 2024-05-14T17:06:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76ee42e740 1 -- 172.21.15.69:0/3087934430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76d80054d0 con 0x560452de5210 2024-05-14T17:06:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76c8ff9640 1 -- 172.21.15.69:0/3087934430 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f76d4003d80 con 0x560452de5210 2024-05-14T17:06:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76ee42e740 10 client.17109 fetch_fsmap learned FSMap version 59 2024-05-14T17:06:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.043+0000 7f76ee42e740 1 -- 172.21.15.69:0/3087934430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560452dfe130 con 0x560452de5210 2024-05-14T17:06:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.044+0000 7f76c8ff9640 1 -- 172.21.15.69:0/3087934430 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76d80054d0 con 0x560452de5210 2024-05-14T17:06:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.044+0000 7f76c8ff9640 1 -- 172.21.15.69:0/3087934430 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f76d40045a0 con 0x560452de5210 2024-05-14T17:06:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.044+0000 7f76ee42e740 10 client.17109 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:06:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.044+0000 7f76ee42e740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.044+0000 7f76ee42e740 20 client.17109 populate_metadata read hostname 'smithi069' 2024-05-14T17:06:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.044+0000 7f76ee42e740 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560452dff640 0x560452e01a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.044+0000 7f76ee42e740 4 client.17109 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.044+0000 7f76ee42e740 1 -- 172.21.15.69:0/3087934430 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560452d59b80 con 0x560452dff640 2024-05-14T17:06:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.044+0000 7f76e8caf640 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560452dff640 0x560452e01a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.045+0000 7f76c8ff9640 10 client.17109 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.045+0000 7f76e8caf640 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560452dff640 0x560452e01a30 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da14d5b5b36558cc server_cookie=ca72ff8f5e07ae22 in_seq=0 out_seq=0 2024-05-14T17:06:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.045+0000 7f76c8ff9640 1 -- 172.21.15.69:0/3087934430 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560452d59b80 con 0x560452dff640 2024-05-14T17:06:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.045+0000 7f76c8ff9640 10 client.17109 handle_command_reply: tid=0 2024-05-14T17:06:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.045+0000 7f76ee42e740 1 client.17109 shutdown 2024-05-14T17:06:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 1 -- 172.21.15.69:0/3087934430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452de5210 msgr2=0x560452dec720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452de5210 0x560452dec720 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f76d402ae90 tx=0x7f76d402fdd0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 1 -- 172.21.15.69:0/3087934430 shutdown_connections 2024-05-14T17:06:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560452dff640 0x560452e01a30 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452defad0 0x560452decc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76c8ff9640 0 client.17109 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:06:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452de5210 0x560452dec720 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 1 --2- 172.21.15.69:0/3087934430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452def430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 1 -- 172.21.15.69:0/3087934430 >> 172.21.15.69:0/3087934430 conn(0x560452de0e00 msgr2=0x560452de0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 1 -- 172.21.15.69:0/3087934430 shutdown_connections 2024-05-14T17:06:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 1 -- 172.21.15.69:0/3087934430 wait complete. 2024-05-14T17:06:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.046+0000 7f76ee42e740 20 client.17109 trim_cache size 0 max 16384 2024-05-14T17:06:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76ee42e740 1 Processor -- start 2024-05-14T17:06:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76ee42e740 1 -- start start 2024-05-14T17:06:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452e07290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de5210 0x560452e077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452de9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76ee42e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f76e41b75f0 con 0x560452defad0 2024-05-14T17:06:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76ee42e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f76e41b4e90 con 0x560452de5210 2024-05-14T17:06:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76ee42e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76e41b2d00 con 0x560452de48f0 2024-05-14T17:06:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76e8caf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452de9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76e8caf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452de9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42618/0 (socket says 172.21.15.69:42618) 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76e8caf640 1 -- 172.21.15.69:0/684643066 learned_addr learned my addr 172.21.15.69:0/684643066 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76e94b0640 1 --2- 172.21.15.69:0/684643066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de5210 0x560452e077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.061+0000 7f76e8caf640 1 --2- 172.21.15.69:0/684643066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452e07290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76e8caf640 1 -- 172.21.15.69:0/684643066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 msgr2=0x560452e07290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76e8caf640 1 --2- 172.21.15.69:0/684643066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452e07290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76e8caf640 1 -- 172.21.15.69:0/684643066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de5210 msgr2=0x560452e077d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76e8caf640 1 --2- 172.21.15.69:0/684643066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de5210 0x560452e077d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76e8caf640 1 -- 172.21.15.69:0/684643066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560452decff0 con 0x560452defad0 2024-05-14T17:06:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76e8caf640 1 --2- 172.21.15.69:0/684643066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452de9980 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f76d80058c0 tx=0x7f76d800c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f282dc6a10a1bf88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76caffd640 1 -- 172.21.15.69:0/684643066 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8031030 con 0x560452defad0 2024-05-14T17:06:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76caffd640 1 -- 172.21.15.69:0/684643066 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76d802a800 con 0x560452defad0 2024-05-14T17:06:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76caffd640 1 -- 172.21.15.69:0/684643066 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d802be60 con 0x560452defad0 2024-05-14T17:06:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76ee42e740 1 -- 172.21.15.69:0/684643066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 msgr2=0x560452de9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76ee42e740 1 --2- 172.21.15.69:0/684643066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452de9980 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f76d80058c0 tx=0x7f76d800c670 comp rx=0 tx=0).stop 2024-05-14T17:06:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76ee42e740 1 -- 172.21.15.69:0/684643066 shutdown_connections 2024-05-14T17:06:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76ee42e740 1 --2- 172.21.15.69:0/684643066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452de9980 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76ee42e740 1 --2- 172.21.15.69:0/684643066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de5210 0x560452e077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.062+0000 7f76ee42e740 1 --2- 172.21.15.69:0/684643066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452e07290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 -- 172.21.15.69:0/684643066 >> 172.21.15.69:0/684643066 conn(0x560452de0e00 msgr2=0x560452dfa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 -- 172.21.15.69:0/684643066 shutdown_connections 2024-05-14T17:06:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 -- 172.21.15.69:0/684643066 wait complete. 2024-05-14T17:06:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 -- start start 2024-05-14T17:06:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 Processor -- start 2024-05-14T17:06:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452df7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de5210 0x560452df8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f76e41b3350 con 0x560452defad0 2024-05-14T17:06:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f76e41b47c0 con 0x560452de5210 2024-05-14T17:06:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76e41b3a20 con 0x560452de48f0 2024-05-14T17:06:59.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76ee42e740 10 client.0 _ll_register_callbacks cb 0x7f76d402fa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76e8caf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76e8caf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42624/0 (socket says 172.21.15.69:42624) 2024-05-14T17:06:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.063+0000 7f76e8caf640 1 -- 172.21.15.69:0/347606432 learned_addr learned my addr 172.21.15.69:0/347606432 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:06:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76e94b0640 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de5210 0x560452df8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76e8caf640 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452df7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76e8caf640 1 -- 172.21.15.69:0/347606432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 msgr2=0x560452df7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76e8caf640 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452df7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76e8caf640 1 -- 172.21.15.69:0/347606432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de5210 msgr2=0x560452df8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76e8caf640 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de5210 0x560452df8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76e8caf640 1 -- 172.21.15.69:0/347606432 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560452df5950 con 0x560452defad0 2024-05-14T17:06:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76e8caf640 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df5150 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f76d800c4e0 tx=0x7f76d802d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a9c6f777e4e670 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:06:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76c8ff9640 1 -- 172.21.15.69:0/347606432 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d803b070 con 0x560452defad0 2024-05-14T17:06:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76ee42e740 1 -- 172.21.15.69:0/347606432 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76d802a800 con 0x560452defad0 2024-05-14T17:06:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76c8ff9640 1 -- 172.21.15.69:0/347606432 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f76d802a9b0 con 0x560452defad0 2024-05-14T17:06:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76c8ff9640 1 -- 172.21.15.69:0/347606432 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f76d8047050 con 0x560452defad0 2024-05-14T17:07:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76ee42e740 10 client.17115 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76ee42e740 1 -- 172.21.15.69:0/347606432 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560452e69af0 con 0x560452defad0 2024-05-14T17:07:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.064+0000 7f76c8ff9640 1 -- 172.21.15.69:0/347606432 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76d802a800 con 0x560452defad0 2024-05-14T17:07:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.065+0000 7f76c8ff9640 1 -- 172.21.15.69:0/347606432 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f76d801e460 con 0x560452defad0 2024-05-14T17:07:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.065+0000 7f76ee42e740 10 client.17115 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.065+0000 7f76ee42e740 10 client.17115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.065+0000 7f76ee42e740 20 client.17115 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.065+0000 7f76ee42e740 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560452e69fc0 0x560452e6c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.065+0000 7f76ee42e740 4 client.17115 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.065+0000 7f76ee42e740 1 -- 172.21.15.69:0/347606432 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560452d53360 con 0x560452e69fc0 2024-05-14T17:07:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.065+0000 7f76e94b0640 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560452e69fc0 0x560452e6c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.065+0000 7f76c8ff9640 10 client.17115 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.065+0000 7f76e94b0640 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560452e69fc0 0x560452e6c3e0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a89daa956fa1ee server_cookie=db52ac55341902ad in_seq=0 out_seq=0 2024-05-14T17:07:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76c8ff9640 1 -- 172.21.15.69:0/347606432 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560452d53360 con 0x560452e69fc0 2024-05-14T17:07:00.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76c8ff9640 10 client.17115 handle_command_reply: tid=0 2024-05-14T17:07:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 client.17115 shutdown 2024-05-14T17:07:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 -- 172.21.15.69:0/347606432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 msgr2=0x560452df5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df5150 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f76d800c4e0 tx=0x7f76d802d040 comp rx=0 tx=0).stop 2024-05-14T17:07:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 -- 172.21.15.69:0/347606432 shutdown_connections 2024-05-14T17:07:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560452e69fc0 0x560452e6c3e0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560452defad0 0x560452df5150 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76c8ff9640 0 client.17115 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560452de5210 0x560452df8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 --2- 172.21.15.69:0/347606432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560452de48f0 0x560452df7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 -- 172.21.15.69:0/347606432 >> 172.21.15.69:0/347606432 conn(0x560452de0e00 msgr2=0x560452dfaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 -- 172.21.15.69:0/347606432 shutdown_connections 2024-05-14T17:07:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 1 -- 172.21.15.69:0/347606432 wait complete. 2024-05-14T17:07:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.081+0000 7f76ee42e740 20 client.17115 trim_cache size 0 max 16384 2024-05-14T17:07:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:07:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:07:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:07:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 -- 172.21.15.69:0/724193127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76cc0b2a90 msgr2=0x7f76cc0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76cc0b2a90 0x7f76cc0b4f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0a6460 tx=0x7f76dc0046c0 comp rx=0 tx=0).stop 2024-05-14T17:07:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 -- 172.21.15.69:0/724193127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76e4106e90 msgr2=0x7f76e419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76e4106e90 0x7f76e419f640 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00d680 tx=0x7f76dc00db40 comp rx=0 tx=0).stop 2024-05-14T17:07:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 -- 172.21.15.69:0/724193127 shutdown_connections 2024-05-14T17:07:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f76cc0b2a90 0x7f76cc0b4f50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76e4106e90 0x7f76e419f640 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76e41023b0 0x7f76e41a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 --2- 172.21.15.69:0/724193127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76e4101a90 0x7f76e41a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 -- 172.21.15.69:0/724193127 >> 172.21.15.69:0/724193127 conn(0x7f76e4078b60 msgr2=0x7f76e4104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 -- 172.21.15.69:0/724193127 shutdown_connections 2024-05-14T17:07:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:29.082+0000 7f76eb714640 1 -- 172.21.15.69:0/724193127 wait complete. 2024-05-14T17:07:02.176 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:30 smithi069 ceph-mon[31507]: pgmap v525: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.1 MiB/s wr, 668 op/s 2024-05-14T17:07:02.225 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:32 smithi069 ceph-mon[31507]: pgmap v526: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-05-14T17:07:02.278 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:34 smithi069 ceph-mon[31507]: pgmap v527: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 4.6 MiB/s wr, 510 op/s 2024-05-14T17:07:02.325 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:36 smithi069 ceph-mon[31507]: pgmap v528: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 222 KiB/s wr, 32 op/s 2024-05-14T17:07:02.392 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:38 smithi069 ceph-mon[31507]: pgmap v529: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 226 KiB/s wr, 35 op/s 2024-05-14T17:07:02.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:39 smithi069 ceph-mon[31507]: pgmap v530: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-05-14T17:07:02.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:53 smithi203 ceph-mon[35787]: pgmap v537: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-05-14T17:07:02.542 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:55 smithi203 ceph-mon[35787]: pgmap v538: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 8 op/s 2024-05-14T17:07:02.592 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:57 smithi203 ceph-mon[35787]: pgmap v539: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-05-14T17:07:02.642 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:06:59 smithi203 ceph-mon[35787]: pgmap v540: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-05-14T17:07:02.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:01 smithi203 ceph-mon[35787]: pgmap v541: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-05-14T17:07:02.742 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:53 smithi092 ceph-mon[35763]: pgmap v537: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-05-14T17:07:02.792 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:55 smithi092 ceph-mon[35763]: pgmap v538: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 8 op/s 2024-05-14T17:07:02.842 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:57 smithi092 ceph-mon[35763]: pgmap v539: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-05-14T17:07:02.901 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:06:59 smithi092 ceph-mon[35763]: pgmap v540: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-05-14T17:07:02.951 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:01 smithi092 ceph-mon[35763]: pgmap v541: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-05-14T17:07:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.179+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2628785714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18f8074d50 msgr2=0x7f18f8075130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.179+0000 7f18ffaa0640 1 --2- 172.21.15.69:0/2628785714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18f8074d50 0x7f18f8075130 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009900 tx=0x7f18f402a480 comp rx=0 tx=0).stop 2024-05-14T17:07:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.179+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2628785714 shutdown_connections 2024-05-14T17:07:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.179+0000 7f18ffaa0640 1 --2- 172.21.15.69:0/2628785714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f8113b10 0x7f18f8115f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.179+0000 7f18ffaa0640 1 --2- 172.21.15.69:0/2628785714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f8075670 0x7f18f81135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.179+0000 7f18ffaa0640 1 --2- 172.21.15.69:0/2628785714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18f8074d50 0x7f18f8075130 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.179+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2628785714 >> 172.21.15.69:0/2628785714 conn(0x7f18f8079ee0 msgr2=0x7f18f807a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.179+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2628785714 shutdown_connections 2024-05-14T17:07:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.179+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2628785714 wait complete. 2024-05-14T17:07:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18ffaa0640 1 Processor -- start 2024-05-14T17:07:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18ffaa0640 1 -- start start 2024-05-14T17:07:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18ffaa0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18f8074d50 0x7f18f81a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18ffaa0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f8075670 0x7f18f81a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18ffaa0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f8113b10 0x7f18f819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18ffaa0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18f8119320 con 0x7f18f8074d50 2024-05-14T17:07:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18ffaa0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81191a0 con 0x7f18f8075670 2024-05-14T17:07:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18ffaa0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81194a0 con 0x7f18f8113b10 2024-05-14T17:07:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18fd83c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18f8074d50 0x7f18f81a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18fd83c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18f8074d50 0x7f18f81a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40686/0 (socket says 172.21.15.69:40686) 2024-05-14T17:07:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.180+0000 7f18fd83c640 1 -- 172.21.15.69:0/2307665822 learned_addr learned my addr 172.21.15.69:0/2307665822 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18fd03b640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f8075670 0x7f18f81a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18fe03d640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f8113b10 0x7f18f819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18fd83c640 1 -- 172.21.15.69:0/2307665822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f8113b10 msgr2=0x7f18f819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f8113b10 0x7f18f819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18fd83c640 1 -- 172.21.15.69:0/2307665822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f8075670 msgr2=0x7f18f81a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f8075670 0x7f18f81a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18fd83c640 1 -- 172.21.15.69:0/2307665822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f819fd60 con 0x7f18f8074d50 2024-05-14T17:07:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18f8074d50 0x7f18f81a5430 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f18f4006010 tx=0x7f18f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18eeffd640 1 -- 172.21.15.69:0/2307665822 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f403f030 con 0x7f18f8074d50 2024-05-14T17:07:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18eeffd640 1 -- 172.21.15.69:0/2307665822 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f4003cb0 con 0x7f18f8074d50 2024-05-14T17:07:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2307665822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f819fff0 con 0x7f18f8074d50 2024-05-14T17:07:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18eeffd640 1 -- 172.21.15.69:0/2307665822 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4035070 con 0x7f18f8074d50 2024-05-14T17:07:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.181+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2307665822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f81ac2f0 con 0x7f18f8074d50 2024-05-14T17:07:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.184+0000 7f18eeffd640 1 -- 172.21.15.69:0/2307665822 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f18f4042030 con 0x7f18f8074d50 2024-05-14T17:07:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.184+0000 7f19027c3740 1 Processor -- start 2024-05-14T17:07:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.184+0000 7f19027c3740 1 -- start start 2024-05-14T17:07:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f19027c3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51afdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f19027c3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51afe210 0x560d51b08590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f19027c3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51b08ad0 0x560d51b0aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f19027c3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560d51a70c30 con 0x560d51afd8f0 2024-05-14T17:07:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f19027c3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560d51a607d0 con 0x560d51b08ad0 2024-05-14T17:07:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f19027c3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560d51a64810 con 0x560d51afe210 2024-05-14T17:07:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fe03d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51afdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fe03d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51afdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40700/0 (socket says 172.21.15.69:40700) 2024-05-14T17:07:05.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fe03d640 1 -- 172.21.15.69:0/2323412406 learned_addr learned my addr 172.21.15.69:0/2323412406 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18eeffd640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f18e40b2a10 0x7f18e40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2323412406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51afe210 0x560d51b08590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fd03b640 1 --2- 172.21.15.69:0/2323412406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51b08ad0 0x560d51b0aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fe03d640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f18e40b2a10 0x7f18e40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fe03d640 1 -- 172.21.15.69:0/2323412406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51afe210 msgr2=0x560d51b08590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fe03d640 1 --2- 172.21.15.69:0/2323412406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51afe210 0x560d51b08590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fe03d640 1 -- 172.21.15.69:0/2323412406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51b08ad0 msgr2=0x560d51b0aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fe03d640 1 --2- 172.21.15.69:0/2323412406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51b08ad0 0x560d51b0aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fe03d640 1 -- 172.21.15.69:0/2323412406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d51775210 con 0x560d51afd8f0 2024-05-14T17:07:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2323412406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51afe210 0x560d51b08590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18eeffd640 1 -- 172.21.15.69:0/2307665822 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f18f4102a90 con 0x7f18f8074d50 2024-05-14T17:07:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.185+0000 7f18fe03d640 1 --2- 172.21.15.69:0/2323412406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51afdcd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f18f0014910 tx=0x7f18f0014dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59cf7e2d23c9eb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f18fe03d640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f18e40b2a10 0x7f18e40b4ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f18f001c000 tx=0x7f18f001e000 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f18e3fff640 1 -- 172.21.15.69:0/2323412406 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0014030 con 0x560d51afd8f0 2024-05-14T17:07:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f18e3fff640 1 -- 172.21.15.69:0/2323412406 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f000cca0 con 0x560d51afd8f0 2024-05-14T17:07:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f18e3fff640 1 -- 172.21.15.69:0/2323412406 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0010780 con 0x560d51afd8f0 2024-05-14T17:07:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f19027c3740 1 -- 172.21.15.69:0/2323412406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 msgr2=0x560d51afdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f19027c3740 1 --2- 172.21.15.69:0/2323412406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51afdcd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f18f0014910 tx=0x7f18f0014dd0 comp rx=0 tx=0).stop 2024-05-14T17:07:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f19027c3740 1 -- 172.21.15.69:0/2323412406 shutdown_connections 2024-05-14T17:07:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f19027c3740 1 --2- 172.21.15.69:0/2323412406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51b08ad0 0x560d51b0aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f19027c3740 1 --2- 172.21.15.69:0/2323412406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51afe210 0x560d51b08590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f19027c3740 1 --2- 172.21.15.69:0/2323412406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51afdcd0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.186+0000 7f19027c3740 1 -- 172.21.15.69:0/2323412406 >> 172.21.15.69:0/2323412406 conn(0x560d51af9e00 msgr2=0x560d51afa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 1 -- 172.21.15.69:0/2323412406 shutdown_connections 2024-05-14T17:07:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 1 -- 172.21.15.69:0/2323412406 wait complete. 2024-05-14T17:07:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 1 -- start start 2024-05-14T17:07:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 1 Processor -- start 2024-05-14T17:07:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afd8f0 0x560d51b08430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b05720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b05c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560d51746790 con 0x560d51afe210 2024-05-14T17:07:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560d51a1c890 con 0x560d51afd8f0 2024-05-14T17:07:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560d51b0b7a0 con 0x560d51b08ad0 2024-05-14T17:07:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.187+0000 7f19027c3740 10 client.0 _ll_register_callbacks cb 0x560d51a1fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fd03b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b05720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fd03b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b05720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40716/0 (socket says 172.21.15.69:40716) 2024-05-14T17:07:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fd03b640 1 -- 172.21.15.69:0/2252909727 learned_addr learned my addr 172.21.15.69:0/2252909727 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afd8f0 0x560d51b08430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fe03d640 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b05c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fd03b640 1 -- 172.21.15.69:0/2252909727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 msgr2=0x560d51b05c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fd03b640 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b05c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fd03b640 1 -- 172.21.15.69:0/2252909727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afd8f0 msgr2=0x560d51b08430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fd03b640 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afd8f0 0x560d51b08430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fd03b640 1 -- 172.21.15.69:0/2252909727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d51841620 con 0x560d51afe210 2024-05-14T17:07:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18fd03b640 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b05720 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f18e800b8f0 tx=0x7f18e800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeea3376680b445d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18e17fa640 1 -- 172.21.15.69:0/2252909727 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8003e20 con 0x560d51afe210 2024-05-14T17:07:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18e17fa640 1 -- 172.21.15.69:0/2252909727 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f18e8005020 con 0x560d51afe210 2024-05-14T17:07:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f19027c3740 1 -- 172.21.15.69:0/2252909727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f000cca0 con 0x560d51afe210 2024-05-14T17:07:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f18e17fa640 1 -- 172.21.15.69:0/2252909727 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f18e8005220 con 0x560d51afe210 2024-05-14T17:07:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f19027c3740 10 client.17127 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.188+0000 7f19027c3740 1 -- 172.21.15.69:0/2252909727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560d51b17290 con 0x560d51afe210 2024-05-14T17:07:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f18e17fa640 1 -- 172.21.15.69:0/2252909727 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f000cca0 con 0x560d51afe210 2024-05-14T17:07:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f18e17fa640 1 -- 172.21.15.69:0/2252909727 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f18e801b470 con 0x560d51afe210 2024-05-14T17:07:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f19027c3740 10 client.17127 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f19027c3740 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f19027c3740 20 client.17127 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f19027c3740 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560d51b187f0 0x560d51b1ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f19027c3740 4 client.17127 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f19027c3740 1 -- 172.21.15.69:0/2252909727 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d51a72b80 con 0x560d51b187f0 2024-05-14T17:07:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560d51b187f0 0x560d51b1ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f18e17fa640 10 client.17127 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.189+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560d51b187f0 0x560d51b1ac20 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593f313fa2795ea5 server_cookie=3f4b20573d44e404 in_seq=0 out_seq=0 2024-05-14T17:07:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.190+0000 7f18e17fa640 1 -- 172.21.15.69:0/2252909727 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560d51a72b80 con 0x560d51b187f0 2024-05-14T17:07:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.190+0000 7f18e17fa640 10 client.17127 handle_command_reply: tid=0 2024-05-14T17:07:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.190+0000 7f19027c3740 1 client.17127 shutdown 2024-05-14T17:07:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.190+0000 7f19027c3740 1 -- 172.21.15.69:0/2252909727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 msgr2=0x560d51b05720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.190+0000 7f19027c3740 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b05720 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f18e800b8f0 tx=0x7f18e800bdb0 comp rx=0 tx=0).stop 2024-05-14T17:07:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.191+0000 7f19027c3740 1 -- 172.21.15.69:0/2252909727 shutdown_connections 2024-05-14T17:07:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.191+0000 7f19027c3740 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560d51b187f0 0x560d51b1ac20 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.191+0000 7f19027c3740 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b05c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.191+0000 7f19027c3740 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b05720 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.191+0000 7f18e17fa640 0 client.17127 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.191+0000 7f19027c3740 1 --2- 172.21.15.69:0/2252909727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afd8f0 0x560d51b08430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.191+0000 7f19027c3740 1 -- 172.21.15.69:0/2252909727 >> 172.21.15.69:0/2252909727 conn(0x560d51af9e00 msgr2=0x560d51af9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.191+0000 7f19027c3740 1 -- 172.21.15.69:0/2252909727 shutdown_connections 2024-05-14T17:07:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.191+0000 7f19027c3740 1 -- 172.21.15.69:0/2252909727 wait complete. 2024-05-14T17:07:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.191+0000 7f19027c3740 20 client.17127 trim_cache size 0 max 16384 2024-05-14T17:07:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f19027c3740 1 Processor -- start 2024-05-14T17:07:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f19027c3740 1 -- start start 2024-05-14T17:07:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f19027c3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afd8f0 0x560d51b146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f19027c3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b07f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f19027c3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f19027c3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81b74f0 con 0x560d51afe210 2024-05-14T17:07:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f19027c3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81b4d90 con 0x560d51afd8f0 2024-05-14T17:07:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f19027c3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81b2c00 con 0x560d51b08ad0 2024-05-14T17:07:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f18fd03b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b07f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f18fd03b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b07f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40732/0 (socket says 172.21.15.69:40732) 2024-05-14T17:07:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f18fd03b640 1 -- 172.21.15.69:0/1840226185 learned_addr learned my addr 172.21.15.69:0/1840226185 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f18fd03b640 1 -- 172.21.15.69:0/1840226185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 msgr2=0x560d51b204f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f18fd03b640 1 --2- 172.21.15.69:0/1840226185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f18fd03b640 1 -- 172.21.15.69:0/1840226185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afd8f0 msgr2=0x560d51b146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f18fd03b640 1 --2- 172.21.15.69:0/1840226185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afd8f0 0x560d51b146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.206+0000 7f18fd03b640 1 -- 172.21.15.69:0/1840226185 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d51af9c30 con 0x560d51afe210 2024-05-14T17:07:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f18fd03b640 1 --2- 172.21.15.69:0/1840226185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b07f60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f18e801ba50 tx=0x7f18e8005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f281394239ab5f9c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f18e37fe640 1 -- 172.21.15.69:0/1840226185 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8031070 con 0x560d51afe210 2024-05-14T17:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f18e37fe640 1 -- 172.21.15.69:0/1840226185 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e801b470 con 0x560d51afe210 2024-05-14T17:07:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f18e37fe640 1 -- 172.21.15.69:0/1840226185 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8028040 con 0x560d51afe210 2024-05-14T17:07:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f19027c3740 1 -- 172.21.15.69:0/1840226185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 msgr2=0x560d51b07f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f19027c3740 1 --2- 172.21.15.69:0/1840226185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b07f60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f18e801ba50 tx=0x7f18e8005cd0 comp rx=0 tx=0).stop 2024-05-14T17:07:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f19027c3740 1 -- 172.21.15.69:0/1840226185 shutdown_connections 2024-05-14T17:07:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f19027c3740 1 --2- 172.21.15.69:0/1840226185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f19027c3740 1 --2- 172.21.15.69:0/1840226185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afe210 0x560d51b07f60 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f19027c3740 1 --2- 172.21.15.69:0/1840226185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afd8f0 0x560d51b146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.207+0000 7f19027c3740 1 -- 172.21.15.69:0/1840226185 >> 172.21.15.69:0/1840226185 conn(0x560d51af9e00 msgr2=0x560d51affb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 1 -- 172.21.15.69:0/1840226185 shutdown_connections 2024-05-14T17:07:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 1 -- 172.21.15.69:0/1840226185 wait complete. 2024-05-14T17:07:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 1 -- start start 2024-05-14T17:07:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 1 Processor -- start 2024-05-14T17:07:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51b0c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afe210 0x560d51b0cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b0d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81b3250 con 0x560d51afd8f0 2024-05-14T17:07:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81b46c0 con 0x560d51afe210 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81b3920 con 0x560d51b08ad0 2024-05-14T17:07:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f19027c3740 10 client.0 _ll_register_callbacks cb 0x7f18e8004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f18fd83c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51b0c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f18fd83c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51b0c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40738/0 (socket says 172.21.15.69:40738) 2024-05-14T17:07:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.208+0000 7f18fd83c640 1 -- 172.21.15.69:0/2495044977 learned_addr learned my addr 172.21.15.69:0/2495044977 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18fd03b640 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afe210 0x560d51b0cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18fe03d640 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b0d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18fd83c640 1 -- 172.21.15.69:0/2495044977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 msgr2=0x560d51b0d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b0d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18fd83c640 1 -- 172.21.15.69:0/2495044977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afe210 msgr2=0x560d51b0cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afe210 0x560d51b0cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18fd83c640 1 -- 172.21.15.69:0/2495044977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d51b820a0 con 0x560d51afd8f0 2024-05-14T17:07:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18fd03b640 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afe210 0x560d51b0cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18fe03d640 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b0d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:07:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18fd83c640 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51b0c600 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f18f4005c40 tx=0x7f18f40044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6749d250a4dccb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:07:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18e17fa640 1 -- 172.21.15.69:0/2495044977 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4049070 con 0x560d51afd8f0 2024-05-14T17:07:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18e17fa640 1 -- 172.21.15.69:0/2495044977 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f18f40c9fc0 con 0x560d51afd8f0 2024-05-14T17:07:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f19027c3740 1 -- 172.21.15.69:0/2495044977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e801b470 con 0x560d51afd8f0 2024-05-14T17:07:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18e17fa640 1 -- 172.21.15.69:0/2495044977 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f18f40c94d0 con 0x560d51afd8f0 2024-05-14T17:07:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f19027c3740 10 client.17133 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f19027c3740 1 -- 172.21.15.69:0/2495044977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560d51b833e0 con 0x560d51afd8f0 2024-05-14T17:07:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18e17fa640 1 -- 172.21.15.69:0/2495044977 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e801b470 con 0x560d51afd8f0 2024-05-14T17:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f18e17fa640 1 -- 172.21.15.69:0/2495044977 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f18f40d2730 con 0x560d51afd8f0 2024-05-14T17:07:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f19027c3740 10 client.17133 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.209+0000 7f19027c3740 10 client.17133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.210+0000 7f19027c3740 20 client.17133 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.210+0000 7f19027c3740 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560d51b83ab0 0x560d51b85ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.210+0000 7f19027c3740 4 client.17133 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.210+0000 7f19027c3740 1 -- 172.21.15.69:0/2495044977 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d51a6c360 con 0x560d51b83ab0 2024-05-14T17:07:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.210+0000 7f18fd03b640 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560d51b83ab0 0x560d51b85ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.210+0000 7f18e17fa640 10 client.17133 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.210+0000 7f18fd03b640 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560d51b83ab0 0x560d51b85ed0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bbad5357f1d563e server_cookie=eceeaf19209c2d65 in_seq=0 out_seq=0 2024-05-14T17:07:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18e17fa640 1 -- 172.21.15.69:0/2495044977 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560d51a6c360 con 0x560d51b83ab0 2024-05-14T17:07:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18e17fa640 10 client.17133 handle_command_reply: tid=0 2024-05-14T17:07:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 client.17133 shutdown 2024-05-14T17:07:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 -- 172.21.15.69:0/2495044977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 msgr2=0x560d51b0c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51b0c600 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f18f4005c40 tx=0x7f18f40044e0 comp rx=0 tx=0).stop 2024-05-14T17:07:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 -- 172.21.15.69:0/2495044977 shutdown_connections 2024-05-14T17:07:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560d51b83ab0 0x560d51b85ed0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d51b08ad0 0x560d51b0d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d51afe210 0x560d51b0cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18e17fa640 0 client.17133 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 --2- 172.21.15.69:0/2495044977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d51afd8f0 0x560d51b0c600 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 -- 172.21.15.69:0/2495044977 >> 172.21.15.69:0/2495044977 conn(0x560d51af9e00 msgr2=0x560d51b00410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 -- 172.21.15.69:0/2495044977 shutdown_connections 2024-05-14T17:07:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 1 -- 172.21.15.69:0/2495044977 wait complete. 2024-05-14T17:07:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f19027c3740 20 client.17133 trim_cache size 0 max 16384 2024-05-14T17:07:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:07:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:07:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:07:14.142 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:03 smithi203 ceph-mon[35787]: pgmap v542: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 KiB/s wr, 28 op/s 2024-05-14T17:07:14.182 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:05 smithi203 ceph-mon[35787]: pgmap v543: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-05-14T17:07:14.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:07 smithi203 ceph-mon[35787]: pgmap v544: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2024-05-14T17:07:14.274 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:09 smithi203 ceph-mon[35787]: pgmap v545: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 23 op/s 2024-05-14T17:07:14.315 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:09 smithi203 ceph-mon[35787]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T17:07:14.357 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:12 smithi203 ceph-mon[35787]: pgmap v547: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-05-14T17:07:14.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:03 smithi092 ceph-mon[35763]: pgmap v542: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 KiB/s wr, 28 op/s 2024-05-14T17:07:14.449 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:05 smithi092 ceph-mon[35763]: pgmap v543: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-05-14T17:07:14.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:07 smithi092 ceph-mon[35763]: pgmap v544: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2024-05-14T17:07:14.532 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:09 smithi092 ceph-mon[35763]: pgmap v545: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 23 op/s 2024-05-14T17:07:14.574 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:09 smithi092 ceph-mon[35763]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T17:07:14.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:12 smithi092 ceph-mon[35763]: pgmap v547: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-05-14T17:07:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2307665822 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f18e40b2a10 msgr2=0x7f18e40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18ffaa0640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f18e40b2a10 0x7f18e40b4ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f18f001c000 tx=0x7f18f001e000 comp rx=0 tx=0).stop 2024-05-14T17:07:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2307665822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18f8074d50 msgr2=0x7f18f81a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18ffaa0640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18f8074d50 0x7f18f81a5430 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f18f4006010 tx=0x7f18f40042f0 comp rx=0 tx=0).stop 2024-05-14T17:07:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2307665822 shutdown_connections 2024-05-14T17:07:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18ffaa0640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f18e40b2a10 0x7f18e40b4ed0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18ffaa0640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18f8113b10 0x7f18f819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18ffaa0640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f8075670 0x7f18f81a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18ffaa0640 1 --2- 172.21.15.69:0/2307665822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18f8074d50 0x7f18f81a5430 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.226+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2307665822 >> 172.21.15.69:0/2307665822 conn(0x7f18f8079ee0 msgr2=0x7f18f80ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.227+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2307665822 shutdown_connections 2024-05-14T17:07:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:42.227+0000 7f18ffaa0640 1 -- 172.21.15.69:0/2307665822 wait complete. 2024-05-14T17:07:15.184 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:41 smithi069 ceph-mon[31507]: pgmap v531: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:07:15.224 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:43 smithi069 ceph-mon[31507]: pgmap v532: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-05-14T17:07:15.266 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:45 smithi069 ceph-mon[31507]: pgmap v533: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-05-14T17:07:15.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:47 smithi069 ceph-mon[31507]: pgmap v534: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-14T17:07:15.349 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:49 smithi069 ceph-mon[31507]: pgmap v535: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-05-14T17:07:15.391 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:51 smithi069 ceph-mon[31507]: pgmap v536: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-05-14T17:07:15.433 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:53 smithi069 ceph-mon[31507]: pgmap v537: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-05-14T17:07:15.485 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:14 smithi203 ceph-mon[35787]: pgmap v548: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:15.524 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:14 smithi203 ceph-mon[35787]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T17:07:15.566 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:14 smithi092 ceph-mon[35763]: pgmap v548: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:15.616 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:14 smithi092 ceph-mon[35763]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T17:07:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.321+0000 7f126cfb7640 1 -- 172.21.15.69:0/3070319874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12681022f0 msgr2=0x7f126810e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.321+0000 7f126cfb7640 1 --2- 172.21.15.69:0/3070319874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12681022f0 0x7f126810e800 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1260009cd0 tx=0x7f126002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:07:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.321+0000 7f126cfb7640 1 -- 172.21.15.69:0/3070319874 shutdown_connections 2024-05-14T17:07:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.321+0000 7f126cfb7640 1 --2- 172.21.15.69:0/3070319874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12681022f0 0x7f126810e800 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.321+0000 7f126cfb7640 1 --2- 172.21.15.69:0/3070319874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1268101930 0x7f1268101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.321+0000 7f126cfb7640 1 --2- 172.21.15.69:0/3070319874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1268107940 0x7f1268107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.321+0000 7f126cfb7640 1 -- 172.21.15.69:0/3070319874 >> 172.21.15.69:0/3070319874 conn(0x7f1268077c90 msgr2=0x7f12680780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.321+0000 7f126cfb7640 1 -- 172.21.15.69:0/3070319874 shutdown_connections 2024-05-14T17:07:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.321+0000 7f126cfb7640 1 -- 172.21.15.69:0/3070319874 wait complete. 2024-05-14T17:07:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f126cfb7640 1 Processor -- start 2024-05-14T17:07:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f126cfb7640 1 -- start start 2024-05-14T17:07:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f126cfb7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1268101930 0x7f1268198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f126cfb7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12681022f0 0x7f1268198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f126cfb7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1268107940 0x7f126819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f126cfb7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1268111c00 con 0x7f1268107940 2024-05-14T17:07:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f126cfb7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1268111a80 con 0x7f1268101930 2024-05-14T17:07:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f126cfb7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1268111d80 con 0x7f12681022f0 2024-05-14T17:07:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f1266d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1268107940 0x7f126819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f1266d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1268107940 0x7f126819c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46320/0 (socket says 172.21.15.69:46320) 2024-05-14T17:07:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f1266d9d640 1 -- 172.21.15.69:0/647474891 learned_addr learned my addr 172.21.15.69:0/647474891 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f126659c640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1268101930 0x7f1268198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.322+0000 7f1265d9b640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12681022f0 0x7f1268198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f1266d9d640 1 -- 172.21.15.69:0/647474891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12681022f0 msgr2=0x7f1268198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f1266d9d640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12681022f0 0x7f1268198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f1266d9d640 1 -- 172.21.15.69:0/647474891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1268101930 msgr2=0x7f1268198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f1266d9d640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1268101930 0x7f1268198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f1266d9d640 1 -- 172.21.15.69:0/647474891 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126819d0a0 con 0x7f1268107940 2024-05-14T17:07:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f1266d9d640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1268107940 0x7f126819c970 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1260005cc0 tx=0x7f1260003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:17.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f125f7fe640 1 -- 172.21.15.69:0/647474891 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260004240 con 0x7f1268107940 2024-05-14T17:07:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f125f7fe640 1 -- 172.21.15.69:0/647474891 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12600305a0 con 0x7f1268107940 2024-05-14T17:07:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f126cfb7640 1 -- 172.21.15.69:0/647474891 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126819d330 con 0x7f1268107940 2024-05-14T17:07:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f125f7fe640 1 -- 172.21.15.69:0/647474891 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600466c0 con 0x7f1268107940 2024-05-14T17:07:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.323+0000 7f126cfb7640 1 -- 172.21.15.69:0/647474891 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12681a4c20 con 0x7f1268107940 2024-05-14T17:07:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.326+0000 7f126fda8740 1 Processor -- start 2024-05-14T17:07:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.326+0000 7f125f7fe640 1 -- 172.21.15.69:0/647474891 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1260030a50 con 0x7f1268107940 2024-05-14T17:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f126fda8740 1 -- start start 2024-05-14T17:07:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f126fda8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fe47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f126fda8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896fe4d80 0x55a896fe5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f126fda8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896ff3c20 0x55a896ff6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f126fda8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a896f57c30 con 0x55a896fe5fa0 2024-05-14T17:07:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f126fda8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a896f477d0 con 0x55a896ff3c20 2024-05-14T17:07:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f126fda8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a896f4b810 con 0x55a896fe4d80 2024-05-14T17:07:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f1265d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fe47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f1265d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fe47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46336/0 (socket says 172.21.15.69:46336) 2024-05-14T17:07:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f1265d9b640 1 -- 172.21.15.69:0/3442790814 learned_addr learned my addr 172.21.15.69:0/3442790814 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f126659c640 1 --2- 172.21.15.69:0/3442790814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896fe4d80 0x55a896fe5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f1265d9b640 1 -- 172.21.15.69:0/3442790814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896fe4d80 msgr2=0x55a896fe5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f1265d9b640 1 --2- 172.21.15.69:0/3442790814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896fe4d80 0x55a896fe5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f1265d9b640 1 -- 172.21.15.69:0/3442790814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896ff3c20 msgr2=0x55a896ff6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f1265d9b640 1 --2- 172.21.15.69:0/3442790814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896ff3c20 0x55a896ff6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f1265d9b640 1 -- 172.21.15.69:0/3442790814 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a896c5c210 con 0x55a896fe5fa0 2024-05-14T17:07:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f125f7fe640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f12540b2a10 0x7f12540b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f1265d9b640 1 --2- 172.21.15.69:0/3442790814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fe47b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1258009ae0 tx=0x7f12580101f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f757daebf46fdc11 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.327+0000 7f126659c640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f12540b2a10 0x7f12540b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f125f7fe640 1 -- 172.21.15.69:0/647474891 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f126002e030 con 0x7f1268107940 2024-05-14T17:07:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f125cff9640 1 -- 172.21.15.69:0/3442790814 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258018020 con 0x55a896fe5fa0 2024-05-14T17:07:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f125cff9640 1 -- 172.21.15.69:0/3442790814 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258010dd0 con 0x55a896fe5fa0 2024-05-14T17:07:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f125cff9640 1 -- 172.21.15.69:0/3442790814 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258013710 con 0x55a896fe5fa0 2024-05-14T17:07:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f126fda8740 1 -- 172.21.15.69:0/3442790814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 msgr2=0x55a896fe47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f126659c640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f12540b2a10 0x7f12540b4ed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55a896fe74e0 tx=0x7f125000a2d0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f126fda8740 1 --2- 172.21.15.69:0/3442790814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fe47b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1258009ae0 tx=0x7f12580101f0 comp rx=0 tx=0).stop 2024-05-14T17:07:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f126fda8740 1 -- 172.21.15.69:0/3442790814 shutdown_connections 2024-05-14T17:07:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f126fda8740 1 --2- 172.21.15.69:0/3442790814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896ff3c20 0x55a896ff6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f126fda8740 1 --2- 172.21.15.69:0/3442790814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896fe4d80 0x55a896fe5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f126fda8740 1 --2- 172.21.15.69:0/3442790814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fe47b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.328+0000 7f126fda8740 1 -- 172.21.15.69:0/3442790814 >> 172.21.15.69:0/3442790814 conn(0x55a896fe0e00 msgr2=0x55a896fe11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:18.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 1 -- 172.21.15.69:0/3442790814 shutdown_connections 2024-05-14T17:07:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 1 -- 172.21.15.69:0/3442790814 wait complete. 2024-05-14T17:07:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 1 -- start start 2024-05-14T17:07:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 1 Processor -- start 2024-05-14T17:07:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe4d80 0x55a896ff35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe5fa0 0x55a896ff0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896ff3c20 0x55a896ff0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a896c2d790 con 0x55a896fe4d80 2024-05-14T17:07:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a896f03890 con 0x55a896fe5fa0 2024-05-14T17:07:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a896ff6b50 con 0x55a896ff3c20 2024-05-14T17:07:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126fda8740 10 client.0 _ll_register_callbacks cb 0x55a896f06d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe4d80 0x55a896ff35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe4d80 0x55a896ff35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46342/0 (socket says 172.21.15.69:46342) 2024-05-14T17:07:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f126659c640 1 -- 172.21.15.69:0/4141402636 learned_addr learned my addr 172.21.15.69:0/4141402636 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.329+0000 7f1265d9b640 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe5fa0 0x55a896ff0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f1266d9d640 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896ff3c20 0x55a896ff0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f126659c640 1 -- 172.21.15.69:0/4141402636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896ff3c20 msgr2=0x55a896ff0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:19.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f126659c640 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896ff3c20 0x55a896ff0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f126659c640 1 -- 172.21.15.69:0/4141402636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe5fa0 msgr2=0x55a896ff0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f126659c640 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe5fa0 0x55a896ff0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f126659c640 1 -- 172.21.15.69:0/4141402636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a896d28620 con 0x55a896fe4d80 2024-05-14T17:07:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f1266d9d640 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896ff3c20 0x55a896ff0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:07:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f126659c640 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe4d80 0x55a896ff35a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1250010ec0 tx=0x7f12500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02a950b56970bc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f124dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:07:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f124dffb640 1 -- 172.21.15.69:0/4141402636 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125002a060 con 0x55a896fe4d80 2024-05-14T17:07:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f126fda8740 1 -- 172.21.15.69:0/4141402636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1258010dd0 con 0x55a896fe4d80 2024-05-14T17:07:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f124dffb640 1 -- 172.21.15.69:0/4141402636 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f125001b460 con 0x55a896fe4d80 2024-05-14T17:07:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f124dffb640 1 -- 172.21.15.69:0/4141402636 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f125001b660 con 0x55a896fe4d80 2024-05-14T17:07:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f126fda8740 10 client.17145 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f126fda8740 1 -- 172.21.15.69:0/4141402636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55a897002740 con 0x55a896fe4d80 2024-05-14T17:07:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f124dffb640 1 -- 172.21.15.69:0/4141402636 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258010dd0 con 0x55a896fe4d80 2024-05-14T17:07:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f124dffb640 1 -- 172.21.15.69:0/4141402636 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f1250019040 con 0x55a896fe4d80 2024-05-14T17:07:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.330+0000 7f126fda8740 10 client.17145 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.331+0000 7f126fda8740 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.331+0000 7f126fda8740 20 client.17145 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.331+0000 7f126fda8740 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a897003c50 0x55a897006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.331+0000 7f126fda8740 4 client.17145 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.331+0000 7f126fda8740 1 -- 172.21.15.69:0/4141402636 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a896f59b80 con 0x55a897003c50 2024-05-14T17:07:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.331+0000 7f1265d9b640 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a897003c50 0x55a897006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.331+0000 7f1265d9b640 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a897003c50 0x55a897006040 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ced6ae67a8d314 server_cookie=5cf6f6353f6233a in_seq=0 out_seq=0 2024-05-14T17:07:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.331+0000 7f124dffb640 10 client.17145 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f124dffb640 1 -- 172.21.15.69:0/4141402636 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a896f59b80 con 0x55a897003c50 2024-05-14T17:07:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f124dffb640 10 client.17145 handle_command_reply: tid=0 2024-05-14T17:07:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 client.17145 shutdown 2024-05-14T17:07:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 -- 172.21.15.69:0/4141402636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe4d80 msgr2=0x55a896ff35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe4d80 0x55a896ff35a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1250010ec0 tx=0x7f12500046c0 comp rx=0 tx=0).stop 2024-05-14T17:07:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 -- 172.21.15.69:0/4141402636 shutdown_connections 2024-05-14T17:07:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a897003c50 0x55a897006040 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896ff3c20 0x55a896ff0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe5fa0 0x55a896ff0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 --2- 172.21.15.69:0/4141402636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe4d80 0x55a896ff35a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f124dffb640 0 client.17145 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 -- 172.21.15.69:0/4141402636 >> 172.21.15.69:0/4141402636 conn(0x55a896fe0e00 msgr2=0x55a896fe6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 -- 172.21.15.69:0/4141402636 shutdown_connections 2024-05-14T17:07:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 1 -- 172.21.15.69:0/4141402636 wait complete. 2024-05-14T17:07:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.332+0000 7f126fda8740 20 client.17145 trim_cache size 0 max 16384 2024-05-14T17:07:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f126fda8740 1 Processor -- start 2024-05-14T17:07:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f126fda8740 1 -- start start 2024-05-14T17:07:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f126fda8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe4d80 0x55a896fe0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f126fda8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fffa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f126fda8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896ff3c20 0x55a896ff30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f126fda8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12681aece0 con 0x55a896fe5fa0 2024-05-14T17:07:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f126fda8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f12681ac580 con 0x55a896fe4d80 2024-05-14T17:07:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f126fda8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f12681aa3f0 con 0x55a896ff3c20 2024-05-14T17:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f1265d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f1265d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46354/0 (socket says 172.21.15.69:46354) 2024-05-14T17:07:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f1265d9b640 1 -- 172.21.15.69:0/2248149201 learned_addr learned my addr 172.21.15.69:0/2248149201 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.347+0000 7f126659c640 1 --2- 172.21.15.69:0/2248149201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe4d80 0x55a896fe0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f1265d9b640 1 -- 172.21.15.69:0/2248149201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896ff3c20 msgr2=0x55a896ff30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f1265d9b640 1 --2- 172.21.15.69:0/2248149201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896ff3c20 0x55a896ff30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f1265d9b640 1 -- 172.21.15.69:0/2248149201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe4d80 msgr2=0x55a896fe0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f1265d9b640 1 --2- 172.21.15.69:0/2248149201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe4d80 0x55a896fe0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f1265d9b640 1 -- 172.21.15.69:0/2248149201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a896feede0 con 0x55a896fe5fa0 2024-05-14T17:07:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f1265d9b640 1 --2- 172.21.15.69:0/2248149201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fffa90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55a896fdbfd0 tx=0x7f1258020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304afb8ed467992c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f124ffff640 1 -- 172.21.15.69:0/2248149201 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258018040 con 0x55a896fe5fa0 2024-05-14T17:07:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f124ffff640 1 -- 172.21.15.69:0/2248149201 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258009590 con 0x55a896fe5fa0 2024-05-14T17:07:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f124ffff640 1 -- 172.21.15.69:0/2248149201 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125802c7d0 con 0x55a896fe5fa0 2024-05-14T17:07:21.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f126fda8740 1 -- 172.21.15.69:0/2248149201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 msgr2=0x55a896fffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f126fda8740 1 --2- 172.21.15.69:0/2248149201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fffa90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55a896fdbfd0 tx=0x7f1258020040 comp rx=0 tx=0).stop 2024-05-14T17:07:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f126fda8740 1 -- 172.21.15.69:0/2248149201 shutdown_connections 2024-05-14T17:07:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f126fda8740 1 --2- 172.21.15.69:0/2248149201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896ff3c20 0x55a896ff30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f126fda8740 1 --2- 172.21.15.69:0/2248149201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896fe5fa0 0x55a896fffa90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f126fda8740 1 --2- 172.21.15.69:0/2248149201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe4d80 0x55a896fe0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.348+0000 7f126fda8740 1 -- 172.21.15.69:0/2248149201 >> 172.21.15.69:0/2248149201 conn(0x55a896fe0e00 msgr2=0x55a896ff5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 1 -- 172.21.15.69:0/2248149201 shutdown_connections 2024-05-14T17:07:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 1 -- 172.21.15.69:0/2248149201 wait complete. 2024-05-14T17:07:21.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 1 -- start start 2024-05-14T17:07:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 1 Processor -- start 2024-05-14T17:07:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe4d80 0x55a896ffbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896fe5fa0 0x55a896ffc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896ff3c20 0x55a89700b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12681aaa40 con 0x55a896ff3c20 2024-05-14T17:07:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f12681abeb0 con 0x55a896fe4d80 2024-05-14T17:07:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f12681ab110 con 0x55a896fe5fa0 2024-05-14T17:07:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126fda8740 10 client.0 _ll_register_callbacks cb 0x7f1250010860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f1266d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896ff3c20 0x55a89700b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f1266d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896ff3c20 0x55a89700b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46364/0 (socket says 172.21.15.69:46364) 2024-05-14T17:07:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f1266d9d640 1 -- 172.21.15.69:0/1735291979 learned_addr learned my addr 172.21.15.69:0/1735291979 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.349+0000 7f126659c640 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe4d80 0x55a896ffbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f1266d9d640 1 -- 172.21.15.69:0/1735291979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896fe5fa0 msgr2=0x55a896ffc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f1266d9d640 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896fe5fa0 0x55a896ffc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f1266d9d640 1 -- 172.21.15.69:0/1735291979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe4d80 msgr2=0x55a896ffbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f1266d9d640 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe4d80 0x55a896ffbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f1266d9d640 1 -- 172.21.15.69:0/1735291979 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a89700bc60 con 0x55a896ff3c20 2024-05-14T17:07:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f1266d9d640 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896ff3c20 0x55a89700b620 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f12600c8060 tx=0x7f12600caa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902811adaca34410 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f124dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:07:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f124dffb640 1 -- 172.21.15.69:0/1735291979 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600cf1a0 con 0x55a896ff3c20 2024-05-14T17:07:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f124dffb640 1 -- 172.21.15.69:0/1735291979 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f12600ca050 con 0x55a896ff3c20 2024-05-14T17:07:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f126fda8740 1 -- 172.21.15.69:0/1735291979 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1258009590 con 0x55a896ff3c20 2024-05-14T17:07:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f124dffb640 1 -- 172.21.15.69:0/1735291979 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f12600d3c50 con 0x55a896ff3c20 2024-05-14T17:07:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f126fda8740 10 client.17151 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f126fda8740 1 -- 172.21.15.69:0/1735291979 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55a896fe7f30 con 0x55a896ff3c20 2024-05-14T17:07:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f124dffb640 1 -- 172.21.15.69:0/1735291979 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258009590 con 0x55a896ff3c20 2024-05-14T17:07:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f124dffb640 1 -- 172.21.15.69:0/1735291979 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f1260030070 con 0x55a896ff3c20 2024-05-14T17:07:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f126fda8740 10 client.17151 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f126fda8740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f126fda8740 20 client.17151 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f126fda8740 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a89706f660 0x55a897071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f126fda8740 4 client.17151 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.350+0000 7f126fda8740 1 -- 172.21.15.69:0/1735291979 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a896f53360 con 0x55a89706f660 2024-05-14T17:07:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.351+0000 7f126659c640 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a89706f660 0x55a897071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.351+0000 7f124dffb640 10 client.17151 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.351+0000 7f126659c640 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a89706f660 0x55a897071a80 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4adc3af655d68224 server_cookie=4030af49f7c6902f in_seq=0 out_seq=0 2024-05-14T17:07:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f124dffb640 1 -- 172.21.15.69:0/1735291979 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a896f53360 con 0x55a89706f660 2024-05-14T17:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f124dffb640 10 client.17151 handle_command_reply: tid=0 2024-05-14T17:07:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 client.17151 shutdown 2024-05-14T17:07:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 -- 172.21.15.69:0/1735291979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896ff3c20 msgr2=0x55a89700b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896ff3c20 0x55a89700b620 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f12600c8060 tx=0x7f12600caa40 comp rx=0 tx=0).stop 2024-05-14T17:07:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 -- 172.21.15.69:0/1735291979 shutdown_connections 2024-05-14T17:07:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a89706f660 0x55a897071a80 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a896ff3c20 0x55a89700b620 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a896fe5fa0 0x55a896ffc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f124dffb640 0 client.17151 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 --2- 172.21.15.69:0/1735291979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a896fe4d80 0x55a896ffbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 -- 172.21.15.69:0/1735291979 >> 172.21.15.69:0/1735291979 conn(0x55a896fe0e00 msgr2=0x55a896ff4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 -- 172.21.15.69:0/1735291979 shutdown_connections 2024-05-14T17:07:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.363+0000 7f126fda8740 1 -- 172.21.15.69:0/1735291979 wait complete. 2024-05-14T17:07:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126fda8740 20 client.17151 trim_cache size 0 max 16384 2024-05-14T17:07:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:07:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:07:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:07:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 -- 172.21.15.69:0/647474891 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f12540b2a10 msgr2=0x7f12540b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f12540b2a10 0x7f12540b4ed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55a896fe74e0 tx=0x7f125000a2d0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 -- 172.21.15.69:0/647474891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1268107940 msgr2=0x7f126819c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1268107940 0x7f126819c970 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1260005cc0 tx=0x7f1260003ba0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 -- 172.21.15.69:0/647474891 shutdown_connections 2024-05-14T17:07:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f12540b2a10 0x7f12540b4ed0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1268107940 0x7f126819c970 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12681022f0 0x7f1268198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 --2- 172.21.15.69:0/647474891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1268101930 0x7f1268198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 -- 172.21.15.69:0/647474891 >> 172.21.15.69:0/647474891 conn(0x7f1268077c90 msgr2=0x7f12680ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 -- 172.21.15.69:0/647474891 shutdown_connections 2024-05-14T17:07:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:06:55.364+0000 7f126cfb7640 1 -- 172.21.15.69:0/647474891 wait complete. 2024-05-14T17:07:24.133 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:55 smithi069 ceph-mon[31507]: pgmap v538: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 8 op/s 2024-05-14T17:07:24.166 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:57 smithi069 ceph-mon[31507]: pgmap v539: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-05-14T17:07:24.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:06:59 smithi069 ceph-mon[31507]: pgmap v540: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-05-14T17:07:24.251 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:01 smithi069 ceph-mon[31507]: pgmap v541: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-05-14T17:07:24.307 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:03 smithi069 ceph-mon[31507]: pgmap v542: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 KiB/s wr, 28 op/s 2024-05-14T17:07:24.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:05 smithi069 ceph-mon[31507]: pgmap v543: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-05-14T17:07:24.388 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:07 smithi069 ceph-mon[31507]: pgmap v544: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2024-05-14T17:07:24.427 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:16 smithi203 ceph-mon[35787]: pgmap v550: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-05-14T17:07:24.457 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:18 smithi203 ceph-mon[35787]: pgmap v551: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:07:24.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:18 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/20720146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:07:24.524 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:20 smithi203 ceph-mon[35787]: pgmap v552: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:24.558 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:22 smithi203 ceph-mon[35787]: pgmap v553: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:24.591 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:16 smithi092 ceph-mon[35763]: pgmap v550: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-05-14T17:07:24.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:18 smithi092 ceph-mon[35763]: pgmap v551: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:07:24.683 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:18 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/20720146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:07:24.716 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:20 smithi092 ceph-mon[35763]: pgmap v552: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:24.749 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:22 smithi092 ceph-mon[35763]: pgmap v553: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:24.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:24 smithi203 ceph-mon[35787]: pgmap v554: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-05-14T17:07:24.816 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:24 smithi092 ceph-mon[35763]: pgmap v554: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-05-14T17:07:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.462+0000 7f6c48abd640 1 -- 172.21.15.69:0/3979798405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c44107940 msgr2=0x7f6c44107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.462+0000 7f6c48abd640 1 --2- 172.21.15.69:0/3979798405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c44107940 0x7f6c44107d20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009930 tx=0x7f6c3402a420 comp rx=0 tx=0).stop 2024-05-14T17:07:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.462+0000 7f6c48abd640 1 -- 172.21.15.69:0/3979798405 shutdown_connections 2024-05-14T17:07:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.462+0000 7f6c48abd640 1 --2- 172.21.15.69:0/3979798405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c44102340 0x7f6c4410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.462+0000 7f6c48abd640 1 --2- 172.21.15.69:0/3979798405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c44101980 0x7f6c44101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.462+0000 7f6c48abd640 1 --2- 172.21.15.69:0/3979798405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c44107940 0x7f6c44107d20 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.462+0000 7f6c48abd640 1 -- 172.21.15.69:0/3979798405 >> 172.21.15.69:0/3979798405 conn(0x7f6c44078ad0 msgr2=0x7f6c44078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.462+0000 7f6c48abd640 1 -- 172.21.15.69:0/3979798405 shutdown_connections 2024-05-14T17:07:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.462+0000 7f6c48abd640 1 -- 172.21.15.69:0/3979798405 wait complete. 2024-05-14T17:07:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.463+0000 7f6c48abd640 1 Processor -- start 2024-05-14T17:07:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.463+0000 7f6c48abd640 1 -- start start 2024-05-14T17:07:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.463+0000 7f6c48abd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c44101980 0x7f6c4419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.463+0000 7f6c48abd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c44102340 0x7f6c4419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.463+0000 7f6c48abd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c44107940 0x7f6c441a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.463+0000 7f6c48abd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44111c00 con 0x7f6c44101980 2024-05-14T17:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.463+0000 7f6c48abd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44111a80 con 0x7f6c44107940 2024-05-14T17:07:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.463+0000 7f6c48abd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44111d80 con 0x7f6c44102340 2024-05-14T17:07:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c44101980 0x7f6c4419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c44101980 0x7f6c4419a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37376/0 (socket says 172.21.15.69:37376) 2024-05-14T17:07:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c4259c640 1 -- 172.21.15.69:0/1652801759 learned_addr learned my addr 172.21.15.69:0/1652801759 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c42d9d640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c44107940 0x7f6c441a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c3bfff640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c44102340 0x7f6c4419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c4259c640 1 -- 172.21.15.69:0/1652801759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c44102340 msgr2=0x7f6c4419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c4259c640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c44102340 0x7f6c4419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:25.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c4259c640 1 -- 172.21.15.69:0/1652801759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c44107940 msgr2=0x7f6c441a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c4259c640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c44107940 0x7f6c441a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c4259c640 1 -- 172.21.15.69:0/1652801759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c441a5620 con 0x7f6c44101980 2024-05-14T17:07:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c4259c640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c44101980 0x7f6c4419a060 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6c34004770 tx=0x7f6c340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c3b7fe640 1 -- 172.21.15.69:0/1652801759 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3403f070 con 0x7f6c44101980 2024-05-14T17:07:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c3b7fe640 1 -- 172.21.15.69:0/1652801759 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3403b460 con 0x7f6c44101980 2024-05-14T17:07:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c3b7fe640 1 -- 172.21.15.69:0/1652801759 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34035070 con 0x7f6c44101980 2024-05-14T17:07:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c48abd640 1 -- 172.21.15.69:0/1652801759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c441a57f0 con 0x7f6c44101980 2024-05-14T17:07:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.464+0000 7f6c48abd640 1 -- 172.21.15.69:0/1652801759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c441a5bf0 con 0x7f6c44101980 2024-05-14T17:07:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.467+0000 7f6c3b7fe640 1 -- 172.21.15.69:0/1652801759 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6c3403b8d0 con 0x7f6c44101980 2024-05-14T17:07:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.467+0000 7f6c4b7d3740 1 Processor -- start 2024-05-14T17:07:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.467+0000 7f6c4b7d3740 1 -- start start 2024-05-14T17:07:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.467+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521560cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.467+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x56452156b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:26.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.467+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x56452156dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:26.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.467+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5645214d3c30 con 0x5645215608f0 2024-05-14T17:07:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.467+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5645214c37d0 con 0x564521561210 2024-05-14T17:07:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.467+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5645214c7810 con 0x56452156bad0 2024-05-14T17:07:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c42d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521560cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c42d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521560cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37392/0 (socket says 172.21.15.69:37392) 2024-05-14T17:07:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c42d9d640 1 -- 172.21.15.69:0/1083297957 learned_addr learned my addr 172.21.15.69:0/1083297957 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c4259c640 1 --2- 172.21.15.69:0/1083297957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x56452156b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c3bfff640 1 --2- 172.21.15.69:0/1083297957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x56452156dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c3b7fe640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6c300b2a10 0x7f6c300b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c42d9d640 1 -- 172.21.15.69:0/1083297957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 msgr2=0x56452156dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c42d9d640 1 --2- 172.21.15.69:0/1083297957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x56452156dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c42d9d640 1 -- 172.21.15.69:0/1083297957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 msgr2=0x56452156b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c42d9d640 1 --2- 172.21.15.69:0/1083297957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x56452156b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c42d9d640 1 -- 172.21.15.69:0/1083297957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645211d8210 con 0x5645215608f0 2024-05-14T17:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c3b7fe640 1 -- 172.21.15.69:0/1652801759 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f6c341029a0 con 0x7f6c44101980 2024-05-14T17:07:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c42d9d640 1 --2- 172.21.15.69:0/1083297957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521560cd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00a430 tx=0x7f6c3c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae6525bc2b95040 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c42d9d640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6c300b2a10 0x7f6c300b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c38ff9640 1 -- 172.21.15.69:0/1083297957 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c010830 con 0x5645215608f0 2024-05-14T17:07:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c38ff9640 1 -- 172.21.15.69:0/1083297957 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3c010d30 con 0x5645215608f0 2024-05-14T17:07:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.468+0000 7f6c38ff9640 1 -- 172.21.15.69:0/1083297957 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c020830 con 0x5645215608f0 2024-05-14T17:07:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.469+0000 7f6c42d9d640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6c300b2a10 0x7f6c300b4ed0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c019000 tx=0x7f6c3c013380 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.469+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/1083297957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 msgr2=0x564521560cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.469+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/1083297957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521560cd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00a430 tx=0x7f6c3c009650 comp rx=0 tx=0).stop 2024-05-14T17:07:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.469+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/1083297957 shutdown_connections 2024-05-14T17:07:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.469+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/1083297957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x56452156dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.469+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/1083297957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x56452156b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.469+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/1083297957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521560cd0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.469+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/1083297957 >> 172.21.15.69:0/1083297957 conn(0x56452155ce00 msgr2=0x56452155d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.469+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/1083297957 shutdown_connections 2024-05-14T17:07:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.469+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/1083297957 wait complete. 2024-05-14T17:07:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4b7d3740 1 -- start start 2024-05-14T17:07:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4b7d3740 1 Processor -- start 2024-05-14T17:07:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x56452156b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x564521568730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x564521568c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5645211a9790 con 0x5645215608f0 2024-05-14T17:07:27.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56452147f890 con 0x564521561210 2024-05-14T17:07:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56452156e7a0 con 0x56452156bad0 2024-05-14T17:07:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4b7d3740 10 client.0 _ll_register_callbacks cb 0x564521482d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x56452156b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x56452156b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37402/0 (socket says 172.21.15.69:37402) 2024-05-14T17:07:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4259c640 1 -- 172.21.15.69:0/689332821 learned_addr learned my addr 172.21.15.69:0/689332821 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c3bfff640 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x564521568730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c42d9d640 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x564521568c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4259c640 1 -- 172.21.15.69:0/689332821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 msgr2=0x564521568c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4259c640 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x564521568c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4259c640 1 -- 172.21.15.69:0/689332821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 msgr2=0x564521568730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4259c640 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x564521568730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.470+0000 7f6c4259c640 1 -- 172.21.15.69:0/689332821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645212a4620 con 0x5645215608f0 2024-05-14T17:07:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c4259c640 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x56452156b440 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6c34005940 tx=0x7f6c3403b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18e7c875cec6b0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:07:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c29ffb640 1 -- 172.21.15.69:0/689332821 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34048070 con 0x5645215608f0 2024-05-14T17:07:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c29ffb640 1 -- 172.21.15.69:0/689332821 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f6c340c5f00 con 0x5645215608f0 2024-05-14T17:07:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/689332821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3c010d30 con 0x5645215608f0 2024-05-14T17:07:28.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c29ffb640 1 -- 172.21.15.69:0/689332821 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6c340c95c0 con 0x5645215608f0 2024-05-14T17:07:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c4b7d3740 10 client.17163 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/689332821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x56452157a1c0 con 0x5645215608f0 2024-05-14T17:07:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c29ffb640 1 -- 172.21.15.69:0/689332821 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3c010d30 con 0x5645215608f0 2024-05-14T17:07:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c29ffb640 1 -- 172.21.15.69:0/689332821 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f6c3404b060 con 0x5645215608f0 2024-05-14T17:07:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c4b7d3740 10 client.17163 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c4b7d3740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c4b7d3740 20 client.17163 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56452157b620 0x56452157da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c4b7d3740 4 client.17163 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.471+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/689332821 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645214d5b80 con 0x56452157b620 2024-05-14T17:07:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.472+0000 7f6c3bfff640 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56452157b620 0x56452157da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.472+0000 7f6c29ffb640 10 client.17163 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.472+0000 7f6c3bfff640 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56452157b620 0x56452157da50 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c7c6a6f468ef44 server_cookie=1ece5a73ffefe2c8 in_seq=0 out_seq=0 2024-05-14T17:07:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c29ffb640 1 -- 172.21.15.69:0/689332821 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645214d5b80 con 0x56452157b620 2024-05-14T17:07:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c29ffb640 10 client.17163 handle_command_reply: tid=0 2024-05-14T17:07:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 client.17163 shutdown 2024-05-14T17:07:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/689332821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 msgr2=0x56452156b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x56452156b440 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6c34005940 tx=0x7f6c3403b840 comp rx=0 tx=0).stop 2024-05-14T17:07:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/689332821 shutdown_connections 2024-05-14T17:07:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56452157b620 0x56452157da50 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x564521568c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x564521568730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c29ffb640 0 client.17163 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/689332821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x56452156b440 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/689332821 >> 172.21.15.69:0/689332821 conn(0x56452155ce00 msgr2=0x56452155c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/689332821 shutdown_connections 2024-05-14T17:07:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/689332821 wait complete. 2024-05-14T17:07:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.473+0000 7f6c4b7d3740 20 client.17163 trim_cache size 0 max 16384 2024-05-14T17:07:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.488+0000 7f6c4b7d3740 1 Processor -- start 2024-05-14T17:07:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.488+0000 7f6c4b7d3740 1 -- start start 2024-05-14T17:07:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.488+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521563410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.488+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564521561210 0x56452156af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.488+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452156bad0 0x564521583360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.488+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441b0ec0 con 0x5645215608f0 2024-05-14T17:07:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.488+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ae760 con 0x56452156bad0 2024-05-14T17:07:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.488+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ac5d0 con 0x564521561210 2024-05-14T17:07:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521563410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521563410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37416/0 (socket says 172.21.15.69:37416) 2024-05-14T17:07:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c4259c640 1 -- 172.21.15.69:0/3817547828 learned_addr learned my addr 172.21.15.69:0/3817547828 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c42d9d640 1 --2- 172.21.15.69:0/3817547828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452156bad0 0x564521583360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c4259c640 1 -- 172.21.15.69:0/3817547828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564521561210 msgr2=0x56452156af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c4259c640 1 --2- 172.21.15.69:0/3817547828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564521561210 0x56452156af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c4259c640 1 -- 172.21.15.69:0/3817547828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452156bad0 msgr2=0x564521583360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c4259c640 1 --2- 172.21.15.69:0/3817547828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452156bad0 0x564521583360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c4259c640 1 -- 172.21.15.69:0/3817547828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56452157e0d0 con 0x5645215608f0 2024-05-14T17:07:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c4259c640 1 --2- 172.21.15.69:0/3817547828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521563410 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6c34102e20 tx=0x7f6c34005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4262929ba1853e24 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:30.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c2bfff640 1 -- 172.21.15.69:0/3817547828 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34048070 con 0x5645215608f0 2024-05-14T17:07:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c2bfff640 1 -- 172.21.15.69:0/3817547828 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c34102e50 con 0x5645215608f0 2024-05-14T17:07:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.489+0000 7f6c2bfff640 1 -- 172.21.15.69:0/3817547828 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340c5400 con 0x5645215608f0 2024-05-14T17:07:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/3817547828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 msgr2=0x564521563410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/3817547828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521563410 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6c34102e20 tx=0x7f6c34005c90 comp rx=0 tx=0).stop 2024-05-14T17:07:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/3817547828 shutdown_connections 2024-05-14T17:07:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/3817547828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56452156bad0 0x564521583360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/3817547828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564521561210 0x56452156af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/3817547828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x564521563410 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/3817547828 >> 172.21.15.69:0/3817547828 conn(0x56452155ce00 msgr2=0x564521562b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/3817547828 shutdown_connections 2024-05-14T17:07:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/3817547828 wait complete. 2024-05-14T17:07:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 -- start start 2024-05-14T17:07:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 Processor -- start 2024-05-14T17:07:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x5645215e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x5645215e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x56452156fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441acc20 con 0x5645215608f0 2024-05-14T17:07:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ae090 con 0x564521561210 2024-05-14T17:07:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ad2f0 con 0x56452156bad0 2024-05-14T17:07:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4b7d3740 10 client.0 _ll_register_callbacks cb 0x7f6c340329f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.490+0000 7f6c4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x5645215e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x5645215e3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37426/0 (socket says 172.21.15.69:37426) 2024-05-14T17:07:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4259c640 1 -- 172.21.15.69:0/2752326789 learned_addr learned my addr 172.21.15.69:0/2752326789 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c42d9d640 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x56452156fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4259c640 1 -- 172.21.15.69:0/2752326789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 msgr2=0x56452156fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4259c640 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x56452156fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c3bfff640 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x5645215e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4259c640 1 -- 172.21.15.69:0/2752326789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 msgr2=0x5645215e3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4259c640 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x5645215e3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4259c640 1 -- 172.21.15.69:0/2752326789 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56452155cc30 con 0x5645215608f0 2024-05-14T17:07:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c42d9d640 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x56452156fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:07:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c3bfff640 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x5645215e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:07:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4259c640 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x5645215e3630 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6c3403bcb0 tx=0x7f6c340d2700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e985cfacfa3081 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:07:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c29ffb640 1 -- 172.21.15.69:0/2752326789 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34048030 con 0x5645215608f0 2024-05-14T17:07:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c29ffb640 1 -- 172.21.15.69:0/2752326789 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8542+0+0 (secure 0 0 0) 0x7f6c340c8ed0 con 0x5645215608f0 2024-05-14T17:07:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/2752326789 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c34102e50 con 0x5645215608f0 2024-05-14T17:07:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c29ffb640 1 -- 172.21.15.69:0/2752326789 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6c340ea070 con 0x5645215608f0 2024-05-14T17:07:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4b7d3740 10 client.17169 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.491+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/2752326789 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x56452155cc30 con 0x5645215608f0 2024-05-14T17:07:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.492+0000 7f6c29ffb640 1 -- 172.21.15.69:0/2752326789 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c34102e50 con 0x5645215608f0 2024-05-14T17:07:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.492+0000 7f6c29ffb640 1 -- 172.21.15.69:0/2752326789 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f6c340c8940 con 0x5645215608f0 2024-05-14T17:07:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.492+0000 7f6c4b7d3740 10 client.17169 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.492+0000 7f6c4b7d3740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.492+0000 7f6c4b7d3740 20 client.17169 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.492+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5645215eaed0 0x5645215ed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.492+0000 7f6c4b7d3740 4 client.17169 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.492+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/2752326789 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645214cf360 con 0x5645215eaed0 2024-05-14T17:07:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.492+0000 7f6c3bfff640 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5645215eaed0 0x5645215ed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.493+0000 7f6c3bfff640 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5645215eaed0 0x5645215ed2c0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a63e9ec8197ed0 server_cookie=f23f6b2f7a3214da in_seq=0 out_seq=0 2024-05-14T17:07:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.493+0000 7f6c29ffb640 10 client.17169 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c29ffb640 1 -- 172.21.15.69:0/2752326789 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5645214cf360 con 0x5645215eaed0 2024-05-14T17:07:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c29ffb640 10 client.17169 handle_command_reply: tid=0 2024-05-14T17:07:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 client.17169 shutdown 2024-05-14T17:07:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/2752326789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 msgr2=0x5645215e3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x5645215e3630 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6c3403bcb0 tx=0x7f6c340d2700 comp rx=0 tx=0).stop 2024-05-14T17:07:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/2752326789 shutdown_connections 2024-05-14T17:07:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5645215eaed0 0x5645215ed2c0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56452156bad0 0x56452156fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564521561210 0x5645215e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 --2- 172.21.15.69:0/2752326789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645215608f0 0x5645215e3630 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c29ffb640 0 client.17169 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/2752326789 >> 172.21.15.69:0/2752326789 conn(0x56452155ce00 msgr2=0x5645215622a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/2752326789 shutdown_connections 2024-05-14T17:07:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.504+0000 7f6c4b7d3740 1 -- 172.21.15.69:0/2752326789 wait complete. 2024-05-14T17:07:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c4b7d3740 20 client.17169 trim_cache size 0 max 16384 2024-05-14T17:07:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:07:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:07:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:07:32.833 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:25 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:07:32.864 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:26 smithi092 ceph-mon[35763]: pgmap v555: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 6 op/s 2024-05-14T17:07:32.898 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:26 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:07:32.931 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:26 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:07:32.965 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:26 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:07:32.998 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:28 smithi092 ceph-mon[35763]: pgmap v556: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-05-14T17:07:33.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:30 smithi092 ceph-mon[35763]: pgmap v557: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-14T17:07:33.090 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:32 smithi092 ceph-mon[35763]: pgmap v558: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-14T17:07:33.123 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:25 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:07:33.156 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:26 smithi203 ceph-mon[35787]: pgmap v555: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 6 op/s 2024-05-14T17:07:33.207 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:26 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:07:33.243 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:26 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:07:33.282 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:26 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:07:33.315 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:28 smithi203 ceph-mon[35787]: pgmap v556: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-05-14T17:07:33.348 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:30 smithi203 ceph-mon[35787]: pgmap v557: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-14T17:07:33.395 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:32 smithi203 ceph-mon[35787]: pgmap v558: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-14T17:07:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 -- 172.21.15.69:0/1652801759 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6c300b2a10 msgr2=0x7f6c300b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6c300b2a10 0x7f6c300b4ed0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c019000 tx=0x7f6c3c013380 comp rx=0 tx=0).stop 2024-05-14T17:07:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 -- 172.21.15.69:0/1652801759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c44101980 msgr2=0x7f6c4419a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c44101980 0x7f6c4419a060 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6c34004770 tx=0x7f6c340042f0 comp rx=0 tx=0).stop 2024-05-14T17:07:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 -- 172.21.15.69:0/1652801759 shutdown_connections 2024-05-14T17:07:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f6c300b2a10 0x7f6c300b4ed0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c44107940 0x7f6c441a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c44102340 0x7f6c4419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 --2- 172.21.15.69:0/1652801759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c44101980 0x7f6c4419a060 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 -- 172.21.15.69:0/1652801759 >> 172.21.15.69:0/1652801759 conn(0x7f6c44078ad0 msgr2=0x7f6c44104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 -- 172.21.15.69:0/1652801759 shutdown_connections 2024-05-14T17:07:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:08.505+0000 7f6c48abd640 1 -- 172.21.15.69:0/1652801759 wait complete. 2024-05-14T17:07:33.883 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:09 smithi069 ceph-mon[31507]: pgmap v545: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 23 op/s 2024-05-14T17:07:33.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:09 smithi069 ceph-mon[31507]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T17:07:33.999 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:12 smithi069 ceph-mon[31507]: pgmap v547: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-05-14T17:07:34.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:14 smithi069 ceph-mon[31507]: pgmap v548: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:34.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:14 smithi069 ceph-mon[31507]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T17:07:34.133 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:07:34.166 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:16 smithi069 ceph-mon[31507]: pgmap v550: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-05-14T17:07:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.179+0000 7f324bd9f700 1 -- 172.21.15.69:0/935009426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244108520 msgr2=0x7f3244108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:34.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.179+0000 7f324bd9f700 1 --2- 172.21.15.69:0/935009426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244108520 0x7f3244108900 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3234009990 tx=0x7f3234009ca0 comp rx=0 tx=0).stop 2024-05-14T17:07:34.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.179+0000 7f324bd9f700 1 -- 172.21.15.69:0/935009426 shutdown_connections 2024-05-14T17:07:34.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.179+0000 7f324bd9f700 1 --2- 172.21.15.69:0/935009426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244108520 0x7f3244108900 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:34.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.179+0000 7f324bd9f700 1 --2- 172.21.15.69:0/935009426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3244102e00 0x7f324410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:34.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.179+0000 7f324bd9f700 1 --2- 172.21.15.69:0/935009426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3244102440 0x7f32441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:34.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.179+0000 7f324bd9f700 1 -- 172.21.15.69:0/935009426 >> 172.21.15.69:0/935009426 conn(0x7f32440793f0 msgr2=0x7f3244079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:34.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.179+0000 7f324bd9f700 1 -- 172.21.15.69:0/935009426 shutdown_connections 2024-05-14T17:07:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.179+0000 7f324bd9f700 1 -- 172.21.15.69:0/935009426 wait complete. 2024-05-14T17:07:34.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.180+0000 7f324bd9f700 1 Processor -- start 2024-05-14T17:07:34.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.180+0000 7f324bd9f700 1 -- start start 2024-05-14T17:07:34.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.180+0000 7f324bd9f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3244102440 0x7f324419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:34.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.180+0000 7f324bd9f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3244102e00 0x7f324419b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324bd9f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244108520 0x7f32441a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:34.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324bd9f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32441126d0 con 0x7f3244108520 2024-05-14T17:07:34.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324bd9f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3244112550 con 0x7f3244102440 2024-05-14T17:07:34.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324bd9f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3244112850 con 0x7f3244102e00 2024-05-14T17:07:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324a33c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244108520 0x7f32441a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324a33c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244108520 0x7f32441a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45776/0 (socket says 172.21.15.69:45776) 2024-05-14T17:07:34.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324a33c700 1 -- 172.21.15.69:0/20720146 learned_addr learned my addr 172.21.15.69:0/20720146 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f3249b3b700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3244102440 0x7f324419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:34.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324933a700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3244102e00 0x7f324419b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:35.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324a33c700 1 -- 172.21.15.69:0/20720146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3244102e00 msgr2=0x7f324419b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:35.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324a33c700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3244102e00 0x7f324419b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:35.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324a33c700 1 -- 172.21.15.69:0/20720146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3244102440 msgr2=0x7f324419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:35.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324a33c700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3244102440 0x7f324419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:35.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324a33c700 1 -- 172.21.15.69:0/20720146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32441a60f0 con 0x7f3244108520 2024-05-14T17:07:35.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.181+0000 7f324a33c700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244108520 0x7f32441a5bb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b970 tx=0x7f323c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:35.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.182+0000 7f323affd700 1 -- 172.21.15.69:0/20720146 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c00d860 con 0x7f3244108520 2024-05-14T17:07:35.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.182+0000 7f323affd700 1 -- 172.21.15.69:0/20720146 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f323c00e430 con 0x7f3244108520 2024-05-14T17:07:35.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.182+0000 7f324bd9f700 1 -- 172.21.15.69:0/20720146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32441a6380 con 0x7f3244108520 2024-05-14T17:07:35.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.182+0000 7f323affd700 1 -- 172.21.15.69:0/20720146 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c017660 con 0x7f3244108520 2024-05-14T17:07:35.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.182+0000 7f324bd9f700 1 -- 172.21.15.69:0/20720146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32441a6890 con 0x7f3244108520 2024-05-14T17:07:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.183+0000 7f324bd9f700 1 -- 172.21.15.69:0/20720146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324410a9b0 con 0x7f3244108520 2024-05-14T17:07:35.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.185+0000 7f323affd700 1 -- 172.21.15.69:0/20720146 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f323c00da00 con 0x7f3244108520 2024-05-14T17:07:35.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.186+0000 7f323affd700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f32300b2a70 0x7f32300b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.186+0000 7f3249b3b700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f32300b2a70 0x7f32300b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:35.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.186+0000 7f323affd700 1 -- 172.21.15.69:0/20720146 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f323c0e2340 con 0x7f3244108520 2024-05-14T17:07:35.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.186+0000 7f3249b3b700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f32300b2a70 0x7f32300b4f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f323400b5c0 tx=0x7f3234005960 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:35.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.189+0000 7f323affd700 1 -- 172.21.15.69:0/20720146 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f323c053df0 con 0x7f3244108520 2024-05-14T17:07:35.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.456+0000 7f324bd9f700 1 -- 172.21.15.69:0/20720146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f324410a390 con 0x7f3244108520 2024-05-14T17:07:35.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.457+0000 7f323affd700 1 -- 172.21.15.69:0/20720146 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11860 (secure 0 0 0) 0x7f323c013030 con 0x7f3244108520 2024-05-14T17:07:35.742 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:07:35.775 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:05:33.721215+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26245":{"gid":26245,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3691870879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3691870879},{"type":"v1","addr":"172.21.15.203:6837","nonce":3691870879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24446,24422,24433]},"id":1}]} 2024-05-14T17:07:35.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 -- 172.21.15.69:0/20720146 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f32300b2a70 msgr2=0x7f32300b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:35.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f32300b2a70 0x7f32300b4f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f323400b5c0 tx=0x7f3234005960 comp rx=0 tx=0).stop 2024-05-14T17:07:35.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 -- 172.21.15.69:0/20720146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244108520 msgr2=0x7f32441a5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:35.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244108520 0x7f32441a5bb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b970 tx=0x7f323c00bc80 comp rx=0 tx=0).stop 2024-05-14T17:07:35.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 -- 172.21.15.69:0/20720146 shutdown_connections 2024-05-14T17:07:36.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f32300b2a70 0x7f32300b4f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:36.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244108520 0x7f32441a5bb0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:36.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3244102440 0x7f324419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:36.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 --2- 172.21.15.69:0/20720146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3244102e00 0x7f324419b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:36.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 -- 172.21.15.69:0/20720146 >> 172.21.15.69:0/20720146 conn(0x7f32440793f0 msgr2=0x7f3244101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:36.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 -- 172.21.15.69:0/20720146 shutdown_connections 2024-05-14T17:07:36.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:07:17.460+0000 7f324bd9f700 1 -- 172.21.15.69:0/20720146 wait complete. 2024-05-14T17:07:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 59 2024-05-14T17:07:36.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:18 smithi069 ceph-mon[31507]: pgmap v551: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:07:36.376 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:18 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/20720146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:07:36.410 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --set-id 471a1ec6 --release --await' 2024-05-14T17:07:36.442 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --set-id 471a1ec6 --release --await 2024-05-14T17:07:36.501 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:20 smithi069 ceph-mon[31507]: pgmap v552: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:36.552 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:34 smithi092 ceph-mon[35763]: pgmap v559: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:07:36.584 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:36 smithi092 ceph-mon[35763]: pgmap v560: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-05-14T17:07:36.617 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:34 smithi203 ceph-mon[35787]: pgmap v559: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:07:36.651 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:36 smithi203 ceph-mon[35787]: pgmap v560: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-05-14T17:07:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 -- 172.21.15.69:0/2553064801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2700102a60 msgr2=0x7f2700102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 --2- 172.21.15.69:0/2553064801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2700102a60 0x7f2700102ee0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009930 tx=0x7f26f402a420 comp rx=0 tx=0).stop 2024-05-14T17:07:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 -- 172.21.15.69:0/2553064801 shutdown_connections 2024-05-14T17:07:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 --2- 172.21.15.69:0/2553064801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700103420 0x7f270010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 --2- 172.21.15.69:0/2553064801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2700102a60 0x7f2700102ee0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 --2- 172.21.15.69:0/2553064801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2700108a70 0x7f2700108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 -- 172.21.15.69:0/2553064801 >> 172.21.15.69:0/2553064801 conn(0x7f2700079ee0 msgr2=0x7f270007a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 -- 172.21.15.69:0/2553064801 shutdown_connections 2024-05-14T17:07:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 -- 172.21.15.69:0/2553064801 wait complete. 2024-05-14T17:07:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 Processor -- start 2024-05-14T17:07:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.604+0000 7f2708119640 1 -- start start 2024-05-14T17:07:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f2708119640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700102a60 0x7f2700074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f2708119640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2700103420 0x7f2700075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f2708119640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2700108a70 0x7f27000757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f2708119640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2700112d30 con 0x7f2700108a70 2024-05-14T17:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f2708119640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2700112bb0 con 0x7f2700103420 2024-05-14T17:07:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f2708119640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2700112eb0 con 0x7f2700102a60 2024-05-14T17:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f27066b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2700108a70 0x7f27000757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f27066b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2700108a70 0x7f27000757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45800/0 (socket says 172.21.15.69:45800) 2024-05-14T17:07:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f27066b6640 1 -- 172.21.15.69:0/4269949473 learned_addr learned my addr 172.21.15.69:0/4269949473 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f2705eb5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700102a60 0x7f2700074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f27056b4640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2700103420 0x7f2700075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f27066b6640 1 -- 172.21.15.69:0/4269949473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700102a60 msgr2=0x7f2700074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f27066b6640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700102a60 0x7f2700074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f27066b6640 1 -- 172.21.15.69:0/4269949473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2700103420 msgr2=0x7f2700075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f27066b6640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2700103420 0x7f2700075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f27066b6640 1 -- 172.21.15.69:0/4269949473 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27001a7690 con 0x7f2700108a70 2024-05-14T17:07:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f27066b6640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2700108a70 0x7f27000757d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f26f800e960 tx=0x7f26f800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f26f2ffd640 1 -- 172.21.15.69:0/4269949473 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8004d20 con 0x7f2700108a70 2024-05-14T17:07:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f26f2ffd640 1 -- 172.21.15.69:0/4269949473 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26f8005020 con 0x7f2700108a70 2024-05-14T17:07:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f2708119640 1 -- 172.21.15.69:0/4269949473 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27001a7980 con 0x7f2700108a70 2024-05-14T17:07:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f26f2ffd640 1 -- 172.21.15.69:0/4269949473 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f80116d0 con 0x7f2700108a70 2024-05-14T17:07:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.605+0000 7f2708119640 1 -- 172.21.15.69:0/4269949473 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27001a7ea0 con 0x7f2700108a70 2024-05-14T17:07:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.608+0000 7f26f2ffd640 1 -- 172.21.15.69:0/4269949473 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f26f80051d0 con 0x7f2700108a70 2024-05-14T17:07:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.608+0000 7f270ae38740 1 Processor -- start 2024-05-14T17:07:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f270ae38740 1 -- start start 2024-05-14T17:07:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f270ae38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59b68f0 0x55abd59b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f270ae38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 0x55abd59c1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f270ae38740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f270ae38740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55abd5929c30 con 0x55abd59c1ad0 2024-05-14T17:07:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f270ae38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55abd59197d0 con 0x55abd59b7210 2024-05-14T17:07:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f270ae38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55abd591d810 con 0x55abd59b68f0 2024-05-14T17:07:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f27056b4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f27056b4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45812/0 (socket says 172.21.15.69:45812) 2024-05-14T17:07:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f27056b4640 1 -- 172.21.15.69:0/1155584008 learned_addr learned my addr 172.21.15.69:0/1155584008 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f2705eb5640 1 --2- 172.21.15.69:0/1155584008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 0x55abd59c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f27056b4640 1 --2- 172.21.15.69:0/1155584008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59b68f0 0x55abd59b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f2705eb5640 1 -- 172.21.15.69:0/1155584008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59b68f0 msgr2=0x55abd59b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f2705eb5640 1 --2- 172.21.15.69:0/1155584008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59b68f0 0x55abd59b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f2705eb5640 1 -- 172.21.15.69:0/1155584008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 msgr2=0x55abd59c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f2705eb5640 1 --2- 172.21.15.69:0/1155584008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f26f2ffd640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f26ec0b2a40 0x7f26ec0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f2705eb5640 1 -- 172.21.15.69:0/1155584008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd562e210 con 0x55abd59b7210 2024-05-14T17:07:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.609+0000 7f27066b6640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f26ec0b2a40 0x7f26ec0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.610+0000 7f2705eb5640 1 --2- 172.21.15.69:0/1155584008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 0x55abd59c1590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00bb70 tx=0x7f26fc0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7ee89639b5b2d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.610+0000 7f26f2ffd640 1 -- 172.21.15.69:0/4269949473 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f26f80e26b0 con 0x7f2700108a70 2024-05-14T17:07:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.610+0000 7f26ebfff640 1 -- 172.21.15.69:0/1155584008 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc00be60 con 0x55abd59b7210 2024-05-14T17:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.610+0000 7f26ebfff640 1 -- 172.21.15.69:0/1155584008 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc0035f0 con 0x55abd59b7210 2024-05-14T17:07:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.610+0000 7f26ebfff640 1 -- 172.21.15.69:0/1155584008 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0116a0 con 0x55abd59b7210 2024-05-14T17:07:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 -- 172.21.15.69:0/1155584008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 msgr2=0x55abd59c1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f27066b6640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f26ec0b2a40 0x7f26ec0b4f00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f26f80a99c0 tx=0x7f26f8076360 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 --2- 172.21.15.69:0/1155584008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 0x55abd59c1590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00bb70 tx=0x7f26fc0107a0 comp rx=0 tx=0).stop 2024-05-14T17:07:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 -- 172.21.15.69:0/1155584008 shutdown_connections 2024-05-14T17:07:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 --2- 172.21.15.69:0/1155584008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 --2- 172.21.15.69:0/1155584008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 0x55abd59c1590 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 --2- 172.21.15.69:0/1155584008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59b68f0 0x55abd59b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 -- 172.21.15.69:0/1155584008 >> 172.21.15.69:0/1155584008 conn(0x55abd59b2e00 msgr2=0x55abd59b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 -- 172.21.15.69:0/1155584008 shutdown_connections 2024-05-14T17:07:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 -- 172.21.15.69:0/1155584008 wait complete. 2024-05-14T17:07:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 -- start start 2024-05-14T17:07:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.611+0000 7f270ae38740 1 Processor -- start 2024-05-14T17:07:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f270ae38740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b68f0 0x55abd59c13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f270ae38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 0x55abd59be750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f270ae38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59c1ad0 0x55abd59becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f270ae38740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55abd55ff790 con 0x55abd59b68f0 2024-05-14T17:07:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f270ae38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55abd58d5890 con 0x55abd59b7210 2024-05-14T17:07:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f270ae38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55abd59c47a0 con 0x55abd59c1ad0 2024-05-14T17:07:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f270ae38740 10 client.0 _ll_register_callbacks cb 0x55abd58d8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f27056b4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b68f0 0x55abd59c13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f27056b4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b68f0 0x55abd59c13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45820/0 (socket says 172.21.15.69:45820) 2024-05-14T17:07:39.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f27056b4640 1 -- 172.21.15.69:0/235779412 learned_addr learned my addr 172.21.15.69:0/235779412 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:39.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f2705eb5640 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 0x55abd59be750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f27056b4640 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59c1ad0 0x55abd59becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f27056b4640 1 -- 172.21.15.69:0/235779412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59c1ad0 msgr2=0x55abd59becd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f27056b4640 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59c1ad0 0x55abd59becd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f27056b4640 1 -- 172.21.15.69:0/235779412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 msgr2=0x55abd59be750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f27056b4640 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 0x55abd59be750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f27056b4640 1 -- 172.21.15.69:0/235779412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd56fa620 con 0x55abd59b68f0 2024-05-14T17:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f2705eb5640 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 0x55abd59be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f27056b4640 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b68f0 0x55abd59c13f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f26f4033000 tx=0x7f26f4041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51dc742d7964d76 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f26e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:07:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f26e97fa640 1 -- 172.21.15.69:0/235779412 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4035050 con 0x55abd59b68f0 2024-05-14T17:07:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f26e97fa640 1 -- 172.21.15.69:0/235779412 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f26f4005450 con 0x55abd59b68f0 2024-05-14T17:07:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.612+0000 7f270ae38740 1 -- 172.21.15.69:0/235779412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26fc0035f0 con 0x55abd59b68f0 2024-05-14T17:07:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f26e97fa640 1 -- 172.21.15.69:0/235779412 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f26f4005650 con 0x55abd59b68f0 2024-05-14T17:07:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f270ae38740 10 client.17187 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f270ae38740 1 -- 172.21.15.69:0/235779412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55abd59d0220 con 0x55abd59b68f0 2024-05-14T17:07:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f26e97fa640 1 -- 172.21.15.69:0/235779412 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc0035f0 con 0x55abd59b68f0 2024-05-14T17:07:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f26e97fa640 1 -- 172.21.15.69:0/235779412 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f26f40035f0 con 0x55abd59b68f0 2024-05-14T17:07:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f270ae38740 10 client.17187 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f270ae38740 10 client.17187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f270ae38740 20 client.17187 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f270ae38740 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55abd59d16e0 0x55abd59d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f270ae38740 4 client.17187 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f270ae38740 1 -- 172.21.15.69:0/235779412 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd592bb80 con 0x55abd59d16e0 2024-05-14T17:07:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.613+0000 7f2705eb5640 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55abd59d16e0 0x55abd59d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.614+0000 7f26e97fa640 10 client.17187 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.614+0000 7f2705eb5640 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55abd59d16e0 0x55abd59d3af0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f551ef67d736822 server_cookie=3468588e8b7b3dbf in_seq=0 out_seq=0 2024-05-14T17:07:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.614+0000 7f26e97fa640 1 -- 172.21.15.69:0/235779412 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55abd592bb80 con 0x55abd59d16e0 2024-05-14T17:07:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.614+0000 7f26e97fa640 10 client.17187 handle_command_reply: tid=0 2024-05-14T17:07:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 client.17187 shutdown 2024-05-14T17:07:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 -- 172.21.15.69:0/235779412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b68f0 msgr2=0x55abd59c13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b68f0 0x55abd59c13f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f26f4033000 tx=0x7f26f4041040 comp rx=0 tx=0).stop 2024-05-14T17:07:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 -- 172.21.15.69:0/235779412 shutdown_connections 2024-05-14T17:07:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55abd59d16e0 0x55abd59d3af0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59c1ad0 0x55abd59becd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b7210 0x55abd59be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f26e97fa640 0 client.17187 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 --2- 172.21.15.69:0/235779412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b68f0 0x55abd59c13f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 -- 172.21.15.69:0/235779412 >> 172.21.15.69:0/235779412 conn(0x55abd59b2e00 msgr2=0x55abd59b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 -- 172.21.15.69:0/235779412 shutdown_connections 2024-05-14T17:07:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 1 -- 172.21.15.69:0/235779412 wait complete. 2024-05-14T17:07:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.615+0000 7f270ae38740 20 client.17187 trim_cache size 0 max 16384 2024-05-14T17:07:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f270ae38740 1 Processor -- start 2024-05-14T17:07:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f270ae38740 1 -- start start 2024-05-14T17:07:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f270ae38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 0x55abd59c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f270ae38740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b7210 0x55abd59b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f270ae38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59c1ad0 0x55abd59d9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f270ae38740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27001b3210 con 0x55abd59b7210 2024-05-14T17:07:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f270ae38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f27001b0ab0 con 0x55abd59b68f0 2024-05-14T17:07:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f270ae38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f27001ae920 con 0x55abd59c1ad0 2024-05-14T17:07:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f2705eb5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b7210 0x55abd59b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f2705eb5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b7210 0x55abd59b9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45828/0 (socket says 172.21.15.69:45828) 2024-05-14T17:07:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.630+0000 7f2705eb5640 1 -- 172.21.15.69:0/128859413 learned_addr learned my addr 172.21.15.69:0/128859413 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f27056b4640 1 --2- 172.21.15.69:0/128859413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 0x55abd59c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f2705eb5640 1 -- 172.21.15.69:0/128859413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59c1ad0 msgr2=0x55abd59d9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f2705eb5640 1 --2- 172.21.15.69:0/128859413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59c1ad0 0x55abd59d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f2705eb5640 1 -- 172.21.15.69:0/128859413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 msgr2=0x55abd59c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f2705eb5640 1 --2- 172.21.15.69:0/128859413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 0x55abd59c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f2705eb5640 1 -- 172.21.15.69:0/128859413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd59bf040 con 0x55abd59b7210 2024-05-14T17:07:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f27056b4640 1 --2- 172.21.15.69:0/128859413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 0x55abd59c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f2705eb5640 1 --2- 172.21.15.69:0/128859413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b7210 0x55abd59b9b00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f26fc011010 tx=0x7f26fc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be7085a6a0f69fe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f26eb7fe640 1 -- 172.21.15.69:0/128859413 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc005240 con 0x55abd59b7210 2024-05-14T17:07:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f26eb7fe640 1 -- 172.21.15.69:0/128859413 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc028e50 con 0x55abd59b7210 2024-05-14T17:07:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.631+0000 7f26eb7fe640 1 -- 172.21.15.69:0/128859413 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc003850 con 0x55abd59b7210 2024-05-14T17:07:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 -- 172.21.15.69:0/128859413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b7210 msgr2=0x55abd59b9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 --2- 172.21.15.69:0/128859413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b7210 0x55abd59b9b00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f26fc011010 tx=0x7f26fc00b390 comp rx=0 tx=0).stop 2024-05-14T17:07:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 -- 172.21.15.69:0/128859413 shutdown_connections 2024-05-14T17:07:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 --2- 172.21.15.69:0/128859413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59c1ad0 0x55abd59d9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 --2- 172.21.15.69:0/128859413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59b7210 0x55abd59b9b00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 --2- 172.21.15.69:0/128859413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 0x55abd59c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 -- 172.21.15.69:0/128859413 >> 172.21.15.69:0/128859413 conn(0x55abd59b2e00 msgr2=0x55abd59b93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 -- 172.21.15.69:0/128859413 shutdown_connections 2024-05-14T17:07:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 -- 172.21.15.69:0/128859413 wait complete. 2024-05-14T17:07:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 -- start start 2024-05-14T17:07:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 Processor -- start 2024-05-14T17:07:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 0x55abd59c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59b7210 0x55abd59c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27001aef70 con 0x55abd59c1ad0 2024-05-14T17:07:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f27001b03e0 con 0x55abd59b68f0 2024-05-14T17:07:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f27001af640 con 0x55abd59b7210 2024-05-14T17:07:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.632+0000 7f270ae38740 10 client.0 _ll_register_callbacks cb 0x7f26f40314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f27056b4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f27056b4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45838/0 (socket says 172.21.15.69:45838) 2024-05-14T17:07:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f27056b4640 1 -- 172.21.15.69:0/1646083182 learned_addr learned my addr 172.21.15.69:0/1646083182 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f27056b4640 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 0x55abd59c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f2705eb5640 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59b7210 0x55abd59c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f27056b4640 1 -- 172.21.15.69:0/1646083182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59b7210 msgr2=0x55abd59c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f27056b4640 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59b7210 0x55abd59c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f27056b4640 1 -- 172.21.15.69:0/1646083182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 msgr2=0x55abd59c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f27056b4640 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 0x55abd59c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f27056b4640 1 -- 172.21.15.69:0/1646083182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd5a3afb0 con 0x55abd59c1ad0 2024-05-14T17:07:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f27056b4640 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c6010 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f26f40315b0 tx=0x7f26f4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8baaec84d6ba55 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f26e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:07:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f26e97fa640 1 -- 172.21.15.69:0/1646083182 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4035070 con 0x55abd59c1ad0 2024-05-14T17:07:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f26e97fa640 1 -- 172.21.15.69:0/1646083182 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f26f4005450 con 0x55abd59c1ad0 2024-05-14T17:07:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f270ae38740 1 -- 172.21.15.69:0/1646083182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26fc028e50 con 0x55abd59c1ad0 2024-05-14T17:07:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f26e97fa640 1 -- 172.21.15.69:0/1646083182 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f26f4057070 con 0x55abd59c1ad0 2024-05-14T17:07:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f270ae38740 10 client.17193 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.633+0000 7f270ae38740 1 -- 172.21.15.69:0/1646083182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55abd5a3c2a0 con 0x55abd59c1ad0 2024-05-14T17:07:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f26e97fa640 1 -- 172.21.15.69:0/1646083182 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc028e50 con 0x55abd59c1ad0 2024-05-14T17:07:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f26e97fa640 1 -- 172.21.15.69:0/1646083182 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f26f405d060 con 0x55abd59c1ad0 2024-05-14T17:07:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f270ae38740 10 client.17193 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f270ae38740 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f270ae38740 20 client.17193 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f270ae38740 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55abd5a3c8a0 0x55abd5a3ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f270ae38740 4 client.17193 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f270ae38740 1 -- 172.21.15.69:0/1646083182 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abd5925360 con 0x55abd5a3c8a0 2024-05-14T17:07:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f2705eb5640 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55abd5a3c8a0 0x55abd5a3ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f26e97fa640 10 client.17193 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.634+0000 7f2705eb5640 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55abd5a3c8a0 0x55abd5a3ecc0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db085564141e8db7 server_cookie=e217202b3028c5e in_seq=0 out_seq=0 2024-05-14T17:07:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f26e97fa640 1 -- 172.21.15.69:0/1646083182 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55abd5925360 con 0x55abd5a3c8a0 2024-05-14T17:07:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f26e97fa640 10 client.17193 handle_command_reply: tid=0 2024-05-14T17:07:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 client.17193 shutdown 2024-05-14T17:07:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 -- 172.21.15.69:0/1646083182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 msgr2=0x55abd59c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c6010 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f26f40315b0 tx=0x7f26f4005fd0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 -- 172.21.15.69:0/1646083182 shutdown_connections 2024-05-14T17:07:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55abd5a3c8a0 0x55abd5a3ecc0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd59c1ad0 0x55abd59c6010 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55abd59b7210 0x55abd59c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 --2- 172.21.15.69:0/1646083182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abd59b68f0 0x55abd59c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 -- 172.21.15.69:0/1646083182 >> 172.21.15.69:0/1646083182 conn(0x55abd59b2e00 msgr2=0x55abd59b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f26e97fa640 0 client.17193 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 -- 172.21.15.69:0/1646083182 shutdown_connections 2024-05-14T17:07:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 1 -- 172.21.15.69:0/1646083182 wait complete. 2024-05-14T17:07:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.645+0000 7f270ae38740 20 client.17193 trim_cache size 0 max 16384 2024-05-14T17:07:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:07:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:07:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:07:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 -- 172.21.15.69:0/4269949473 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f26ec0b2a40 msgr2=0x7f26ec0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f26ec0b2a40 0x7f26ec0b4f00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f26f80a99c0 tx=0x7f26f8076360 comp rx=0 tx=0).stop 2024-05-14T17:07:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 -- 172.21.15.69:0/4269949473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2700108a70 msgr2=0x7f27000757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2700108a70 0x7f27000757d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f26f800e960 tx=0x7f26f800ee20 comp rx=0 tx=0).stop 2024-05-14T17:07:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 -- 172.21.15.69:0/4269949473 shutdown_connections 2024-05-14T17:07:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f26ec0b2a40 0x7f26ec0b4f00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2700108a70 0x7f27000757d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2700103420 0x7f2700075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 --2- 172.21.15.69:0/4269949473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2700102a60 0x7f2700074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 -- 172.21.15.69:0/4269949473 >> 172.21.15.69:0/4269949473 conn(0x7f2700079ee0 msgr2=0x7f2700100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 -- 172.21.15.69:0/4269949473 shutdown_connections 2024-05-14T17:07:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:21.646+0000 7f2708119640 1 -- 172.21.15.69:0/4269949473 wait complete. 2024-05-14T17:07:47.252 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:22 smithi069 ceph-mon[31507]: pgmap v553: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:47.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:24 smithi069 ceph-mon[31507]: pgmap v554: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-05-14T17:07:47.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:25 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:07:47.401 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:26 smithi069 ceph-mon[31507]: pgmap v555: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 6 op/s 2024-05-14T17:07:47.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:26 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:07:47.501 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:26 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:07:47.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:26 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:07:47.636 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:28 smithi069 ceph-mon[31507]: pgmap v556: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-05-14T17:07:47.693 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:30 smithi069 ceph-mon[31507]: pgmap v557: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-14T17:07:47.743 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:32 smithi069 ceph-mon[31507]: pgmap v558: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-14T17:07:47.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:34 smithi069 ceph-mon[31507]: pgmap v559: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:07:47.846 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:38 smithi092 ceph-mon[35763]: pgmap v561: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-05-14T17:07:47.893 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:40 smithi092 ceph-mon[35763]: pgmap v562: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-05-14T17:07:47.951 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:42 smithi092 ceph-mon[35763]: pgmap v563: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-14T17:07:48.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:44 smithi092 ceph-mon[35763]: pgmap v564: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:48.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:45 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:07:48.101 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:46 smithi092 ceph-mon[35763]: pgmap v565: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:07:48.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:38 smithi203 ceph-mon[35787]: pgmap v561: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-05-14T17:07:48.201 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:40 smithi203 ceph-mon[35787]: pgmap v562: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-05-14T17:07:48.252 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:42 smithi203 ceph-mon[35787]: pgmap v563: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-14T17:07:48.301 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:44 smithi203 ceph-mon[35787]: pgmap v564: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:07:48.352 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:45 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:07:48.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:46 smithi203 ceph-mon[35787]: pgmap v565: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:07:48.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:48 smithi203 ceph-mon[35787]: pgmap v566: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-14T17:07:48.529 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:48 smithi092 ceph-mon[35763]: pgmap v566: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-14T17:07:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.742+0000 7f746dd32640 1 -- 172.21.15.69:0/1481360949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f746810d0d0 msgr2=0x7f746810f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.742+0000 7f746dd32640 1 --2- 172.21.15.69:0/1481360949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f746810d0d0 0x7f746810f500 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7460009ca0 tx=0x7f746002a950 comp rx=0 tx=0).stop 2024-05-14T17:07:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.743+0000 7f746dd32640 1 -- 172.21.15.69:0/1481360949 shutdown_connections 2024-05-14T17:07:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.743+0000 7f746dd32640 1 --2- 172.21.15.69:0/1481360949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f746810d0d0 0x7f746810f500 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.743+0000 7f746dd32640 1 --2- 172.21.15.69:0/1481360949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7468104620 0x7f746810cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.743+0000 7f746dd32640 1 --2- 172.21.15.69:0/1481360949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468103d00 0x7f74681040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.743+0000 7f746dd32640 1 -- 172.21.15.69:0/1481360949 >> 172.21.15.69:0/1481360949 conn(0x7f7468077cd0 msgr2=0x7f74680780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.743+0000 7f746dd32640 1 -- 172.21.15.69:0/1481360949 shutdown_connections 2024-05-14T17:07:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.743+0000 7f746dd32640 1 -- 172.21.15.69:0/1481360949 wait complete. 2024-05-14T17:07:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.743+0000 7f746dd32640 1 Processor -- start 2024-05-14T17:07:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.743+0000 7f746dd32640 1 -- start start 2024-05-14T17:07:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f746dd32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7468103d00 0x7f7468199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f746dd32640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468104620 0x7f7468199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f746dd32640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f746810d0d0 0x7f746819d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f746dd32640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7468112d10 con 0x7f746810d0d0 2024-05-14T17:07:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f746dd32640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7468112b90 con 0x7f7468103d00 2024-05-14T17:07:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f746dd32640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7468112e90 con 0x7f7468104620 2024-05-14T17:07:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7467fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f746810d0d0 0x7f746819d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7467fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f746810d0d0 0x7f746819d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43268/0 (socket says 172.21.15.69:43268) 2024-05-14T17:07:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7467fff640 1 -- 172.21.15.69:0/3532770676 learned_addr learned my addr 172.21.15.69:0/3532770676 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f74677fe640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7468103d00 0x7f7468199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7466ffd640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468104620 0x7f7468199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7467fff640 1 -- 172.21.15.69:0/3532770676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468104620 msgr2=0x7f7468199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7467fff640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468104620 0x7f7468199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7467fff640 1 -- 172.21.15.69:0/3532770676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7468103d00 msgr2=0x7f7468199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7467fff640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7468103d00 0x7f7468199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7467fff640 1 -- 172.21.15.69:0/3532770676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746819e100 con 0x7f746810d0d0 2024-05-14T17:07:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7466ffd640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468104620 0x7f7468199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:07:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.744+0000 7f7467fff640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f746810d0d0 0x7f746819d9d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7460004af0 tx=0x7f746002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.745+0000 7f7464ff9640 1 -- 172.21.15.69:0/3532770676 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460034040 con 0x7f746810d0d0 2024-05-14T17:07:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.745+0000 7f7464ff9640 1 -- 172.21.15.69:0/3532770676 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f746002fe80 con 0x7f746810d0d0 2024-05-14T17:07:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.745+0000 7f746dd32640 1 -- 172.21.15.69:0/3532770676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746819e390 con 0x7f746810d0d0 2024-05-14T17:07:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.745+0000 7f7464ff9640 1 -- 172.21.15.69:0/3532770676 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460004670 con 0x7f746810d0d0 2024-05-14T17:07:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.745+0000 7f746dd32640 1 -- 172.21.15.69:0/3532770676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74681a5c80 con 0x7f746810d0d0 2024-05-14T17:07:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.747+0000 7f7464ff9640 1 -- 172.21.15.69:0/3532770676 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7460037020 con 0x7f746810d0d0 2024-05-14T17:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.747+0000 7f7470a4e740 1 Processor -- start 2024-05-14T17:07:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.747+0000 7f7470a4e740 1 -- start start 2024-05-14T17:07:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb58f0 0x55dfdfeb5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfec0ad0 0x55dfdfec2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7470a4e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55dfdfe28c30 con 0x55dfdfeb6210 2024-05-14T17:07:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7470a4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dfdfe187d0 con 0x55dfdfeb58f0 2024-05-14T17:07:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7470a4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55dfdfe1c810 con 0x55dfdfec0ad0 2024-05-14T17:07:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f74677fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f74677fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43270/0 (socket says 172.21.15.69:43270) 2024-05-14T17:07:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f74677fe640 1 -- 172.21.15.69:0/1567054495 learned_addr learned my addr 172.21.15.69:0/1567054495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7466ffd640 1 --2- 172.21.15.69:0/1567054495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfec0ad0 0x55dfdfec2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7466ffd640 1 --2- 172.21.15.69:0/1567054495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb58f0 0x55dfdfeb5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7464ff9640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f74500b2a10 0x7f74500b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f74677fe640 1 -- 172.21.15.69:0/1567054495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfec0ad0 msgr2=0x55dfdfec2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f74677fe640 1 --2- 172.21.15.69:0/1567054495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfec0ad0 0x55dfdfec2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f74677fe640 1 -- 172.21.15.69:0/1567054495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb58f0 msgr2=0x55dfdfeb5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f74677fe640 1 --2- 172.21.15.69:0/1567054495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb58f0 0x55dfdfeb5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f74677fe640 1 -- 172.21.15.69:0/1567054495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfdfb2d210 con 0x55dfdfeb6210 2024-05-14T17:07:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7467fff640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f74500b2a10 0x7f74500b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f74677fe640 1 --2- 172.21.15.69:0/1567054495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0590 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f745800bcd0 tx=0x7f74580107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5912bb2e556543d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f7464ff9640 1 -- 172.21.15.69:0/3532770676 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f7460103320 con 0x7f746810d0d0 2024-05-14T17:07:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.748+0000 7f744dffb640 1 -- 172.21.15.69:0/1567054495 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458005020 con 0x55dfdfeb6210 2024-05-14T17:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f744dffb640 1 -- 172.21.15.69:0/1567054495 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74580047a0 con 0x55dfdfeb6210 2024-05-14T17:07:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f744dffb640 1 -- 172.21.15.69:0/1567054495 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458011660 con 0x55dfdfeb6210 2024-05-14T17:07:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f7470a4e740 1 -- 172.21.15.69:0/1567054495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb6210 msgr2=0x55dfdfec0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f7467fff640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f74500b2a10 0x7f74500b4ed0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7460100c20 tx=0x7f74600f9d10 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1567054495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0590 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f745800bcd0 tx=0x7f74580107a0 comp rx=0 tx=0).stop 2024-05-14T17:07:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f7470a4e740 1 -- 172.21.15.69:0/1567054495 shutdown_connections 2024-05-14T17:07:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1567054495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfec0ad0 0x55dfdfec2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1567054495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0590 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1567054495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb58f0 0x55dfdfeb5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f7470a4e740 1 -- 172.21.15.69:0/1567054495 >> 172.21.15.69:0/1567054495 conn(0x55dfdfeb1e00 msgr2=0x55dfdfeb21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f7470a4e740 1 -- 172.21.15.69:0/1567054495 shutdown_connections 2024-05-14T17:07:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.749+0000 7f7470a4e740 1 -- 172.21.15.69:0/1567054495 wait complete. 2024-05-14T17:07:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7470a4e740 1 -- start start 2024-05-14T17:07:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7470a4e740 1 Processor -- start 2024-05-14T17:07:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfeb58f0 0x55dfdfec04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 0x55dfdfebd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfec0ad0 0x55dfdfebddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7470a4e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55dfdfafe790 con 0x55dfdfec0ad0 2024-05-14T17:07:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7470a4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dfdfdd4890 con 0x55dfdfeb6210 2024-05-14T17:07:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7470a4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55dfdfec37a0 con 0x55dfdfeb58f0 2024-05-14T17:07:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7470a4e740 10 client.0 _ll_register_callbacks cb 0x55dfdfdd7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfec0ad0 0x55dfdfebddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f74677fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 0x55dfdfebd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfec0ad0 0x55dfdfebddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43286/0 (socket says 172.21.15.69:43286) 2024-05-14T17:07:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7466ffd640 1 -- 172.21.15.69:0/601277267 learned_addr learned my addr 172.21.15.69:0/601277267 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.750+0000 7f7466ffd640 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfeb58f0 0x55dfdfec04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7466ffd640 1 -- 172.21.15.69:0/601277267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfeb58f0 msgr2=0x55dfdfec04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:53.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7466ffd640 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfeb58f0 0x55dfdfec04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7466ffd640 1 -- 172.21.15.69:0/601277267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 msgr2=0x55dfdfebd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7466ffd640 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 0x55dfdfebd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7466ffd640 1 -- 172.21.15.69:0/601277267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfdfbf9620 con 0x55dfdfec0ad0 2024-05-14T17:07:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7466ffd640 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfec0ad0 0x55dfdfebddd0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f745c012f30 tx=0x7f745c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f493f9286b601d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7446ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:07:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7446ffd640 1 -- 172.21.15.69:0/601277267 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c004030 con 0x55dfdfec0ad0 2024-05-14T17:07:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7446ffd640 1 -- 172.21.15.69:0/601277267 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f745c0041d0 con 0x55dfdfec0ad0 2024-05-14T17:07:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7470a4e740 1 -- 172.21.15.69:0/601277267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74580047a0 con 0x55dfdfec0ad0 2024-05-14T17:07:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7446ffd640 1 -- 172.21.15.69:0/601277267 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f745c029050 con 0x55dfdfec0ad0 2024-05-14T17:07:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7470a4e740 10 client.17205 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7470a4e740 1 -- 172.21.15.69:0/601277267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55dfdfecf280 con 0x55dfdfec0ad0 2024-05-14T17:07:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7446ffd640 1 -- 172.21.15.69:0/601277267 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74580047a0 con 0x55dfdfec0ad0 2024-05-14T17:07:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7446ffd640 1 -- 172.21.15.69:0/601277267 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f745c020070 con 0x55dfdfec0ad0 2024-05-14T17:07:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.751+0000 7f7470a4e740 10 client.17205 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.752+0000 7f7470a4e740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.752+0000 7f7470a4e740 20 client.17205 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.752+0000 7f7470a4e740 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55dfdfed0740 0x55dfdfed2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.752+0000 7f7470a4e740 4 client.17205 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.752+0000 7f7470a4e740 1 -- 172.21.15.69:0/601277267 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfdfe2ab80 con 0x55dfdfed0740 2024-05-14T17:07:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.752+0000 7f74677fe640 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55dfdfed0740 0x55dfdfed2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.752+0000 7f7446ffd640 10 client.17205 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.752+0000 7f74677fe640 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55dfdfed0740 0x55dfdfed2b30 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d7620e7173829d server_cookie=9dc604ee6fc5e5d2 in_seq=0 out_seq=0 2024-05-14T17:07:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7446ffd640 1 -- 172.21.15.69:0/601277267 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dfdfe2ab80 con 0x55dfdfed0740 2024-05-14T17:07:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7446ffd640 10 client.17205 handle_command_reply: tid=0 2024-05-14T17:07:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 client.17205 shutdown 2024-05-14T17:07:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 -- 172.21.15.69:0/601277267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfec0ad0 msgr2=0x55dfdfebddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfec0ad0 0x55dfdfebddd0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f745c012f30 tx=0x7f745c0046c0 comp rx=0 tx=0).stop 2024-05-14T17:07:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 -- 172.21.15.69:0/601277267 shutdown_connections 2024-05-14T17:07:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55dfdfed0740 0x55dfdfed2b30 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfec0ad0 0x55dfdfebddd0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 0x55dfdfebd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7446ffd640 0 client.17205 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 --2- 172.21.15.69:0/601277267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfeb58f0 0x55dfdfec04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 -- 172.21.15.69:0/601277267 >> 172.21.15.69:0/601277267 conn(0x55dfdfeb1e00 msgr2=0x55dfdfeb1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 -- 172.21.15.69:0/601277267 shutdown_connections 2024-05-14T17:07:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 1 -- 172.21.15.69:0/601277267 wait complete. 2024-05-14T17:07:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.753+0000 7f7470a4e740 20 client.17205 trim_cache size 0 max 16384 2024-05-14T17:07:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.768+0000 7f7470a4e740 1 Processor -- start 2024-05-14T17:07:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.768+0000 7f7470a4e740 1 -- start start 2024-05-14T17:07:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfecc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfec0ad0 0x55dfdfed83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7470a4e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74681b0f30 con 0x55dfdfeb58f0 2024-05-14T17:07:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7470a4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74681ae7d0 con 0x55dfdfeb6210 2024-05-14T17:07:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7470a4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f74681ac640 con 0x55dfdfec0ad0 2024-05-14T17:07:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfecc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfecc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43288/0 (socket says 172.21.15.69:43288) 2024-05-14T17:07:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7466ffd640 1 -- 172.21.15.69:0/1607231472 learned_addr learned my addr 172.21.15.69:0/1607231472 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7466ffd640 1 -- 172.21.15.69:0/1607231472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfec0ad0 msgr2=0x55dfdfed83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f74677fe640 1 --2- 172.21.15.69:0/1607231472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7466ffd640 1 --2- 172.21.15.69:0/1607231472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfec0ad0 0x55dfdfed83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7466ffd640 1 -- 172.21.15.69:0/1607231472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 msgr2=0x55dfdfec0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7466ffd640 1 --2- 172.21.15.69:0/1607231472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7466ffd640 1 -- 172.21.15.69:0/1607231472 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfdfeb6040 con 0x55dfdfeb58f0 2024-05-14T17:07:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f74677fe640 1 --2- 172.21.15.69:0/1607231472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:07:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f7466ffd640 1 --2- 172.21.15.69:0/1607231472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfecc670 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f745c00fce0 tx=0x7f745c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb9ae95d0070165 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f744d7fa640 1 -- 172.21.15.69:0/1607231472 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c031020 con 0x55dfdfeb58f0 2024-05-14T17:07:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f744d7fa640 1 -- 172.21.15.69:0/1607231472 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f745c005850 con 0x55dfdfeb58f0 2024-05-14T17:07:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.769+0000 7f744d7fa640 1 -- 172.21.15.69:0/1607231472 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c005040 con 0x55dfdfeb58f0 2024-05-14T17:07:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 -- 172.21.15.69:0/1607231472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 msgr2=0x55dfdfecc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1607231472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfecc670 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f745c00fce0 tx=0x7f745c005fd0 comp rx=0 tx=0).stop 2024-05-14T17:07:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 -- 172.21.15.69:0/1607231472 shutdown_connections 2024-05-14T17:07:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1607231472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfec0ad0 0x55dfdfed83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1607231472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfeb6210 0x55dfdfec0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1607231472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfecc670 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 -- 172.21.15.69:0/1607231472 >> 172.21.15.69:0/1607231472 conn(0x55dfdfeb1e00 msgr2=0x55dfdfeb7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 -- 172.21.15.69:0/1607231472 shutdown_connections 2024-05-14T17:07:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 -- 172.21.15.69:0/1607231472 wait complete. 2024-05-14T17:07:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 -- start start 2024-05-14T17:07:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.770+0000 7f7470a4e740 1 Processor -- start 2024-05-14T17:07:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfec44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfeb6210 0x55dfdfec4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7470a4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfec0ad0 0x55dfdfec4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7470a4e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74681acc90 con 0x55dfdfeb58f0 2024-05-14T17:07:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7470a4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74681ae100 con 0x55dfdfec0ad0 2024-05-14T17:07:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7470a4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f74681ad360 con 0x55dfdfeb6210 2024-05-14T17:07:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7470a4e740 10 client.0 _ll_register_callbacks cb 0x7f745c012b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfec44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfec44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43296/0 (socket says 172.21.15.69:43296) 2024-05-14T17:07:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7466ffd640 1 -- 172.21.15.69:0/1447391892 learned_addr learned my addr 172.21.15.69:0/1447391892 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:07:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7466ffd640 1 -- 172.21.15.69:0/1447391892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfeb6210 msgr2=0x55dfdfec4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f74677fe640 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfeb6210 0x55dfdfec4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7466ffd640 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfeb6210 0x55dfdfec4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7466ffd640 1 -- 172.21.15.69:0/1447391892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfec0ad0 msgr2=0x55dfdfec4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7466ffd640 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfec0ad0 0x55dfdfec4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7466ffd640 1 -- 172.21.15.69:0/1447391892 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfdff39fa0 con 0x55dfdfeb58f0 2024-05-14T17:07:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7466ffd640 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfec44e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f745c004280 tx=0x7f745c0041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e34919e21f3ddf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.771+0000 7f7446ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:07:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7446ffd640 1 -- 172.21.15.69:0/1447391892 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c031030 con 0x55dfdfeb58f0 2024-05-14T17:07:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7446ffd640 1 -- 172.21.15.69:0/1447391892 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f745c011760 con 0x55dfdfeb58f0 2024-05-14T17:07:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7470a4e740 1 -- 172.21.15.69:0/1447391892 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745c005850 con 0x55dfdfeb58f0 2024-05-14T17:07:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7446ffd640 1 -- 172.21.15.69:0/1447391892 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f745c0119e0 con 0x55dfdfeb58f0 2024-05-14T17:07:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7470a4e740 10 client.17211 fetch_fsmap learned FSMap version 59 2024-05-14T17:07:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7470a4e740 1 -- 172.21.15.69:0/1447391892 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55dfdff3b160 con 0x55dfdfeb58f0 2024-05-14T17:07:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7446ffd640 1 -- 172.21.15.69:0/1447391892 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f745c005850 con 0x55dfdfeb58f0 2024-05-14T17:07:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7446ffd640 1 -- 172.21.15.69:0/1447391892 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8342+0+0 (secure 0 0 0) 0x7f745c020050 con 0x55dfdfeb58f0 2024-05-14T17:07:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7470a4e740 10 client.17211 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:07:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7470a4e740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7470a4e740 20 client.17211 populate_metadata read hostname 'smithi069' 2024-05-14T17:07:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55dfdff3b720 0x55dfdff3db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7470a4e740 4 client.17211 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f7470a4e740 1 -- 172.21.15.69:0/1447391892 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfdfe24360 con 0x55dfdff3b720 2024-05-14T17:07:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.772+0000 7f74677fe640 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55dfdff3b720 0x55dfdff3db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.773+0000 7f7446ffd640 10 client.17211 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.773+0000 7f74677fe640 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55dfdff3b720 0x55dfdff3db40 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c2fa95c6984d5 server_cookie=3a77fb82cc637c24 in_seq=0 out_seq=0 2024-05-14T17:07:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7446ffd640 1 -- 172.21.15.69:0/1447391892 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dfdfe24360 con 0x55dfdff3b720 2024-05-14T17:07:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7446ffd640 10 client.17211 handle_command_reply: tid=0 2024-05-14T17:07:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 client.17211 shutdown 2024-05-14T17:07:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 -- 172.21.15.69:0/1447391892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 msgr2=0x55dfdfec44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfec44e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f745c004280 tx=0x7f745c0041a0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 -- 172.21.15.69:0/1447391892 shutdown_connections 2024-05-14T17:07:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55dfdff3b720 0x55dfdff3db40 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dfdfec0ad0 0x55dfdfec4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dfdfeb6210 0x55dfdfec4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 --2- 172.21.15.69:0/1447391892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dfdfeb58f0 0x55dfdfec44e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7446ffd640 0 client.17211 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:07:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 -- 172.21.15.69:0/1447391892 >> 172.21.15.69:0/1447391892 conn(0x55dfdfeb1e00 msgr2=0x55dfdfeb8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 -- 172.21.15.69:0/1447391892 shutdown_connections 2024-05-14T17:07:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 1 -- 172.21.15.69:0/1447391892 wait complete. 2024-05-14T17:07:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.783+0000 7f7470a4e740 20 client.17211 trim_cache size 0 max 16384 2024-05-14T17:07:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:07:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:07:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:07:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 -- 172.21.15.69:0/3532770676 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f74500b2a10 msgr2=0x7f74500b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f74500b2a10 0x7f74500b4ed0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7460100c20 tx=0x7f74600f9d10 comp rx=0 tx=0).stop 2024-05-14T17:07:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 -- 172.21.15.69:0/3532770676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f746810d0d0 msgr2=0x7f746819d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f746810d0d0 0x7f746819d9d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7460004af0 tx=0x7f746002f5d0 comp rx=0 tx=0).stop 2024-05-14T17:08:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 -- 172.21.15.69:0/3532770676 shutdown_connections 2024-05-14T17:08:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f74500b2a10 0x7f74500b4ed0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f746810d0d0 0x7f746819d9d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468104620 0x7f7468199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 --2- 172.21.15.69:0/3532770676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7468103d00 0x7f7468199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 -- 172.21.15.69:0/3532770676 >> 172.21.15.69:0/3532770676 conn(0x7f7468077cd0 msgr2=0x7f74680fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 -- 172.21.15.69:0/3532770676 shutdown_connections 2024-05-14T17:08:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:34.784+0000 7f746dd32640 1 -- 172.21.15.69:0/3532770676 wait complete. 2024-05-14T17:08:00.461 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:36 smithi069 ceph-mon[31507]: pgmap v560: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-05-14T17:08:00.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:38 smithi069 ceph-mon[31507]: pgmap v561: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-05-14T17:08:00.559 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:40 smithi069 ceph-mon[31507]: pgmap v562: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-05-14T17:08:00.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:42 smithi069 ceph-mon[31507]: pgmap v563: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-14T17:08:00.659 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:44 smithi069 ceph-mon[31507]: pgmap v564: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:08:00.709 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:45 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:08:00.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:46 smithi069 ceph-mon[31507]: pgmap v565: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:08:00.839 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:50 smithi092 ceph-mon[35763]: pgmap v567: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:08:00.884 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:52 smithi092 ceph-mon[35763]: pgmap v568: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:08:00.934 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:54 smithi092 ceph-mon[35763]: pgmap v569: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-14T17:08:00.984 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:56 smithi092 ceph-mon[35763]: pgmap v570: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:01.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:07:58 smithi092 ceph-mon[35763]: pgmap v571: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-14T17:08:01.084 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:00 smithi092 ceph-mon[35763]: pgmap v572: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:01.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:50 smithi203 ceph-mon[35787]: pgmap v567: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:08:01.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:52 smithi203 ceph-mon[35787]: pgmap v568: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:08:01.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:54 smithi203 ceph-mon[35787]: pgmap v569: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-14T17:08:01.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:56 smithi203 ceph-mon[35787]: pgmap v570: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:01.334 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:07:58 smithi203 ceph-mon[35787]: pgmap v571: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-14T17:08:01.401 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:00 smithi203 ceph-mon[35787]: pgmap v572: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.880+0000 7f8d38e23640 1 -- 172.21.15.69:0/1585015459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34101ca0 msgr2=0x7f8d34102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.880+0000 7f8d38e23640 1 --2- 172.21.15.69:0/1585015459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34101ca0 0x7f8d34102120 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009900 tx=0x7f8d1c02a410 comp rx=0 tx=0).stop 2024-05-14T17:08:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.880+0000 7f8d38e23640 1 -- 172.21.15.69:0/1585015459 shutdown_connections 2024-05-14T17:08:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.880+0000 7f8d38e23640 1 --2- 172.21.15.69:0/1585015459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d34102660 0x7f8d3410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.880+0000 7f8d38e23640 1 --2- 172.21.15.69:0/1585015459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34101ca0 0x7f8d34102120 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.880+0000 7f8d38e23640 1 --2- 172.21.15.69:0/1585015459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d341068f0 0x7f8d34106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.880+0000 7f8d38e23640 1 -- 172.21.15.69:0/1585015459 >> 172.21.15.69:0/1585015459 conn(0x7f8d34077750 msgr2=0x7f8d34077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.880+0000 7f8d38e23640 1 -- 172.21.15.69:0/1585015459 shutdown_connections 2024-05-14T17:08:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.880+0000 7f8d38e23640 1 -- 172.21.15.69:0/1585015459 wait complete. 2024-05-14T17:08:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.881+0000 7f8d38e23640 1 Processor -- start 2024-05-14T17:08:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.881+0000 7f8d38e23640 1 -- start start 2024-05-14T17:08:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.881+0000 7f8d38e23640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d34101ca0 0x7f8d34199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.881+0000 7f8d38e23640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34102660 0x7f8d3419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.881+0000 7f8d38e23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d341068f0 0x7f8d341a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.881+0000 7f8d38e23640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d34111bd0 con 0x7f8d34102660 2024-05-14T17:08:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.881+0000 7f8d38e23640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8d34111a50 con 0x7f8d341068f0 2024-05-14T17:08:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.881+0000 7f8d38e23640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8d34111d50 con 0x7f8d34101ca0 2024-05-14T17:08:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34102660 0x7f8d3419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34102660 0x7f8d3419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52328/0 (socket says 172.21.15.69:52328) 2024-05-14T17:08:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2bfff640 1 -- 172.21.15.69:0/2202781904 learned_addr learned my addr 172.21.15.69:0/2202781904 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d3259c640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d34101ca0 0x7f8d34199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d341068f0 0x7f8d341a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2bfff640 1 -- 172.21.15.69:0/2202781904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d34101ca0 msgr2=0x7f8d34199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d34101ca0 0x7f8d34199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2bfff640 1 -- 172.21.15.69:0/2202781904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d341068f0 msgr2=0x7f8d341a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d341068f0 0x7f8d341a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2bfff640 1 -- 172.21.15.69:0/2202781904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d341a5530 con 0x7f8d34102660 2024-05-14T17:08:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d341068f0 0x7f8d341a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34102660 0x7f8d3419a4b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c005940 tx=0x7f8d1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2b7fe640 1 -- 172.21.15.69:0/2202781904 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c03f030 con 0x7f8d34102660 2024-05-14T17:08:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2b7fe640 1 -- 172.21.15.69:0/2202781904 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d1c003aa0 con 0x7f8d34102660 2024-05-14T17:08:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d38e23640 1 -- 172.21.15.69:0/2202781904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d341a5700 con 0x7f8d34102660 2024-05-14T17:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d2b7fe640 1 -- 172.21.15.69:0/2202781904 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c035070 con 0x7f8d34102660 2024-05-14T17:08:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.882+0000 7f8d38e23640 1 -- 172.21.15.69:0/2202781904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d341a5bc0 con 0x7f8d34102660 2024-05-14T17:08:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.885+0000 7f8d3bc1d740 1 Processor -- start 2024-05-14T17:08:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3bc1d740 1 -- start start 2024-05-14T17:08:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d2b7fe640 1 -- 172.21.15.69:0/2202781904 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8d1c042030 con 0x7f8d34102660 2024-05-14T17:08:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad438f0 0x5580cad43cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad44210 0x5580cad4e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad50ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5580cacb6c30 con 0x5580cad44210 2024-05-14T17:08:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5580caca67d0 con 0x5580cad4ead0 2024-05-14T17:08:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5580cacaa810 con 0x5580cad438f0 2024-05-14T17:08:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad44210 0x5580cad4e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad44210 0x5580cad4e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52340/0 (socket says 172.21.15.69:52340) 2024-05-14T17:08:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3259c640 1 -- 172.21.15.69:0/254540570 learned_addr learned my addr 172.21.15.69:0/254540570 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad50ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad438f0 0x5580cad43cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3259c640 1 -- 172.21.15.69:0/254540570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad438f0 msgr2=0x5580cad43cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3259c640 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad438f0 0x5580cad43cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3259c640 1 -- 172.21.15.69:0/254540570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 msgr2=0x5580cad50ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3259c640 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad50ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d3259c640 1 -- 172.21.15.69:0/254540570 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580ca9bb210 con 0x5580cad44210 2024-05-14T17:08:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad50ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.886+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad438f0 0x5580cad43cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.887+0000 7f8d3259c640 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad44210 0x5580cad4e590 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400bc90 tx=0x7f8d240107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e9b397c2751974 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.887+0000 7f8d2b7fe640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8d200b2a90 0x7f8d200b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.887+0000 7f8d28ff9640 1 -- 172.21.15.69:0/254540570 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2401c480 con 0x5580cad44210 2024-05-14T17:08:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.887+0000 7f8d28ff9640 1 -- 172.21.15.69:0/254540570 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d2401cad0 con 0x5580cad44210 2024-05-14T17:08:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.887+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8d200b2a90 0x7f8d200b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.887+0000 7f8d28ff9640 1 -- 172.21.15.69:0/254540570 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24011ae0 con 0x5580cad44210 2024-05-14T17:08:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.887+0000 7f8d2b7fe640 1 -- 172.21.15.69:0/2202781904 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f8d1c102cb0 con 0x7f8d34102660 2024-05-14T17:08:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/254540570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad44210 msgr2=0x5580cad4e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8d200b2a90 0x7f8d200b4f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c005fd0 tx=0x7f8d2c009500 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad44210 0x5580cad4e590 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400bc90 tx=0x7f8d240107a0 comp rx=0 tx=0).stop 2024-05-14T17:08:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/254540570 shutdown_connections 2024-05-14T17:08:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad50ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad44210 0x5580cad4e590 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/254540570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad438f0 0x5580cad43cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/254540570 >> 172.21.15.69:0/254540570 conn(0x5580cad3fe00 msgr2=0x5580cad401e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/254540570 shutdown_connections 2024-05-14T17:08:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/254540570 wait complete. 2024-05-14T17:08:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 -- start start 2024-05-14T17:08:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 Processor -- start 2024-05-14T17:08:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad438f0 0x5580cad4e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad4b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad4bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5580ca98c790 con 0x5580cad438f0 2024-05-14T17:08:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5580cac62890 con 0x5580cad4ead0 2024-05-14T17:08:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5580cad517a0 con 0x5580cad44210 2024-05-14T17:08:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.888+0000 7f8d3bc1d740 10 client.0 _ll_register_callbacks cb 0x5580cac65d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad438f0 0x5580cad4e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad438f0 0x5580cad4e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52344/0 (socket says 172.21.15.69:52344) 2024-05-14T17:08:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3259c640 1 -- 172.21.15.69:0/478211116 learned_addr learned my addr 172.21.15.69:0/478211116 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad4bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad4b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3259c640 1 -- 172.21.15.69:0/478211116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 msgr2=0x5580cad4b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3259c640 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad4b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3259c640 1 -- 172.21.15.69:0/478211116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 msgr2=0x5580cad4bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3259c640 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad4bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3259c640 1 -- 172.21.15.69:0/478211116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580caa87620 con 0x5580cad438f0 2024-05-14T17:08:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad4bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3259c640 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad438f0 0x5580cad4e3f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8d24010ca0 tx=0x7f8d2401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c4579265946ef server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d19ffb640 1 -- 172.21.15.69:0/478211116 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24005250 con 0x5580cad438f0 2024-05-14T17:08:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d19ffb640 1 -- 172.21.15.69:0/478211116 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f8d240053f0 con 0x5580cad438f0 2024-05-14T17:08:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/478211116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d2401cad0 con 0x5580cad438f0 2024-05-14T17:08:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d19ffb640 1 -- 172.21.15.69:0/478211116 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8d240055f0 con 0x5580cad438f0 2024-05-14T17:08:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3bc1d740 10 client.17223 fetch_fsmap learned FSMap version 60 2024-05-14T17:08:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.889+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/478211116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5580cad5d0e0 con 0x5580cad438f0 2024-05-14T17:08:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.890+0000 7f8d19ffb640 1 -- 172.21.15.69:0/478211116 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d2401cad0 con 0x5580cad438f0 2024-05-14T17:08:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.890+0000 7f8d19ffb640 1 -- 172.21.15.69:0/478211116 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 60) ==== 8338+0+0 (secure 0 0 0) 0x7f8d24004b80 con 0x5580cad438f0 2024-05-14T17:08:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.890+0000 7f8d3bc1d740 10 client.17223 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:08:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.890+0000 7f8d3bc1d740 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.890+0000 7f8d3bc1d740 20 client.17223 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.890+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5580cad5e490 0x5580cad60880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.890+0000 7f8d3bc1d740 4 client.17223 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.890+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/478211116 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580cacb8b80 con 0x5580cad5e490 2024-05-14T17:08:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.890+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5580cad5e490 0x5580cad60880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.891+0000 7f8d19ffb640 10 client.17223 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.891+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5580cad5e490 0x5580cad60880 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f93501b82cc073 server_cookie=243ac6040c51f19e in_seq=0 out_seq=0 2024-05-14T17:08:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.891+0000 7f8d19ffb640 1 -- 172.21.15.69:0/478211116 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580cacb8b80 con 0x5580cad5e490 2024-05-14T17:08:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.891+0000 7f8d19ffb640 10 client.17223 handle_command_reply: tid=0 2024-05-14T17:08:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.891+0000 7f8d3bc1d740 1 client.17223 shutdown 2024-05-14T17:08:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.891+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/478211116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad438f0 msgr2=0x5580cad4e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.891+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad438f0 0x5580cad4e3f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8d24010ca0 tx=0x7f8d2401c730 comp rx=0 tx=0).stop 2024-05-14T17:08:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.892+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/478211116 shutdown_connections 2024-05-14T17:08:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.892+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5580cad5e490 0x5580cad60880 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.892+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad4bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.892+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad4b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.892+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/478211116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad438f0 0x5580cad4e3f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.892+0000 7f8d19ffb640 0 client.17223 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.892+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/478211116 >> 172.21.15.69:0/478211116 conn(0x5580cad3fe00 msgr2=0x5580cad3f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.892+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/478211116 shutdown_connections 2024-05-14T17:08:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.892+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/478211116 wait complete. 2024-05-14T17:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.892+0000 7f8d3bc1d740 20 client.17223 trim_cache size 0 max 16384 2024-05-14T17:08:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d3bc1d740 1 Processor -- start 2024-05-14T17:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d3bc1d740 1 -- start start 2024-05-14T17:08:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad5a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad3ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad4df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341b0f40 con 0x5580cad3f4c0 2024-05-14T17:08:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341ae810 con 0x5580cad4ead0 2024-05-14T17:08:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341ac680 con 0x5580cad44210 2024-05-14T17:08:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d32d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad4df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d32d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad4df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52358/0 (socket says 172.21.15.69:52358) 2024-05-14T17:08:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d32d9d640 1 -- 172.21.15.69:0/1346088044 learned_addr learned my addr 172.21.15.69:0/1346088044 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/1346088044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad3ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d32d9d640 1 -- 172.21.15.69:0/1346088044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 msgr2=0x5580cad5a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/1346088044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad5a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d32d9d640 1 -- 172.21.15.69:0/1346088044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 msgr2=0x5580cad3ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/1346088044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad3ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d32d9d640 1 -- 172.21.15.69:0/1346088044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cad3fc30 con 0x5580cad3f4c0 2024-05-14T17:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.907+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/1346088044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad3ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/1346088044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad4df20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c012aa0 tx=0x7f8d2c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eac2c20f85c12d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d1bfff640 1 -- 172.21.15.69:0/1346088044 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c01fc00 con 0x5580cad3f4c0 2024-05-14T17:08:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d1bfff640 1 -- 172.21.15.69:0/1346088044 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d2c01fda0 con 0x5580cad3f4c0 2024-05-14T17:08:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d1bfff640 1 -- 172.21.15.69:0/1346088044 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c028690 con 0x5580cad3f4c0 2024-05-14T17:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/1346088044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 msgr2=0x5580cad4df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/1346088044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad4df20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c012aa0 tx=0x7f8d2c014000 comp rx=0 tx=0).stop 2024-05-14T17:08:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/1346088044 shutdown_connections 2024-05-14T17:08:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/1346088044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad4df20 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/1346088044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad3ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/1346088044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad5a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/1346088044 >> 172.21.15.69:0/1346088044 conn(0x5580cad3fe00 msgr2=0x5580cad3e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/1346088044 shutdown_connections 2024-05-14T17:08:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.908+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/1346088044 wait complete. 2024-05-14T17:08:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3bc1d740 1 -- start start 2024-05-14T17:08:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3bc1d740 1 Processor -- start 2024-05-14T17:08:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad52730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad52c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3bc1d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341accd0 con 0x5580cad3f4c0 2024-05-14T17:08:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341ae140 con 0x5580cad4ead0 2024-05-14T17:08:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3bc1d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341ad3a0 con 0x5580cad44210 2024-05-14T17:08:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3bc1d740 10 client.0 _ll_register_callbacks cb 0x7f8d2400b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad52730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad52730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52374/0 (socket says 172.21.15.69:52374) 2024-05-14T17:08:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3259c640 1 -- 172.21.15.69:0/3507804686 learned_addr learned my addr 172.21.15.69:0/3507804686 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad52c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3259c640 1 -- 172.21.15.69:0/3507804686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 msgr2=0x5580cad52c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3259c640 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad52c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3259c640 1 -- 172.21.15.69:0/3507804686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 msgr2=0x5580cad531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3259c640 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d3259c640 1 -- 172.21.15.69:0/3507804686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580cadc7d00 con 0x5580cad3f4c0 2024-05-14T17:08:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d32d9d640 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.909+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad52c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d3259c640 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad52730 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8d24022040 tx=0x7f8d2401da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bab4041a18d072 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d19ffb640 1 -- 172.21.15.69:0/3507804686 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24011440 con 0x5580cad3f4c0 2024-05-14T17:08:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d19ffb640 1 -- 172.21.15.69:0/3507804686 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f8d2402fb60 con 0x5580cad3f4c0 2024-05-14T17:08:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/3507804686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d2c01fda0 con 0x5580cad3f4c0 2024-05-14T17:08:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d19ffb640 1 -- 172.21.15.69:0/3507804686 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8d2403d050 con 0x5580cad3f4c0 2024-05-14T17:08:10.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d3bc1d740 10 client.17229 fetch_fsmap learned FSMap version 60 2024-05-14T17:08:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/3507804686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5580cadc8f60 con 0x5580cad3f4c0 2024-05-14T17:08:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d19ffb640 1 -- 172.21.15.69:0/3507804686 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d2c01fda0 con 0x5580cad3f4c0 2024-05-14T17:08:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d19ffb640 1 -- 172.21.15.69:0/3507804686 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 60) ==== 8338+0+0 (secure 0 0 0) 0x7f8d24011ba0 con 0x5580cad3f4c0 2024-05-14T17:08:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d3bc1d740 10 client.17229 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:08:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d3bc1d740 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d3bc1d740 20 client.17229 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5580cadc9670 0x5580cadcba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d3bc1d740 4 client.17229 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.910+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/3507804686 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580cacb2360 con 0x5580cadc9670 2024-05-14T17:08:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.911+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5580cadc9670 0x5580cadcba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.911+0000 7f8d19ffb640 10 client.17229 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.911+0000 7f8d2bfff640 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5580cadc9670 0x5580cadcba60 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1287dc89612070e8 server_cookie=3bd047b2dee61365 in_seq=0 out_seq=0 2024-05-14T17:08:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d19ffb640 1 -- 172.21.15.69:0/3507804686 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5580cacb2360 con 0x5580cadc9670 2024-05-14T17:08:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d19ffb640 10 client.17229 handle_command_reply: tid=0 2024-05-14T17:08:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 client.17229 shutdown 2024-05-14T17:08:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/3507804686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 msgr2=0x5580cad52730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad52730 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8d24022040 tx=0x7f8d2401da70 comp rx=0 tx=0).stop 2024-05-14T17:08:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/3507804686 shutdown_connections 2024-05-14T17:08:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5580cadc9670 0x5580cadcba60 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580cad4ead0 0x5580cad531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5580cad44210 0x5580cad52c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d19ffb640 0 client.17229 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 --2- 172.21.15.69:0/3507804686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5580cad3f4c0 0x5580cad52730 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/3507804686 >> 172.21.15.69:0/3507804686 conn(0x5580cad3fe00 msgr2=0x5580cad59660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/3507804686 shutdown_connections 2024-05-14T17:08:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 1 -- 172.21.15.69:0/3507804686 wait complete. 2024-05-14T17:08:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.916+0000 7f8d3bc1d740 20 client.17229 trim_cache size 0 max 16384 2024-05-14T17:08:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:08:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:08:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:08:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 -- 172.21.15.69:0/2202781904 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8d200b2a90 msgr2=0x7f8d200b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8d200b2a90 0x7f8d200b4f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c005fd0 tx=0x7f8d2c009500 comp rx=0 tx=0).stop 2024-05-14T17:08:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 -- 172.21.15.69:0/2202781904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34102660 msgr2=0x7f8d3419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34102660 0x7f8d3419a4b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c005940 tx=0x7f8d1c004270 comp rx=0 tx=0).stop 2024-05-14T17:08:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 -- 172.21.15.69:0/2202781904 shutdown_connections 2024-05-14T17:08:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8d200b2a90 0x7f8d200b4f50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d341068f0 0x7f8d341a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34102660 0x7f8d3419a4b0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 --2- 172.21.15.69:0/2202781904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d34101ca0 0x7f8d34199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 -- 172.21.15.69:0/2202781904 >> 172.21.15.69:0/2202781904 conn(0x7f8d34077750 msgr2=0x7f8d340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 -- 172.21.15.69:0/2202781904 shutdown_connections 2024-05-14T17:08:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:07:47.917+0000 7f8d38e23640 1 -- 172.21.15.69:0/2202781904 wait complete. 2024-05-14T17:08:12.173 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:02 smithi203 ceph-mon[35787]: pgmap v573: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:12.223 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: *** Caught signal (Segmentation fault) ** 2024-05-14T17:08:12.255 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: in thread 7f97d82eb700 thread_name:md_log_replay 2024-05-14T17:08:12.305 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:08:12.338 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f97e96add20] 2024-05-14T17:08:12.372 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x55a342b6fc1f] 2024-05-14T17:08:12.427 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a342b733b3] 2024-05-14T17:08:12.502 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 4: (MDLog::_replay_thread()+0x617) [0x55a342bce197] 2024-05-14T17:08:12.538 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a342bd7a11] 2024-05-14T17:08:12.572 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f97e96a31ca] 2024-05-14T17:08:12.630 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 7: clone() 2024-05-14T17:08:12.664 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2024-05-14T17:08:03.087+0000 7f97d82eb700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:08:12.704 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: in thread 7f97d82eb700 thread_name:md_log_replay 2024-05-14T17:08:12.730 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2024-05-14T17:08:12.793 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:08:12.830 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f97e96add20] 2024-05-14T17:08:12.868 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x55a342b6fc1f] 2024-05-14T17:08:12.947 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a342b733b3] 2024-05-14T17:08:12.980 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 4: (MDLog::_replay_thread()+0x617) [0x55a342bce197] 2024-05-14T17:08:13.014 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a342bd7a11] 2024-05-14T17:08:13.047 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f97e96a31ca] 2024-05-14T17:08:13.080 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 7: clone() 2024-05-14T17:08:13.114 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:08:13.147 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2024-05-14T17:08:13.181 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 0> 2024-05-14T17:08:03.087+0000 7f97d82eb700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:08:13.214 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: in thread 7f97d82eb700 thread_name:md_log_replay 2024-05-14T17:08:13.247 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2024-05-14T17:08:13.281 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:08:13.314 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f97e96add20] 2024-05-14T17:08:13.347 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x55a342b6fc1f] 2024-05-14T17:08:13.381 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a342b733b3] 2024-05-14T17:08:13.449 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 4: (MDLog::_replay_thread()+0x617) [0x55a342bce197] 2024-05-14T17:08:13.481 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a342bd7a11] 2024-05-14T17:08:13.539 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f97e96a31ca] 2024-05-14T17:08:13.573 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 7: clone() 2024-05-14T17:08:13.606 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:08:13.639 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2024-05-14T17:08:13.673 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 0> 2024-05-14T17:08:03.087+0000 7f97d82eb700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:08:13.706 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: in thread 7f97d82eb700 thread_name:md_log_replay 2024-05-14T17:08:13.739 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2024-05-14T17:08:13.781 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:08:13.839 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f97e96add20] 2024-05-14T17:08:13.873 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x55a342b6fc1f] 2024-05-14T17:08:13.906 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a342b733b3] 2024-05-14T17:08:13.956 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 4: (MDLog::_replay_thread()+0x617) [0x55a342bce197] 2024-05-14T17:08:13.989 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a342bd7a11] 2024-05-14T17:08:14.025 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f97e96a31ca] 2024-05-14T17:08:14.056 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 7: clone() 2024-05-14T17:08:14.133 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:08:14.173 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c[73687]: 2024-05-14T17:08:14.233 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 podman[74201]: 2024-05-14 17:08:03.140135027 +0000 UTC m=+0.017604637 container died 92f0d2387ee8312c8f2eb29edd4c5ee7d4bfd480ce290e7b630dacd4eb105b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:08:14.265 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:48 smithi069 ceph-mon[31507]: pgmap v566: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-14T17:08:14.298 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:50 smithi069 ceph-mon[31507]: pgmap v567: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:08:14.333 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:52 smithi069 ceph-mon[31507]: pgmap v568: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:08:14.365 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:54 smithi069 ceph-mon[31507]: pgmap v569: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-14T17:08:14.398 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:56 smithi069 ceph-mon[31507]: pgmap v570: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:14.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:07:58 smithi069 ceph-mon[31507]: pgmap v571: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-14T17:08:14.491 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:00 smithi069 ceph-mon[31507]: pgmap v572: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:14.570 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:04 smithi203 ceph-mon[35787]: pgmap v574: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-05-14T17:08:14.607 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:08:14.641 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:02 smithi092 ceph-mon[35763]: pgmap v573: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:14.673 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:04 smithi092 ceph-mon[35763]: pgmap v574: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-05-14T17:08:14.706 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:05 smithi092 ceph-mon[35763]: osdmap e109: 12 total, 12 up, 12 in 2024-05-14T17:08:14.740 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:06 smithi092 ceph-mon[35763]: pgmap v576: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 65 KiB/s wr, 26 op/s 2024-05-14T17:08:14.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:08 smithi092 ceph-mon[35763]: pgmap v577: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 141 KiB/s wr, 40 op/s 2024-05-14T17:08:14.819 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:10 smithi092 ceph-mon[35763]: pgmap v578: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 159 KiB/s wr, 44 op/s 2024-05-14T17:08:14.857 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:12 smithi092 ceph-mon[35763]: pgmap v579: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 159 KiB/s wr, 44 op/s 2024-05-14T17:08:14.899 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:14 smithi092 ceph-mon[35763]: pgmap v580: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-05-14T17:08:14.932 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:14 smithi092 ceph-mon[35763]: osdmap e110: 12 total, 12 up, 12 in 2024-05-14T17:08:14.965 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 podman[74201]: 2024-05-14 17:08:03.890216194 +0000 UTC m=+0.767685793 container remove 92f0d2387ee8312c8f2eb29edd4c5ee7d4bfd480ce290e7b630dacd4eb105b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-c, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:08:14.998 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:03 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-05-14T17:08:15.032 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:04 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-14T17:08:15.067 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:05 smithi203 ceph-mon[35787]: osdmap e109: 12 total, 12 up, 12 in 2024-05-14T17:08:15.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:06 smithi203 ceph-mon[35787]: pgmap v576: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 65 KiB/s wr, 26 op/s 2024-05-14T17:08:15.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:08 smithi203 ceph-mon[35787]: pgmap v577: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 141 KiB/s wr, 40 op/s 2024-05-14T17:08:15.199 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:10 smithi203 ceph-mon[35787]: pgmap v578: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 159 KiB/s wr, 44 op/s 2024-05-14T17:08:15.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:12 smithi203 ceph-mon[35787]: pgmap v579: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 159 KiB/s wr, 44 op/s 2024-05-14T17:08:15.265 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:14 smithi203 ceph-mon[35787]: pgmap v580: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-05-14T17:08:15.299 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:14 smithi203 ceph-mon[35787]: osdmap e110: 12 total, 12 up, 12 in 2024-05-14T17:08:15.333 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:14 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 6. 2024-05-14T17:08:15.411 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:14 smithi203 systemd[1]: Stopped Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:08:15.465 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:14 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Start request repeated too quickly. 2024-05-14T17:08:15.502 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:14 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-14T17:08:15.532 INFO:journalctl@ceph.mds.c.smithi203.stdout:May 14 17:08:14 smithi203 systemd[1]: Failed to start Ceph mds.c for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:08:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 -- 172.21.15.69:0/2246754125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e14102a50 msgr2=0x7f0e14102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 --2- 172.21.15.69:0/2246754125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e14102a50 0x7f0e14102ed0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009930 tx=0x7f0e0802a420 comp rx=0 tx=0).stop 2024-05-14T17:08:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 -- 172.21.15.69:0/2246754125 shutdown_connections 2024-05-14T17:08:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 --2- 172.21.15.69:0/2246754125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e14103410 0x7f0e1410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 --2- 172.21.15.69:0/2246754125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e14102a50 0x7f0e14102ed0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 --2- 172.21.15.69:0/2246754125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14108a60 0x7f0e14108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 -- 172.21.15.69:0/2246754125 >> 172.21.15.69:0/2246754125 conn(0x7f0e140fe540 msgr2=0x7f0e14100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 -- 172.21.15.69:0/2246754125 shutdown_connections 2024-05-14T17:08:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 -- 172.21.15.69:0/2246754125 wait complete. 2024-05-14T17:08:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 Processor -- start 2024-05-14T17:08:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.009+0000 7f0e1a62a640 1 -- start start 2024-05-14T17:08:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e1a62a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e14102a50 0x7f0e141992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e1a62a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14103410 0x7f0e14199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e1a62a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e14108a60 0x7f0e1419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e1a62a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14112d20 con 0x7f0e14108a60 2024-05-14T17:08:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e1a62a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14112ba0 con 0x7f0e14103410 2024-05-14T17:08:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e1a62a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14112ea0 con 0x7f0e14102a50 2024-05-14T17:08:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e18bc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e14108a60 0x7f0e1419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e18bc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e14108a60 0x7f0e1419db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33116/0 (socket says 172.21.15.69:33116) 2024-05-14T17:08:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e18bc7640 1 -- 172.21.15.69:0/3998408833 learned_addr learned my addr 172.21.15.69:0/3998408833 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e13fff640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e14102a50 0x7f0e141992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e137fe640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14103410 0x7f0e14199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e18bc7640 1 -- 172.21.15.69:0/3998408833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e14102a50 msgr2=0x7f0e141992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e18bc7640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e14102a50 0x7f0e141992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e18bc7640 1 -- 172.21.15.69:0/3998408833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14103410 msgr2=0x7f0e14199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e18bc7640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14103410 0x7f0e14199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e18bc7640 1 -- 172.21.15.69:0/3998408833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1419e260 con 0x7f0e14108a60 2024-05-14T17:08:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e13fff640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e14102a50 0x7f0e141992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e18bc7640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e14108a60 0x7f0e1419db30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00b970 tx=0x7f0e0c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e117fa640 1 -- 172.21.15.69:0/3998408833 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c019070 con 0x7f0e14108a60 2024-05-14T17:08:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e1a62a640 1 -- 172.21.15.69:0/3998408833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1419e550 con 0x7f0e14108a60 2024-05-14T17:08:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e117fa640 1 -- 172.21.15.69:0/3998408833 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e0c014440 con 0x7f0e14108a60 2024-05-14T17:08:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.010+0000 7f0e117fa640 1 -- 172.21.15.69:0/3998408833 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0075d0 con 0x7f0e14108a60 2024-05-14T17:08:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.011+0000 7f0e1a62a640 1 -- 172.21.15.69:0/3998408833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e141a5de0 con 0x7f0e14108a60 2024-05-14T17:08:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e117fa640 1 -- 172.21.15.69:0/3998408833 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0e0c01c070 con 0x7f0e14108a60 2024-05-14T17:08:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e1d429740 1 Processor -- start 2024-05-14T17:08:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e1d429740 1 -- start start 2024-05-14T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:17.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a85590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a87ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e1d429740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d1c29edc30 con 0x55d1c2a7a8f0 2024-05-14T17:08:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e1d429740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d1c29dd7d0 con 0x55d1c2a85ad0 2024-05-14T17:08:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e1d429740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d1c29e1810 con 0x55d1c2a7b210 2024-05-14T17:08:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e137fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e137fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33118/0 (socket says 172.21.15.69:33118) 2024-05-14T17:08:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e137fe640 1 -- 172.21.15.69:0/3717956436 learned_addr learned my addr 172.21.15.69:0/3717956436 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.013+0000 7f0e13fff640 1 --2- 172.21.15.69:0/3717956436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a85590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0e137fe640 1 --2- 172.21.15.69:0/3717956436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a87ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0e137fe640 1 -- 172.21.15.69:0/3717956436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 msgr2=0x55d1c2a85590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0e137fe640 1 --2- 172.21.15.69:0/3717956436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a85590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0e137fe640 1 -- 172.21.15.69:0/3717956436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a85ad0 msgr2=0x55d1c2a87ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0e137fe640 1 --2- 172.21.15.69:0/3717956436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a87ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0e137fe640 1 -- 172.21.15.69:0/3717956436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c26f2210 con 0x55d1c2a7a8f0 2024-05-14T17:08:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0e117fa640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0dfc0b2a10 0x7f0dfc0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0e13fff640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0dfc0b2a10 0x7f0dfc0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0e137fe640 1 --2- 172.21.15.69:0/3717956436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7acd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0e08005ab0 tx=0x7f0e080058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7302c22fa82c2e72 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0e117fa640 1 -- 172.21.15.69:0/3998408833 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f0e0c0e1b60 con 0x7f0e14108a60 2024-05-14T17:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0dfa7fc640 1 -- 172.21.15.69:0/3717956436 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08035050 con 0x55d1c2a7a8f0 2024-05-14T17:08:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0dfa7fc640 1 -- 172.21.15.69:0/3717956436 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e08031870 con 0x55d1c2a7a8f0 2024-05-14T17:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.014+0000 7f0dfa7fc640 1 -- 172.21.15.69:0/3717956436 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08004d70 con 0x55d1c2a7a8f0 2024-05-14T17:08:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 -- 172.21.15.69:0/3717956436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 msgr2=0x55d1c2a7acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e13fff640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0dfc0b2a10 0x7f0dfc0b4ed0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55d1c2a7be90 tx=0x7f0e0400c4f0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 --2- 172.21.15.69:0/3717956436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7acd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0e08005ab0 tx=0x7f0e080058b0 comp rx=0 tx=0).stop 2024-05-14T17:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 -- 172.21.15.69:0/3717956436 shutdown_connections 2024-05-14T17:08:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 --2- 172.21.15.69:0/3717956436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a87ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 --2- 172.21.15.69:0/3717956436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a85590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 --2- 172.21.15.69:0/3717956436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7acd0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 -- 172.21.15.69:0/3717956436 >> 172.21.15.69:0/3717956436 conn(0x55d1c2a76e00 msgr2=0x55d1c2a771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 -- 172.21.15.69:0/3717956436 shutdown_connections 2024-05-14T17:08:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 -- 172.21.15.69:0/3717956436 wait complete. 2024-05-14T17:08:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 -- start start 2024-05-14T17:08:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.015+0000 7f0e1d429740 1 Processor -- start 2024-05-14T17:08:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a827b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a82dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e1d429740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d1c26c3790 con 0x55d1c2a85ad0 2024-05-14T17:08:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e1d429740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d1c2999890 con 0x55d1c2a7a8f0 2024-05-14T17:08:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e1d429740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d1c2a88740 con 0x55d1c2a7b210 2024-05-14T17:08:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e1d429740 10 client.0 _ll_register_callbacks cb 0x55d1c299cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e18bc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a82dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e18bc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a82dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33134/0 (socket says 172.21.15.69:33134) 2024-05-14T17:08:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e18bc7640 1 -- 172.21.15.69:0/249769259 learned_addr learned my addr 172.21.15.69:0/249769259 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e137fe640 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a827b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e13fff640 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e18bc7640 1 -- 172.21.15.69:0/249769259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 msgr2=0x55d1c2a827b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e18bc7640 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a827b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e18bc7640 1 -- 172.21.15.69:0/249769259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 msgr2=0x55d1c2a854c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e18bc7640 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a854c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e18bc7640 1 -- 172.21.15.69:0/249769259 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c27be620 con 0x55d1c2a85ad0 2024-05-14T17:08:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e137fe640 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a827b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e13fff640 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a854c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e18bc7640 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a82dc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000f80 tx=0x7f0e0c0a8b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6440ea38b2a04ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:19.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0df37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0df37fe640 1 -- 172.21.15.69:0/249769259 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0ad2f0 con 0x55d1c2a85ad0 2024-05-14T17:08:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0df37fe640 1 -- 172.21.15.69:0/249769259 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f0e0c0b1910 con 0x55d1c2a85ad0 2024-05-14T17:08:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0e1d429740 1 -- 172.21.15.69:0/249769259 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e08031870 con 0x55d1c2a85ad0 2024-05-14T17:08:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.016+0000 7f0df37fe640 1 -- 172.21.15.69:0/249769259 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0e0c0b1da0 con 0x55d1c2a85ad0 2024-05-14T17:08:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0e1d429740 10 client.17241 fetch_fsmap learned FSMap version 60 2024-05-14T17:08:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0e1d429740 1 -- 172.21.15.69:0/249769259 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55d1c2a94240 con 0x55d1c2a85ad0 2024-05-14T17:08:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0df37fe640 1 -- 172.21.15.69:0/249769259 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e08031870 con 0x55d1c2a85ad0 2024-05-14T17:08:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0df37fe640 1 -- 172.21.15.69:0/249769259 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 60) ==== 8338+0+0 (secure 0 0 0) 0x7f0e0c0a4940 con 0x55d1c2a85ad0 2024-05-14T17:08:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0e1d429740 10 client.17241 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:08:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0e1d429740 10 client.17241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0e1d429740 20 client.17241 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0e1d429740 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d1c2a95750 0x55d1c2a97b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0e1d429740 4 client.17241 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0e1d429740 1 -- 172.21.15.69:0/249769259 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1c29efb80 con 0x55d1c2a95750 2024-05-14T17:08:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0e13fff640 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d1c2a95750 0x55d1c2a97b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0df37fe640 10 client.17241 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.017+0000 7f0e13fff640 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d1c2a95750 0x55d1c2a97b40 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5e6524f50b7c16 server_cookie=ec57f6f454d044be in_seq=0 out_seq=0 2024-05-14T17:08:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0df37fe640 1 -- 172.21.15.69:0/249769259 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1c29efb80 con 0x55d1c2a95750 2024-05-14T17:08:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0df37fe640 10 client.17241 handle_command_reply: tid=0 2024-05-14T17:08:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 client.17241 shutdown 2024-05-14T17:08:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 -- 172.21.15.69:0/249769259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 msgr2=0x55d1c2a82dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a82dc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000f80 tx=0x7f0e0c0a8b10 comp rx=0 tx=0).stop 2024-05-14T17:08:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 -- 172.21.15.69:0/249769259 shutdown_connections 2024-05-14T17:08:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d1c2a95750 0x55d1c2a97b40 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a82dc0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a827b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0df37fe640 0 client.17241 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 --2- 172.21.15.69:0/249769259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a854c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 -- 172.21.15.69:0/249769259 >> 172.21.15.69:0/249769259 conn(0x55d1c2a76e00 msgr2=0x55d1c2a76100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 -- 172.21.15.69:0/249769259 shutdown_connections 2024-05-14T17:08:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.018+0000 7f0e1d429740 1 -- 172.21.15.69:0/249769259 wait complete. 2024-05-14T17:08:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.019+0000 7f0e1d429740 20 client.17241 trim_cache size 0 max 16384 2024-05-14T17:08:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.033+0000 7f0e1d429740 1 Processor -- start 2024-05-14T17:08:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.033+0000 7f0e1d429740 1 -- start start 2024-05-14T17:08:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.033+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.033+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7b210 0x55d1c2a84ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a7db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e1d429740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141b0ee0 con 0x55d1c2a7a8f0 2024-05-14T17:08:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e1d429740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141ae7b0 con 0x55d1c2a7b210 2024-05-14T17:08:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e1d429740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141ac620 con 0x55d1c2a85ad0 2024-05-14T17:08:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e13fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e13fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33150/0 (socket says 172.21.15.69:33150) 2024-05-14T17:08:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e13fff640 1 -- 172.21.15.69:0/3823041040 learned_addr learned my addr 172.21.15.69:0/3823041040 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e13fff640 1 -- 172.21.15.69:0/3823041040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a85ad0 msgr2=0x55d1c2a7db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e13fff640 1 --2- 172.21.15.69:0/3823041040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a7db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e13fff640 1 -- 172.21.15.69:0/3823041040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7b210 msgr2=0x55d1c2a84ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e137fe640 1 --2- 172.21.15.69:0/3823041040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7b210 0x55d1c2a84ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e13fff640 1 --2- 172.21.15.69:0/3823041040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7b210 0x55d1c2a84ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e13fff640 1 -- 172.21.15.69:0/3823041040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c2a981a0 con 0x55d1c2a7a8f0 2024-05-14T17:08:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0e13fff640 1 --2- 172.21.15.69:0/3823041040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7cb30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0e04012fd0 tx=0x7f0e04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e75f634eb3e49f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0df9ffb640 1 -- 172.21.15.69:0/3823041040 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04004220 con 0x55d1c2a7a8f0 2024-05-14T17:08:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0df9ffb640 1 -- 172.21.15.69:0/3823041040 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e04029e50 con 0x55d1c2a7a8f0 2024-05-14T17:08:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.034+0000 7f0df9ffb640 1 -- 172.21.15.69:0/3823041040 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04003cf0 con 0x55d1c2a7a8f0 2024-05-14T17:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.035+0000 7f0e1d429740 1 -- 172.21.15.69:0/3823041040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 msgr2=0x55d1c2a7cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.035+0000 7f0e1d429740 1 --2- 172.21.15.69:0/3823041040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7cb30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0e04012fd0 tx=0x7f0e04014000 comp rx=0 tx=0).stop 2024-05-14T17:08:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.035+0000 7f0e1d429740 1 -- 172.21.15.69:0/3823041040 shutdown_connections 2024-05-14T17:08:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.035+0000 7f0e1d429740 1 --2- 172.21.15.69:0/3823041040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a7db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.035+0000 7f0e1d429740 1 --2- 172.21.15.69:0/3823041040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7b210 0x55d1c2a84ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.035+0000 7f0e1d429740 1 --2- 172.21.15.69:0/3823041040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a7cb30 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.035+0000 7f0e1d429740 1 -- 172.21.15.69:0/3823041040 >> 172.21.15.69:0/3823041040 conn(0x55d1c2a76e00 msgr2=0x55d1c2a7d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.035+0000 7f0e1d429740 1 -- 172.21.15.69:0/3823041040 shutdown_connections 2024-05-14T17:08:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.035+0000 7f0e1d429740 1 -- 172.21.15.69:0/3823041040 wait complete. 2024-05-14T17:08:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.035+0000 7f0e1d429740 1 -- start start 2024-05-14T17:08:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e1d429740 1 Processor -- start 2024-05-14T17:08:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a894e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a89a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e1d429740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a89f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e1d429740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141acc70 con 0x55d1c2a85ad0 2024-05-14T17:08:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e1d429740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141ae0e0 con 0x55d1c2a7a8f0 2024-05-14T17:08:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e1d429740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141ad340 con 0x55d1c2a7b210 2024-05-14T17:08:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e1d429740 10 client.0 _ll_register_callbacks cb 0x7f0e0c0a86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e18bc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a89f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e18bc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a89f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33164/0 (socket says 172.21.15.69:33164) 2024-05-14T17:08:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e18bc7640 1 -- 172.21.15.69:0/1084044714 learned_addr learned my addr 172.21.15.69:0/1084044714 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e137fe640 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a89a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e13fff640 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a894e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e18bc7640 1 -- 172.21.15.69:0/1084044714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 msgr2=0x55d1c2a89a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e18bc7640 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a89a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e18bc7640 1 -- 172.21.15.69:0/1084044714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 msgr2=0x55d1c2a894e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e18bc7640 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a894e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e18bc7640 1 -- 172.21.15.69:0/1084044714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c2afefe0 con 0x55d1c2a85ad0 2024-05-14T17:08:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e137fe640 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a89a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e13fff640 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a894e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e18bc7640 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a89f60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0a8ed0 tx=0x7f0e0c0a8760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2182cdba1f164b70 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0df37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0df37fe640 1 -- 172.21.15.69:0/1084044714 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c088060 con 0x55d1c2a85ad0 2024-05-14T17:08:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0df37fe640 1 -- 172.21.15.69:0/1084044714 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f0e0c0a7d80 con 0x55d1c2a85ad0 2024-05-14T17:08:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.036+0000 7f0e1d429740 1 -- 172.21.15.69:0/1084044714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e04029e50 con 0x55d1c2a85ad0 2024-05-14T17:08:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0df37fe640 1 -- 172.21.15.69:0/1084044714 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0e0c0b1b20 con 0x55d1c2a85ad0 2024-05-14T17:08:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0e1d429740 10 client.17247 fetch_fsmap learned FSMap version 60 2024-05-14T17:08:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0e1d429740 1 -- 172.21.15.69:0/1084044714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55d1c2b00290 con 0x55d1c2a85ad0 2024-05-14T17:08:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0df37fe640 1 -- 172.21.15.69:0/1084044714 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e04029e50 con 0x55d1c2a85ad0 2024-05-14T17:08:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0df37fe640 1 -- 172.21.15.69:0/1084044714 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 60) ==== 8338+0+0 (secure 0 0 0) 0x7f0e0c0a48b0 con 0x55d1c2a85ad0 2024-05-14T17:08:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0e1d429740 10 client.17247 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:08:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0e1d429740 10 client.17247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0e1d429740 20 client.17247 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0e1d429740 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d1c2b00980 0x55d1c2b02da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0e1d429740 4 client.17247 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0e1d429740 1 -- 172.21.15.69:0/1084044714 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1c29e9360 con 0x55d1c2b00980 2024-05-14T17:08:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.037+0000 7f0e13fff640 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d1c2b00980 0x55d1c2b02da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.038+0000 7f0df37fe640 10 client.17247 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:01.038+0000 7f0e13fff640 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d1c2b00980 0x55d1c2b02da0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca193c035540c6b server_cookie=afcb700b6b270756 in_seq=0 out_seq=0 2024-05-14T17:08:23.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:02 smithi069 ceph-mon[31507]: pgmap v573: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:23.340 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:16 smithi092 ceph-mon[35763]: pgmap v582: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-05-14T17:08:23.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:18 smithi092 ceph-mon[35763]: pgmap v583: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-05-14T17:08:23.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:19 smithi092 ceph-mon[35763]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-14T17:08:23.438 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:19 smithi092 ceph-mon[35763]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 5 up:standby 2024-05-14T17:08:23.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:20 smithi092 ceph-mon[35763]: pgmap v584: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:08:23.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:22 smithi092 ceph-mon[35763]: pgmap v585: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:08:23.558 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:16 smithi203 ceph-mon[35787]: pgmap v582: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-05-14T17:08:23.589 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:18 smithi203 ceph-mon[35787]: pgmap v583: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-05-14T17:08:23.630 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:19 smithi203 ceph-mon[35787]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-14T17:08:23.680 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:19 smithi203 ceph-mon[35787]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 5 up:standby 2024-05-14T17:08:23.714 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:20 smithi203 ceph-mon[35787]: pgmap v584: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:08:23.747 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:22 smithi203 ceph-mon[35787]: pgmap v585: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:08:23.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.496+0000 7f049fc46700 1 -- 172.21.15.69:0/2466359436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0498102440 msgr2=0x7f04981028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:23.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.497+0000 7f049fc46700 1 --2- 172.21.15.69:0/2466359436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0498102440 0x7f04981028c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0488009990 tx=0x7f0488009ca0 comp rx=0 tx=0).stop 2024-05-14T17:08:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.497+0000 7f049fc46700 1 -- 172.21.15.69:0/2466359436 shutdown_connections 2024-05-14T17:08:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.497+0000 7f049fc46700 1 --2- 172.21.15.69:0/2466359436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0498102e00 0x7f049810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:23.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.497+0000 7f049fc46700 1 --2- 172.21.15.69:0/2466359436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0498108520 0x7f0498108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:23.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.497+0000 7f049fc46700 1 --2- 172.21.15.69:0/2466359436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0498102440 0x7f04981028c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:24.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.497+0000 7f049fc46700 1 -- 172.21.15.69:0/2466359436 >> 172.21.15.69:0/2466359436 conn(0x7f04980793f0 msgr2=0x7f0498079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:24.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.497+0000 7f049fc46700 1 -- 172.21.15.69:0/2466359436 shutdown_connections 2024-05-14T17:08:24.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.497+0000 7f049fc46700 1 -- 172.21.15.69:0/2466359436 wait complete. 2024-05-14T17:08:24.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.497+0000 7f049fc46700 1 Processor -- start 2024-05-14T17:08:24.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.497+0000 7f049fc46700 1 -- start start 2024-05-14T17:08:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049fc46700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0498102440 0x7f0498104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:24.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049fc46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0498102e00 0x7f04981045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:24.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049fc46700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0498108520 0x7f0498104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049fc46700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04981125d0 con 0x7f0498108520 2024-05-14T17:08:24.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049fc46700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0498112450 con 0x7f0498102e00 2024-05-14T17:08:24.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049fc46700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0498112750 con 0x7f0498102440 2024-05-14T17:08:24.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049e1e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0498108520 0x7f0498104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:24.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049e1e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0498108520 0x7f0498104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33196/0 (socket says 172.21.15.69:33196) 2024-05-14T17:08:24.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049e1e3700 1 -- 172.21.15.69:0/3766489828 learned_addr learned my addr 172.21.15.69:0/3766489828 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:24.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049d1e1700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0498102e00 0x7f04981045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:24.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049d9e2700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0498102440 0x7f0498104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049e1e3700 1 -- 172.21.15.69:0/3766489828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0498102440 msgr2=0x7f0498104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:24.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049e1e3700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0498102440 0x7f0498104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:24.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049e1e3700 1 -- 172.21.15.69:0/3766489828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0498102e00 msgr2=0x7f04981045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049e1e3700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0498102e00 0x7f04981045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:24.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049e1e3700 1 -- 172.21.15.69:0/3766489828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04981a80e0 con 0x7f0498108520 2024-05-14T17:08:24.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049e1e3700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0498108520 0x7f0498104b00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0490007ee0 tx=0x7f049000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:24.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f048effd700 1 -- 172.21.15.69:0/3766489828 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490004bb0 con 0x7f0498108520 2024-05-14T17:08:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f049fc46700 1 -- 172.21.15.69:0/3766489828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04981a83d0 con 0x7f0498108520 2024-05-14T17:08:24.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f048effd700 1 -- 172.21.15.69:0/3766489828 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0490004d50 con 0x7f0498108520 2024-05-14T17:08:24.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.498+0000 7f048effd700 1 -- 172.21.15.69:0/3766489828 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490005920 con 0x7f0498108520 2024-05-14T17:08:25.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.499+0000 7f049fc46700 1 -- 172.21.15.69:0/3766489828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04981a88e0 con 0x7f0498108520 2024-05-14T17:08:25.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.502+0000 7f048effd700 1 -- 172.21.15.69:0/3766489828 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f049001a020 con 0x7f0498108520 2024-05-14T17:08:25.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.502+0000 7f048effd700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04840b29f0 0x7f04840b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f04aa4dcb80 1 Processor -- start 2024-05-14T17:08:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f049d9e2700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04840b29f0 0x7f04840b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f048effd700 1 -- 172.21.15.69:0/3766489828 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f04900e1ce0 con 0x7f0498108520 2024-05-14T17:08:25.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f04aa4dcb80 1 -- start start 2024-05-14T17:08:25.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f049d9e2700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04840b29f0 0x7f04840b4eb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f049400abd0 tx=0x7f049400a380 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:25.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x562705517180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627055176c0 0x5627055114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705511ad0 0x562705511f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56270546fd90 con 0x562705516da0 2024-05-14T17:08:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56270545f930 con 0x5627055176c0 2024-05-14T17:08:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562705463970 con 0x562705511ad0 2024-05-14T17:08:25.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f049d9e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x562705517180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f049d9e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x562705517180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33210/0 (socket says 172.21.15.69:33210) 2024-05-14T17:08:25.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f049d9e2700 1 -- 172.21.15.69:0/3152039135 learned_addr learned my addr 172.21.15.69:0/3152039135 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:25.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.503+0000 7f049d1e1700 1 --2- 172.21.15.69:0/3152039135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627055176c0 0x5627055114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.504+0000 7f049d9e2700 1 -- 172.21.15.69:0/3152039135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705511ad0 msgr2=0x562705511f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.504+0000 7f049d9e2700 1 --2- 172.21.15.69:0/3152039135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705511ad0 0x562705511f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.504+0000 7f049d9e2700 1 -- 172.21.15.69:0/3152039135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627055176c0 msgr2=0x5627055114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:25.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.504+0000 7f049d9e2700 1 --2- 172.21.15.69:0/3152039135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627055176c0 0x5627055114b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.504+0000 7f049d9e2700 1 -- 172.21.15.69:0/3152039135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627053af9d0 con 0x562705516da0 2024-05-14T17:08:26.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.504+0000 7f049d1e1700 1 --2- 172.21.15.69:0/3152039135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627055176c0 0x5627055114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:26.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.504+0000 7f049d9e2700 1 --2- 172.21.15.69:0/3152039135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x562705517180 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0494004530 tx=0x7f0494012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdafcbc06a400d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:26.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.504+0000 7f0483fff700 1 -- 172.21.15.69:0/3152039135 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494016070 con 0x562705516da0 2024-05-14T17:08:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.504+0000 7f0483fff700 1 -- 172.21.15.69:0/3152039135 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0494014070 con 0x562705516da0 2024-05-14T17:08:26.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.504+0000 7f0483fff700 1 -- 172.21.15.69:0/3152039135 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049401d400 con 0x562705516da0 2024-05-14T17:08:26.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/3152039135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 msgr2=0x562705517180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:26.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/3152039135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x562705517180 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0494004530 tx=0x7f0494012000 comp rx=0 tx=0).stop 2024-05-14T17:08:26.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/3152039135 shutdown_connections 2024-05-14T17:08:26.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/3152039135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x562705517180 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:26.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/3152039135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705511ad0 0x562705511f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:26.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/3152039135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627055176c0 0x5627055114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:26.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/3152039135 >> 172.21.15.69:0/3152039135 conn(0x56270543c350 msgr2=0x56270550bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:26.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/3152039135 shutdown_connections 2024-05-14T17:08:26.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/3152039135 wait complete. 2024-05-14T17:08:26.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 -- start start 2024-05-14T17:08:26.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.505+0000 7f04aa4dcb80 1 Processor -- start 2024-05-14T17:08:26.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x56270551d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x56270551a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:26.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627055176c0 0x56270551acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:26.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5627053660a0 con 0x562705516da0 2024-05-14T17:08:26.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5627054018c0 con 0x562705511ad0 2024-05-14T17:08:26.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56270551e100 con 0x5627055176c0 2024-05-14T17:08:26.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f04aa4dcb80 10 client.0 _ll_register_callbacks cb 0x5627053f6680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:26.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d1e1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x56270551a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:26.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d1e1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x56270551a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33216/0 (socket says 172.21.15.69:33216) 2024-05-14T17:08:27.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d1e1700 1 -- 172.21.15.69:0/1373094405 learned_addr learned my addr 172.21.15.69:0/1373094405 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:27.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d9e2700 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x56270551d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:27.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049e1e3700 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627055176c0 0x56270551acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:27.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d1e1700 1 -- 172.21.15.69:0/1373094405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627055176c0 msgr2=0x56270551acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:27.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d1e1700 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627055176c0 0x56270551acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:27.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d1e1700 1 -- 172.21.15.69:0/1373094405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 msgr2=0x56270551d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:27.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d1e1700 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x56270551d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:27.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d1e1700 1 -- 172.21.15.69:0/1373094405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627053fe790 con 0x562705516da0 2024-05-14T17:08:27.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d9e2700 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x56270551d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:27.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049e1e3700 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627055176c0 0x56270551acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:27.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f049d1e1700 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x56270551a7b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0488009990 tx=0x7f048800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28004283321cef32 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:27.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.506+0000 7f04817fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:27.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04817fa700 1 -- 172.21.15.69:0/1373094405 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488003c70 con 0x562705516da0 2024-05-14T17:08:27.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/1373094405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0494014070 con 0x562705516da0 2024-05-14T17:08:27.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04817fa700 1 -- 172.21.15.69:0/1373094405 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f0488005070 con 0x562705516da0 2024-05-14T17:08:27.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04817fa700 1 -- 172.21.15.69:0/1373094405 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0488030050 con 0x562705516da0 2024-05-14T17:08:27.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04aa4dcb80 10 client.17259 fetch_fsmap learned FSMap version 60 2024-05-14T17:08:27.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/1373094405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5627055288a0 con 0x562705516da0 2024-05-14T17:08:27.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04817fa700 1 -- 172.21.15.69:0/1373094405 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0494014070 con 0x562705516da0 2024-05-14T17:08:27.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04817fa700 1 -- 172.21.15.69:0/1373094405 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 60) ==== 8338+0+0 (secure 0 0 0) 0x7f048802b030 con 0x562705516da0 2024-05-14T17:08:27.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04aa4dcb80 10 client.17259 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:08:27.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04aa4dcb80 10 client.17259 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:08:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04aa4dcb80 20 client.17259 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:27.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56270552aee0 0x56270552d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04aa4dcb80 4 client.17259 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:27.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/1373094405 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627054dd460 con 0x56270552aee0 2024-05-14T17:08:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.507+0000 7f049d9e2700 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56270552aee0 0x56270552d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:28.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.508+0000 7f04817fa700 10 client.17259 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:28.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.508+0000 7f049d9e2700 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56270552aee0 0x56270552d310 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eb91367766ff05c server_cookie=9feabd36d8b0467a in_seq=0 out_seq=0 2024-05-14T17:08:28.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.508+0000 7f04817fa700 1 -- 172.21.15.69:0/1373094405 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5627054dd460 con 0x56270552aee0 2024-05-14T17:08:28.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04817fa700 10 client.17259 handle_command_reply: tid=0 2024-05-14T17:08:28.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 client.17259 shutdown 2024-05-14T17:08:28.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/1373094405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 msgr2=0x56270551a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:28.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x56270551a7b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0488009990 tx=0x7f048800bfd0 comp rx=0 tx=0).stop 2024-05-14T17:08:28.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/1373094405 shutdown_connections 2024-05-14T17:08:28.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56270552aee0 0x56270552d310 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:28.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627055176c0 0x56270551acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:28.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x56270551d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04817fa700 0 client.17259 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/1373094405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562705516da0 0x56270551a7b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:28.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/1373094405 >> 172.21.15.69:0/1373094405 conn(0x56270543c350 msgr2=0x5627055067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:28.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/1373094405 shutdown_connections 2024-05-14T17:08:28.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/1373094405 wait complete. 2024-05-14T17:08:28.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.509+0000 7f04aa4dcb80 20 client.17259 trim_cache size 0 max 16384 2024-05-14T17:08:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f04aa4dcb80 1 Processor -- start 2024-05-14T17:08:28.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f04aa4dcb80 1 -- start start 2024-05-14T17:08:28.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x562705527010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:28.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705516da0 0x562705518330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x5627055068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:28.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04981b27f0 con 0x5627055176c0 2024-05-14T17:08:29.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f04981b0080 con 0x562705511ad0 2024-05-14T17:08:29.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f04981adef0 con 0x562705516da0 2024-05-14T17:08:29.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f049e1e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x5627055068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:29.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f049e1e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x5627055068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33228/0 (socket says 172.21.15.69:33228) 2024-05-14T17:08:29.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f049e1e3700 1 -- 172.21.15.69:0/2061890568 learned_addr learned my addr 172.21.15.69:0/2061890568 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:29.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f049d9e2700 1 --2- 172.21.15.69:0/2061890568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x562705527010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:29.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f049e1e3700 1 -- 172.21.15.69:0/2061890568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705516da0 msgr2=0x562705518330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:29.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f049e1e3700 1 --2- 172.21.15.69:0/2061890568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705516da0 0x562705518330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:29.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f049e1e3700 1 -- 172.21.15.69:0/2061890568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 msgr2=0x562705527010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:29.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f049e1e3700 1 --2- 172.21.15.69:0/2061890568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x562705527010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:29.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.529+0000 7f049e1e3700 1 -- 172.21.15.69:0/2061890568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56270551b6b0 con 0x5627055176c0 2024-05-14T17:08:29.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f049e1e3700 1 --2- 172.21.15.69:0/2061890568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x5627055068e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f04900a8cb0 tx=0x7f04900a89d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772a3d3f8f8ef555 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:29.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04837fe700 1 -- 172.21.15.69:0/2061890568 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04900a96b0 con 0x5627055176c0 2024-05-14T17:08:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04837fe700 1 -- 172.21.15.69:0/2061890568 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04900a8fb0 con 0x5627055176c0 2024-05-14T17:08:29.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04837fe700 1 -- 172.21.15.69:0/2061890568 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04900b1be0 con 0x5627055176c0 2024-05-14T17:08:29.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/2061890568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 msgr2=0x5627055068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:29.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/2061890568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x5627055068e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f04900a8cb0 tx=0x7f04900a89d0 comp rx=0 tx=0).stop 2024-05-14T17:08:29.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/2061890568 shutdown_connections 2024-05-14T17:08:29.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/2061890568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x5627055068e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/2061890568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x562705527010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:29.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/2061890568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705516da0 0x562705518330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:29.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/2061890568 >> 172.21.15.69:0/2061890568 conn(0x56270543c350 msgr2=0x562705515940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:29.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/2061890568 shutdown_connections 2024-05-14T17:08:29.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.530+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/2061890568 wait complete. 2024-05-14T17:08:29.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f04aa4dcb80 1 -- start start 2024-05-14T17:08:29.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f04aa4dcb80 1 Processor -- start 2024-05-14T17:08:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x562705520360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:30.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705516da0 0x5627055208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:30.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f04aa4dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x562705520de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04981ae540 con 0x5627055176c0 2024-05-14T17:08:30.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f04981af9b0 con 0x562705511ad0 2024-05-14T17:08:30.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f04aa4dcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f04981aec10 con 0x562705516da0 2024-05-14T17:08:30.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f04aa4dcb80 10 client.0 _ll_register_callbacks cb 0x5627053f6680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:30.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f049e1e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x562705520de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f049e1e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x562705520de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33238/0 (socket says 172.21.15.69:33238) 2024-05-14T17:08:30.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f049d9e2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x562705520360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.531+0000 7f049e1e3700 1 -- 172.21.15.69:0/829069471 learned_addr learned my addr 172.21.15.69:0/829069471 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:30.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f049d1e1700 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705516da0 0x5627055208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:30.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f049e1e3700 1 -- 172.21.15.69:0/829069471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705516da0 msgr2=0x5627055208a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f049e1e3700 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705516da0 0x5627055208a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f049e1e3700 1 -- 172.21.15.69:0/829069471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 msgr2=0x562705520360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:30.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f049e1e3700 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x562705520360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:30.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f049e1e3700 1 -- 172.21.15.69:0/829069471 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627055957e0 con 0x5627055176c0 2024-05-14T17:08:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f049d9e2700 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x562705520360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:08:30.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f049e1e3700 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x562705520de0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f04900a6220 tx=0x7f04900053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c536beae057d83b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:30.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f04817fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:30.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f04817fa700 1 -- 172.21.15.69:0/829069471 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04900a96b0 con 0x5627055176c0 2024-05-14T17:08:30.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f04817fa700 1 -- 172.21.15.69:0/829069471 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8542+0+0 (secure 0 0 0) 0x7f04900b1b90 con 0x5627055176c0 2024-05-14T17:08:30.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/829069471 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04900a8fb0 con 0x5627055176c0 2024-05-14T17:08:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f04817fa700 1 -- 172.21.15.69:0/829069471 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f04900ad360 con 0x5627055176c0 2024-05-14T17:08:30.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f04aa4dcb80 10 client.17265 fetch_fsmap learned FSMap version 60 2024-05-14T17:08:31.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/829069471 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x562705596b00 con 0x5627055176c0 2024-05-14T17:08:31.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f04817fa700 1 -- 172.21.15.69:0/829069471 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04900a8fb0 con 0x5627055176c0 2024-05-14T17:08:31.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.532+0000 7f04817fa700 1 -- 172.21.15.69:0/829069471 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 60) ==== 8338+0+0 (secure 0 0 0) 0x7f04900a8210 con 0x5627055176c0 2024-05-14T17:08:31.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.533+0000 7f04aa4dcb80 10 client.17265 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:08:31.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.533+0000 7f04aa4dcb80 10 client.17265 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:08:31.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.533+0000 7f04aa4dcb80 20 client.17265 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.533+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562705597110 0x562705599530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:31.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.533+0000 7f04aa4dcb80 4 client.17265 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "471a1ec6", "release": true, "await": true}] 2024-05-14T17:08:31.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.533+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/829069471 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "set_id": "471a1ec6", "release": true, "await": true}) -- 0x562705471ce0 con 0x562705597110 2024-05-14T17:08:31.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.533+0000 7f049d9e2700 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562705597110 0x562705599530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:31.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.533+0000 7f04817fa700 10 client.17265 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:31.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.533+0000 7f049d9e2700 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562705597110 0x562705599530 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44eb7527e9046135 server_cookie=f2c4bc08d91ac0c2 in_seq=0 out_seq=0 2024-05-14T17:08:31.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.539+0000 7f04817fa700 1 -- 172.21.15.69:0/829069471 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562705471ce0 con 0x562705597110 2024-05-14T17:08:31.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.539+0000 7f04817fa700 10 client.17265 handle_command_reply: tid=0 2024-05-14T17:08:31.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.539+0000 7f04aa4dcb80 1 client.17265 shutdown 2024-05-14T17:08:31.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.539+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/829069471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 msgr2=0x562705520de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:31.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.539+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x562705520de0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f04900a6220 tx=0x7f04900053c0 comp rx=0 tx=0).stop 2024-05-14T17:08:31.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/829069471 shutdown_connections 2024-05-14T17:08:31.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562705597110 0x562705599530 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:31.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627055176c0 0x562705520de0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:31.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562705511ad0 0x562705520360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:31.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f04817fa700 0 client.17265 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:31.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f04aa4dcb80 1 --2- 172.21.15.69:0/829069471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562705516da0 0x5627055208a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:31.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/829069471 >> 172.21.15.69:0/829069471 conn(0x56270543c350 msgr2=0x5627055161d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:31.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/829069471 shutdown_connections 2024-05-14T17:08:31.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f04aa4dcb80 1 -- 172.21.15.69:0/829069471 wait complete. 2024-05-14T17:08:32.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f04aa4dcb80 20 client.17265 trim_cache size 0 max 16384 2024-05-14T17:08:32.077 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:08:32.112 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 60, 2024-05-14T17:08:32.145 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T17:08:32.179 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 44, 2024-05-14T17:08:32.212 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T17:08:32.252 INFO:teuthology.orchestra.run.smithi069.stdout: "471a1ec6": { 2024-05-14T17:08:32.287 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 44, 2024-05-14T17:08:32.336 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T17:08:32.370 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:08:32.458 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T17:08:32.495 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:08:32.529 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:08:32.637 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T17:08:32.704 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 149.7, 2024-05-14T17:08:32.737 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T17:08:32.771 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T17:08:32.827 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T17:08:32.886 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:08:32.921 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-05-14T17:08:32.954 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:08:33.012 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:08:33.046 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:08:33.079 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:08:33.113 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:08:33.146 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:08:33.179 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:08:33.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f049fc46700 1 -- 172.21.15.69:0/3766489828 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04840b29f0 msgr2=0x7f04840b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:33.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f049fc46700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04840b29f0 0x7f04840b4eb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f049400abd0 tx=0x7f049400a380 comp rx=0 tx=0).stop 2024-05-14T17:08:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f049fc46700 1 -- 172.21.15.69:0/3766489828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0498108520 msgr2=0x7f0498104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f049fc46700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0498108520 0x7f0498104b00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0490007ee0 tx=0x7f049000d3b0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f049fc46700 1 -- 172.21.15.69:0/3766489828 shutdown_connections 2024-05-14T17:08:33.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f049fc46700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f04840b29f0 0x7f04840b4eb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f049fc46700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0498108520 0x7f0498104b00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f049fc46700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0498102440 0x7f0498104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f049fc46700 1 --2- 172.21.15.69:0/3766489828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0498102e00 0x7f04981045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.540+0000 7f049fc46700 1 -- 172.21.15.69:0/3766489828 >> 172.21.15.69:0/3766489828 conn(0x7f04980793f0 msgr2=0x7f049806b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:33.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.541+0000 7f049fc46700 1 -- 172.21.15.69:0/3766489828 shutdown_connections 2024-05-14T17:08:33.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:08:02.541+0000 7f049fc46700 1 -- 172.21.15.69:0/3766489828 wait complete. 2024-05-14T17:08:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0df37fe640 1 -- 172.21.15.69:0/1084044714 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d1c29e9360 con 0x55d1c2b00980 2024-05-14T17:08:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0df37fe640 10 client.17247 handle_command_reply: tid=0 2024-05-14T17:08:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 client.17247 shutdown 2024-05-14T17:08:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 -- 172.21.15.69:0/1084044714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 msgr2=0x55d1c2a89f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a89f60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0a8ed0 tx=0x7f0e0c0a8760 comp rx=0 tx=0).stop 2024-05-14T17:08:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 -- 172.21.15.69:0/1084044714 shutdown_connections 2024-05-14T17:08:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d1c2b00980 0x55d1c2b02da0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1c2a85ad0 0x55d1c2a89f60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0df37fe640 0 client.17247 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1c2a7b210 0x55d1c2a89a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 --2- 172.21.15.69:0/1084044714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d1c2a7a8f0 0x55d1c2a894e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 -- 172.21.15.69:0/1084044714 >> 172.21.15.69:0/1084044714 conn(0x55d1c2a76e00 msgr2=0x55d1c2a7c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 -- 172.21.15.69:0/1084044714 shutdown_connections 2024-05-14T17:08:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 1 -- 172.21.15.69:0/1084044714 wait complete. 2024-05-14T17:08:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.556+0000 7f0e1d429740 20 client.17247 trim_cache size 0 max 16384 2024-05-14T17:08:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:08:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:08:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:08:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 -- 172.21.15.69:0/3998408833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0dfc0b2a10 msgr2=0x7f0dfc0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0dfc0b2a10 0x7f0dfc0b4ed0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55d1c2a7be90 tx=0x7f0e0400c4f0 comp rx=0 tx=0).stop 2024-05-14T17:08:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 -- 172.21.15.69:0/3998408833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e14108a60 msgr2=0x7f0e1419db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e14108a60 0x7f0e1419db30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00b970 tx=0x7f0e0c00be30 comp rx=0 tx=0).stop 2024-05-14T17:08:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 -- 172.21.15.69:0/3998408833 shutdown_connections 2024-05-14T17:08:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0dfc0b2a10 0x7f0dfc0b4ed0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e14108a60 0x7f0e1419db30 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14103410 0x7f0e14199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 --2- 172.21.15.69:0/3998408833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e14102a50 0x7f0e141992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 -- 172.21.15.69:0/3998408833 >> 172.21.15.69:0/3998408833 conn(0x7f0e140fe540 msgr2=0x7f0e140fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 -- 172.21.15.69:0/3998408833 shutdown_connections 2024-05-14T17:08:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:02.557+0000 7f0e1a62a640 1 -- 172.21.15.69:0/3998408833 wait complete. 2024-05-14T17:08:34.919 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:24 smithi092 ceph-mon[35763]: pgmap v586: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-05-14T17:08:34.956 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:26 smithi092 ceph-mon[35763]: pgmap v587: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-14T17:08:34.989 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:26 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:08:35.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:26 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:08:35.107 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:26 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:08:35.140 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:26 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:08:35.172 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:28 smithi092 ceph-mon[35763]: pgmap v588: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:08:35.206 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:30 smithi092 ceph-mon[35763]: pgmap v589: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-14T17:08:35.242 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:32 smithi092 ceph-mon[35763]: pgmap v590: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-14T17:08:35.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:34 smithi092 ceph-mon[35763]: pgmap v591: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-14T17:08:35.339 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:24 smithi203 ceph-mon[35787]: pgmap v586: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-05-14T17:08:35.372 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:26 smithi203 ceph-mon[35787]: pgmap v587: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-14T17:08:35.406 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:26 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:08:35.439 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:26 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:08:35.481 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:26 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:08:35.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:26 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:08:35.547 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:28 smithi203 ceph-mon[35787]: pgmap v588: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:08:35.595 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:30 smithi203 ceph-mon[35787]: pgmap v589: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-14T17:08:35.631 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:32 smithi203 ceph-mon[35787]: pgmap v590: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-14T17:08:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:34 smithi203 ceph-mon[35787]: pgmap v591: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-14T17:08:35.699 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-14T17:08:35.750 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:04 smithi069 ceph-mon[31507]: pgmap v574: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-05-14T17:08:35.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:05 smithi069 ceph-mon[31507]: osdmap e109: 12 total, 12 up, 12 in 2024-05-14T17:08:35.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:06 smithi069 ceph-mon[31507]: pgmap v576: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 65 KiB/s wr, 26 op/s 2024-05-14T17:08:35.856 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:08 smithi069 ceph-mon[31507]: pgmap v577: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 141 KiB/s wr, 40 op/s 2024-05-14T17:08:35.914 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:10 smithi069 ceph-mon[31507]: pgmap v578: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 159 KiB/s wr, 44 op/s 2024-05-14T17:08:35.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:12 smithi069 ceph-mon[31507]: pgmap v579: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 159 KiB/s wr, 44 op/s 2024-05-14T17:08:35.990 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:14 smithi069 ceph-mon[31507]: pgmap v580: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-05-14T17:08:36.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:14 smithi069 ceph-mon[31507]: osdmap e110: 12 total, 12 up, 12 in 2024-05-14T17:08:36.075 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '471a1ec6', total seconds elapsed: 218.9 2024-05-14T17:08:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.661+0000 7f8ac1293640 1 -- 172.21.15.69:0/3457901369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc108a80 msgr2=0x7f8abc108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.661+0000 7f8ac1293640 1 --2- 172.21.15.69:0/3457901369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc108a80 0x7f8abc108e60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009cd0 tx=0x7f8ab402a8e0 comp rx=0 tx=0).stop 2024-05-14T17:08:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.661+0000 7f8ac1293640 1 -- 172.21.15.69:0/3457901369 shutdown_connections 2024-05-14T17:08:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.661+0000 7f8ac1293640 1 --2- 172.21.15.69:0/3457901369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8abc103430 0x7f8abc10f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.661+0000 7f8ac1293640 1 --2- 172.21.15.69:0/3457901369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc102a70 0x7f8abc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.661+0000 7f8ac1293640 1 --2- 172.21.15.69:0/3457901369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc108a80 0x7f8abc108e60 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.661+0000 7f8ac1293640 1 -- 172.21.15.69:0/3457901369 >> 172.21.15.69:0/3457901369 conn(0x7f8abc0fe500 msgr2=0x7f8abc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.661+0000 7f8ac1293640 1 -- 172.21.15.69:0/3457901369 shutdown_connections 2024-05-14T17:08:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.661+0000 7f8ac1293640 1 -- 172.21.15.69:0/3457901369 wait complete. 2024-05-14T17:08:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.661+0000 7f8ac1293640 1 Processor -- start 2024-05-14T17:08:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8ac1293640 1 -- start start 2024-05-14T17:08:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8ac1293640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc102a70 0x7f8abc199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8ac1293640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8abc103430 0x7f8abc199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8ac1293640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc108a80 0x7f8abc19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8ac1293640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc112d90 con 0x7f8abc102a70 2024-05-14T17:08:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8ac1293640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc112c10 con 0x7f8abc103430 2024-05-14T17:08:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8ac1293640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc112f10 con 0x7f8abc108a80 2024-05-14T17:08:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc102a70 0x7f8abc199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc102a70 0x7f8abc199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48876/0 (socket says 172.21.15.69:48876) 2024-05-14T17:08:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8abaffd640 1 -- 172.21.15.69:0/1091479074 learned_addr learned my addr 172.21.15.69:0/1091479074 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8abc103430 0x7f8abc199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8abb7fe640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc108a80 0x7f8abc19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8abaffd640 1 -- 172.21.15.69:0/1091479074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc108a80 msgr2=0x7f8abc19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8abaffd640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc108a80 0x7f8abc19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8abaffd640 1 -- 172.21.15.69:0/1091479074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8abc103430 msgr2=0x7f8abc199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8abaffd640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8abc103430 0x7f8abc199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8abaffd640 1 -- 172.21.15.69:0/1091479074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc19e220 con 0x7f8abc102a70 2024-05-14T17:08:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.662+0000 7f8abb7fe640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc108a80 0x7f8abc19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.663+0000 7f8abaffd640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc102a70 0x7f8abc199230 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8ab402aed0 tx=0x7f8ab4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.663+0000 7f8ab3fff640 1 -- 172.21.15.69:0/1091479074 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4004450 con 0x7f8abc102a70 2024-05-14T17:08:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.663+0000 7f8ab3fff640 1 -- 172.21.15.69:0/1091479074 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab40045f0 con 0x7f8abc102a70 2024-05-14T17:08:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.663+0000 7f8ac1293640 1 -- 172.21.15.69:0/1091479074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc19e4b0 con 0x7f8abc102a70 2024-05-14T17:08:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.663+0000 7f8ab3fff640 1 -- 172.21.15.69:0/1091479074 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4045690 con 0x7f8abc102a70 2024-05-14T17:08:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.663+0000 7f8ac1293640 1 -- 172.21.15.69:0/1091479074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc1a5da0 con 0x7f8abc102a70 2024-05-14T17:08:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.665+0000 7f8ab3fff640 1 -- 172.21.15.69:0/1091479074 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8ab40305a0 con 0x7f8abc102a70 2024-05-14T17:08:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8ac3fa9740 1 Processor -- start 2024-05-14T17:08:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8ac3fa9740 1 -- start start 2024-05-14T17:08:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c17d0 0x56181c9c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 0x56181c9d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56181c934c30 con 0x56181c9d0ee0 2024-05-14T17:08:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56181c9247d0 con 0x56181c9c17d0 2024-05-14T17:08:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56181c928810 con 0x56181c9c20f0 2024-05-14T17:08:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8aba7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8aba7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9d32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48882/0 (socket says 172.21.15.69:48882) 2024-05-14T17:08:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8aba7fc640 1 -- 172.21.15.69:0/2235682782 learned_addr learned my addr 172.21.15.69:0/2235682782 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:37.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8abaffd640 1 --2- 172.21.15.69:0/2235682782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 0x56181c9d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.666+0000 7f8abb7fe640 1 --2- 172.21.15.69:0/2235682782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c17d0 0x56181c9c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8aba7fc640 1 -- 172.21.15.69:0/2235682782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 msgr2=0x56181c9d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8ab3fff640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8aa80b2a10 0x7f8aa80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/2235682782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 0x56181c9d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8aba7fc640 1 -- 172.21.15.69:0/2235682782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c17d0 msgr2=0x56181c9c1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/2235682782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c17d0 0x56181c9c1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8aba7fc640 1 -- 172.21.15.69:0/2235682782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56181c639210 con 0x56181c9d0ee0 2024-05-14T17:08:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8abb7fe640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8aa80b2a10 0x7f8aa80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/2235682782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9d32d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400cbe0 tx=0x7f8aa400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5edc25a41324c0f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8ab3fff640 1 -- 172.21.15.69:0/1091479074 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7f8ab402e030 con 0x7f8abc102a70 2024-05-14T17:08:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8ab17fa640 1 -- 172.21.15.69:0/2235682782 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4010630 con 0x56181c9d0ee0 2024-05-14T17:08:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8ab17fa640 1 -- 172.21.15.69:0/2235682782 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aa4010c80 con 0x56181c9d0ee0 2024-05-14T17:08:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.667+0000 7f8ab17fa640 1 -- 172.21.15.69:0/2235682782 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa401aaf0 con 0x56181c9d0ee0 2024-05-14T17:08:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2235682782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 msgr2=0x56181c9d32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8abb7fe640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8aa80b2a10 0x7f8aa80b4ed0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8aac005fd0 tx=0x7f8aac0046c0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/2235682782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9d32d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400cbe0 tx=0x7f8aa400e720 comp rx=0 tx=0).stop 2024-05-14T17:08:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2235682782 shutdown_connections 2024-05-14T17:08:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/2235682782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9d32d0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/2235682782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 0x56181c9d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/2235682782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c17d0 0x56181c9c1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2235682782 >> 172.21.15.69:0/2235682782 conn(0x56181c9bde00 msgr2=0x56181c9be1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2235682782 shutdown_connections 2024-05-14T17:08:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2235682782 wait complete. 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 -- start start 2024-05-14T17:08:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.668+0000 7f8ac3fa9740 1 Processor -- start 2024-05-14T17:08:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c17d0 0x56181c9d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 0x56181c9cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56181c60a790 con 0x56181c9d0ee0 2024-05-14T17:08:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56181c8e0890 con 0x56181c9c20f0 2024-05-14T17:08:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56181c9d3bb0 con 0x56181c9c17d0 2024-05-14T17:08:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8ac3fa9740 10 client.0 _ll_register_callbacks cb 0x56181c8e3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8abb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8abb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48898/0 (socket says 172.21.15.69:48898) 2024-05-14T17:08:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8abb7fe640 1 -- 172.21.15.69:0/612819474 learned_addr learned my addr 172.21.15.69:0/612819474 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 0x56181c9cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8abaffd640 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c17d0 0x56181c9d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8abb7fe640 1 -- 172.21.15.69:0/612819474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c17d0 msgr2=0x56181c9d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8abb7fe640 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c17d0 0x56181c9d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8abb7fe640 1 -- 172.21.15.69:0/612819474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 msgr2=0x56181c9cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8abb7fe640 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 0x56181c9cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8abb7fe640 1 -- 172.21.15.69:0/612819474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56181c705620 con 0x56181c9d0ee0 2024-05-14T17:08:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8abb7fe640 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9ce0e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0116c0 tx=0x7f8aac005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82170bdf65f2eb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8aa27fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8aa27fc640 1 -- 172.21.15.69:0/612819474 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac01c9c0 con 0x56181c9d0ee0 2024-05-14T17:08:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8aa27fc640 1 -- 172.21.15.69:0/612819474 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7f8aac01cb60 con 0x56181c9d0ee0 2024-05-14T17:08:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.669+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/612819474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa4010c80 con 0x56181c9d0ee0 2024-05-14T17:08:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8aa27fc640 1 -- 172.21.15.69:0/612819474 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8aac01cd60 con 0x56181c9d0ee0 2024-05-14T17:08:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8ac3fa9740 10 client.17277 fetch_fsmap learned FSMap version 60 2024-05-14T17:08:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/612819474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56181c9df750 con 0x56181c9d0ee0 2024-05-14T17:08:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8aa27fc640 1 -- 172.21.15.69:0/612819474 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aa4010c80 con 0x56181c9d0ee0 2024-05-14T17:08:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8aa27fc640 1 -- 172.21.15.69:0/612819474 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 60) ==== 8338+0+0 (secure 0 0 0) 0x7f8aac023070 con 0x56181c9d0ee0 2024-05-14T17:08:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8ac3fa9740 10 client.17277 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:08:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8ac3fa9740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8ac3fa9740 20 client.17277 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56181c9e0cd0 0x56181c9e30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8ac3fa9740 4 client.17277 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:40.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/612819474 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56181c936b80 con 0x56181c9e0cd0 2024-05-14T17:08:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.670+0000 7f8abaffd640 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56181c9e0cd0 0x56181c9e30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8aa27fc640 10 client.17277 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8abaffd640 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56181c9e0cd0 0x56181c9e30c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=699d600f8fa7678f server_cookie=62a5bd6ad3d7edb7 in_seq=0 out_seq=0 2024-05-14T17:08:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8aa27fc640 1 -- 172.21.15.69:0/612819474 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56181c936b80 con 0x56181c9e0cd0 2024-05-14T17:08:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8aa27fc640 10 client.17277 handle_command_reply: tid=0 2024-05-14T17:08:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8ac3fa9740 1 client.17277 shutdown 2024-05-14T17:08:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/612819474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 msgr2=0x56181c9ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9ce0e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0116c0 tx=0x7f8aac005e60 comp rx=0 tx=0).stop 2024-05-14T17:08:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/612819474 shutdown_connections 2024-05-14T17:08:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56181c9e0cd0 0x56181c9e30c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9d0ee0 0x56181c9ce0e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 0x56181c9cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8aa27fc640 0 client.17277 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/612819474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c17d0 0x56181c9d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.671+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/612819474 >> 172.21.15.69:0/612819474 conn(0x56181c9bde00 msgr2=0x56181c9c3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.672+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/612819474 shutdown_connections 2024-05-14T17:08:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.672+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/612819474 wait complete. 2024-05-14T17:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.672+0000 7f8ac3fa9740 20 client.17277 trim_cache size 0 max 16384 2024-05-14T17:08:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8ac3fa9740 1 Processor -- start 2024-05-14T17:08:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8ac3fa9740 1 -- start start 2024-05-14T17:08:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 0x56181c9e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9d0ee0 0x56181c9e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1aecb0 con 0x56181c9c17d0 2024-05-14T17:08:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1ac580 con 0x56181c9d0ee0 2024-05-14T17:08:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1aa3d0 con 0x56181c9c20f0 2024-05-14T17:08:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:41.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48900/0 (socket says 172.21.15.69:48900) 2024-05-14T17:08:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8abaffd640 1 -- 172.21.15.69:0/1118165267 learned_addr learned my addr 172.21.15.69:0/1118165267 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/1118165267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 0x56181c9e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8abaffd640 1 -- 172.21.15.69:0/1118165267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 msgr2=0x56181c9e89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8abaffd640 1 --2- 172.21.15.69:0/1118165267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 0x56181c9e89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8abaffd640 1 -- 172.21.15.69:0/1118165267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9d0ee0 msgr2=0x56181c9e8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8abaffd640 1 --2- 172.21.15.69:0/1118165267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9d0ee0 0x56181c9e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8abaffd640 1 -- 172.21.15.69:0/1118165267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56181c9c1f20 con 0x56181c9c17d0 2024-05-14T17:08:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.687+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/1118165267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 0x56181c9e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8abaffd640 1 --2- 172.21.15.69:0/1118165267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d03a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8ab41018e0 tx=0x7f8ab4109550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164e76f919902782 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ab0ff9640 1 -- 172.21.15.69:0/1118165267 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40ca8d0 con 0x56181c9c17d0 2024-05-14T17:08:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ab0ff9640 1 -- 172.21.15.69:0/1118165267 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab4103e50 con 0x56181c9c17d0 2024-05-14T17:08:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ab0ff9640 1 -- 172.21.15.69:0/1118165267 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40ca6f0 con 0x56181c9c17d0 2024-05-14T17:08:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/1118165267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 msgr2=0x56181c9d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/1118165267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d03a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8ab41018e0 tx=0x7f8ab4109550 comp rx=0 tx=0).stop 2024-05-14T17:08:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/1118165267 shutdown_connections 2024-05-14T17:08:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/1118165267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9d0ee0 0x56181c9e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/1118165267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9c20f0 0x56181c9e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/1118165267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d03a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/1118165267 >> 172.21.15.69:0/1118165267 conn(0x56181c9bde00 msgr2=0x56181c9bd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.688+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/1118165267 shutdown_connections 2024-05-14T17:08:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/1118165267 wait complete. 2024-05-14T17:08:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8ac3fa9740 1 -- start start 2024-05-14T17:08:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8ac3fa9740 1 Processor -- start 2024-05-14T17:08:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 0x56181c9d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8ac3fa9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9d0ee0 0x56181ca4a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1aaa40 con 0x56181c9c17d0 2024-05-14T17:08:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1abeb0 con 0x56181c9c20f0 2024-05-14T17:08:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8ac3fa9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1ab110 con 0x56181c9d0ee0 2024-05-14T17:08:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8ac3fa9740 10 client.0 _ll_register_callbacks cb 0x7f8aac00ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48902/0 (socket says 172.21.15.69:48902) 2024-05-14T17:08:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8abaffd640 1 -- 172.21.15.69:0/2851041631 learned_addr learned my addr 172.21.15.69:0/2851041631 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8abb7fe640 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9d0ee0 0x56181ca4a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 0x56181c9d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8abaffd640 1 -- 172.21.15.69:0/2851041631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9d0ee0 msgr2=0x56181ca4a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8abaffd640 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9d0ee0 0x56181ca4a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8abaffd640 1 -- 172.21.15.69:0/2851041631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 msgr2=0x56181c9d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8abaffd640 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 0x56181c9d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8abaffd640 1 -- 172.21.15.69:0/2851041631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56181c9e3430 con 0x56181c9c17d0 2024-05-14T17:08:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8abb7fe640 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9d0ee0 0x56181ca4a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.689+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 0x56181c9d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8abaffd640 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d5210 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40caf10 tx=0x7f8ab4101790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1758c5bfacdac31a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8aa27fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8aa27fc640 1 -- 172.21.15.69:0/2851041631 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40ca6f0 con 0x56181c9c17d0 2024-05-14T17:08:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8aa27fc640 1 -- 172.21.15.69:0/2851041631 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7f8ab4030890 con 0x56181c9c17d0 2024-05-14T17:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2851041631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ab4103e50 con 0x56181c9c17d0 2024-05-14T17:08:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8aa27fc640 1 -- 172.21.15.69:0/2851041631 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8ab40ca8d0 con 0x56181c9c17d0 2024-05-14T17:08:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8ac3fa9740 10 client.17283 fetch_fsmap learned FSMap version 60 2024-05-14T17:08:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2851041631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56181ca4bc20 con 0x56181c9c17d0 2024-05-14T17:08:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8aa27fc640 1 -- 172.21.15.69:0/2851041631 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab4103e50 con 0x56181c9c17d0 2024-05-14T17:08:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8aa27fc640 1 -- 172.21.15.69:0/2851041631 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 60) ==== 8338+0+0 (secure 0 0 0) 0x7f8ab404d070 con 0x56181c9c17d0 2024-05-14T17:08:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8ac3fa9740 10 client.17283 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:08:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8ac3fa9740 10 client.17283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8ac3fa9740 20 client.17283 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56181ca4c270 0x56181ca4e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8ac3fa9740 4 client.17283 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2851041631 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56181c930360 con 0x56181ca4c270 2024-05-14T17:08:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.690+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56181ca4c270 0x56181ca4e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.691+0000 7f8aa27fc640 10 client.17283 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.691+0000 7f8aba7fc640 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56181ca4c270 0x56181ca4e690 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=901529977a68eba1 server_cookie=8d37a99d4de4fc6d in_seq=0 out_seq=0 2024-05-14T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8aa27fc640 1 -- 172.21.15.69:0/2851041631 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56181c930360 con 0x56181ca4c270 2024-05-14T17:08:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8aa27fc640 10 client.17283 handle_command_reply: tid=0 2024-05-14T17:08:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 client.17283 shutdown 2024-05-14T17:08:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2851041631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 msgr2=0x56181c9d5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d5210 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40caf10 tx=0x7f8ab4101790 comp rx=0 tx=0).stop 2024-05-14T17:08:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2851041631 shutdown_connections 2024-05-14T17:08:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56181ca4c270 0x56181ca4e690 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56181c9d0ee0 0x56181ca4a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56181c9c20f0 0x56181c9d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 --2- 172.21.15.69:0/2851041631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56181c9c17d0 0x56181c9d5210 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8aa27fc640 0 client.17283 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2851041631 >> 172.21.15.69:0/2851041631 conn(0x56181c9bde00 msgr2=0x56181c9c3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2851041631 shutdown_connections 2024-05-14T17:08:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 1 -- 172.21.15.69:0/2851041631 wait complete. 2024-05-14T17:08:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac3fa9740 20 client.17283 trim_cache size 0 max 16384 2024-05-14T17:08:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:08:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:08:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:08:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac1293640 1 -- 172.21.15.69:0/1091479074 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8aa80b2a10 msgr2=0x7f8aa80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac1293640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8aa80b2a10 0x7f8aa80b4ed0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8aac005fd0 tx=0x7f8aac0046c0 comp rx=0 tx=0).stop 2024-05-14T17:08:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac1293640 1 -- 172.21.15.69:0/1091479074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc102a70 msgr2=0x7f8abc199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac1293640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc102a70 0x7f8abc199230 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8ab402aed0 tx=0x7f8ab4003c20 comp rx=0 tx=0).stop 2024-05-14T17:08:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac1293640 1 -- 172.21.15.69:0/1091479074 shutdown_connections 2024-05-14T17:08:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac1293640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8aa80b2a10 0x7f8aa80b4ed0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac1293640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc108a80 0x7f8abc19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac1293640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8abc103430 0x7f8abc199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:44.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac1293640 1 --2- 172.21.15.69:0/1091479074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8abc102a70 0x7f8abc199230 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.712+0000 7f8ac1293640 1 -- 172.21.15.69:0/1091479074 >> 172.21.15.69:0/1091479074 conn(0x7f8abc0fe500 msgr2=0x7f8abc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.713+0000 7f8ac1293640 1 -- 172.21.15.69:0/1091479074 shutdown_connections 2024-05-14T17:08:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:15.713+0000 7f8ac1293640 1 -- 172.21.15.69:0/1091479074 wait complete. 2024-05-14T17:08:44.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:16 smithi069 ceph-mon[31507]: pgmap v582: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 108 KiB/s wr, 24 op/s 2024-05-14T17:08:44.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:18 smithi069 ceph-mon[31507]: pgmap v583: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-05-14T17:08:44.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:19 smithi069 ceph-mon[31507]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-14T17:08:44.888 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:19 smithi069 ceph-mon[31507]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 5 up:standby 2024-05-14T17:08:44.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:20 smithi069 ceph-mon[31507]: pgmap v584: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:08:44.963 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:22 smithi069 ceph-mon[31507]: pgmap v585: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-05-14T17:08:45.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:24 smithi069 ceph-mon[31507]: pgmap v586: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-05-14T17:08:45.047 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:26 smithi069 ceph-mon[31507]: pgmap v587: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-14T17:08:45.080 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:26 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:08:45.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:26 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:08:45.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:26 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:08:45.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:26 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:08:45.230 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:28 smithi069 ceph-mon[31507]: pgmap v588: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:08:45.265 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:36 smithi092 ceph-mon[35763]: pgmap v592: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:08:45.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:38 smithi092 ceph-mon[35763]: pgmap v593: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-14T17:08:45.330 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:40 smithi092 ceph-mon[35763]: pgmap v594: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:08:45.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:42 smithi092 ceph-mon[35763]: pgmap v595: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:08:45.438 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:44 smithi092 ceph-mon[35763]: pgmap v596: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T17:08:45.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:36 smithi203 ceph-mon[35787]: pgmap v592: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:08:45.505 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:38 smithi203 ceph-mon[35787]: pgmap v593: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-14T17:08:45.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:40 smithi203 ceph-mon[35787]: pgmap v594: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:08:45.606 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:42 smithi203 ceph-mon[35787]: pgmap v595: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:08:45.639 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:44 smithi203 ceph-mon[35787]: pgmap v596: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T17:08:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.809+0000 7fd427c3f640 1 -- 172.21.15.69:0/483264246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd420102340 msgr2=0x7fd42010e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.809+0000 7fd427c3f640 1 --2- 172.21.15.69:0/483264246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd420102340 0x7fd42010e800 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd418009cd0 tx=0x7fd41802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:08:45.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.809+0000 7fd427c3f640 1 -- 172.21.15.69:0/483264246 shutdown_connections 2024-05-14T17:08:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.809+0000 7fd427c3f640 1 --2- 172.21.15.69:0/483264246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd420102340 0x7fd42010e800 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.809+0000 7fd427c3f640 1 --2- 172.21.15.69:0/483264246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd420101980 0x7fd420101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.809+0000 7fd427c3f640 1 --2- 172.21.15.69:0/483264246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420107940 0x7fd420107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.809+0000 7fd427c3f640 1 -- 172.21.15.69:0/483264246 >> 172.21.15.69:0/483264246 conn(0x7fd420078ad0 msgr2=0x7fd420078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.809+0000 7fd427c3f640 1 -- 172.21.15.69:0/483264246 shutdown_connections 2024-05-14T17:08:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.809+0000 7fd427c3f640 1 -- 172.21.15.69:0/483264246 wait complete. 2024-05-14T17:08:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.810+0000 7fd427c3f640 1 Processor -- start 2024-05-14T17:08:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.810+0000 7fd427c3f640 1 -- start start 2024-05-14T17:08:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.810+0000 7fd427c3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420101980 0x7fd420199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.810+0000 7fd427c3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd420102340 0x7fd420199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.810+0000 7fd427c3f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd420107940 0x7fd42019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.810+0000 7fd427c3f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd420111c00 con 0x7fd420102340 2024-05-14T17:08:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.810+0000 7fd427c3f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd420111a80 con 0x7fd420107940 2024-05-14T17:08:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.810+0000 7fd427c3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd420111d80 con 0x7fd420101980 2024-05-14T17:08:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.810+0000 7fd4251da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd420102340 0x7fd420199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd4251da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd420102340 0x7fd420199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56090/0 (socket says 172.21.15.69:56090) 2024-05-14T17:08:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd4251da640 1 -- 172.21.15.69:0/463771602 learned_addr learned my addr 172.21.15.69:0/463771602 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd4261dc640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd420107940 0x7fd42019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd4259db640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420101980 0x7fd420199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd4251da640 1 -- 172.21.15.69:0/463771602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420101980 msgr2=0x7fd420199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd4251da640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420101980 0x7fd420199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd4251da640 1 -- 172.21.15.69:0/463771602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd420107940 msgr2=0x7fd42019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd4251da640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd420107940 0x7fd42019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd4251da640 1 -- 172.21.15.69:0/463771602 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42019e230 con 0x7fd420102340 2024-05-14T17:08:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd4251da640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd420102340 0x7fd420199780 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00d950 tx=0x7fd41c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd416ffd640 1 -- 172.21.15.69:0/463771602 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c017070 con 0x7fd420102340 2024-05-14T17:08:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd427c3f640 1 -- 172.21.15.69:0/463771602 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42019e520 con 0x7fd420102340 2024-05-14T17:08:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd416ffd640 1 -- 172.21.15.69:0/463771602 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c009c30 con 0x7fd420102340 2024-05-14T17:08:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd416ffd640 1 -- 172.21.15.69:0/463771602 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00f620 con 0x7fd420102340 2024-05-14T17:08:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.811+0000 7fd427c3f640 1 -- 172.21.15.69:0/463771602 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4201a5db0 con 0x7fd420102340 2024-05-14T17:08:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.814+0000 7fd42aa31740 1 Processor -- start 2024-05-14T17:08:46.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd42aa31740 1 -- start start 2024-05-14T17:08:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd416ffd640 1 -- 172.21.15.69:0/463771602 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd41c010460 con 0x7fd420102340 2024-05-14T17:08:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d520648f0 0x556d52064cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d52065210 0x556d5206f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d52071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd42aa31740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556d51fd7c30 con 0x556d5206fad0 2024-05-14T17:08:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd42aa31740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556d51fc77d0 con 0x556d520648f0 2024-05-14T17:08:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd42aa31740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556d51fcb810 con 0x556d52065210 2024-05-14T17:08:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd4251da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d52071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd4251da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d52071ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56092/0 (socket says 172.21.15.69:56092) 2024-05-14T17:08:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd4251da640 1 -- 172.21.15.69:0/2179589044 learned_addr learned my addr 172.21.15.69:0/2179589044 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd4259db640 1 --2- 172.21.15.69:0/2179589044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d52065210 0x556d5206f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.815+0000 7fd4261dc640 1 --2- 172.21.15.69:0/2179589044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d520648f0 0x556d52064cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd4251da640 1 -- 172.21.15.69:0/2179589044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d52065210 msgr2=0x556d5206f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd4251da640 1 --2- 172.21.15.69:0/2179589044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d52065210 0x556d5206f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd4251da640 1 -- 172.21.15.69:0/2179589044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d520648f0 msgr2=0x556d52064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd4251da640 1 --2- 172.21.15.69:0/2179589044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d520648f0 0x556d52064cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd4251da640 1 -- 172.21.15.69:0/2179589044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d51cdc210 con 0x556d5206fad0 2024-05-14T17:08:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd4251da640 1 --2- 172.21.15.69:0/2179589044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d52071ec0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0df910 tx=0x7fd41c0ea890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6cd2f4a76c2ee2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd416ffd640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd40c0b2a10 0x7fd40c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd40bfff640 1 -- 172.21.15.69:0/2179589044 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0f5070 con 0x556d5206fad0 2024-05-14T17:08:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd4261dc640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd40c0b2a10 0x7fd40c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd40bfff640 1 -- 172.21.15.69:0/2179589044 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c0a9040 con 0x556d5206fad0 2024-05-14T17:08:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd40bfff640 1 -- 172.21.15.69:0/2179589044 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c07e180 con 0x556d5206fad0 2024-05-14T17:08:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.816+0000 7fd416ffd640 1 -- 172.21.15.69:0/463771602 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7fd41c0e2260 con 0x7fd420102340 2024-05-14T17:08:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd42aa31740 1 -- 172.21.15.69:0/2179589044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 msgr2=0x556d52071ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd4261dc640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd40c0b2a10 0x7fd40c0b4ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd418005e10 tx=0x7fd41803d040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2179589044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d52071ec0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0df910 tx=0x7fd41c0ea890 comp rx=0 tx=0).stop 2024-05-14T17:08:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd42aa31740 1 -- 172.21.15.69:0/2179589044 shutdown_connections 2024-05-14T17:08:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2179589044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d52071ec0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2179589044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d52065210 0x556d5206f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2179589044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d520648f0 0x556d52064cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd42aa31740 1 -- 172.21.15.69:0/2179589044 >> 172.21.15.69:0/2179589044 conn(0x556d52060e00 msgr2=0x556d520611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd42aa31740 1 -- 172.21.15.69:0/2179589044 shutdown_connections 2024-05-14T17:08:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd42aa31740 1 -- 172.21.15.69:0/2179589044 wait complete. 2024-05-14T17:08:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.817+0000 7fd42aa31740 1 -- start start 2024-05-14T17:08:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd42aa31740 1 Processor -- start 2024-05-14T17:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d520648f0 0x556d5206f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d5206c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d5206fad0 0x556d5206cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd42aa31740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556d51cad790 con 0x556d52065210 2024-05-14T17:08:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd42aa31740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556d51f83890 con 0x556d520648f0 2024-05-14T17:08:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd42aa31740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556d520727a0 con 0x556d5206fad0 2024-05-14T17:08:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd42aa31740 10 client.0 _ll_register_callbacks cb 0x556d51f86d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd4251da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d5206c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd4251da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d5206c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56096/0 (socket says 172.21.15.69:56096) 2024-05-14T17:08:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd4251da640 1 -- 172.21.15.69:0/2766670258 learned_addr learned my addr 172.21.15.69:0/2766670258 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd4261dc640 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d5206fad0 0x556d5206cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd4259db640 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d520648f0 0x556d5206f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd4251da640 1 -- 172.21.15.69:0/2766670258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d5206fad0 msgr2=0x556d5206cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd4251da640 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d5206fad0 0x556d5206cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd4251da640 1 -- 172.21.15.69:0/2766670258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d520648f0 msgr2=0x556d5206f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd4251da640 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d520648f0 0x556d5206f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.818+0000 7fd4251da640 1 -- 172.21.15.69:0/2766670258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d51da8620 con 0x556d52065210 2024-05-14T17:08:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd4251da640 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d5206c6e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0a67a0 tx=0x7fd41c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd2776185f12ed1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd4097fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd4097fa640 1 -- 172.21.15.69:0/2766670258 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c020030 con 0x556d52065210 2024-05-14T17:08:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd4097fa640 1 -- 172.21.15.69:0/2766670258 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7fd41c0a57c0 con 0x556d52065210 2024-05-14T17:08:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd42aa31740 1 -- 172.21.15.69:0/2766670258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41c0a9040 con 0x556d52065210 2024-05-14T17:08:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd4097fa640 1 -- 172.21.15.69:0/2766670258 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd41c07d230 con 0x556d52065210 2024-05-14T17:08:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd42aa31740 10 client.17295 fetch_fsmap learned FSMap version 61 2024-05-14T17:08:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd42aa31740 1 -- 172.21.15.69:0/2766670258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x556d5207e170 con 0x556d52065210 2024-05-14T17:08:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd4097fa640 1 -- 172.21.15.69:0/2766670258 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c0a9040 con 0x556d52065210 2024-05-14T17:08:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd4097fa640 1 -- 172.21.15.69:0/2766670258 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fd41c0a9790 con 0x556d52065210 2024-05-14T17:08:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd42aa31740 10 client.17295 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:08:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd42aa31740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd42aa31740 20 client.17295 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556d5207f4c0 0x556d520818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd42aa31740 4 client.17295 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.819+0000 7fd42aa31740 1 -- 172.21.15.69:0/2766670258 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d51fd9b80 con 0x556d5207f4c0 2024-05-14T17:08:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.820+0000 7fd4259db640 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556d5207f4c0 0x556d520818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.820+0000 7fd4097fa640 10 client.17295 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.820+0000 7fd4259db640 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556d5207f4c0 0x556d520818b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef877d9792a5629 server_cookie=c793a69dbef41f3b in_seq=0 out_seq=0 2024-05-14T17:08:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd4097fa640 1 -- 172.21.15.69:0/2766670258 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d51fd9b80 con 0x556d5207f4c0 2024-05-14T17:08:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd4097fa640 10 client.17295 handle_command_reply: tid=0 2024-05-14T17:08:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 client.17295 shutdown 2024-05-14T17:08:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 -- 172.21.15.69:0/2766670258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 msgr2=0x556d5206c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d5206c6e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0a67a0 tx=0x7fd41c010cb0 comp rx=0 tx=0).stop 2024-05-14T17:08:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 -- 172.21.15.69:0/2766670258 shutdown_connections 2024-05-14T17:08:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556d5207f4c0 0x556d520818b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d5206fad0 0x556d5206cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:49.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d5206c6e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd4097fa640 0 client.17295 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2766670258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d520648f0 0x556d5206f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 -- 172.21.15.69:0/2766670258 >> 172.21.15.69:0/2766670258 conn(0x556d52060e00 msgr2=0x556d52060100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 -- 172.21.15.69:0/2766670258 shutdown_connections 2024-05-14T17:08:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 1 -- 172.21.15.69:0/2766670258 wait complete. 2024-05-14T17:08:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.821+0000 7fd42aa31740 20 client.17295 trim_cache size 0 max 16384 2024-05-14T17:08:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.836+0000 7fd42aa31740 1 Processor -- start 2024-05-14T17:08:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.836+0000 7fd42aa31740 1 -- start start 2024-05-14T17:08:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.836+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d520648f0 0x556d5206ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.836+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d52069940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.836+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d5206fad0 0x556d52069e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.836+0000 7fd42aa31740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201b0100 con 0x556d52065210 2024-05-14T17:08:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.836+0000 7fd42aa31740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201b07d0 con 0x556d5206fad0 2024-05-14T17:08:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.836+0000 7fd42aa31740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201ae0e0 con 0x556d520648f0 2024-05-14T17:08:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4251da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d52069940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4251da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d52069940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56112/0 (socket says 172.21.15.69:56112) 2024-05-14T17:08:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4251da640 1 -- 172.21.15.69:0/889634723 learned_addr learned my addr 172.21.15.69:0/889634723 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4261dc640 1 --2- 172.21.15.69:0/889634723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d5206fad0 0x556d52069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4259db640 1 --2- 172.21.15.69:0/889634723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d520648f0 0x556d5206ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4251da640 1 -- 172.21.15.69:0/889634723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d520648f0 msgr2=0x556d5206ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:50.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4251da640 1 --2- 172.21.15.69:0/889634723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d520648f0 0x556d5206ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4251da640 1 -- 172.21.15.69:0/889634723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d5206fad0 msgr2=0x556d52069e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4251da640 1 --2- 172.21.15.69:0/889634723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d5206fad0 0x556d52069e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4251da640 1 -- 172.21.15.69:0/889634723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d52065040 con 0x556d52065210 2024-05-14T17:08:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd4251da640 1 --2- 172.21.15.69:0/889634723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d52069940 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd41c07dbb0 tx=0x7fd41c0ea890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db07e056d816cf4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd40b7fe640 1 -- 172.21.15.69:0/889634723 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c020070 con 0x556d52065210 2024-05-14T17:08:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd40b7fe640 1 -- 172.21.15.69:0/889634723 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c0e5b80 con 0x556d52065210 2024-05-14T17:08:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.837+0000 7fd40b7fe640 1 -- 172.21.15.69:0/889634723 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c07d420 con 0x556d52065210 2024-05-14T17:08:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 -- 172.21.15.69:0/889634723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 msgr2=0x556d52069940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 --2- 172.21.15.69:0/889634723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d52069940 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd41c07dbb0 tx=0x7fd41c0ea890 comp rx=0 tx=0).stop 2024-05-14T17:08:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 -- 172.21.15.69:0/889634723 shutdown_connections 2024-05-14T17:08:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 --2- 172.21.15.69:0/889634723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d5206fad0 0x556d52069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 --2- 172.21.15.69:0/889634723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d52065210 0x556d52069940 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 --2- 172.21.15.69:0/889634723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d520648f0 0x556d5206ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 -- 172.21.15.69:0/889634723 >> 172.21.15.69:0/889634723 conn(0x556d52060e00 msgr2=0x556d52060100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 -- 172.21.15.69:0/889634723 shutdown_connections 2024-05-14T17:08:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 -- 172.21.15.69:0/889634723 wait complete. 2024-05-14T17:08:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 -- start start 2024-05-14T17:08:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.838+0000 7fd42aa31740 1 Processor -- start 2024-05-14T17:08:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d520648f0 0x556d520e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d52065210 0x556d520e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd42aa31740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d520e0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd42aa31740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201ada10 con 0x556d5206fad0 2024-05-14T17:08:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd42aa31740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201ac620 con 0x556d52065210 2024-05-14T17:08:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd42aa31740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201acc70 con 0x556d520648f0 2024-05-14T17:08:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd42aa31740 10 client.0 _ll_register_callbacks cb 0x7fd41c0ea340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4261dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d520e0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4261dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d520e0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56128/0 (socket says 172.21.15.69:56128) 2024-05-14T17:08:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4261dc640 1 -- 172.21.15.69:0/2141241985 learned_addr learned my addr 172.21.15.69:0/2141241985 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4259db640 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d520648f0 0x556d520e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4251da640 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d52065210 0x556d520e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4261dc640 1 -- 172.21.15.69:0/2141241985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d520648f0 msgr2=0x556d520e67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4261dc640 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d520648f0 0x556d520e67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4261dc640 1 -- 172.21.15.69:0/2141241985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d52065210 msgr2=0x556d520e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4261dc640 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d52065210 0x556d520e6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4261dc640 1 -- 172.21.15.69:0/2141241985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d520e12b0 con 0x556d5206fad0 2024-05-14T17:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4261dc640 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d520e0bc0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd41802ae00 tx=0x7fd4180095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13db735d097845e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4097fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4097fa640 1 -- 172.21.15.69:0/2141241985 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418058060 con 0x556d5206fad0 2024-05-14T17:08:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd4097fa640 1 -- 172.21.15.69:0/2141241985 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7fd4180307a0 con 0x556d5206fad0 2024-05-14T17:08:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.839+0000 7fd42aa31740 1 -- 172.21.15.69:0/2141241985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41c0e5b80 con 0x556d5206fad0 2024-05-14T17:08:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd4097fa640 1 -- 172.21.15.69:0/2141241985 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd418004820 con 0x556d5206fad0 2024-05-14T17:08:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd42aa31740 10 client.17301 fetch_fsmap learned FSMap version 61 2024-05-14T17:08:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd42aa31740 1 -- 172.21.15.69:0/2141241985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x556d520edfc0 con 0x556d5206fad0 2024-05-14T17:08:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd4097fa640 1 -- 172.21.15.69:0/2141241985 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c0e5b80 con 0x556d5206fad0 2024-05-14T17:08:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd4097fa640 1 -- 172.21.15.69:0/2141241985 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fd418030070 con 0x556d5206fad0 2024-05-14T17:08:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd42aa31740 10 client.17301 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:08:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd42aa31740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd42aa31740 20 client.17301 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556d520ee740 0x556d520f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd42aa31740 4 client.17301 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd42aa31740 1 -- 172.21.15.69:0/2141241985 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d51fd3360 con 0x556d520ee740 2024-05-14T17:08:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.840+0000 7fd4259db640 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556d520ee740 0x556d520f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.841+0000 7fd4097fa640 10 client.17301 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.841+0000 7fd4259db640 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556d520ee740 0x556d520f0b60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd4e16044dff3cf server_cookie=85140d4e3a9d1698 in_seq=0 out_seq=0 2024-05-14T17:08:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd4097fa640 1 -- 172.21.15.69:0/2141241985 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556d51fd3360 con 0x556d520ee740 2024-05-14T17:08:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd4097fa640 10 client.17301 handle_command_reply: tid=0 2024-05-14T17:08:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 client.17301 shutdown 2024-05-14T17:08:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 -- 172.21.15.69:0/2141241985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 msgr2=0x556d520e0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d520e0bc0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd41802ae00 tx=0x7fd4180095c0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 -- 172.21.15.69:0/2141241985 shutdown_connections 2024-05-14T17:08:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x556d520ee740 0x556d520f0b60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d5206fad0 0x556d520e0bc0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d52065210 0x556d520e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 --2- 172.21.15.69:0/2141241985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d520648f0 0x556d520e67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd4097fa640 0 client.17301 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 -- 172.21.15.69:0/2141241985 >> 172.21.15.69:0/2141241985 conn(0x556d52060e00 msgr2=0x556d5207a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 -- 172.21.15.69:0/2141241985 shutdown_connections 2024-05-14T17:08:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 1 -- 172.21.15.69:0/2141241985 wait complete. 2024-05-14T17:08:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.850+0000 7fd42aa31740 20 client.17301 trim_cache size 0 max 16384 2024-05-14T17:08:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:08:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:08:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:08:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 -- 172.21.15.69:0/463771602 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd40c0b2a10 msgr2=0x7fd40c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd40c0b2a10 0x7fd40c0b4ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd418005e10 tx=0x7fd41803d040 comp rx=0 tx=0).stop 2024-05-14T17:08:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 -- 172.21.15.69:0/463771602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd420102340 msgr2=0x7fd420199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd420102340 0x7fd420199780 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00d950 tx=0x7fd41c00de10 comp rx=0 tx=0).stop 2024-05-14T17:08:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 -- 172.21.15.69:0/463771602 shutdown_connections 2024-05-14T17:08:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd40c0b2a10 0x7fd40c0b4ed0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd420107940 0x7fd42019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd420102340 0x7fd420199780 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 --2- 172.21.15.69:0/463771602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420101980 0x7fd420199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 -- 172.21.15.69:0/463771602 >> 172.21.15.69:0/463771602 conn(0x7fd420078ad0 msgr2=0x7fd42006a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 -- 172.21.15.69:0/463771602 shutdown_connections 2024-05-14T17:08:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:28.851+0000 7fd427c3f640 1 -- 172.21.15.69:0/463771602 wait complete. 2024-05-14T17:08:53.956 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:30 smithi069 ceph-mon[31507]: pgmap v589: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-14T17:08:54.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:32 smithi069 ceph-mon[31507]: pgmap v590: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-14T17:08:54.047 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:34 smithi069 ceph-mon[31507]: pgmap v591: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-14T17:08:54.080 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:36 smithi069 ceph-mon[31507]: pgmap v592: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:08:54.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:38 smithi069 ceph-mon[31507]: pgmap v593: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-14T17:08:54.147 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:40 smithi069 ceph-mon[31507]: pgmap v594: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:08:54.216 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:46 smithi092 ceph-mon[35763]: pgmap v597: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:08:54.247 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:48 smithi092 ceph-mon[35763]: pgmap v598: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-05-14T17:08:54.280 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:50 smithi092 ceph-mon[35763]: pgmap v599: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 6 op/s 2024-05-14T17:08:54.314 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:52 smithi092 ceph-mon[35763]: pgmap v600: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 6 op/s 2024-05-14T17:08:54.347 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:46 smithi203 ceph-mon[35787]: pgmap v597: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:08:54.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:48 smithi203 ceph-mon[35787]: pgmap v598: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-05-14T17:08:54.439 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:50 smithi203 ceph-mon[35787]: pgmap v599: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 6 op/s 2024-05-14T17:08:54.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:52 smithi203 ceph-mon[35787]: pgmap v600: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 6 op/s 2024-05-14T17:08:54.507 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:54 smithi092 ceph-mon[35763]: pgmap v601: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-05-14T17:08:54.556 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:54 smithi203 ceph-mon[35787]: pgmap v601: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-05-14T17:08:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.949+0000 7f7a2032b640 1 -- 172.21.15.69:0/672400916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a18102a30 msgr2=0x7f7a18102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:54.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.949+0000 7f7a2032b640 1 --2- 172.21.15.69:0/672400916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a18102a30 0x7f7a18102eb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009930 tx=0x7f7a0c02a420 comp rx=0 tx=0).stop 2024-05-14T17:08:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.949+0000 7f7a2032b640 1 -- 172.21.15.69:0/672400916 shutdown_connections 2024-05-14T17:08:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.949+0000 7f7a2032b640 1 --2- 172.21.15.69:0/672400916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a181033f0 0x7f7a1810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.949+0000 7f7a2032b640 1 --2- 172.21.15.69:0/672400916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a18102a30 0x7f7a18102eb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.949+0000 7f7a2032b640 1 --2- 172.21.15.69:0/672400916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a18108a40 0x7f7a18108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.949+0000 7f7a2032b640 1 -- 172.21.15.69:0/672400916 >> 172.21.15.69:0/672400916 conn(0x7f7a18079ee0 msgr2=0x7f7a1807a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.949+0000 7f7a2032b640 1 -- 172.21.15.69:0/672400916 shutdown_connections 2024-05-14T17:08:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.949+0000 7f7a2032b640 1 -- 172.21.15.69:0/672400916 wait complete. 2024-05-14T17:08:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a2032b640 1 Processor -- start 2024-05-14T17:08:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a2032b640 1 -- start start 2024-05-14T17:08:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a2032b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a18102a30 0x7f7a18199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a2032b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a181033f0 0x7f7a18199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a2032b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a18108a40 0x7f7a1819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a2032b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18112d00 con 0x7f7a18102a30 2024-05-14T17:08:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a2032b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18112b80 con 0x7f7a18108a40 2024-05-14T17:08:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a2032b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18112e80 con 0x7f7a181033f0 2024-05-14T17:08:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a1e0c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a18102a30 0x7f7a18199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a1e0c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a18102a30 0x7f7a18199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37976/0 (socket says 172.21.15.69:37976) 2024-05-14T17:08:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a1e0c7640 1 -- 172.21.15.69:0/2576853600 learned_addr learned my addr 172.21.15.69:0/2576853600 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a181033f0 0x7f7a18199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.950+0000 7f7a1e8c8640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a18108a40 0x7f7a1819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a1e0c7640 1 -- 172.21.15.69:0/2576853600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a181033f0 msgr2=0x7f7a18199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a181033f0 0x7f7a18199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a1e0c7640 1 -- 172.21.15.69:0/2576853600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a18108a40 msgr2=0x7f7a1819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a18108a40 0x7f7a1819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a1e0c7640 1 -- 172.21.15.69:0/2576853600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1819e130 con 0x7f7a18102a30 2024-05-14T17:08:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a18102a30 0x7f7a18199140 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800c920 tx=0x7f7a0800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a177fe640 1 -- 172.21.15.69:0/2576853600 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08007c70 con 0x7f7a18102a30 2024-05-14T17:08:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a177fe640 1 -- 172.21.15.69:0/2576853600 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a08007e10 con 0x7f7a18102a30 2024-05-14T17:08:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a177fe640 1 -- 172.21.15.69:0/2576853600 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08018820 con 0x7f7a18102a30 2024-05-14T17:08:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a2032b640 1 -- 172.21.15.69:0/2576853600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1819e420 con 0x7f7a18102a30 2024-05-14T17:08:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.951+0000 7f7a2032b640 1 -- 172.21.15.69:0/2576853600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a181a5cb0 con 0x7f7a18102a30 2024-05-14T17:08:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.954+0000 7f7a23049740 1 Processor -- start 2024-05-14T17:08:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a23049740 1 -- start start 2024-05-14T17:08:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a177fe640 1 -- 172.21.15.69:0/2576853600 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7a0800f450 con 0x7f7a18102a30 2024-05-14T17:08:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a23049740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fc9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a23049740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a23049740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fd6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a23049740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cd02f3cc30 con 0x55cd02fca210 2024-05-14T17:08:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a23049740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cd02f2c7d0 con 0x55cd02fd4ad0 2024-05-14T17:08:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a23049740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cd02f30810 con 0x55cd02fc98f0 2024-05-14T17:08:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a1e0c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a1e0c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37992/0 (socket says 172.21.15.69:37992) 2024-05-14T17:08:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a1e0c7640 1 -- 172.21.15.69:0/3953104898 learned_addr learned my addr 172.21.15.69:0/3953104898 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a1e8c8640 1 --2- 172.21.15.69:0/3953104898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fc9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/3953104898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fd6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.955+0000 7f7a1e0c7640 1 -- 172.21.15.69:0/3953104898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 msgr2=0x55cd02fc9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/3953104898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fc9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a1e0c7640 1 -- 172.21.15.69:0/3953104898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 msgr2=0x55cd02fd6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/3953104898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fd6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a1e0c7640 1 -- 172.21.15.69:0/3953104898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd02c41210 con 0x55cd02fca210 2024-05-14T17:08:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/3953104898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fd6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/3953104898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd4590 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7a080dfd00 tx=0x7f7a080e8cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f507db9bd9bee22 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a177fe640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7a000b2a10 0x7f7a000b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a14bfe640 1 -- 172.21.15.69:0/3953104898 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a080f1070 con 0x55cd02fca210 2024-05-14T17:08:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a1e8c8640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7a000b2a10 0x7f7a000b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a14bfe640 1 -- 172.21.15.69:0/3953104898 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a080a9430 con 0x55cd02fca210 2024-05-14T17:08:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a14bfe640 1 -- 172.21.15.69:0/3953104898 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a080a96f0 con 0x55cd02fca210 2024-05-14T17:08:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.956+0000 7f7a177fe640 1 -- 172.21.15.69:0/2576853600 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7f7a080e2650 con 0x7f7a18102a30 2024-05-14T17:08:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 -- 172.21.15.69:0/3953104898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 msgr2=0x55cd02fd4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a1e8c8640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7a000b2a10 0x7f7a000b4ed0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a10005fd0 tx=0x7f7a10009550 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 --2- 172.21.15.69:0/3953104898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd4590 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7a080dfd00 tx=0x7f7a080e8cd0 comp rx=0 tx=0).stop 2024-05-14T17:08:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 -- 172.21.15.69:0/3953104898 shutdown_connections 2024-05-14T17:08:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 --2- 172.21.15.69:0/3953104898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fd6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 --2- 172.21.15.69:0/3953104898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd4590 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 --2- 172.21.15.69:0/3953104898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fc9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 -- 172.21.15.69:0/3953104898 >> 172.21.15.69:0/3953104898 conn(0x55cd02fc5e00 msgr2=0x55cd02fc61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 -- 172.21.15.69:0/3953104898 shutdown_connections 2024-05-14T17:08:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 -- 172.21.15.69:0/3953104898 wait complete. 2024-05-14T17:08:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 -- start start 2024-05-14T17:08:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.957+0000 7f7a23049740 1 Processor -- start 2024-05-14T17:08:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a23049740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fd4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a23049740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a23049740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fd1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a23049740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cd02c12790 con 0x55cd02fca210 2024-05-14T17:08:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a23049740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cd02ee8890 con 0x55cd02fd4ad0 2024-05-14T17:08:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a23049740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cd02fd77a0 con 0x55cd02fc98f0 2024-05-14T17:08:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a23049740 10 client.0 _ll_register_callbacks cb 0x55cd02eebd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1d8c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1d8c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38006/0 (socket says 172.21.15.69:38006) 2024-05-14T17:08:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/3892089536 learned_addr learned my addr 172.21.15.69:0/3892089536 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fd4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1e8c8640 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fd1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/3892089536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 msgr2=0x55cd02fd4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fd4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/3892089536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 msgr2=0x55cd02fd1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fd1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/3892089536 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd02d0d620 con 0x55cd02fca210 2024-05-14T17:08:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1e8c8640 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fd1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd1640 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009930 tx=0x7f7a0c031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23db9c233e61b332 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f79fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:08:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f79fdffb640 1 -- 172.21.15.69:0/3892089536 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c035040 con 0x55cd02fca210 2024-05-14T17:08:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f79fdffb640 1 -- 172.21.15.69:0/3892089536 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7f7a0c031820 con 0x55cd02fca210 2024-05-14T17:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a23049740 1 -- 172.21.15.69:0/3892089536 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a080a9430 con 0x55cd02fca210 2024-05-14T17:08:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f79fdffb640 1 -- 172.21.15.69:0/3892089536 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7a0c004040 con 0x55cd02fca210 2024-05-14T17:08:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.958+0000 7f7a23049740 10 client.17313 fetch_fsmap learned FSMap version 61 2024-05-14T17:08:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.959+0000 7f7a23049740 1 -- 172.21.15.69:0/3892089536 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55cd02fe30d0 con 0x55cd02fca210 2024-05-14T17:08:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.959+0000 7f79fdffb640 1 -- 172.21.15.69:0/3892089536 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a080a9430 con 0x55cd02fca210 2024-05-14T17:08:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.959+0000 7f79fdffb640 1 -- 172.21.15.69:0/3892089536 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f7a0c045480 con 0x55cd02fca210 2024-05-14T17:08:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.959+0000 7f7a23049740 10 client.17313 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:08:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.959+0000 7f7a23049740 10 client.17313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.959+0000 7f7a23049740 20 client.17313 populate_metadata read hostname 'smithi069' 2024-05-14T17:08:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.959+0000 7f7a23049740 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd02fe4630 0x55cd02fe6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.959+0000 7f7a23049740 4 client.17313 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.959+0000 7f7a23049740 1 -- 172.21.15.69:0/3892089536 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd02f3eb80 con 0x55cd02fe4630 2024-05-14T17:08:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.959+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd02fe4630 0x55cd02fe6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.960+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd02fe4630 0x55cd02fe6ab0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993b090c80b46bd5 server_cookie=80c28c6ec61b730f in_seq=0 out_seq=0 2024-05-14T17:08:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.960+0000 7f79fdffb640 10 client.17313 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.960+0000 7f79fdffb640 1 -- 172.21.15.69:0/3892089536 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cd02f3eb80 con 0x55cd02fe4630 2024-05-14T17:08:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.960+0000 7f79fdffb640 10 client.17313 handle_command_reply: tid=0 2024-05-14T17:08:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.960+0000 7f7a23049740 1 client.17313 shutdown 2024-05-14T17:08:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.960+0000 7f7a23049740 1 -- 172.21.15.69:0/3892089536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 msgr2=0x55cd02fd1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.960+0000 7f7a23049740 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd1640 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009930 tx=0x7f7a0c031f50 comp rx=0 tx=0).stop 2024-05-14T17:08:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.961+0000 7f7a23049740 1 -- 172.21.15.69:0/3892089536 shutdown_connections 2024-05-14T17:08:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.961+0000 7f7a23049740 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd02fe4630 0x55cd02fe6ab0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.961+0000 7f7a23049740 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fd1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.961+0000 7f7a23049740 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fd1640 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.961+0000 7f79fdffb640 0 client.17313 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:08:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.961+0000 7f7a23049740 1 --2- 172.21.15.69:0/3892089536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fd4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.961+0000 7f7a23049740 1 -- 172.21.15.69:0/3892089536 >> 172.21.15.69:0/3892089536 conn(0x55cd02fc5e00 msgr2=0x55cd02fc5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.961+0000 7f7a23049740 1 -- 172.21.15.69:0/3892089536 shutdown_connections 2024-05-14T17:08:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.961+0000 7f7a23049740 1 -- 172.21.15.69:0/3892089536 wait complete. 2024-05-14T17:08:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.961+0000 7f7a23049740 20 client.17313 trim_cache size 0 max 16384 2024-05-14T17:08:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a23049740 1 Processor -- start 2024-05-14T17:08:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a23049740 1 -- start start 2024-05-14T17:08:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a23049740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fe0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a23049740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fec380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a23049740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a23049740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181b0160 con 0x55cd02fca210 2024-05-14T17:08:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a23049740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181b0830 con 0x55cd02fd4ad0 2024-05-14T17:08:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a23049740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181ae140 con 0x55cd02fc98f0 2024-05-14T17:08:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a1d8c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fec380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:59.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a1d8c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fec380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38008/0 (socket says 172.21.15.69:38008) 2024-05-14T17:08:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/1131976307 learned_addr learned my addr 172.21.15.69:0/1131976307 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:08:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/1131976307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fe0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/1131976307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 msgr2=0x55cd02fe0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/1131976307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fe0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/1131976307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 msgr2=0x55cd02fec8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/1131976307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/1131976307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd02fc5c30 con 0x55cd02fca210 2024-05-14T17:09:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.976+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/1131976307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fec380 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0458d0 tx=0x7f7a0c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c12a48d279a650a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f79fffff640 1 -- 172.21.15.69:0/1131976307 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c000e60 con 0x55cd02fca210 2024-05-14T17:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f79fffff640 1 -- 172.21.15.69:0/1131976307 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a0c031950 con 0x55cd02fca210 2024-05-14T17:09:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f79fffff640 1 -- 172.21.15.69:0/1131976307 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c035030 con 0x55cd02fca210 2024-05-14T17:09:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f7a23049740 1 -- 172.21.15.69:0/1131976307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 msgr2=0x55cd02fec380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f7a23049740 1 --2- 172.21.15.69:0/1131976307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fec380 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0458d0 tx=0x7f7a0c031400 comp rx=0 tx=0).stop 2024-05-14T17:09:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f7a23049740 1 -- 172.21.15.69:0/1131976307 shutdown_connections 2024-05-14T17:09:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f7a23049740 1 --2- 172.21.15.69:0/1131976307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fd4ad0 0x55cd02fec8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f7a23049740 1 --2- 172.21.15.69:0/1131976307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fec380 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f7a23049740 1 --2- 172.21.15.69:0/1131976307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fc98f0 0x55cd02fe0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f7a23049740 1 -- 172.21.15.69:0/1131976307 >> 172.21.15.69:0/1131976307 conn(0x55cd02fc5e00 msgr2=0x55cd02fc5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f7a23049740 1 -- 172.21.15.69:0/1131976307 shutdown_connections 2024-05-14T17:09:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f7a23049740 1 -- 172.21.15.69:0/1131976307 wait complete. 2024-05-14T17:09:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.977+0000 7f7a23049740 1 -- start start 2024-05-14T17:09:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a23049740 1 Processor -- start 2024-05-14T17:09:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a23049740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fc98f0 0x55cd02fdcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:00.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a23049740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fda010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a23049740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fd4ad0 0x55cd02fda550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a23049740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181ada70 con 0x55cd02fca210 2024-05-14T17:09:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a23049740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181ac680 con 0x55cd02fc98f0 2024-05-14T17:09:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a23049740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181accd0 con 0x55cd02fd4ad0 2024-05-14T17:09:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a23049740 10 client.0 _ll_register_callbacks cb 0x7f7a0c031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1d8c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fda010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1d8c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fda010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38016/0 (socket says 172.21.15.69:38016) 2024-05-14T17:09:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/3509518536 learned_addr learned my addr 172.21.15.69:0/3509518536 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:09:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fc98f0 0x55cd02fdcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1e8c8640 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fd4ad0 0x55cd02fda550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/3509518536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fd4ad0 msgr2=0x55cd02fda550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fd4ad0 0x55cd02fda550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/3509518536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fc98f0 msgr2=0x55cd02fdcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fc98f0 0x55cd02fdcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1d8c6640 1 -- 172.21.15.69:0/3509518536 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd02fdaa90 con 0x55cd02fca210 2024-05-14T17:09:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1e8c8640 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fd4ad0 0x55cd02fda550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:09:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a1d8c6640 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fda010 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c045a40 tx=0x7f7a0c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8467ae43545f24c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f79fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:09:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f79fdffb640 1 -- 172.21.15.69:0/3509518536 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c035070 con 0x55cd02fca210 2024-05-14T17:09:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f79fdffb640 1 -- 172.21.15.69:0/3509518536 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7f7a0c0314a0 con 0x55cd02fca210 2024-05-14T17:09:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.978+0000 7f7a23049740 1 -- 172.21.15.69:0/3509518536 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cd02f38360 con 0x55cd02fca210 2024-05-14T17:09:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f79fdffb640 1 -- 172.21.15.69:0/3509518536 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7a0c064050 con 0x55cd02fca210 2024-05-14T17:09:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f7a23049740 10 client.17319 fetch_fsmap learned FSMap version 61 2024-05-14T17:09:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f7a23049740 1 -- 172.21.15.69:0/3509518536 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55cd0304ec80 con 0x55cd02fca210 2024-05-14T17:09:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f79fdffb640 1 -- 172.21.15.69:0/3509518536 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cd02f38360 con 0x55cd02fca210 2024-05-14T17:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f79fdffb640 1 -- 172.21.15.69:0/3509518536 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f7a0c0564c0 con 0x55cd02fca210 2024-05-14T17:09:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f7a23049740 10 client.17319 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:09:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f7a23049740 10 client.17319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:09:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f7a23049740 20 client.17319 populate_metadata read hostname 'smithi069' 2024-05-14T17:09:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f7a23049740 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd0304f380 0x55cd030517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f7a23049740 4 client.17319 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:09:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f7a23049740 1 -- 172.21.15.69:0/3509518536 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd02eb25f0 con 0x55cd0304f380 2024-05-14T17:09:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.979+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd0304f380 0x55cd030517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.980+0000 7f7a1e0c7640 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd0304f380 0x55cd030517a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b5996ad800116d4 server_cookie=a20e51f7ac180b76 in_seq=0 out_seq=0 2024-05-14T17:09:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.980+0000 7f79fdffb640 10 client.17319 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:09:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f79fdffb640 1 -- 172.21.15.69:0/3509518536 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cd02eb25f0 con 0x55cd0304f380 2024-05-14T17:09:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f79fdffb640 10 client.17319 handle_command_reply: tid=0 2024-05-14T17:09:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 client.17319 shutdown 2024-05-14T17:09:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 -- 172.21.15.69:0/3509518536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 msgr2=0x55cd02fda010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fda010 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c045a40 tx=0x7f7a0c0035f0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 -- 172.21.15.69:0/3509518536 shutdown_connections 2024-05-14T17:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55cd0304f380 0x55cd030517a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd02fd4ad0 0x55cd02fda550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd02fca210 0x55cd02fda010 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 --2- 172.21.15.69:0/3509518536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cd02fc98f0 0x55cd02fdcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f79fdffb640 0 client.17319 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:09:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 -- 172.21.15.69:0/3509518536 >> 172.21.15.69:0/3509518536 conn(0x55cd02fc5e00 msgr2=0x55cd02fdf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 -- 172.21.15.69:0/3509518536 shutdown_connections 2024-05-14T17:09:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 1 -- 172.21.15.69:0/3509518536 wait complete. 2024-05-14T17:09:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.985+0000 7f7a23049740 20 client.17319 trim_cache size 0 max 16384 2024-05-14T17:09:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:09:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:09:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:09:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 -- 172.21.15.69:0/2576853600 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7a000b2a10 msgr2=0x7f7a000b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7a000b2a10 0x7f7a000b4ed0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a10005fd0 tx=0x7f7a10009550 comp rx=0 tx=0).stop 2024-05-14T17:09:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 -- 172.21.15.69:0/2576853600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a18102a30 msgr2=0x7f7a18199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a18102a30 0x7f7a18199140 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800c920 tx=0x7f7a0800cde0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 -- 172.21.15.69:0/2576853600 shutdown_connections 2024-05-14T17:09:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f7a000b2a10 0x7f7a000b4ed0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a18108a40 0x7f7a1819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a181033f0 0x7f7a18199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 --2- 172.21.15.69:0/2576853600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a18102a30 0x7f7a18199140 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 -- 172.21.15.69:0/2576853600 >> 172.21.15.69:0/2576853600 conn(0x7f7a18079ee0 msgr2=0x7f7a18100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 -- 172.21.15.69:0/2576853600 shutdown_connections 2024-05-14T17:09:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:41.986+0000 7f7a2032b640 1 -- 172.21.15.69:0/2576853600 wait complete. 2024-05-14T17:09:02.963 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:42 smithi069 ceph-mon[31507]: pgmap v595: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:09:02.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:44 smithi069 ceph-mon[31507]: pgmap v596: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T17:09:03.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:46 smithi069 ceph-mon[31507]: pgmap v597: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:09:03.063 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:48 smithi069 ceph-mon[31507]: pgmap v598: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-05-14T17:09:03.096 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:50 smithi069 ceph-mon[31507]: pgmap v599: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 6 op/s 2024-05-14T17:09:03.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:52 smithi069 ceph-mon[31507]: pgmap v600: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 6 op/s 2024-05-14T17:09:03.166 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:56 smithi092 ceph-mon[35763]: pgmap v602: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 6 op/s 2024-05-14T17:09:03.221 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:08:58 smithi092 ceph-mon[35763]: pgmap v603: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-05-14T17:09:03.255 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:00 smithi092 ceph-mon[35763]: pgmap v604: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-05-14T17:09:03.288 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:02 smithi092 ceph-mon[35763]: pgmap v605: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-05-14T17:09:03.363 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:56 smithi203 ceph-mon[35787]: pgmap v602: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 6 op/s 2024-05-14T17:09:03.396 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:08:58 smithi203 ceph-mon[35787]: pgmap v603: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-05-14T17:09:03.430 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:00 smithi203 ceph-mon[35787]: pgmap v604: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-05-14T17:09:03.463 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:02 smithi203 ceph-mon[35787]: pgmap v605: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-05-14T17:09:03.515 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/: 2024-05-14T17:09:03.538 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:03.572 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 . 2024-05-14T17:09:03.624 INFO:teuthology.orchestra.run.smithi069.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 .. 2024-05-14T17:09:03.655 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 data 2024-05-14T17:09:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:03.745 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data: 2024-05-14T17:09:03.772 INFO:teuthology.orchestra.run.smithi069.stdout:total 36 2024-05-14T17:09:03.805 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 . 2024-05-14T17:09:03.874 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 .. 2024-05-14T17:09:03.905 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:50 base 2024-05-14T17:09:03.930 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 14 16:50 current_logfiles 2024-05-14T17:09:03.955 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 May 14 16:50 global 2024-05-14T17:09:04.017 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 log 2024-05-14T17:09:04.044 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_commit_ts 2024-05-14T17:09:04.072 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_dynshmem 2024-05-14T17:09:04.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 14 16:50 pg_hba.conf 2024-05-14T17:09:04.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 14 16:50 pg_ident.conf 2024-05-14T17:09:04.164 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:08 pg_logical 2024-05-14T17:09:04.195 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 pg_multixact 2024-05-14T17:09:04.289 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_notify 2024-05-14T17:09:04.314 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_replslot 2024-05-14T17:09:04.339 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_serial 2024-05-14T17:09:04.372 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_snapshots 2024-05-14T17:09:04.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:08 pg_stat 2024-05-14T17:09:04.456 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 17:08 pg_stat_tmp 2024-05-14T17:09:04.489 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 17:05 pg_subtrans 2024-05-14T17:09:04.523 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_tblspc 2024-05-14T17:09:04.556 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pg_twophase 2024-05-14T17:09:04.589 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 PG_VERSION 2024-05-14T17:09:04.670 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 14 17:08 pg_wal 2024-05-14T17:09:04.698 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:59 pg_xact 2024-05-14T17:09:04.731 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 14 16:50 postgresql.auto.conf 2024-05-14T17:09:04.764 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 14 16:50 postgresql.conf 2024-05-14T17:09:04.798 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 14 16:50 postmaster.opts 2024-05-14T17:09:04.831 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:04.865 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base: 2024-05-14T17:09:04.904 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:04.931 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:50 . 2024-05-14T17:09:04.965 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:04.998 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:50 1 2024-05-14T17:09:05.048 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:50 13493 2024-05-14T17:09:05.081 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 May 14 16:55 13494 2024-05-14T17:09:05.115 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 pgsql_tmp 2024-05-14T17:09:05.148 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:05.182 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-14T17:09:05.215 INFO:teuthology.orchestra.run.smithi069.stdout:total 7777 2024-05-14T17:09:05.282 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:50 . 2024-05-14T17:09:05.315 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:50 .. 2024-05-14T17:09:05.348 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 112 2024-05-14T17:09:05.382 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 113 2024-05-14T17:09:05.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 1247 2024-05-14T17:09:05.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1247_fsm 2024-05-14T17:09:05.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1247_vm 2024-05-14T17:09:05.521 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:50 1249 2024-05-14T17:09:05.548 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1249_fsm 2024-05-14T17:09:05.582 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1249_vm 2024-05-14T17:09:05.632 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:50 1255 2024-05-14T17:09:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1255_fsm 2024-05-14T17:09:05.698 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1255_vm 2024-05-14T17:09:05.732 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 16:50 1259 2024-05-14T17:09:05.765 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1259_fsm 2024-05-14T17:09:05.799 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1259_vm 2024-05-14T17:09:05.832 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:50 13344 2024-05-14T17:09:05.881 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13344_fsm 2024-05-14T17:09:05.907 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13344_vm 2024-05-14T17:09:05.941 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13346 2024-05-14T17:09:05.974 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13348 2024-05-14T17:09:06.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349 2024-05-14T17:09:06.086 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13349_fsm 2024-05-14T17:09:06.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349_vm 2024-05-14T17:09:06.192 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13351 2024-05-14T17:09:06.237 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13353 2024-05-14T17:09:06.312 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354 2024-05-14T17:09:06.378 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13354_fsm 2024-05-14T17:09:06.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354_vm 2024-05-14T17:09:06.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13356 2024-05-14T17:09:06.528 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13358 2024-05-14T17:09:06.578 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359 2024-05-14T17:09:06.628 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13359_fsm 2024-05-14T17:09:06.678 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359_vm 2024-05-14T17:09:06.739 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13361 2024-05-14T17:09:06.787 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13363 2024-05-14T17:09:06.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1417 2024-05-14T17:09:06.887 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1418 2024-05-14T17:09:06.937 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 174 2024-05-14T17:09:06.987 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 175 2024-05-14T17:09:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2187 2024-05-14T17:09:07.087 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2224 2024-05-14T17:09:07.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2328 2024-05-14T17:09:07.212 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2336 2024-05-14T17:09:07.262 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2337 2024-05-14T17:09:07.312 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2579 2024-05-14T17:09:07.362 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2600 2024-05-14T17:09:07.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2600_fsm 2024-05-14T17:09:07.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2600_vm 2024-05-14T17:09:07.512 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601 2024-05-14T17:09:07.563 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2601_fsm 2024-05-14T17:09:07.618 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601_vm 2024-05-14T17:09:07.662 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2602 2024-05-14T17:09:07.746 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2602_fsm 2024-05-14T17:09:07.796 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2602_vm 2024-05-14T17:09:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2603 2024-05-14T17:09:07.896 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2603_fsm 2024-05-14T17:09:07.946 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2603_vm 2024-05-14T17:09:08.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2604 2024-05-14T17:09:08.079 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2605 2024-05-14T17:09:08.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2605_fsm 2024-05-14T17:09:08.204 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2605_vm 2024-05-14T17:09:08.254 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606 2024-05-14T17:09:08.304 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2606_fsm 2024-05-14T17:09:08.355 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606_vm 2024-05-14T17:09:08.405 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2607 2024-05-14T17:09:08.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2607_fsm 2024-05-14T17:09:08.505 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2607_vm 2024-05-14T17:09:08.555 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:50 2608 2024-05-14T17:09:08.605 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2608_fsm 2024-05-14T17:09:08.655 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2608_vm 2024-05-14T17:09:08.705 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:50 2609 2024-05-14T17:09:08.755 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2609_fsm 2024-05-14T17:09:08.805 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2609_vm 2024-05-14T17:09:08.872 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2610 2024-05-14T17:09:08.923 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2610_fsm 2024-05-14T17:09:08.971 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2610_vm 2024-05-14T17:09:09.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2611 2024-05-14T17:09:09.080 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612 2024-05-14T17:09:09.130 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2612_fsm 2024-05-14T17:09:09.180 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612_vm 2024-05-14T17:09:09.230 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2613 2024-05-14T17:09:09.280 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615 2024-05-14T17:09:09.330 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2615_fsm 2024-05-14T17:09:09.380 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615_vm 2024-05-14T17:09:09.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2616 2024-05-14T17:09:09.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2616_fsm 2024-05-14T17:09:09.530 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2616_vm 2024-05-14T17:09:09.580 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:50 2617 2024-05-14T17:09:09.630 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2617_fsm 2024-05-14T17:09:09.705 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2617_vm 2024-05-14T17:09:09.755 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:50 2618 2024-05-14T17:09:09.807 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2618_fsm 2024-05-14T17:09:09.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2618_vm 2024-05-14T17:09:09.906 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 14 16:50 2619 2024-05-14T17:09:09.956 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2619_fsm 2024-05-14T17:09:10.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2619_vm 2024-05-14T17:09:10.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2620 2024-05-14T17:09:10.106 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2650 2024-05-14T17:09:10.156 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2651 2024-05-14T17:09:10.206 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2652 2024-05-14T17:09:10.256 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2653 2024-05-14T17:09:10.317 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2654 2024-05-14T17:09:10.364 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2655 2024-05-14T17:09:10.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2656 2024-05-14T17:09:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2657 2024-05-14T17:09:10.539 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:50 2658 2024-05-14T17:09:10.590 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 2659 2024-05-14T17:09:10.640 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2660 2024-05-14T17:09:10.689 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2661 2024-05-14T17:09:10.740 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2662 2024-05-14T17:09:10.790 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2663 2024-05-14T17:09:10.840 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2664 2024-05-14T17:09:10.890 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2665 2024-05-14T17:09:10.940 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2666 2024-05-14T17:09:10.990 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2667 2024-05-14T17:09:11.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2668 2024-05-14T17:09:11.090 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2669 2024-05-14T17:09:11.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2670 2024-05-14T17:09:11.217 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:50 2673 2024-05-14T17:09:11.257 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:50 2674 2024-05-14T17:09:11.307 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:50 2675 2024-05-14T17:09:11.373 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2678 2024-05-14T17:09:11.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2679 2024-05-14T17:09:11.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2680 2024-05-14T17:09:11.523 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2681 2024-05-14T17:09:11.573 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2682 2024-05-14T17:09:11.623 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2683 2024-05-14T17:09:11.673 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2684 2024-05-14T17:09:11.723 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2685 2024-05-14T17:09:11.773 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2686 2024-05-14T17:09:11.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2687 2024-05-14T17:09:11.873 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2688 2024-05-14T17:09:11.924 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2689 2024-05-14T17:09:11.974 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:50 2690 2024-05-14T17:09:12.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:50 2691 2024-05-14T17:09:12.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2692 2024-05-14T17:09:12.124 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2693 2024-05-14T17:09:12.207 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2696 2024-05-14T17:09:12.257 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2699 2024-05-14T17:09:12.307 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2701 2024-05-14T17:09:12.357 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2702 2024-05-14T17:09:12.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2703 2024-05-14T17:09:12.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2704 2024-05-14T17:09:12.516 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2753 2024-05-14T17:09:12.566 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2753_fsm 2024-05-14T17:09:12.616 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2753_vm 2024-05-14T17:09:12.666 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2754 2024-05-14T17:09:12.716 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2755 2024-05-14T17:09:12.766 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2756 2024-05-14T17:09:12.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2757 2024-05-14T17:09:12.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2830 2024-05-14T17:09:12.916 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2831 2024-05-14T17:09:12.989 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2832 2024-05-14T17:09:13.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2833 2024-05-14T17:09:13.108 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2834 2024-05-14T17:09:13.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2835 2024-05-14T17:09:13.208 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2836 2024-05-14T17:09:13.258 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2837 2024-05-14T17:09:13.322 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:50 2838 2024-05-14T17:09:13.366 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2838_fsm 2024-05-14T17:09:13.408 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2838_vm 2024-05-14T17:09:13.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2839 2024-05-14T17:09:13.508 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840 2024-05-14T17:09:13.558 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840_fsm 2024-05-14T17:09:13.608 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2840_vm 2024-05-14T17:09:13.650 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2841 2024-05-14T17:09:13.691 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2995 2024-05-14T17:09:13.741 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2996 2024-05-14T17:09:13.791 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079 2024-05-14T17:09:13.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3079_fsm 2024-05-14T17:09:13.941 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079_vm 2024-05-14T17:09:13.992 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3080 2024-05-14T17:09:14.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3081 2024-05-14T17:09:14.092 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3085 2024-05-14T17:09:14.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3118 2024-05-14T17:09:14.267 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3119 2024-05-14T17:09:14.308 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:50 3164 2024-05-14T17:09:14.350 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3256 2024-05-14T17:09:14.392 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3257 2024-05-14T17:09:14.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3258 2024-05-14T17:09:14.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3350 2024-05-14T17:09:14.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3351 2024-05-14T17:09:14.567 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3379 2024-05-14T17:09:14.642 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3380 2024-05-14T17:09:14.685 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3381 2024-05-14T17:09:14.750 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3394 2024-05-14T17:09:14.800 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3394_fsm 2024-05-14T17:09:14.850 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3394_vm 2024-05-14T17:09:14.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3395 2024-05-14T17:09:14.934 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3429 2024-05-14T17:09:14.984 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3430 2024-05-14T17:09:15.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3431 2024-05-14T17:09:15.084 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3433 2024-05-14T17:09:15.125 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3439 2024-05-14T17:09:15.176 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3440 2024-05-14T17:09:15.226 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3455 2024-05-14T17:09:15.268 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:50 3456 2024-05-14T17:09:15.317 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3456_fsm 2024-05-14T17:09:15.359 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3456_vm 2024-05-14T17:09:15.409 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3466 2024-05-14T17:09:15.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3467 2024-05-14T17:09:15.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3468 2024-05-14T17:09:15.568 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3501 2024-05-14T17:09:15.618 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3502 2024-05-14T17:09:15.668 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3503 2024-05-14T17:09:15.718 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3534 2024-05-14T17:09:15.768 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541 2024-05-14T17:09:15.818 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3541_fsm 2024-05-14T17:09:15.868 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541_vm 2024-05-14T17:09:15.918 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3542 2024-05-14T17:09:15.968 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3574 2024-05-14T17:09:16.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3575 2024-05-14T17:09:16.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3576 2024-05-14T17:09:16.118 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3596 2024-05-14T17:09:16.168 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3597 2024-05-14T17:09:16.218 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3598 2024-05-14T17:09:16.268 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3599 2024-05-14T17:09:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600 2024-05-14T17:09:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3600_fsm 2024-05-14T17:09:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600_vm 2024-05-14T17:09:16.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601 2024-05-14T17:09:16.543 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3601_fsm 2024-05-14T17:09:16.585 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601_vm 2024-05-14T17:09:16.635 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602 2024-05-14T17:09:16.685 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3602_fsm 2024-05-14T17:09:16.727 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602_vm 2024-05-14T17:09:16.777 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603 2024-05-14T17:09:16.827 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603_fsm 2024-05-14T17:09:16.877 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3603_vm 2024-05-14T17:09:16.927 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3604 2024-05-14T17:09:16.977 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3605 2024-05-14T17:09:17.027 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3606 2024-05-14T17:09:17.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3607 2024-05-14T17:09:17.152 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3608 2024-05-14T17:09:17.202 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 3609 2024-05-14T17:09:17.252 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3712 2024-05-14T17:09:17.302 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764 2024-05-14T17:09:17.355 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3764_fsm 2024-05-14T17:09:17.402 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764_vm 2024-05-14T17:09:17.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3766 2024-05-14T17:09:17.502 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3767 2024-05-14T17:09:17.553 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3997 2024-05-14T17:09:17.602 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4143 2024-05-14T17:09:17.678 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4144 2024-05-14T17:09:17.727 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4145 2024-05-14T17:09:17.777 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4146 2024-05-14T17:09:17.827 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4147 2024-05-14T17:09:17.877 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4148 2024-05-14T17:09:17.927 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4149 2024-05-14T17:09:18.002 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4150 2024-05-14T17:09:18.052 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4151 2024-05-14T17:09:18.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4152 2024-05-14T17:09:18.153 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4153 2024-05-14T17:09:18.202 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4154 2024-05-14T17:09:18.259 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4155 2024-05-14T17:09:18.302 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4156 2024-05-14T17:09:18.353 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4157 2024-05-14T17:09:18.403 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4158 2024-05-14T17:09:18.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4159 2024-05-14T17:09:18.503 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4160 2024-05-14T17:09:18.553 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4161 2024-05-14T17:09:18.603 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4162 2024-05-14T17:09:18.653 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4163 2024-05-14T17:09:18.703 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4164 2024-05-14T17:09:18.766 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4165 2024-05-14T17:09:18.836 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4166 2024-05-14T17:09:18.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4167 2024-05-14T17:09:18.936 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4168 2024-05-14T17:09:18.986 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4169 2024-05-14T17:09:19.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4170 2024-05-14T17:09:19.086 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4171 2024-05-14T17:09:19.146 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4172 2024-05-14T17:09:19.195 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4173 2024-05-14T17:09:19.245 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4174 2024-05-14T17:09:19.303 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 5002 2024-05-14T17:09:19.345 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 548 2024-05-14T17:09:19.387 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 549 2024-05-14T17:09:19.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6102 2024-05-14T17:09:19.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6104 2024-05-14T17:09:19.520 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6106 2024-05-14T17:09:19.562 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6110 2024-05-14T17:09:19.628 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6111 2024-05-14T17:09:19.670 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6112 2024-05-14T17:09:19.712 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6113 2024-05-14T17:09:19.754 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6117 2024-05-14T17:09:19.803 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 826 2024-05-14T17:09:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 827 2024-05-14T17:09:19.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 828 2024-05-14T17:09:19.976 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:50 pg_filenode.map 2024-05-14T17:09:20.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 PG_VERSION 2024-05-14T17:09:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:20.104 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-14T17:09:20.154 INFO:teuthology.orchestra.run.smithi069.stdout:total 7777 2024-05-14T17:09:20.204 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:50 . 2024-05-14T17:09:20.254 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:50 .. 2024-05-14T17:09:20.304 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 112 2024-05-14T17:09:20.354 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 113 2024-05-14T17:09:20.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 1247 2024-05-14T17:09:20.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1247_fsm 2024-05-14T17:09:20.521 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1247_vm 2024-05-14T17:09:20.571 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:50 1249 2024-05-14T17:09:20.621 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1249_fsm 2024-05-14T17:09:20.671 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1249_vm 2024-05-14T17:09:20.787 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:50 1255 2024-05-14T17:09:20.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1255_fsm 2024-05-14T17:09:20.887 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1255_vm 2024-05-14T17:09:20.938 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 16:50 1259 2024-05-14T17:09:20.988 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1259_fsm 2024-05-14T17:09:21.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1259_vm 2024-05-14T17:09:21.088 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:50 13344 2024-05-14T17:09:21.163 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13344_fsm 2024-05-14T17:09:21.213 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13344_vm 2024-05-14T17:09:21.263 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13346 2024-05-14T17:09:21.330 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13348 2024-05-14T17:09:21.380 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349 2024-05-14T17:09:21.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13349_fsm 2024-05-14T17:09:21.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349_vm 2024-05-14T17:09:21.530 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13351 2024-05-14T17:09:21.580 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13353 2024-05-14T17:09:21.630 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354 2024-05-14T17:09:21.680 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13354_fsm 2024-05-14T17:09:21.730 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354_vm 2024-05-14T17:09:21.805 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13356 2024-05-14T17:09:21.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13358 2024-05-14T17:09:21.905 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359 2024-05-14T17:09:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13359_fsm 2024-05-14T17:09:22.005 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359_vm 2024-05-14T17:09:22.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13361 2024-05-14T17:09:22.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13363 2024-05-14T17:09:22.180 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1417 2024-05-14T17:09:22.230 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1418 2024-05-14T17:09:22.280 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 174 2024-05-14T17:09:22.330 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 175 2024-05-14T17:09:22.380 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2187 2024-05-14T17:09:22.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2224 2024-05-14T17:09:22.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2328 2024-05-14T17:09:22.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2336 2024-05-14T17:09:22.572 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2337 2024-05-14T17:09:22.622 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2579 2024-05-14T17:09:22.672 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2600 2024-05-14T17:09:22.722 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2600_fsm 2024-05-14T17:09:22.772 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2600_vm 2024-05-14T17:09:22.822 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601 2024-05-14T17:09:22.872 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2601_fsm 2024-05-14T17:09:22.923 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601_vm 2024-05-14T17:09:22.997 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2602 2024-05-14T17:09:23.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2602_fsm 2024-05-14T17:09:23.097 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2602_vm 2024-05-14T17:09:23.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2603 2024-05-14T17:09:23.198 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2603_fsm 2024-05-14T17:09:23.248 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2603_vm 2024-05-14T17:09:23.298 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2604 2024-05-14T17:09:23.348 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2605 2024-05-14T17:09:23.398 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2605_fsm 2024-05-14T17:09:23.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2605_vm 2024-05-14T17:09:23.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606 2024-05-14T17:09:23.548 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2606_fsm 2024-05-14T17:09:23.598 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606_vm 2024-05-14T17:09:23.648 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2607 2024-05-14T17:09:23.698 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2607_fsm 2024-05-14T17:09:23.748 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2607_vm 2024-05-14T17:09:23.815 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:50 2608 2024-05-14T17:09:23.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2608_fsm 2024-05-14T17:09:23.915 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2608_vm 2024-05-14T17:09:23.965 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:50 2609 2024-05-14T17:09:24.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2609_fsm 2024-05-14T17:09:24.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2609_vm 2024-05-14T17:09:24.115 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2610 2024-05-14T17:09:24.165 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2610_fsm 2024-05-14T17:09:24.273 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2610_vm 2024-05-14T17:09:24.323 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2611 2024-05-14T17:09:24.373 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612 2024-05-14T17:09:24.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2612_fsm 2024-05-14T17:09:24.484 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612_vm 2024-05-14T17:09:24.540 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2613 2024-05-14T17:09:24.590 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615 2024-05-14T17:09:24.648 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2615_fsm 2024-05-14T17:09:24.715 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615_vm 2024-05-14T17:09:24.765 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2616 2024-05-14T17:09:24.815 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2616_fsm 2024-05-14T17:09:24.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2616_vm 2024-05-14T17:09:24.915 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:50 2617 2024-05-14T17:09:24.965 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2617_fsm 2024-05-14T17:09:25.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2617_vm 2024-05-14T17:09:25.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:50 2618 2024-05-14T17:09:25.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2618_fsm 2024-05-14T17:09:25.157 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2618_vm 2024-05-14T17:09:25.207 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 14 16:50 2619 2024-05-14T17:09:25.249 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2619_fsm 2024-05-14T17:09:25.299 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2619_vm 2024-05-14T17:09:25.349 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2620 2024-05-14T17:09:25.399 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2650 2024-05-14T17:09:25.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2651 2024-05-14T17:09:25.516 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2652 2024-05-14T17:09:25.566 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2653 2024-05-14T17:09:25.616 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2654 2024-05-14T17:09:25.666 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2655 2024-05-14T17:09:25.716 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2656 2024-05-14T17:09:25.766 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2657 2024-05-14T17:09:25.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:50 2658 2024-05-14T17:09:25.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 2659 2024-05-14T17:09:25.916 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2660 2024-05-14T17:09:25.966 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2661 2024-05-14T17:09:26.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2662 2024-05-14T17:09:26.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2663 2024-05-14T17:09:26.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2664 2024-05-14T17:09:26.166 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2665 2024-05-14T17:09:26.216 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2666 2024-05-14T17:09:26.266 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2667 2024-05-14T17:09:26.333 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2668 2024-05-14T17:09:26.383 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2669 2024-05-14T17:09:26.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2670 2024-05-14T17:09:26.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:50 2673 2024-05-14T17:09:26.500 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:50 2674 2024-05-14T17:09:26.550 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:50 2675 2024-05-14T17:09:26.591 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2678 2024-05-14T17:09:26.633 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2679 2024-05-14T17:09:26.683 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2680 2024-05-14T17:09:26.733 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2681 2024-05-14T17:09:26.775 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2682 2024-05-14T17:09:26.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2683 2024-05-14T17:09:26.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2684 2024-05-14T17:09:26.917 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2685 2024-05-14T17:09:26.958 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2686 2024-05-14T17:09:27.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2687 2024-05-14T17:09:27.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2688 2024-05-14T17:09:27.125 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2689 2024-05-14T17:09:27.175 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:50 2690 2024-05-14T17:09:27.225 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:50 2691 2024-05-14T17:09:27.275 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2692 2024-05-14T17:09:27.327 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2693 2024-05-14T17:09:27.373 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2696 2024-05-14T17:09:27.409 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2699 2024-05-14T17:09:27.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2701 2024-05-14T17:09:27.509 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2702 2024-05-14T17:09:27.559 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2703 2024-05-14T17:09:27.609 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2704 2024-05-14T17:09:27.676 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2753 2024-05-14T17:09:27.725 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2753_fsm 2024-05-14T17:09:27.775 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2753_vm 2024-05-14T17:09:27.825 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2754 2024-05-14T17:09:27.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2755 2024-05-14T17:09:27.942 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2756 2024-05-14T17:09:27.994 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2757 2024-05-14T17:09:28.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2830 2024-05-14T17:09:28.092 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2831 2024-05-14T17:09:28.142 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2832 2024-05-14T17:09:28.192 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2833 2024-05-14T17:09:28.242 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2834 2024-05-14T17:09:28.292 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2835 2024-05-14T17:09:28.342 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2836 2024-05-14T17:09:28.392 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2837 2024-05-14T17:09:28.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:50 2838 2024-05-14T17:09:28.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2838_fsm 2024-05-14T17:09:28.543 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2838_vm 2024-05-14T17:09:28.593 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2839 2024-05-14T17:09:28.643 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840 2024-05-14T17:09:28.709 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840_fsm 2024-05-14T17:09:28.766 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2840_vm 2024-05-14T17:09:28.809 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2841 2024-05-14T17:09:28.859 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2995 2024-05-14T17:09:28.909 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2996 2024-05-14T17:09:28.960 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079 2024-05-14T17:09:29.009 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3079_fsm 2024-05-14T17:09:29.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079_vm 2024-05-14T17:09:29.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3080 2024-05-14T17:09:29.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3081 2024-05-14T17:09:29.210 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3085 2024-05-14T17:09:29.260 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3118 2024-05-14T17:09:29.310 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3119 2024-05-14T17:09:29.360 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:50 3164 2024-05-14T17:09:29.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3256 2024-05-14T17:09:29.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3257 2024-05-14T17:09:29.543 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3258 2024-05-14T17:09:29.593 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3350 2024-05-14T17:09:29.644 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3351 2024-05-14T17:09:29.685 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3379 2024-05-14T17:09:29.727 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3380 2024-05-14T17:09:29.768 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3381 2024-05-14T17:09:29.818 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3394 2024-05-14T17:09:29.860 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3394_fsm 2024-05-14T17:09:29.910 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3394_vm 2024-05-14T17:09:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3395 2024-05-14T17:09:30.002 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3429 2024-05-14T17:09:30.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3430 2024-05-14T17:09:30.094 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3431 2024-05-14T17:09:30.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3433 2024-05-14T17:09:30.185 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3439 2024-05-14T17:09:30.227 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3440 2024-05-14T17:09:30.294 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3455 2024-05-14T17:09:30.344 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:50 3456 2024-05-14T17:09:30.394 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3456_fsm 2024-05-14T17:09:30.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3456_vm 2024-05-14T17:09:30.527 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3466 2024-05-14T17:09:30.577 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3467 2024-05-14T17:09:30.627 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3468 2024-05-14T17:09:30.686 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3501 2024-05-14T17:09:30.736 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3502 2024-05-14T17:09:30.785 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3503 2024-05-14T17:09:30.836 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3534 2024-05-14T17:09:30.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541 2024-05-14T17:09:30.936 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3541_fsm 2024-05-14T17:09:30.986 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541_vm 2024-05-14T17:09:31.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3542 2024-05-14T17:09:31.086 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3574 2024-05-14T17:09:31.161 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3575 2024-05-14T17:09:31.194 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3576 2024-05-14T17:09:31.244 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3596 2024-05-14T17:09:31.294 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3597 2024-05-14T17:09:31.354 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3598 2024-05-14T17:09:31.394 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3599 2024-05-14T17:09:31.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600 2024-05-14T17:09:31.486 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3600_fsm 2024-05-14T17:09:31.528 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600_vm 2024-05-14T17:09:31.569 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601 2024-05-14T17:09:31.611 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3601_fsm 2024-05-14T17:09:31.653 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601_vm 2024-05-14T17:09:31.695 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602 2024-05-14T17:09:31.736 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3602_fsm 2024-05-14T17:09:31.778 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602_vm 2024-05-14T17:09:31.820 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603 2024-05-14T17:09:31.895 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603_fsm 2024-05-14T17:09:31.937 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3603_vm 2024-05-14T17:09:31.979 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3604 2024-05-14T17:09:32.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3605 2024-05-14T17:09:32.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3606 2024-05-14T17:09:32.123 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3607 2024-05-14T17:09:32.162 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3608 2024-05-14T17:09:32.204 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 3609 2024-05-14T17:09:32.254 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3712 2024-05-14T17:09:32.287 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764 2024-05-14T17:09:32.329 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3764_fsm 2024-05-14T17:09:32.380 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764_vm 2024-05-14T17:09:32.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3766 2024-05-14T17:09:32.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3767 2024-05-14T17:09:32.505 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3997 2024-05-14T17:09:32.538 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4143 2024-05-14T17:09:32.597 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4144 2024-05-14T17:09:32.630 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4145 2024-05-14T17:09:32.663 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4146 2024-05-14T17:09:32.696 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4147 2024-05-14T17:09:32.730 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4148 2024-05-14T17:09:32.763 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4149 2024-05-14T17:09:32.822 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4150 2024-05-14T17:09:32.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4151 2024-05-14T17:09:32.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4152 2024-05-14T17:09:32.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4153 2024-05-14T17:09:32.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4154 2024-05-14T17:09:32.988 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4155 2024-05-14T17:09:33.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4156 2024-05-14T17:09:33.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4157 2024-05-14T17:09:33.088 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4158 2024-05-14T17:09:33.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4159 2024-05-14T17:09:33.164 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4160 2024-05-14T17:09:33.197 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4161 2024-05-14T17:09:33.230 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4162 2024-05-14T17:09:33.263 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4163 2024-05-14T17:09:33.297 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4164 2024-05-14T17:09:33.330 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4165 2024-05-14T17:09:33.364 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4166 2024-05-14T17:09:33.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4167 2024-05-14T17:09:33.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4168 2024-05-14T17:09:33.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4169 2024-05-14T17:09:33.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4170 2024-05-14T17:09:33.555 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4171 2024-05-14T17:09:33.589 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4172 2024-05-14T17:09:33.622 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4173 2024-05-14T17:09:33.655 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4174 2024-05-14T17:09:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 5002 2024-05-14T17:09:33.764 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 548 2024-05-14T17:09:33.797 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 549 2024-05-14T17:09:33.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6102 2024-05-14T17:09:33.864 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6104 2024-05-14T17:09:33.897 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6106 2024-05-14T17:09:33.931 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6110 2024-05-14T17:09:33.964 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6111 2024-05-14T17:09:33.998 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6112 2024-05-14T17:09:34.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6113 2024-05-14T17:09:34.081 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6117 2024-05-14T17:09:34.114 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 826 2024-05-14T17:09:34.148 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 827 2024-05-14T17:09:34.181 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 828 2024-05-14T17:09:34.214 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:50 pg_filenode.map 2024-05-14T17:09:34.248 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 PG_VERSION 2024-05-14T17:09:34.281 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:34.323 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-14T17:09:34.356 INFO:teuthology.orchestra.run.smithi069.stdout:total 592894 2024-05-14T17:09:34.400 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 May 14 16:55 . 2024-05-14T17:09:34.431 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:50 .. 2024-05-14T17:09:34.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 112 2024-05-14T17:09:34.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 113 2024-05-14T17:09:34.531 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 1247 2024-05-14T17:09:34.565 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1247_fsm 2024-05-14T17:09:34.598 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1247_vm 2024-05-14T17:09:34.660 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:50 1249 2024-05-14T17:09:34.698 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1249_fsm 2024-05-14T17:09:34.731 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1249_vm 2024-05-14T17:09:34.765 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:50 1255 2024-05-14T17:09:34.798 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1255_fsm 2024-05-14T17:09:34.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1255_vm 2024-05-14T17:09:34.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 17:06 1259 2024-05-14T17:09:34.907 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1259_fsm 2024-05-14T17:09:34.940 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1259_vm 2024-05-14T17:09:34.974 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:50 13344 2024-05-14T17:09:35.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13344_fsm 2024-05-14T17:09:35.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13344_vm 2024-05-14T17:09:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13346 2024-05-14T17:09:35.107 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13348 2024-05-14T17:09:35.140 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349 2024-05-14T17:09:35.173 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13349_fsm 2024-05-14T17:09:35.207 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13349_vm 2024-05-14T17:09:35.252 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13351 2024-05-14T17:09:35.290 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13353 2024-05-14T17:09:35.324 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354 2024-05-14T17:09:35.360 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13354_fsm 2024-05-14T17:09:35.399 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13354_vm 2024-05-14T17:09:35.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13356 2024-05-14T17:09:35.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13358 2024-05-14T17:09:35.516 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359 2024-05-14T17:09:35.549 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 13359_fsm 2024-05-14T17:09:35.582 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13359_vm 2024-05-14T17:09:35.616 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 13361 2024-05-14T17:09:35.657 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 13363 2024-05-14T17:09:35.691 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1417 2024-05-14T17:09:35.724 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 1418 2024-05-14T17:09:35.757 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 443301888 May 14 17:06 16396 2024-05-14T17:09:35.791 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 14 17:05 16396_fsm 2024-05-14T17:09:35.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 16396_vm 2024-05-14T17:09:35.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1671168 May 14 17:06 16397 2024-05-14T17:09:35.924 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 17:06 16397_fsm 2024-05-14T17:09:35.957 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 17:06 16397_vm 2024-05-14T17:09:35.991 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1785856 May 14 17:06 16399 2024-05-14T17:09:36.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 17:06 16399_fsm 2024-05-14T17:09:36.083 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 17:06 16399_vm 2024-05-14T17:09:36.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 17:06 16400 2024-05-14T17:09:36.158 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 17:06 16402 2024-05-14T17:09:36.200 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 May 14 17:06 16404 2024-05-14T17:09:36.233 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 79970304 May 14 17:06 16406 2024-05-14T17:09:36.267 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 17:06 16406_fsm 2024-05-14T17:09:36.300 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 17:02 16406_vm 2024-05-14T17:09:36.333 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 174 2024-05-14T17:09:36.367 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 175 2024-05-14T17:09:36.400 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2187 2024-05-14T17:09:36.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2224 2024-05-14T17:09:36.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2328 2024-05-14T17:09:36.533 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2336 2024-05-14T17:09:36.577 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2337 2024-05-14T17:09:36.617 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2579 2024-05-14T17:09:36.650 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2600 2024-05-14T17:09:36.693 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2600_fsm 2024-05-14T17:09:36.725 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2600_vm 2024-05-14T17:09:36.784 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601 2024-05-14T17:09:36.817 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2601_fsm 2024-05-14T17:09:36.850 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2601_vm 2024-05-14T17:09:36.884 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2602 2024-05-14T17:09:36.917 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2602_fsm 2024-05-14T17:09:36.950 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2602_vm 2024-05-14T17:09:36.984 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2603 2024-05-14T17:09:37.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2603_fsm 2024-05-14T17:09:37.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2603_vm 2024-05-14T17:09:37.084 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2604 2024-05-14T17:09:37.125 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2605 2024-05-14T17:09:37.193 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2605_fsm 2024-05-14T17:09:37.226 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2605_vm 2024-05-14T17:09:37.259 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606 2024-05-14T17:09:37.317 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2606_fsm 2024-05-14T17:09:37.351 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2606_vm 2024-05-14T17:09:37.384 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2607 2024-05-14T17:09:37.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2607_fsm 2024-05-14T17:09:37.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2607_vm 2024-05-14T17:09:37.501 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:50 2608 2024-05-14T17:09:37.534 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2608_fsm 2024-05-14T17:09:37.568 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2608_vm 2024-05-14T17:09:37.601 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:50 2609 2024-05-14T17:09:37.634 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2609_fsm 2024-05-14T17:09:37.668 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2609_vm 2024-05-14T17:09:37.701 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2610 2024-05-14T17:09:37.734 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2610_fsm 2024-05-14T17:09:37.768 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2610_vm 2024-05-14T17:09:37.820 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2611 2024-05-14T17:09:37.851 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612 2024-05-14T17:09:37.909 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2612_fsm 2024-05-14T17:09:37.943 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2612_vm 2024-05-14T17:09:37.976 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2613 2024-05-14T17:09:38.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615 2024-05-14T17:09:38.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2615_fsm 2024-05-14T17:09:38.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2615_vm 2024-05-14T17:09:38.110 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2616 2024-05-14T17:09:38.143 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2616_fsm 2024-05-14T17:09:38.195 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2616_vm 2024-05-14T17:09:38.226 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:50 2617 2024-05-14T17:09:38.260 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2617_fsm 2024-05-14T17:09:38.293 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2617_vm 2024-05-14T17:09:38.326 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:50 2618 2024-05-14T17:09:38.360 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2618_fsm 2024-05-14T17:09:38.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2618_vm 2024-05-14T17:09:38.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 May 14 17:06 2619 2024-05-14T17:09:38.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:53 2619_fsm 2024-05-14T17:09:38.510 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2619_vm 2024-05-14T17:09:38.543 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2620 2024-05-14T17:09:38.577 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2650 2024-05-14T17:09:38.610 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2651 2024-05-14T17:09:38.647 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2652 2024-05-14T17:09:38.685 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2653 2024-05-14T17:09:38.718 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2654 2024-05-14T17:09:38.752 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2655 2024-05-14T17:09:38.785 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2656 2024-05-14T17:09:38.819 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2657 2024-05-14T17:09:38.852 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:50 2658 2024-05-14T17:09:38.889 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:50 2659 2024-05-14T17:09:38.935 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2660 2024-05-14T17:09:38.969 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2661 2024-05-14T17:09:39.002 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2662 2024-05-14T17:09:39.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2663 2024-05-14T17:09:39.102 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2664 2024-05-14T17:09:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2665 2024-05-14T17:09:39.169 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2666 2024-05-14T17:09:39.202 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2667 2024-05-14T17:09:39.236 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2668 2024-05-14T17:09:39.280 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2669 2024-05-14T17:09:39.313 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2670 2024-05-14T17:09:39.352 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:50 2673 2024-05-14T17:09:39.386 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:50 2674 2024-05-14T17:09:39.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:50 2675 2024-05-14T17:09:39.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2678 2024-05-14T17:09:39.553 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2679 2024-05-14T17:09:39.586 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2680 2024-05-14T17:09:39.619 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2681 2024-05-14T17:09:39.653 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2682 2024-05-14T17:09:39.703 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2683 2024-05-14T17:09:39.755 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2684 2024-05-14T17:09:39.811 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2685 2024-05-14T17:09:39.844 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2686 2024-05-14T17:09:39.878 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2687 2024-05-14T17:09:39.911 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2688 2024-05-14T17:09:39.944 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:50 2689 2024-05-14T17:09:39.979 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:50 2690 2024-05-14T17:09:40.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:50 2691 2024-05-14T17:09:40.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2692 2024-05-14T17:09:40.078 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2693 2024-05-14T17:09:40.111 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:59 2696 2024-05-14T17:09:40.144 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2699 2024-05-14T17:09:40.178 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2701 2024-05-14T17:09:40.211 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2702 2024-05-14T17:09:40.245 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2703 2024-05-14T17:09:40.286 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 2704 2024-05-14T17:09:40.320 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2753 2024-05-14T17:09:40.383 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2753_fsm 2024-05-14T17:09:40.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2753_vm 2024-05-14T17:09:40.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2754 2024-05-14T17:09:40.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2755 2024-05-14T17:09:40.520 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2756 2024-05-14T17:09:40.553 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 2757 2024-05-14T17:09:40.598 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2830 2024-05-14T17:09:40.628 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2831 2024-05-14T17:09:40.662 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2832 2024-05-14T17:09:40.695 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2833 2024-05-14T17:09:40.728 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2834 2024-05-14T17:09:40.762 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2835 2024-05-14T17:09:40.795 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2836 2024-05-14T17:09:40.829 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2837 2024-05-14T17:09:40.870 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:50 2838 2024-05-14T17:09:40.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2838_fsm 2024-05-14T17:09:40.937 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2838_vm 2024-05-14T17:09:41.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2839 2024-05-14T17:09:41.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840 2024-05-14T17:09:41.079 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2840_fsm 2024-05-14T17:09:41.120 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2840_vm 2024-05-14T17:09:41.154 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2841 2024-05-14T17:09:41.187 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2995 2024-05-14T17:09:41.220 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2996 2024-05-14T17:09:41.254 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079 2024-05-14T17:09:41.287 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3079_fsm 2024-05-14T17:09:41.320 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3079_vm 2024-05-14T17:09:41.354 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3080 2024-05-14T17:09:41.387 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3081 2024-05-14T17:09:41.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3085 2024-05-14T17:09:41.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3118 2024-05-14T17:09:41.507 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3119 2024-05-14T17:09:41.537 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:50 3164 2024-05-14T17:09:41.571 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3256 2024-05-14T17:09:41.639 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3257 2024-05-14T17:09:41.671 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3258 2024-05-14T17:09:41.704 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3350 2024-05-14T17:09:41.738 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3351 2024-05-14T17:09:41.771 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3379 2024-05-14T17:09:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3380 2024-05-14T17:09:41.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3381 2024-05-14T17:09:41.871 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3394 2024-05-14T17:09:41.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3394_fsm 2024-05-14T17:09:41.938 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3394_vm 2024-05-14T17:09:41.971 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3395 2024-05-14T17:09:42.005 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3429 2024-05-14T17:09:42.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3430 2024-05-14T17:09:42.090 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3431 2024-05-14T17:09:42.121 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3433 2024-05-14T17:09:42.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3439 2024-05-14T17:09:42.188 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3440 2024-05-14T17:09:42.242 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:50 3455 2024-05-14T17:09:42.280 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:50 3456 2024-05-14T17:09:42.313 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3456_fsm 2024-05-14T17:09:42.347 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3456_vm 2024-05-14T17:09:42.380 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3466 2024-05-14T17:09:42.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3467 2024-05-14T17:09:42.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3468 2024-05-14T17:09:42.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3501 2024-05-14T17:09:42.513 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3502 2024-05-14T17:09:42.547 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3503 2024-05-14T17:09:42.580 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3534 2024-05-14T17:09:42.622 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541 2024-05-14T17:09:42.682 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3541_fsm 2024-05-14T17:09:42.713 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3541_vm 2024-05-14T17:09:42.747 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3542 2024-05-14T17:09:42.780 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3574 2024-05-14T17:09:42.830 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3575 2024-05-14T17:09:42.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3576 2024-05-14T17:09:42.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3596 2024-05-14T17:09:42.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3597 2024-05-14T17:09:42.988 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3598 2024-05-14T17:09:43.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3599 2024-05-14T17:09:43.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600 2024-05-14T17:09:43.089 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3600_fsm 2024-05-14T17:09:43.122 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3600_vm 2024-05-14T17:09:43.155 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601 2024-05-14T17:09:43.189 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3601_fsm 2024-05-14T17:09:43.231 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3601_vm 2024-05-14T17:09:43.264 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602 2024-05-14T17:09:43.297 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3602_fsm 2024-05-14T17:09:43.331 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3602_vm 2024-05-14T17:09:43.364 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603 2024-05-14T17:09:43.397 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3603_fsm 2024-05-14T17:09:43.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3603_vm 2024-05-14T17:09:43.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3604 2024-05-14T17:09:43.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3605 2024-05-14T17:09:43.556 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3606 2024-05-14T17:09:43.589 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3607 2024-05-14T17:09:43.622 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3608 2024-05-14T17:09:43.656 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:50 3609 2024-05-14T17:09:43.696 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3712 2024-05-14T17:09:43.735 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764 2024-05-14T17:09:43.773 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 3764_fsm 2024-05-14T17:09:43.814 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3764_vm 2024-05-14T17:09:43.848 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3766 2024-05-14T17:09:43.881 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 3767 2024-05-14T17:09:43.914 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3997 2024-05-14T17:09:43.948 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4143 2024-05-14T17:09:43.981 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4144 2024-05-14T17:09:44.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4145 2024-05-14T17:09:44.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4146 2024-05-14T17:09:44.103 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4147 2024-05-14T17:09:44.139 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4148 2024-05-14T17:09:44.173 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4149 2024-05-14T17:09:44.206 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4150 2024-05-14T17:09:44.240 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4151 2024-05-14T17:09:44.273 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4152 2024-05-14T17:09:44.306 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4153 2024-05-14T17:09:44.340 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4154 2024-05-14T17:09:44.388 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4155 2024-05-14T17:09:44.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4156 2024-05-14T17:09:44.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4157 2024-05-14T17:09:44.506 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4158 2024-05-14T17:09:44.540 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4159 2024-05-14T17:09:44.573 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4160 2024-05-14T17:09:44.606 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4161 2024-05-14T17:09:44.648 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4162 2024-05-14T17:09:44.682 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4163 2024-05-14T17:09:44.728 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4164 2024-05-14T17:09:44.761 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4165 2024-05-14T17:09:44.782 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4166 2024-05-14T17:09:44.799 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4167 2024-05-14T17:09:44.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4168 2024-05-14T17:09:44.840 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4169 2024-05-14T17:09:44.873 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4170 2024-05-14T17:09:44.899 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4171 2024-05-14T17:09:44.927 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4172 2024-05-14T17:09:44.949 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4173 2024-05-14T17:09:44.982 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4174 2024-05-14T17:09:45.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 5002 2024-05-14T17:09:45.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 548 2024-05-14T17:09:45.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 549 2024-05-14T17:09:45.091 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6102 2024-05-14T17:09:45.116 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6104 2024-05-14T17:09:45.141 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6106 2024-05-14T17:09:45.195 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6110 2024-05-14T17:09:45.224 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6111 2024-05-14T17:09:45.249 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6112 2024-05-14T17:09:45.274 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6113 2024-05-14T17:09:45.308 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6117 2024-05-14T17:09:45.333 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 826 2024-05-14T17:09:45.358 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 827 2024-05-14T17:09:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 828 2024-05-14T17:09:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:50 pg_filenode.map 2024-05-14T17:09:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 May 14 16:50 pg_internal.init 2024-05-14T17:09:45.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:50 PG_VERSION 2024-05-14T17:09:45.508 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:45.533 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-14T17:09:45.566 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:45.591 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:45.616 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:50 .. 2024-05-14T17:09:45.641 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:45.702 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/global: 2024-05-14T17:09:45.725 INFO:teuthology.orchestra.run.smithi069.stdout:total 560 2024-05-14T17:09:45.750 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 May 14 16:50 . 2024-05-14T17:09:45.775 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:45.800 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1213 2024-05-14T17:09:45.842 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1213_fsm 2024-05-14T17:09:45.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1213_vm 2024-05-14T17:09:45.908 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1214 2024-05-14T17:09:45.933 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1214_fsm 2024-05-14T17:09:45.959 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1214_vm 2024-05-14T17:09:45.983 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 1232 2024-05-14T17:09:46.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 1233 2024-05-14T17:09:46.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1260 2024-05-14T17:09:46.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1260_fsm 2024-05-14T17:09:46.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1260_vm 2024-05-14T17:09:46.092 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1261 2024-05-14T17:09:46.117 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1261_fsm 2024-05-14T17:09:46.194 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1261_vm 2024-05-14T17:09:46.217 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1262 2024-05-14T17:09:46.242 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 1262_fsm 2024-05-14T17:09:46.267 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 1262_vm 2024-05-14T17:09:46.294 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2396 2024-05-14T17:09:46.342 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:50 2396_fsm 2024-05-14T17:09:46.370 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2396_vm 2024-05-14T17:09:46.392 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2397 2024-05-14T17:09:46.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2671 2024-05-14T17:09:46.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2672 2024-05-14T17:09:46.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2676 2024-05-14T17:09:46.492 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2677 2024-05-14T17:09:46.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2694 2024-05-14T17:09:46.542 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2695 2024-05-14T17:09:46.567 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2697 2024-05-14T17:09:46.598 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:50 2698 2024-05-14T17:09:46.617 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2846 2024-05-14T17:09:46.672 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2847 2024-05-14T17:09:46.692 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2964 2024-05-14T17:09:46.718 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2965 2024-05-14T17:09:46.743 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 2966 2024-05-14T17:09:46.776 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 2967 2024-05-14T17:09:46.801 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 3592 2024-05-14T17:09:46.826 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 3593 2024-05-14T17:09:46.852 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4060 2024-05-14T17:09:46.884 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4061 2024-05-14T17:09:46.918 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4175 2024-05-14T17:09:46.943 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4176 2024-05-14T17:09:46.968 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4177 2024-05-14T17:09:46.993 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4178 2024-05-14T17:09:47.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4181 2024-05-14T17:09:47.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4182 2024-05-14T17:09:47.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4183 2024-05-14T17:09:47.093 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4184 2024-05-14T17:09:47.159 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 4185 2024-05-14T17:09:47.176 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 4186 2024-05-14T17:09:47.201 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6000 2024-05-14T17:09:47.243 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6001 2024-05-14T17:09:47.268 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6002 2024-05-14T17:09:47.293 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 6100 2024-05-14T17:09:47.318 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6114 2024-05-14T17:09:47.343 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 6115 2024-05-14T17:09:47.368 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 17:08 pg_control 2024-05-14T17:09:47.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:50 pg_filenode.map 2024-05-14T17:09:47.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 May 14 16:50 pg_internal.init 2024-05-14T17:09:47.443 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:47.470 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/log: 2024-05-14T17:09:47.518 INFO:teuthology.orchestra.run.smithi069.stdout:total 2 2024-05-14T17:09:47.544 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 . 2024-05-14T17:09:47.585 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:47.610 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1571 May 14 17:06 postgresql-Tue.log 2024-05-14T17:09:47.650 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:47.669 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-14T17:09:47.710 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:47.735 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:47.760 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:47.785 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:47.831 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-14T17:09:47.852 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:47.884 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:47.902 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:47.952 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-14T17:09:47.977 INFO:teuthology.orchestra.run.smithi069.stdout:total 1 2024-05-14T17:09:48.002 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:08 . 2024-05-14T17:09:48.027 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:48.052 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 mappings 2024-05-14T17:09:48.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 14 17:08 replorigin_checkpoint 2024-05-14T17:09:48.134 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 snapshots 2024-05-14T17:09:48.177 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:48.203 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-14T17:09:48.227 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:48.253 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:48.277 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:08 .. 2024-05-14T17:09:48.309 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:48.337 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-14T17:09:48.361 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:48.386 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:48.411 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:08 .. 2024-05-14T17:09:48.436 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:48.461 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-14T17:09:48.486 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:48.511 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 . 2024-05-14T17:09:48.536 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:48.561 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 members 2024-05-14T17:09:48.614 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 offsets 2024-05-14T17:09:48.636 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:48.661 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-14T17:09:48.686 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-05-14T17:09:48.711 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 . 2024-05-14T17:09:48.736 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 .. 2024-05-14T17:09:48.778 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:50 0000 2024-05-14T17:09:48.795 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:48.820 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-14T17:09:48.856 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-05-14T17:09:48.878 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:50 . 2024-05-14T17:09:48.903 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:50 .. 2024-05-14T17:09:48.928 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 0000 2024-05-14T17:09:48.953 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:48.978 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-14T17:09:49.004 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:49.045 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:49.088 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:49.112 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:49.137 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-14T17:09:49.162 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:49.187 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:49.212 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:49.239 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-14T17:09:49.304 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:49.370 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:49.404 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:49.437 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-14T17:09:49.471 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:49.504 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:49.571 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:49.604 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:49.637 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-14T17:09:49.671 INFO:teuthology.orchestra.run.smithi069.stdout:total 15 2024-05-14T17:09:49.704 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:08 . 2024-05-14T17:09:49.738 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:49.771 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 May 14 17:08 db_0.stat 2024-05-14T17:09:49.804 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 May 14 17:08 db_13494.stat 2024-05-14T17:09:49.845 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 May 14 17:08 global.stat 2024-05-14T17:09:49.879 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:49.913 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-14T17:09:49.954 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:49.988 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 17:08 . 2024-05-14T17:09:50.021 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:50.054 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:50.088 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-14T17:09:50.129 INFO:teuthology.orchestra.run.smithi069.stdout:total 96 2024-05-14T17:09:50.163 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 17:05 . 2024-05-14T17:09:50.196 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:50.273 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 17:06 0017 2024-05-14T17:09:50.313 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:50.338 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-14T17:09:50.363 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:50.380 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:50.413 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:50.457 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:50.497 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-14T17:09:50.530 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:50.563 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:50.597 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:50.630 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:50.663 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-14T17:09:50.705 INFO:teuthology.orchestra.run.smithi069.stdout:total 1048576 2024-05-14T17:09:50.730 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 14 17:08 . 2024-05-14T17:09:50.764 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:50.789 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 000000010000000000000092 2024-05-14T17:09:50.859 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:55 000000010000000000000093 2024-05-14T17:09:50.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:55 000000010000000000000094 2024-05-14T17:09:50.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 000000010000000000000095 2024-05-14T17:09:50.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 000000010000000000000096 2024-05-14T17:09:50.989 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 000000010000000000000097 2024-05-14T17:09:51.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:01 000000010000000000000098 2024-05-14T17:09:51.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:02 000000010000000000000099 2024-05-14T17:09:51.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:02 00000001000000000000009A 2024-05-14T17:09:51.080 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:01 00000001000000000000009B 2024-05-14T17:09:51.105 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 00000001000000000000009C 2024-05-14T17:09:51.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 00000001000000000000009D 2024-05-14T17:09:51.164 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 00000001000000000000009E 2024-05-14T17:09:51.206 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:02 00000001000000000000009F 2024-05-14T17:09:51.240 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000A0 2024-05-14T17:09:51.281 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000A1 2024-05-14T17:09:51.306 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000A2 2024-05-14T17:09:51.331 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:00 0000000100000000000000A3 2024-05-14T17:09:51.384 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000A4 2024-05-14T17:09:51.406 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000A5 2024-05-14T17:09:51.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000A6 2024-05-14T17:09:51.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000A7 2024-05-14T17:09:51.523 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000A8 2024-05-14T17:09:51.548 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:01 0000000100000000000000A9 2024-05-14T17:09:51.582 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:00 0000000100000000000000AA 2024-05-14T17:09:51.623 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000AB 2024-05-14T17:09:51.648 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000AC 2024-05-14T17:09:51.681 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000AD 2024-05-14T17:09:51.714 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000AE 2024-05-14T17:09:51.773 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000AF 2024-05-14T17:09:51.806 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000B0 2024-05-14T17:09:51.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:00 0000000100000000000000B1 2024-05-14T17:09:51.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000B2 2024-05-14T17:09:51.906 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000B3 2024-05-14T17:09:51.959 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000B4 2024-05-14T17:09:51.998 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000B5 2024-05-14T17:09:52.031 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000B6 2024-05-14T17:09:52.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:01 0000000100000000000000B7 2024-05-14T17:09:52.098 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000B8 2024-05-14T17:09:52.131 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:01 0000000100000000000000B9 2024-05-14T17:09:52.173 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:59 0000000100000000000000BA 2024-05-14T17:09:52.207 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000BB 2024-05-14T17:09:52.240 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000BC 2024-05-14T17:09:52.273 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000BD 2024-05-14T17:09:52.298 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000BE 2024-05-14T17:09:52.332 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000BF 2024-05-14T17:09:52.365 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000C0 2024-05-14T17:09:52.398 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000C1 2024-05-14T17:09:52.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000C2 2024-05-14T17:09:52.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000C3 2024-05-14T17:09:52.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000C4 2024-05-14T17:09:52.521 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000C5 2024-05-14T17:09:52.599 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000C6 2024-05-14T17:09:52.640 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000C7 2024-05-14T17:09:52.673 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000C8 2024-05-14T17:09:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000C9 2024-05-14T17:09:52.740 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000CA 2024-05-14T17:09:52.782 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000CB 2024-05-14T17:09:52.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000CC 2024-05-14T17:09:52.857 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000CD 2024-05-14T17:09:52.899 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000CE 2024-05-14T17:09:52.949 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000CF 2024-05-14T17:09:52.974 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000D0 2024-05-14T17:09:53.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000D1 2024-05-14T17:09:53.032 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 archive_status 2024-05-14T17:09:53.057 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:53.090 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-14T17:09:53.124 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-05-14T17:09:53.157 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:50 . 2024-05-14T17:09:53.191 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 14 17:08 .. 2024-05-14T17:09:53.224 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:09:53.241 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-14T17:09:53.271 INFO:teuthology.orchestra.run.smithi069.stdout:total 376 2024-05-14T17:09:53.307 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:59 . 2024-05-14T17:09:53.341 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:08 .. 2024-05-14T17:09:53.360 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 May 14 16:59 0000 2024-05-14T17:09:53.382 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 17:06 0001 2024-05-14T17:09:53.408 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-14T17:09:53.443 INFO:tasks.quiescer:joining Quiescers 2024-05-14T17:09:53.466 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-14T17:09:53.499 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:09:53.525 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:54 smithi069 ceph-mon[31507]: pgmap v601: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-05-14T17:09:53.550 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:04 smithi203 ceph-mon[35787]: pgmap v606: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 89 KiB/s wr, 36 op/s 2024-05-14T17:09:53.566 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:06 smithi203 ceph-mon[35787]: pgmap v607: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 85 KiB/s wr, 33 op/s 2024-05-14T17:09:53.599 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:08 smithi203 ceph-mon[35787]: pgmap v608: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 90 KiB/s wr, 36 op/s 2024-05-14T17:09:53.645 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:10 smithi203 ceph-mon[35787]: pgmap v609: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 29 op/s 2024-05-14T17:09:53.674 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:10 smithi203 ceph-mon[35787]: osdmap e111: 12 total, 12 up, 12 in 2024-05-14T17:09:53.699 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:12 smithi203 ceph-mon[35787]: pgmap v611: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 50 KiB/s wr, 28 op/s 2024-05-14T17:09:53.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:14 smithi203 ceph-mon[35787]: pgmap v612: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-05-14T17:09:53.758 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:14 smithi203 ceph-mon[35787]: osdmap e112: 12 total, 12 up, 12 in 2024-05-14T17:09:53.800 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:16 smithi203 ceph-mon[35787]: pgmap v614: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-05-14T17:09:53.833 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:18 smithi203 ceph-mon[35787]: pgmap v615: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-14T17:09:53.875 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:20 smithi203 ceph-mon[35787]: pgmap v616: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:09:53.916 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:22 smithi203 ceph-mon[35787]: pgmap v617: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:09:53.950 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:24 smithi203 ceph-mon[35787]: pgmap v618: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-05-14T17:09:53.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:26 smithi203 ceph-mon[35787]: pgmap v619: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9.5 KiB/s wr, 5 op/s 2024-05-14T17:09:54.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:26 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:09:54.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:27 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:09:54.092 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:27 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:09:54.125 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:27 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:09:54.167 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:28 smithi203 ceph-mon[35787]: pgmap v620: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 7 op/s 2024-05-14T17:09:54.208 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:30 smithi203 ceph-mon[35787]: pgmap v621: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2024-05-14T17:09:54.261 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:32 smithi203 ceph-mon[35787]: pgmap v622: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2024-05-14T17:09:54.300 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:34 smithi203 ceph-mon[35787]: pgmap v623: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-14T17:09:54.342 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:36 smithi203 ceph-mon[35787]: pgmap v624: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-14T17:09:54.375 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:38 smithi203 ceph-mon[35787]: pgmap v625: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 KiB/s wr, 8 op/s 2024-05-14T17:09:54.417 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:40 smithi203 ceph-mon[35787]: pgmap v626: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 6 op/s 2024-05-14T17:09:54.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:42 smithi203 ceph-mon[35787]: pgmap v627: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 6 op/s 2024-05-14T17:09:54.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:44 smithi203 ceph-mon[35787]: pgmap v628: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 7 op/s 2024-05-14T17:09:54.532 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:46 smithi203 ceph-mon[35787]: pgmap v629: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 5 op/s 2024-05-14T17:09:54.592 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:48 smithi203 ceph-mon[35787]: pgmap v630: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 7 op/s 2024-05-14T17:09:54.625 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:50 smithi203 ceph-mon[35787]: pgmap v631: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:09:54.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:52 smithi203 ceph-mon[35787]: pgmap v632: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:09:54.700 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:04 smithi092 ceph-mon[35763]: pgmap v606: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 89 KiB/s wr, 36 op/s 2024-05-14T17:09:54.734 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:06 smithi092 ceph-mon[35763]: pgmap v607: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 85 KiB/s wr, 33 op/s 2024-05-14T17:09:54.767 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:08 smithi092 ceph-mon[35763]: pgmap v608: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 90 KiB/s wr, 36 op/s 2024-05-14T17:09:54.800 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:10 smithi092 ceph-mon[35763]: pgmap v609: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 29 op/s 2024-05-14T17:09:54.834 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:10 smithi092 ceph-mon[35763]: osdmap e111: 12 total, 12 up, 12 in 2024-05-14T17:09:54.867 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:12 smithi092 ceph-mon[35763]: pgmap v611: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 50 KiB/s wr, 28 op/s 2024-05-14T17:09:54.936 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:14 smithi092 ceph-mon[35763]: pgmap v612: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-05-14T17:09:54.975 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:14 smithi092 ceph-mon[35763]: osdmap e112: 12 total, 12 up, 12 in 2024-05-14T17:09:55.009 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:16 smithi092 ceph-mon[35763]: pgmap v614: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-05-14T17:09:55.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:18 smithi092 ceph-mon[35763]: pgmap v615: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-14T17:09:55.084 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:20 smithi092 ceph-mon[35763]: pgmap v616: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:09:55.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:22 smithi092 ceph-mon[35763]: pgmap v617: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:09:55.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:24 smithi092 ceph-mon[35763]: pgmap v618: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-05-14T17:09:55.204 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:26 smithi092 ceph-mon[35763]: pgmap v619: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9.5 KiB/s wr, 5 op/s 2024-05-14T17:09:55.242 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:26 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:09:55.276 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:27 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:09:55.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:27 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:09:55.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:27 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:09:55.426 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:28 smithi092 ceph-mon[35763]: pgmap v620: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 7 op/s 2024-05-14T17:09:55.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:30 smithi092 ceph-mon[35763]: pgmap v621: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2024-05-14T17:09:55.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:32 smithi092 ceph-mon[35763]: pgmap v622: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2024-05-14T17:09:55.543 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:34 smithi092 ceph-mon[35763]: pgmap v623: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-14T17:09:55.584 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:36 smithi092 ceph-mon[35763]: pgmap v624: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-14T17:09:55.618 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:38 smithi092 ceph-mon[35763]: pgmap v625: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 KiB/s wr, 8 op/s 2024-05-14T17:09:55.651 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:40 smithi092 ceph-mon[35763]: pgmap v626: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 6 op/s 2024-05-14T17:09:55.684 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:42 smithi092 ceph-mon[35763]: pgmap v627: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 6 op/s 2024-05-14T17:09:55.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:44 smithi092 ceph-mon[35763]: pgmap v628: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 7 op/s 2024-05-14T17:09:55.743 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:46 smithi092 ceph-mon[35763]: pgmap v629: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 5 op/s 2024-05-14T17:09:55.793 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:48 smithi092 ceph-mon[35763]: pgmap v630: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 7 op/s 2024-05-14T17:09:55.826 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:50 smithi092 ceph-mon[35763]: pgmap v631: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:09:55.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:52 smithi092 ceph-mon[35763]: pgmap v632: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:09:55.912 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:54 smithi092 ceph-mon[35763]: pgmap v633: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-14T17:09:55.943 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:54 smithi203 ceph-mon[35787]: pgmap v633: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-14T17:09:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 -- 172.21.15.69:0/3174599859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f84101c30 msgr2=0x7f8f841020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 --2- 172.21.15.69:0/3174599859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f84101c30 0x7f8f841020b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009900 tx=0x7f8f7802a410 comp rx=0 tx=0).stop 2024-05-14T17:09:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 -- 172.21.15.69:0/3174599859 shutdown_connections 2024-05-14T17:09:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 --2- 172.21.15.69:0/3174599859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f841025f0 0x7f8f8410e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 --2- 172.21.15.69:0/3174599859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f84101c30 0x7f8f841020b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 --2- 172.21.15.69:0/3174599859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f84106920 0x7f8f84106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 -- 172.21.15.69:0/3174599859 >> 172.21.15.69:0/3174599859 conn(0x7f8f84078730 msgr2=0x7f8f84078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 -- 172.21.15.69:0/3174599859 shutdown_connections 2024-05-14T17:09:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 -- 172.21.15.69:0/3174599859 wait complete. 2024-05-14T17:09:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 Processor -- start 2024-05-14T17:09:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.080+0000 7f8f8ae09640 1 -- start start 2024-05-14T17:09:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f8ae09640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f84101c30 0x7f8f84198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f8ae09640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f841025f0 0x7f8f84198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f8ae09640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f84106920 0x7f8f8419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f8ae09640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f84111bc0 con 0x7f8f84106920 2024-05-14T17:09:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f8ae09640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f84111a40 con 0x7f8f841025f0 2024-05-14T17:09:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f8ae09640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f84111d40 con 0x7f8f84101c30 2024-05-14T17:09:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f893a6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f84106920 0x7f8f8419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f893a6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f84106920 0x7f8f8419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52184/0 (socket says 172.21.15.69:52184) 2024-05-14T17:09:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f893a6640 1 -- 172.21.15.69:0/499801465 learned_addr learned my addr 172.21.15.69:0/499801465 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:09:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f83fff640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f841025f0 0x7f8f84198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f84101c30 0x7f8f84198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f893a6640 1 -- 172.21.15.69:0/499801465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f84101c30 msgr2=0x7f8f84198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f893a6640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f84101c30 0x7f8f84198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f893a6640 1 -- 172.21.15.69:0/499801465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f841025f0 msgr2=0x7f8f84198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f893a6640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f841025f0 0x7f8f84198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f893a6640 1 -- 172.21.15.69:0/499801465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8419cfe0 con 0x7f8f84106920 2024-05-14T17:09:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f84101c30 0x7f8f84198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f893a6640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f84106920 0x7f8f8419c8b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00ebb0 tx=0x7f8f7c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f81ffb640 1 -- 172.21.15.69:0/499801465 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c007ca0 con 0x7f8f84106920 2024-05-14T17:09:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f81ffb640 1 -- 172.21.15.69:0/499801465 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f7c007e40 con 0x7f8f84106920 2024-05-14T17:09:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f8ae09640 1 -- 172.21.15.69:0/499801465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8419d270 con 0x7f8f84106920 2024-05-14T17:09:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.081+0000 7f8f81ffb640 1 -- 172.21.15.69:0/499801465 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0057d0 con 0x7f8f84106920 2024-05-14T17:09:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.082+0000 7f8f8ae09640 1 -- 172.21.15.69:0/499801465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8406b2a0 con 0x7f8f84106920 2024-05-14T17:09:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.084+0000 7f8f81ffb640 1 -- 172.21.15.69:0/499801465 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8f7c004a40 con 0x7f8f84106920 2024-05-14T17:09:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.084+0000 7f8f8dbfe740 1 Processor -- start 2024-05-14T17:09:57.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.084+0000 7f8f8dbfe740 1 -- start start 2024-05-14T17:09:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.084+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bf2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.084+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bf3210 0x561335bfd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.084+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bfdad0 0x561335bffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.084+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561335b65c30 con 0x561335bf28f0 2024-05-14T17:09:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.084+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561335b557d0 con 0x561335bfdad0 2024-05-14T17:09:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.084+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561335b59810 con 0x561335bf3210 2024-05-14T17:09:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f81ffb640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8f6c0b2a10 0x7f8f6c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bf2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bf2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52192/0 (socket says 172.21.15.69:52192) 2024-05-14T17:09:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f83fff640 1 -- 172.21.15.69:0/4141156085 learned_addr learned my addr 172.21.15.69:0/4141156085 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:09:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f893a6640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8f6c0b2a10 0x7f8f6c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/4141156085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bf3210 0x561335bfd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f83fff640 1 --2- 172.21.15.69:0/4141156085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bfdad0 0x561335bffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f81ffb640 1 -- 172.21.15.69:0/499801465 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7f8f7c0e3c20 con 0x7f8f84106920 2024-05-14T17:09:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f83fff640 1 -- 172.21.15.69:0/4141156085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bf3210 msgr2=0x561335bfd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f83fff640 1 --2- 172.21.15.69:0/4141156085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bf3210 0x561335bfd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f83fff640 1 -- 172.21.15.69:0/4141156085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bfdad0 msgr2=0x561335bffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f83fff640 1 --2- 172.21.15.69:0/4141156085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bfdad0 0x561335bffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f83fff640 1 -- 172.21.15.69:0/4141156085 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133586a210 con 0x561335bf28f0 2024-05-14T17:09:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f893a6640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8f6c0b2a10 0x7f8f6c0b4ed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0e1960 tx=0x7f8f7c0756c0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f83fff640 1 --2- 172.21.15.69:0/4141156085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bf2cd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8f7802a930 tx=0x7f8f78005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1630b64a575d2973 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f6affd640 1 -- 172.21.15.69:0/4141156085 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78044020 con 0x561335bf28f0 2024-05-14T17:09:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f6affd640 1 -- 172.21.15.69:0/4141156085 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f780317e0 con 0x561335bf28f0 2024-05-14T17:09:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.085+0000 7f8f6affd640 1 -- 172.21.15.69:0/4141156085 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78035040 con 0x561335bf28f0 2024-05-14T17:09:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.086+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/4141156085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 msgr2=0x561335bf2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.086+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/4141156085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bf2cd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8f7802a930 tx=0x7f8f78005c00 comp rx=0 tx=0).stop 2024-05-14T17:09:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.086+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/4141156085 shutdown_connections 2024-05-14T17:09:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.086+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/4141156085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bfdad0 0x561335bffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.086+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/4141156085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bf3210 0x561335bfd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.086+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/4141156085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bf2cd0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.086+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/4141156085 >> 172.21.15.69:0/4141156085 conn(0x561335beee00 msgr2=0x561335bef1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.086+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/4141156085 shutdown_connections 2024-05-14T17:09:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.086+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/4141156085 wait complete. 2024-05-14T17:09:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.086+0000 7f8f8dbfe740 1 -- start start 2024-05-14T17:09:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f8dbfe740 1 Processor -- start 2024-05-14T17:09:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bf28f0 0x561335bfd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 0x561335bfa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bfdad0 0x561335bfadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56133583b790 con 0x561335bfdad0 2024-05-14T17:09:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561335b11890 con 0x561335bf3210 2024-05-14T17:09:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561335c00740 con 0x561335bf28f0 2024-05-14T17:09:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f8dbfe740 10 client.0 _ll_register_callbacks cb 0x561335b14d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bfdad0 0x561335bfadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bfdad0 0x561335bfadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52208/0 (socket says 172.21.15.69:52208) 2024-05-14T17:09:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f83fff640 1 -- 172.21.15.69:0/1764168197 learned_addr learned my addr 172.21.15.69:0/1764168197 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:09:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f83fff640 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bf28f0 0x561335bfd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 0x561335bfa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f83fff640 1 -- 172.21.15.69:0/1764168197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bf28f0 msgr2=0x561335bfd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f83fff640 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bf28f0 0x561335bfd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f83fff640 1 -- 172.21.15.69:0/1764168197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 msgr2=0x561335bfa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f83fff640 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 0x561335bfa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.087+0000 7f8f83fff640 1 -- 172.21.15.69:0/1764168197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561335936620 con 0x561335bfdad0 2024-05-14T17:09:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f83fff640 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bfdad0 0x561335bfadc0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8f78031a90 tx=0x7f8f78040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce4f0c36162f87c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f63fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:09:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f63fff640 1 -- 172.21.15.69:0/1764168197 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78035030 con 0x561335bfdad0 2024-05-14T17:09:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f63fff640 1 -- 172.21.15.69:0/1764168197 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7f8f78004050 con 0x561335bfdad0 2024-05-14T17:09:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1764168197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f780317e0 con 0x561335bfdad0 2024-05-14T17:09:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f63fff640 1 -- 172.21.15.69:0/1764168197 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8f78044070 con 0x561335bfdad0 2024-05-14T17:09:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f8dbfe740 10 client.17331 fetch_fsmap learned FSMap version 61 2024-05-14T17:09:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1764168197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x561335c0c1e0 con 0x561335bfdad0 2024-05-14T17:09:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f63fff640 1 -- 172.21.15.69:0/1764168197 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f780317e0 con 0x561335bfdad0 2024-05-14T17:09:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f63fff640 1 -- 172.21.15.69:0/1764168197 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f8f780035f0 con 0x561335bfdad0 2024-05-14T17:09:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f8dbfe740 10 client.17331 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:09:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f8dbfe740 10 client.17331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:09:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f8dbfe740 20 client.17331 populate_metadata read hostname 'smithi069' 2024-05-14T17:09:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561335c0d530 0x561335c0f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f8dbfe740 4 client.17331 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:09:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.088+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1764168197 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561335b67b80 con 0x561335c0d530 2024-05-14T17:10:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.089+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561335c0d530 0x561335c0f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.089+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561335c0d530 0x561335c0f920 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d53c7a319622890 server_cookie=a27440f772dbe039 in_seq=0 out_seq=0 2024-05-14T17:10:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.089+0000 7f8f63fff640 10 client.17331 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f63fff640 1 -- 172.21.15.69:0/1764168197 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561335b67b80 con 0x561335c0d530 2024-05-14T17:10:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f63fff640 10 client.17331 handle_command_reply: tid=0 2024-05-14T17:10:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 client.17331 shutdown 2024-05-14T17:10:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1764168197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bfdad0 msgr2=0x561335bfadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bfdad0 0x561335bfadc0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8f78031a90 tx=0x7f8f78040040 comp rx=0 tx=0).stop 2024-05-14T17:10:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1764168197 shutdown_connections 2024-05-14T17:10:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561335c0d530 0x561335c0f920 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bfdad0 0x561335bfadc0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 0x561335bfa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f63fff640 0 client.17331 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1764168197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bf28f0 0x561335bfd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1764168197 >> 172.21.15.69:0/1764168197 conn(0x561335beee00 msgr2=0x561335bee100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1764168197 shutdown_connections 2024-05-14T17:10:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1764168197 wait complete. 2024-05-14T17:10:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.090+0000 7f8f8dbfe740 20 client.17331 trim_cache size 0 max 16384 2024-05-14T17:10:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f8dbfe740 1 Processor -- start 2024-05-14T17:10:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f8dbfe740 1 -- start start 2024-05-14T17:10:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bfcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 0x561335bf5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bfdad0 0x561335bf5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841add90 con 0x561335bf28f0 2024-05-14T17:10:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841ae460 con 0x561335bf3210 2024-05-14T17:10:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841abd70 con 0x561335bfdad0 2024-05-14T17:10:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bfcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bfcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52216/0 (socket says 172.21.15.69:52216) 2024-05-14T17:10:01.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f83fff640 1 -- 172.21.15.69:0/3693936955 learned_addr learned my addr 172.21.15.69:0/3693936955 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/3693936955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 0x561335bf5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f83fff640 1 -- 172.21.15.69:0/3693936955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bfdad0 msgr2=0x561335bf5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f83fff640 1 --2- 172.21.15.69:0/3693936955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bfdad0 0x561335bf5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f83fff640 1 -- 172.21.15.69:0/3693936955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 msgr2=0x561335bf5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:01.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f83fff640 1 --2- 172.21.15.69:0/3693936955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 0x561335bf5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f83fff640 1 -- 172.21.15.69:0/3693936955 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561335bee880 con 0x561335bf28f0 2024-05-14T17:10:01.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.105+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/3693936955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 0x561335bf5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f83fff640 1 --2- 172.21.15.69:0/3693936955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bfcff0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005d00 tx=0x7f8f78005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75b7c1a10f69ae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f6a7fc640 1 -- 172.21.15.69:0/3693936955 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78035070 con 0x561335bf28f0 2024-05-14T17:10:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f6a7fc640 1 -- 172.21.15.69:0/3693936955 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f780035f0 con 0x561335bf28f0 2024-05-14T17:10:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f6a7fc640 1 -- 172.21.15.69:0/3693936955 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78004c90 con 0x561335bf28f0 2024-05-14T17:10:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/3693936955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 msgr2=0x561335bfcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/3693936955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bfcff0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005d00 tx=0x7f8f78005bd0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/3693936955 shutdown_connections 2024-05-14T17:10:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/3693936955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bfdad0 0x561335bf5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/3693936955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf3210 0x561335bf5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/3693936955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf28f0 0x561335bfcff0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.106+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/3693936955 >> 172.21.15.69:0/3693936955 conn(0x561335beee00 msgr2=0x561335bf5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.107+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/3693936955 shutdown_connections 2024-05-14T17:10:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.107+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/3693936955 wait complete. 2024-05-14T17:10:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.107+0000 7f8f8dbfe740 1 -- start start 2024-05-14T17:10:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.107+0000 7f8f8dbfe740 1 Processor -- start 2024-05-14T17:10:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.107+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf28f0 0x561335c10630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.107+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf3210 0x561335c10b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.107+0000 7f8f8dbfe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bfdad0 0x561335c110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.107+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841ab6a0 con 0x561335bf3210 2024-05-14T17:10:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.107+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841aa2b0 con 0x561335bf28f0 2024-05-14T17:10:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.107+0000 7f8f8dbfe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841aa900 con 0x561335bfdad0 2024-05-14T17:10:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f8dbfe740 10 client.0 _ll_register_callbacks cb 0x7f8f78032c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f88ba5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf3210 0x561335c10b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf28f0 0x561335c10630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f88ba5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf3210 0x561335c10b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52222/0 (socket says 172.21.15.69:52222) 2024-05-14T17:10:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f88ba5640 1 -- 172.21.15.69:0/1587541061 learned_addr learned my addr 172.21.15.69:0/1587541061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f83fff640 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bfdad0 0x561335c110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f88ba5640 1 -- 172.21.15.69:0/1587541061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bfdad0 msgr2=0x561335c110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bfdad0 0x561335c110b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f88ba5640 1 -- 172.21.15.69:0/1587541061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf28f0 msgr2=0x561335c10630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf28f0 0x561335c10630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f88ba5640 1 -- 172.21.15.69:0/1587541061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561335c76b50 con 0x561335bf3210 2024-05-14T17:10:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f83fff640 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf28f0 0x561335c10630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f88ba5640 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf3210 0x561335c10b70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400bfd0 tx=0x7f8f740128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cc5f7ac3f24f26 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f63fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f63fff640 1 -- 172.21.15.69:0/1587541061 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7401b790 con 0x561335bf3210 2024-05-14T17:10:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f63fff640 1 -- 172.21.15.69:0/1587541061 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7f8f7401b930 con 0x561335bf3210 2024-05-14T17:10:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1587541061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f780035f0 con 0x561335bf3210 2024-05-14T17:10:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f63fff640 1 -- 172.21.15.69:0/1587541061 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8f7402b050 con 0x561335bf3210 2024-05-14T17:10:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f8dbfe740 10 client.17337 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.108+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1587541061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x561335c77e60 con 0x561335bf3210 2024-05-14T17:10:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f63fff640 1 -- 172.21.15.69:0/1587541061 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f780035f0 con 0x561335bf3210 2024-05-14T17:10:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f63fff640 1 -- 172.21.15.69:0/1587541061 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f8f740233d0 con 0x561335bf3210 2024-05-14T17:10:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f8dbfe740 10 client.17337 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f8dbfe740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f8dbfe740 20 client.17337 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561335c78610 0x561335c7aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:03.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f8dbfe740 4 client.17337 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1587541061 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561335b61360 con 0x561335c78610 2024-05-14T17:10:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f83fff640 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561335c78610 0x561335c7aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f63fff640 10 client.17337 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.109+0000 7f8f83fff640 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561335c78610 0x561335c7aa30 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e30ce9ba3d5978cc server_cookie=9e61070feeaba8a0 in_seq=0 out_seq=0 2024-05-14T17:10:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.119+0000 7f8f63fff640 1 -- 172.21.15.69:0/1587541061 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561335b61360 con 0x561335c78610 2024-05-14T17:10:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.119+0000 7f8f63fff640 10 client.17337 handle_command_reply: tid=0 2024-05-14T17:10:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 client.17337 shutdown 2024-05-14T17:10:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1587541061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf3210 msgr2=0x561335c10b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf3210 0x561335c10b70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400bfd0 tx=0x7f8f740128e0 comp rx=0 tx=0).stop 2024-05-14T17:10:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1587541061 shutdown_connections 2024-05-14T17:10:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x561335c78610 0x561335c7aa30 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561335bfdad0 0x561335c110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561335bf3210 0x561335c10b70 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f63fff640 0 client.17337 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 --2- 172.21.15.69:0/1587541061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561335bf28f0 0x561335c10630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1587541061 >> 172.21.15.69:0/1587541061 conn(0x561335beee00 msgr2=0x561335bee100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1587541061 shutdown_connections 2024-05-14T17:10:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 1 -- 172.21.15.69:0/1587541061 wait complete. 2024-05-14T17:10:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8dbfe740 20 client.17337 trim_cache size 0 max 16384 2024-05-14T17:10:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:10:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:10:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:10:04.009 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:56 smithi092 ceph-mon[35763]: pgmap v634: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-05-14T17:10:04.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:09:58 smithi092 ceph-mon[35763]: pgmap v635: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-14T17:10:04.075 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:00 smithi092 ceph-mon[35763]: pgmap v636: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-05-14T17:10:04.141 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:00 smithi092 ceph-mon[35763]: overall HEALTH_OK 2024-05-14T17:10:04.174 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:02 smithi092 ceph-mon[35763]: pgmap v637: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 KiB/s wr, 7 op/s 2024-05-14T17:10:04.208 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:56 smithi203 ceph-mon[35787]: pgmap v634: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-05-14T17:10:04.256 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:09:58 smithi203 ceph-mon[35787]: pgmap v635: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-14T17:10:04.291 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:00 smithi203 ceph-mon[35787]: pgmap v636: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-05-14T17:10:04.358 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:00 smithi203 ceph-mon[35787]: overall HEALTH_OK 2024-05-14T17:10:04.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:02 smithi203 ceph-mon[35787]: pgmap v637: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 KiB/s wr, 7 op/s 2024-05-14T17:10:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8ae09640 1 -- 172.21.15.69:0/499801465 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8f6c0b2a10 msgr2=0x7f8f6c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8ae09640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8f6c0b2a10 0x7f8f6c0b4ed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0e1960 tx=0x7f8f7c0756c0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8ae09640 1 -- 172.21.15.69:0/499801465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f84106920 msgr2=0x7f8f8419c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.120+0000 7f8f8ae09640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f84106920 0x7f8f8419c8b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00ebb0 tx=0x7f8f7c007590 comp rx=0 tx=0).stop 2024-05-14T17:10:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.121+0000 7f8f8ae09640 1 -- 172.21.15.69:0/499801465 shutdown_connections 2024-05-14T17:10:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.121+0000 7f8f8ae09640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f8f6c0b2a10 0x7f8f6c0b4ed0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.121+0000 7f8f8ae09640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f84106920 0x7f8f8419c8b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.121+0000 7f8f8ae09640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f841025f0 0x7f8f84198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.121+0000 7f8f8ae09640 1 --2- 172.21.15.69:0/499801465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f84101c30 0x7f8f84198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.121+0000 7f8f8ae09640 1 -- 172.21.15.69:0/499801465 >> 172.21.15.69:0/499801465 conn(0x7f8f84078730 msgr2=0x7f8f840ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.121+0000 7f8f8ae09640 1 -- 172.21.15.69:0/499801465 shutdown_connections 2024-05-14T17:10:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:08:55.121+0000 7f8f8ae09640 1 -- 172.21.15.69:0/499801465 wait complete. 2024-05-14T17:10:04.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:56 smithi069 ceph-mon[31507]: pgmap v602: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 6 op/s 2024-05-14T17:10:04.892 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:08:58 smithi069 ceph-mon[31507]: pgmap v603: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-05-14T17:10:04.933 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:00 smithi069 ceph-mon[31507]: pgmap v604: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-05-14T17:10:04.967 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:02 smithi069 ceph-mon[31507]: pgmap v605: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-05-14T17:10:05.000 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:04 smithi069 ceph-mon[31507]: pgmap v606: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 89 KiB/s wr, 36 op/s 2024-05-14T17:10:05.042 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:06 smithi069 ceph-mon[31507]: pgmap v607: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 85 KiB/s wr, 33 op/s 2024-05-14T17:10:05.076 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:04 smithi092 ceph-mon[35763]: pgmap v638: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-14T17:10:05.109 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:04 smithi203 ceph-mon[35787]: pgmap v638: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-14T17:10:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.219+0000 7fa848d89640 1 -- 172.21.15.69:0/203602019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa84406bc40 msgr2=0x7fa84410f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.219+0000 7fa848d89640 1 --2- 172.21.15.69:0/203602019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa84406bc40 0x7fa84410f8f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009ca0 tx=0x7fa83c02a950 comp rx=0 tx=0).stop 2024-05-14T17:10:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.220+0000 7fa848d89640 1 -- 172.21.15.69:0/203602019 shutdown_connections 2024-05-14T17:10:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.220+0000 7fa848d89640 1 --2- 172.21.15.69:0/203602019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa84406bc40 0x7fa84410f8f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.220+0000 7fa848d89640 1 --2- 172.21.15.69:0/203602019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84406b280 0x7fa84406b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.220+0000 7fa848d89640 1 --2- 172.21.15.69:0/203602019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa844108c50 0x7fa844109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.220+0000 7fa848d89640 1 -- 172.21.15.69:0/203602019 >> 172.21.15.69:0/203602019 conn(0x7fa844077750 msgr2=0x7fa844077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.220+0000 7fa848d89640 1 -- 172.21.15.69:0/203602019 shutdown_connections 2024-05-14T17:10:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.220+0000 7fa848d89640 1 -- 172.21.15.69:0/203602019 wait complete. 2024-05-14T17:10:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.220+0000 7fa848d89640 1 Processor -- start 2024-05-14T17:10:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa848d89640 1 -- start start 2024-05-14T17:10:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa848d89640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84406b280 0x7fa8441990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa848d89640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa84406bc40 0x7fa844199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa848d89640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa844108c50 0x7fa84419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa848d89640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa844112d70 con 0x7fa844108c50 2024-05-14T17:10:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa848d89640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa844112bf0 con 0x7fa84406b280 2024-05-14T17:10:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa848d89640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa844112ef0 con 0x7fa84406bc40 2024-05-14T17:10:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa842d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa844108c50 0x7fa84419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa842d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa844108c50 0x7fa84419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40662/0 (socket says 172.21.15.69:40662) 2024-05-14T17:10:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa842d9d640 1 -- 172.21.15.69:0/736984375 learned_addr learned my addr 172.21.15.69:0/736984375 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa84259c640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84406b280 0x7fa8441990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa83bfff640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa84406bc40 0x7fa844199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa842d9d640 1 -- 172.21.15.69:0/736984375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa84406bc40 msgr2=0x7fa844199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa842d9d640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa84406bc40 0x7fa844199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa842d9d640 1 -- 172.21.15.69:0/736984375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84406b280 msgr2=0x7fa8441990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa842d9d640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84406b280 0x7fa8441990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa842d9d640 1 -- 172.21.15.69:0/736984375 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84419e0c0 con 0x7fa844108c50 2024-05-14T17:10:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.221+0000 7fa842d9d640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa844108c50 0x7fa84419d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009dc0 tx=0x7fa83c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.222+0000 7fa83b7fe640 1 -- 172.21.15.69:0/736984375 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c034040 con 0x7fa844108c50 2024-05-14T17:10:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.222+0000 7fa83b7fe640 1 -- 172.21.15.69:0/736984375 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa83c003e50 con 0x7fa844108c50 2024-05-14T17:10:06.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.222+0000 7fa848d89640 1 -- 172.21.15.69:0/736984375 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84419e350 con 0x7fa844108c50 2024-05-14T17:10:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.222+0000 7fa83b7fe640 1 -- 172.21.15.69:0/736984375 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c004600 con 0x7fa844108c50 2024-05-14T17:10:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.222+0000 7fa848d89640 1 -- 172.21.15.69:0/736984375 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441a5c40 con 0x7fa844108c50 2024-05-14T17:10:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.224+0000 7fa83b7fe640 1 -- 172.21.15.69:0/736984375 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa83c037020 con 0x7fa844108c50 2024-05-14T17:10:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.224+0000 7fa84bb7d740 1 Processor -- start 2024-05-14T17:10:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa84bb7d740 1 -- start start 2024-05-14T17:10:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e651cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 0x55a36e65c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e65eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a36e5c4c30 con 0x55a36e6518f0 2024-05-14T17:10:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a36e5b47d0 con 0x55a36e652210 2024-05-14T17:10:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a36e5b8810 con 0x55a36e65cad0 2024-05-14T17:10:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa83b7fe640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa8300b2a90 0x7fa8300b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e651cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e651cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40678/0 (socket says 172.21.15.69:40678) 2024-05-14T17:10:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa83bfff640 1 -- 172.21.15.69:0/1722952032 learned_addr learned my addr 172.21.15.69:0/1722952032 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa842d9d640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa8300b2a90 0x7fa8300b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa84259c640 1 --2- 172.21.15.69:0/1722952032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 0x55a36e65c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.225+0000 7fa83bfff640 1 --2- 172.21.15.69:0/1722952032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e65eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa83b7fe640 1 -- 172.21.15.69:0/736984375 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7fa83c1030d0 con 0x7fa844108c50 2024-05-14T17:10:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa83bfff640 1 -- 172.21.15.69:0/1722952032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 msgr2=0x55a36e65eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa83bfff640 1 --2- 172.21.15.69:0/1722952032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e65eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa83bfff640 1 -- 172.21.15.69:0/1722952032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 msgr2=0x55a36e65c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa83bfff640 1 --2- 172.21.15.69:0/1722952032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 0x55a36e65c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa83bfff640 1 -- 172.21.15.69:0/1722952032 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36e2c9210 con 0x55a36e6518f0 2024-05-14T17:10:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa83bfff640 1 --2- 172.21.15.69:0/1722952032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e651cd0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa82c005bc0 tx=0x7fa82c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a21a08b22e8cef server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa842d9d640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa8300b2a90 0x7fa8300b4f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa83c100a70 tx=0x7fa83c096e80 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa838ff9640 1 -- 172.21.15.69:0/1722952032 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c004050 con 0x55a36e6518f0 2024-05-14T17:10:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa838ff9640 1 -- 172.21.15.69:0/1722952032 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82c0041f0 con 0x55a36e6518f0 2024-05-14T17:10:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.226+0000 7fa838ff9640 1 -- 172.21.15.69:0/1722952032 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c00f740 con 0x55a36e6518f0 2024-05-14T17:10:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1722952032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 msgr2=0x55a36e651cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/1722952032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e651cd0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa82c005bc0 tx=0x7fa82c009850 comp rx=0 tx=0).stop 2024-05-14T17:10:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1722952032 shutdown_connections 2024-05-14T17:10:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/1722952032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e65eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/1722952032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 0x55a36e65c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/1722952032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e651cd0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1722952032 >> 172.21.15.69:0/1722952032 conn(0x55a36e64de00 msgr2=0x55a36e64e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1722952032 shutdown_connections 2024-05-14T17:10:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1722952032 wait complete. 2024-05-14T17:10:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 -- start start 2024-05-14T17:10:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 Processor -- start 2024-05-14T17:10:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.227+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e6518f0 0x55a36e65c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e652210 0x55a36e659700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e659c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a36e29a790 con 0x55a36e652210 2024-05-14T17:10:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a36e570890 con 0x55a36e6518f0 2024-05-14T17:10:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a36e65f7a0 con 0x55a36e65cad0 2024-05-14T17:10:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84bb7d740 10 client.0 _ll_register_callbacks cb 0x55a36e573d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e652210 0x55a36e659700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e652210 0x55a36e659700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40684/0 (socket says 172.21.15.69:40684) 2024-05-14T17:10:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84259c640 1 -- 172.21.15.69:0/3616408999 learned_addr learned my addr 172.21.15.69:0/3616408999 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa83bfff640 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e6518f0 0x55a36e65c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa83bfff640 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e659c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84259c640 1 -- 172.21.15.69:0/3616408999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 msgr2=0x55a36e659c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84259c640 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e659c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84259c640 1 -- 172.21.15.69:0/3616408999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e6518f0 msgr2=0x55a36e65c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84259c640 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e6518f0 0x55a36e65c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84259c640 1 -- 172.21.15.69:0/3616408999 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36e395620 con 0x55a36e652210 2024-05-14T17:10:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa83bfff640 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e659c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84259c640 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e652210 0x55a36e659700 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa834012740 tx=0x7fa834012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cecc63b6605308 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa829ffb640 1 -- 172.21.15.69:0/3616408999 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83400f920 con 0x55a36e652210 2024-05-14T17:10:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa829ffb640 1 -- 172.21.15.69:0/3616408999 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7fa83400fac0 con 0x55a36e652210 2024-05-14T17:10:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84bb7d740 1 -- 172.21.15.69:0/3616408999 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82c0041f0 con 0x55a36e652210 2024-05-14T17:10:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa829ffb640 1 -- 172.21.15.69:0/3616408999 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa83400fd00 con 0x55a36e652210 2024-05-14T17:10:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84bb7d740 10 client.17349 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.228+0000 7fa84bb7d740 1 -- 172.21.15.69:0/3616408999 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a36e66b240 con 0x55a36e652210 2024-05-14T17:10:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.229+0000 7fa829ffb640 1 -- 172.21.15.69:0/3616408999 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82c0041f0 con 0x55a36e652210 2024-05-14T17:10:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.229+0000 7fa829ffb640 1 -- 172.21.15.69:0/3616408999 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fa83401c420 con 0x55a36e652210 2024-05-14T17:10:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.229+0000 7fa84bb7d740 10 client.17349 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.229+0000 7fa84bb7d740 10 client.17349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.229+0000 7fa84bb7d740 20 client.17349 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.229+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a36e66c7c0 0x55a36e66ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.229+0000 7fa84bb7d740 4 client.17349 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.229+0000 7fa84bb7d740 1 -- 172.21.15.69:0/3616408999 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a36e5c6b80 con 0x55a36e66c7c0 2024-05-14T17:10:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.229+0000 7fa83bfff640 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a36e66c7c0 0x55a36e66ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa829ffb640 10 client.17349 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa83bfff640 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a36e66c7c0 0x55a36e66ebd0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a664d62fb2c2f019 server_cookie=79ca36ddae88f9c in_seq=0 out_seq=0 2024-05-14T17:10:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa829ffb640 1 -- 172.21.15.69:0/3616408999 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a36e5c6b80 con 0x55a36e66c7c0 2024-05-14T17:10:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa829ffb640 10 client.17349 handle_command_reply: tid=0 2024-05-14T17:10:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa84bb7d740 1 client.17349 shutdown 2024-05-14T17:10:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa84bb7d740 1 -- 172.21.15.69:0/3616408999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e652210 msgr2=0x55a36e659700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e652210 0x55a36e659700 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa834012740 tx=0x7fa834012c00 comp rx=0 tx=0).stop 2024-05-14T17:10:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa84bb7d740 1 -- 172.21.15.69:0/3616408999 shutdown_connections 2024-05-14T17:10:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a36e66c7c0 0x55a36e66ebd0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e659c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e652210 0x55a36e659700 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa829ffb640 0 client.17349 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/3616408999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e6518f0 0x55a36e65c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.230+0000 7fa84bb7d740 1 -- 172.21.15.69:0/3616408999 >> 172.21.15.69:0/3616408999 conn(0x55a36e64de00 msgr2=0x55a36e64d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.231+0000 7fa84bb7d740 1 -- 172.21.15.69:0/3616408999 shutdown_connections 2024-05-14T17:10:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.231+0000 7fa84bb7d740 1 -- 172.21.15.69:0/3616408999 wait complete. 2024-05-14T17:10:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.231+0000 7fa84bb7d740 20 client.17349 trim_cache size 0 max 16384 2024-05-14T17:10:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa84bb7d740 1 Processor -- start 2024-05-14T17:10:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa84bb7d740 1 -- start start 2024-05-14T17:10:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e6518f0 0x55a36e668670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e652210 0x55a36e6744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e65cad0 0x55a36e6749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441b0010 con 0x55a36e65cad0 2024-05-14T17:10:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441b06e0 con 0x55a36e6518f0 2024-05-14T17:10:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441adff0 con 0x55a36e652210 2024-05-14T17:10:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e65cad0 0x55a36e6749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e65cad0 0x55a36e6749e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40696/0 (socket says 172.21.15.69:40696) 2024-05-14T17:10:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa83bfff640 1 -- 172.21.15.69:0/2470253119 learned_addr learned my addr 172.21.15.69:0/2470253119 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa83bfff640 1 --2- 172.21.15.69:0/2470253119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e6518f0 0x55a36e668670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa84259c640 1 --2- 172.21.15.69:0/2470253119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e652210 0x55a36e6744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa83bfff640 1 -- 172.21.15.69:0/2470253119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e652210 msgr2=0x55a36e6744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa83bfff640 1 --2- 172.21.15.69:0/2470253119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e652210 0x55a36e6744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa83bfff640 1 -- 172.21.15.69:0/2470253119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e6518f0 msgr2=0x55a36e668670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa83bfff640 1 --2- 172.21.15.69:0/2470253119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e6518f0 0x55a36e668670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa83bfff640 1 -- 172.21.15.69:0/2470253119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36e648510 con 0x55a36e65cad0 2024-05-14T17:10:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa84259c640 1 --2- 172.21.15.69:0/2470253119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e652210 0x55a36e6744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa83bfff640 1 --2- 172.21.15.69:0/2470253119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e65cad0 0x55a36e6749e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa82c003da0 tx=0x7fa82c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d32b345c28d364 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa82bfff640 1 -- 172.21.15.69:0/2470253119 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0050e0 con 0x55a36e65cad0 2024-05-14T17:10:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.246+0000 7fa82bfff640 1 -- 172.21.15.69:0/2470253119 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82c003aa0 con 0x55a36e65cad0 2024-05-14T17:10:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa82bfff640 1 -- 172.21.15.69:0/2470253119 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c02d7a0 con 0x55a36e65cad0 2024-05-14T17:10:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 -- 172.21.15.69:0/2470253119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e65cad0 msgr2=0x55a36e6749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/2470253119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e65cad0 0x55a36e6749e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa82c003da0 tx=0x7fa82c009850 comp rx=0 tx=0).stop 2024-05-14T17:10:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 -- 172.21.15.69:0/2470253119 shutdown_connections 2024-05-14T17:10:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/2470253119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e65cad0 0x55a36e6749e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/2470253119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e652210 0x55a36e6744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/2470253119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e6518f0 0x55a36e668670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 -- 172.21.15.69:0/2470253119 >> 172.21.15.69:0/2470253119 conn(0x55a36e64de00 msgr2=0x55a36e64d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 -- 172.21.15.69:0/2470253119 shutdown_connections 2024-05-14T17:10:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 -- 172.21.15.69:0/2470253119 wait complete. 2024-05-14T17:10:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 -- start start 2024-05-14T17:10:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.247+0000 7fa84bb7d740 1 Processor -- start 2024-05-14T17:10:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e6624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 0x55a36e662a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84bb7d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e662f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441ad920 con 0x55a36e6518f0 2024-05-14T17:10:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441ac530 con 0x55a36e652210 2024-05-14T17:10:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84bb7d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441acb80 con 0x55a36e65cad0 2024-05-14T17:10:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84bb7d740 10 client.0 _ll_register_callbacks cb 0x7fa834009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e6624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e6624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40698/0 (socket says 172.21.15.69:40698) 2024-05-14T17:10:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa83bfff640 1 -- 172.21.15.69:0/1231844328 learned_addr learned my addr 172.21.15.69:0/1231844328 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84259c640 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 0x55a36e662a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa83bfff640 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e662f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa83bfff640 1 -- 172.21.15.69:0/1231844328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 msgr2=0x55a36e662f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa83bfff640 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e662f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa83bfff640 1 -- 172.21.15.69:0/1231844328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 msgr2=0x55a36e662a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa83bfff640 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 0x55a36e662a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa83bfff640 1 -- 172.21.15.69:0/1231844328 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36e6d6060 con 0x55a36e6518f0 2024-05-14T17:10:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84259c640 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 0x55a36e662a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa83bfff640 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e6624d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa82c003e10 tx=0x7fa82c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bdd193131d298c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa829ffb640 1 -- 172.21.15.69:0/1231844328 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c034020 con 0x55a36e6518f0 2024-05-14T17:10:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa829ffb640 1 -- 172.21.15.69:0/1231844328 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8542+0+0 (secure 0 0 0) 0x7fa82c0050c0 con 0x55a36e6518f0 2024-05-14T17:10:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1231844328 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82c003aa0 con 0x55a36e6518f0 2024-05-14T17:10:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa829ffb640 1 -- 172.21.15.69:0/1231844328 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa82c03f050 con 0x55a36e6518f0 2024-05-14T17:10:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84bb7d740 10 client.17355 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.248+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1231844328 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a36e6d71e0 con 0x55a36e6518f0 2024-05-14T17:10:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa829ffb640 1 -- 172.21.15.69:0/1231844328 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82c003aa0 con 0x55a36e6518f0 2024-05-14T17:10:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa829ffb640 1 -- 172.21.15.69:0/1231844328 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fa82c00f500 con 0x55a36e6518f0 2024-05-14T17:10:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa84bb7d740 10 client.17355 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa84bb7d740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa84bb7d740 20 client.17355 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a36e6d7650 0x55a36e6d9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa84bb7d740 4 client.17355 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1231844328 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a36e5c0360 con 0x55a36e6d7650 2024-05-14T17:10:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa84259c640 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a36e6d7650 0x55a36e6d9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa829ffb640 10 client.17355 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.249+0000 7fa84259c640 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a36e6d7650 0x55a36e6d9a40 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269b239b0c1146b4 server_cookie=caf6d702fb64108c in_seq=0 out_seq=0 2024-05-14T17:10:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa829ffb640 1 -- 172.21.15.69:0/1231844328 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a36e5c0360 con 0x55a36e6d7650 2024-05-14T17:10:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa829ffb640 10 client.17355 handle_command_reply: tid=0 2024-05-14T17:10:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 client.17355 shutdown 2024-05-14T17:10:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1231844328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 msgr2=0x55a36e6624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e6624d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa82c003e10 tx=0x7fa82c005850 comp rx=0 tx=0).stop 2024-05-14T17:10:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1231844328 shutdown_connections 2024-05-14T17:10:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55a36e6d7650 0x55a36e6d9a40 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a36e65cad0 0x55a36e662f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa829ffb640 0 client.17355 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a36e652210 0x55a36e662a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 --2- 172.21.15.69:0/1231844328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a36e6518f0 0x55a36e6624d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1231844328 >> 172.21.15.69:0/1231844328 conn(0x55a36e64de00 msgr2=0x55a36e667660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1231844328 shutdown_connections 2024-05-14T17:10:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 1 -- 172.21.15.69:0/1231844328 wait complete. 2024-05-14T17:10:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.260+0000 7fa84bb7d740 20 client.17355 trim_cache size 0 max 16384 2024-05-14T17:10:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:10:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:10:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:10:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 -- 172.21.15.69:0/736984375 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa8300b2a90 msgr2=0x7fa8300b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa8300b2a90 0x7fa8300b4f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa83c100a70 tx=0x7fa83c096e80 comp rx=0 tx=0).stop 2024-05-14T17:10:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 -- 172.21.15.69:0/736984375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa844108c50 msgr2=0x7fa84419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa844108c50 0x7fa84419d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009dc0 tx=0x7fa83c003ce0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 -- 172.21.15.69:0/736984375 shutdown_connections 2024-05-14T17:10:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa8300b2a90 0x7fa8300b4f50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa844108c50 0x7fa84419d990 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa84406bc40 0x7fa844199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 --2- 172.21.15.69:0/736984375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84406b280 0x7fa8441990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 -- 172.21.15.69:0/736984375 >> 172.21.15.69:0/736984375 conn(0x7fa844077750 msgr2=0x7fa844100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 -- 172.21.15.69:0/736984375 shutdown_connections 2024-05-14T17:10:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:08.261+0000 7fa848d89640 1 -- 172.21.15.69:0/736984375 wait complete. 2024-05-14T17:10:13.892 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:08 smithi069 ceph-mon[31507]: pgmap v608: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 90 KiB/s wr, 36 op/s 2024-05-14T17:10:13.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:10 smithi069 ceph-mon[31507]: pgmap v609: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 29 op/s 2024-05-14T17:10:13.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:10 smithi069 ceph-mon[31507]: osdmap e111: 12 total, 12 up, 12 in 2024-05-14T17:10:14.025 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:12 smithi069 ceph-mon[31507]: pgmap v611: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 50 KiB/s wr, 28 op/s 2024-05-14T17:10:14.058 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:14 smithi069 ceph-mon[31507]: pgmap v612: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-05-14T17:10:14.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:14 smithi069 ceph-mon[31507]: osdmap e112: 12 total, 12 up, 12 in 2024-05-14T17:10:14.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:16 smithi069 ceph-mon[31507]: pgmap v614: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-05-14T17:10:14.159 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:18 smithi069 ceph-mon[31507]: pgmap v615: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-14T17:10:14.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:20 smithi069 ceph-mon[31507]: pgmap v616: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:10:14.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:06 smithi092 ceph-mon[35763]: pgmap v639: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 51 KiB/s wr, 24 op/s 2024-05-14T17:10:14.258 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:08 smithi092 ceph-mon[35763]: pgmap v640: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 63 KiB/s wr, 28 op/s 2024-05-14T17:10:14.317 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:10 smithi092 ceph-mon[35763]: pgmap v641: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 63 KiB/s wr, 26 op/s 2024-05-14T17:10:14.350 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:10 smithi092 ceph-mon[35763]: osdmap e113: 12 total, 12 up, 12 in 2024-05-14T17:10:14.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:12 smithi092 ceph-mon[35763]: pgmap v643: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 62 KiB/s wr, 29 op/s 2024-05-14T17:10:14.417 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:06 smithi203 ceph-mon[35787]: pgmap v639: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 51 KiB/s wr, 24 op/s 2024-05-14T17:10:14.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:08 smithi203 ceph-mon[35787]: pgmap v640: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 63 KiB/s wr, 28 op/s 2024-05-14T17:10:14.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:10 smithi203 ceph-mon[35787]: pgmap v641: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 63 KiB/s wr, 26 op/s 2024-05-14T17:10:14.517 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:10 smithi203 ceph-mon[35787]: osdmap e113: 12 total, 12 up, 12 in 2024-05-14T17:10:14.576 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:12 smithi203 ceph-mon[35787]: pgmap v643: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 62 KiB/s wr, 29 op/s 2024-05-14T17:10:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.353+0000 7f80ca4da640 1 -- 172.21.15.69:0/3105981245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4104600 msgr2=0x7f80c410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.353+0000 7f80ca4da640 1 --2- 172.21.15.69:0/3105981245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4104600 0x7f80c410cae0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009900 tx=0x7f80b802a510 comp rx=0 tx=0).stop 2024-05-14T17:10:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.353+0000 7f80ca4da640 1 -- 172.21.15.69:0/3105981245 shutdown_connections 2024-05-14T17:10:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.353+0000 7f80ca4da640 1 --2- 172.21.15.69:0/3105981245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c410d020 0x7f80c410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.353+0000 7f80ca4da640 1 --2- 172.21.15.69:0/3105981245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4104600 0x7f80c410cae0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.353+0000 7f80ca4da640 1 --2- 172.21.15.69:0/3105981245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80c4103ce0 0x7f80c41040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.353+0000 7f80ca4da640 1 -- 172.21.15.69:0/3105981245 >> 172.21.15.69:0/3105981245 conn(0x7f80c4078ad0 msgr2=0x7f80c4078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.353+0000 7f80ca4da640 1 -- 172.21.15.69:0/3105981245 shutdown_connections 2024-05-14T17:10:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.353+0000 7f80ca4da640 1 -- 172.21.15.69:0/3105981245 wait complete. 2024-05-14T17:10:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.354+0000 7f80ca4da640 1 Processor -- start 2024-05-14T17:10:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.354+0000 7f80ca4da640 1 -- start start 2024-05-14T17:10:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.354+0000 7f80ca4da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c4103ce0 0x7f80c41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.354+0000 7f80ca4da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4104600 0x7f80c4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.354+0000 7f80ca4da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80c410d020 0x7f80c419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.354+0000 7f80ca4da640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4112cd0 con 0x7f80c4104600 2024-05-14T17:10:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.354+0000 7f80ca4da640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4112b50 con 0x7f80c4103ce0 2024-05-14T17:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.354+0000 7f80ca4da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4112e50 con 0x7f80c410d020 2024-05-14T17:10:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4104600 0x7f80c4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4104600 0x7f80c4199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58960/0 (socket says 172.21.15.69:58960) 2024-05-14T17:10:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c37fe640 1 -- 172.21.15.69:0/1774516223 learned_addr learned my addr 172.21.15.69:0/1774516223 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c8a77640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80c410d020 0x7f80c419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c3fff640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c4103ce0 0x7f80c41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c37fe640 1 -- 172.21.15.69:0/1774516223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80c410d020 msgr2=0x7f80c419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c37fe640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80c410d020 0x7f80c419dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c37fe640 1 -- 172.21.15.69:0/1774516223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c4103ce0 msgr2=0x7f80c41991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c37fe640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c4103ce0 0x7f80c41991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c37fe640 1 -- 172.21.15.69:0/1774516223 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c419e1e0 con 0x7f80c4104600 2024-05-14T17:10:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c8a77640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80c410d020 0x7f80c419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c3fff640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c4103ce0 0x7f80c41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c37fe640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4104600 0x7f80c4199730 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005940 tx=0x7f80b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c17fa640 1 -- 172.21.15.69:0/1774516223 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b803f030 con 0x7f80c4104600 2024-05-14T17:10:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80ca4da640 1 -- 172.21.15.69:0/1774516223 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c419e470 con 0x7f80c4104600 2024-05-14T17:10:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c17fa640 1 -- 172.21.15.69:0/1774516223 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b8003cb0 con 0x7f80c4104600 2024-05-14T17:10:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80c17fa640 1 -- 172.21.15.69:0/1774516223 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8035070 con 0x7f80c4104600 2024-05-14T17:10:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.355+0000 7f80ca4da640 1 -- 172.21.15.69:0/1774516223 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c41a5d60 con 0x7f80c4104600 2024-05-14T17:10:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80cd1fb740 1 Processor -- start 2024-05-14T17:10:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80cd1fb740 1 -- start start 2024-05-14T17:10:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c17fa640 1 -- 172.21.15.69:0/1774516223 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f80b8042030 con 0x7f80c4104600 2024-05-14T17:10:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd098f0 0x55d0fdd09cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd0a210 0x55d0fdd14590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd16ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d0fdc7cc30 con 0x55d0fdd14ad0 2024-05-14T17:10:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d0fdc6c7d0 con 0x55d0fdd098f0 2024-05-14T17:10:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d0fdc70810 con 0x55d0fdd0a210 2024-05-14T17:10:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd16ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd16ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58966/0 (socket says 172.21.15.69:58966) 2024-05-14T17:10:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c37fe640 1 -- 172.21.15.69:0/3595918204 learned_addr learned my addr 172.21.15.69:0/3595918204 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c8a77640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd098f0 0x55d0fdd09cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c3fff640 1 --2- 172.21.15.69:0/3595918204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd0a210 0x55d0fdd14590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c37fe640 1 -- 172.21.15.69:0/3595918204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd0a210 msgr2=0x55d0fdd14590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c37fe640 1 --2- 172.21.15.69:0/3595918204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd0a210 0x55d0fdd14590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c37fe640 1 -- 172.21.15.69:0/3595918204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd098f0 msgr2=0x55d0fdd09cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c37fe640 1 --2- 172.21.15.69:0/3595918204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd098f0 0x55d0fdd09cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.358+0000 7f80c37fe640 1 -- 172.21.15.69:0/3595918204 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fd981210 con 0x55d0fdd14ad0 2024-05-14T17:10:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.359+0000 7f80c37fe640 1 --2- 172.21.15.69:0/3595918204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd16ec0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f80b8100620 tx=0x7f80b810b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5861b5d10c49e2ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.359+0000 7f80c17fa640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80ac0b2a10 0x7f80ac0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.359+0000 7f80aa7fc640 1 -- 172.21.15.69:0/3595918204 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80c6890 con 0x55d0fdd14ad0 2024-05-14T17:10:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.359+0000 7f80c8a77640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80ac0b2a10 0x7f80ac0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.359+0000 7f80aa7fc640 1 -- 172.21.15.69:0/3595918204 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b8110030 con 0x55d0fdd14ad0 2024-05-14T17:10:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.359+0000 7f80aa7fc640 1 -- 172.21.15.69:0/3595918204 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80c9b90 con 0x55d0fdd14ad0 2024-05-14T17:10:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.359+0000 7f80c17fa640 1 -- 172.21.15.69:0/1774516223 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7f80b8102f70 con 0x7f80c4104600 2024-05-14T17:10:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.359+0000 7f80c8a77640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80ac0b2a10 0x7f80ac0b4ed0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80bc011fd0 tx=0x7f80bc0119b0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.359+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3595918204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 msgr2=0x55d0fdd16ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.359+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3595918204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd16ec0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f80b8100620 tx=0x7f80b810b320 comp rx=0 tx=0).stop 2024-05-14T17:10:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3595918204 shutdown_connections 2024-05-14T17:10:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3595918204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd16ec0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3595918204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd0a210 0x55d0fdd14590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3595918204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd098f0 0x55d0fdd09cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3595918204 >> 172.21.15.69:0/3595918204 conn(0x55d0fdd05e00 msgr2=0x55d0fdd061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3595918204 shutdown_connections 2024-05-14T17:10:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3595918204 wait complete. 2024-05-14T17:10:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 -- start start 2024-05-14T17:10:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 Processor -- start 2024-05-14T17:10:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd098f0 0x55d0fdd143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd11c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d0fd952790 con 0x55d0fdd0a210 2024-05-14T17:10:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d0fdc28890 con 0x55d0fdd14ad0 2024-05-14T17:10:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d0fdd174c0 con 0x55d0fdd098f0 2024-05-14T17:10:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.360+0000 7f80cd1fb740 10 client.0 _ll_register_callbacks cb 0x55d0fdc2bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd116d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58968/0 (socket says 172.21.15.69:58968) 2024-05-14T17:10:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c37fe640 1 -- 172.21.15.69:0/3932403262 learned_addr learned my addr 172.21.15.69:0/3932403262 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c8a77640 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd11c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c3fff640 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd098f0 0x55d0fdd143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c37fe640 1 -- 172.21.15.69:0/3932403262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd098f0 msgr2=0x55d0fdd143e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c37fe640 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd098f0 0x55d0fdd143e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c37fe640 1 -- 172.21.15.69:0/3932403262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd14ad0 msgr2=0x55d0fdd11c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c37fe640 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd11c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c37fe640 1 -- 172.21.15.69:0/3932403262 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fda4d620 con 0x55d0fdd0a210 2024-05-14T17:10:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c8a77640 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd11c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80c37fe640 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd116d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f80b804b000 tx=0x7f80b8100650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b77d641e34864f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80a37fe640 1 -- 172.21.15.69:0/3932403262 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80ece60 con 0x55d0fdd0a210 2024-05-14T17:10:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80a37fe640 1 -- 172.21.15.69:0/3932403262 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7f80b80c6830 con 0x55d0fdd0a210 2024-05-14T17:10:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3932403262 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80b8110030 con 0x55d0fdd0a210 2024-05-14T17:10:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80a37fe640 1 -- 172.21.15.69:0/3932403262 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f80b80c9d80 con 0x55d0fdd0a210 2024-05-14T17:10:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80cd1fb740 10 client.17367 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3932403262 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55d0fdd22f00 con 0x55d0fdd0a210 2024-05-14T17:10:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.361+0000 7f80a37fe640 1 -- 172.21.15.69:0/3932403262 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b8110030 con 0x55d0fdd0a210 2024-05-14T17:10:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.362+0000 7f80a37fe640 1 -- 172.21.15.69:0/3932403262 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f80b8100930 con 0x55d0fdd0a210 2024-05-14T17:10:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.362+0000 7f80cd1fb740 10 client.17367 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.362+0000 7f80cd1fb740 10 client.17367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.362+0000 7f80cd1fb740 20 client.17367 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.362+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d0fdd24250 0x55d0fdd26640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.362+0000 7f80cd1fb740 4 client.17367 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.362+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3932403262 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0fdc7eb80 con 0x55d0fdd24250 2024-05-14T17:10:18.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.362+0000 7f80c3fff640 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d0fdd24250 0x55d0fdd26640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.362+0000 7f80a37fe640 10 client.17367 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.362+0000 7f80c3fff640 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d0fdd24250 0x55d0fdd26640 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ab1470c4ec822d server_cookie=b3ca0604d97d120e in_seq=0 out_seq=0 2024-05-14T17:10:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.363+0000 7f80a37fe640 1 -- 172.21.15.69:0/3932403262 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0fdc7eb80 con 0x55d0fdd24250 2024-05-14T17:10:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.363+0000 7f80a37fe640 10 client.17367 handle_command_reply: tid=0 2024-05-14T17:10:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.363+0000 7f80cd1fb740 1 client.17367 shutdown 2024-05-14T17:10:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.363+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3932403262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 msgr2=0x55d0fdd116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd116d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f80b804b000 tx=0x7f80b8100650 comp rx=0 tx=0).stop 2024-05-14T17:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3932403262 shutdown_connections 2024-05-14T17:10:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d0fdd24250 0x55d0fdd26640 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd11c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd116d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80a37fe640 0 client.17367 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3932403262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd098f0 0x55d0fdd143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3932403262 >> 172.21.15.69:0/3932403262 conn(0x55d0fdd05e00 msgr2=0x55d0fdd05100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3932403262 shutdown_connections 2024-05-14T17:10:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3932403262 wait complete. 2024-05-14T17:10:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.364+0000 7f80cd1fb740 20 client.17367 trim_cache size 0 max 16384 2024-05-14T17:10:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.378+0000 7f80cd1fb740 1 Processor -- start 2024-05-14T17:10:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.378+0000 7f80cd1fb740 1 -- start start 2024-05-14T17:10:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd098f0 0x55d0fdd203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd2be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd2c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80c41adf30 con 0x55d0fdd0a210 2024-05-14T17:10:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80c41ae600 con 0x55d0fdd098f0 2024-05-14T17:10:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80c41abf10 con 0x55d0fdd14ad0 2024-05-14T17:10:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd2be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd2be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58974/0 (socket says 172.21.15.69:58974) 2024-05-14T17:10:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80c37fe640 1 -- 172.21.15.69:0/3258792909 learned_addr learned my addr 172.21.15.69:0/3258792909 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80c3fff640 1 --2- 172.21.15.69:0/3258792909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd098f0 0x55d0fdd203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80c37fe640 1 -- 172.21.15.69:0/3258792909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd14ad0 msgr2=0x55d0fdd2c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80c37fe640 1 --2- 172.21.15.69:0/3258792909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd2c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80c37fe640 1 -- 172.21.15.69:0/3258792909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd098f0 msgr2=0x55d0fdd203c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80c37fe640 1 --2- 172.21.15.69:0/3258792909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd098f0 0x55d0fdd203c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80c37fe640 1 -- 172.21.15.69:0/3258792909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fdd0d2b0 con 0x55d0fdd0a210 2024-05-14T17:10:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80c37fe640 1 --2- 172.21.15.69:0/3258792909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd2be10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f80b80ca1d0 tx=0x7f80b810b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52afd1bbdf4f30f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80a9ffb640 1 -- 172.21.15.69:0/3258792909 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8049020 con 0x55d0fdd0a210 2024-05-14T17:10:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80a9ffb640 1 -- 172.21.15.69:0/3258792909 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b8110030 con 0x55d0fdd0a210 2024-05-14T17:10:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.379+0000 7f80a9ffb640 1 -- 172.21.15.69:0/3258792909 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80ece60 con 0x55d0fdd0a210 2024-05-14T17:10:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3258792909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 msgr2=0x55d0fdd2be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3258792909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd2be10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f80b80ca1d0 tx=0x7f80b810b320 comp rx=0 tx=0).stop 2024-05-14T17:10:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3258792909 shutdown_connections 2024-05-14T17:10:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3258792909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd2c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3258792909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd0a210 0x55d0fdd2be10 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/3258792909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd098f0 0x55d0fdd203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3258792909 >> 172.21.15.69:0/3258792909 conn(0x55d0fdd05e00 msgr2=0x55d0fdd05100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3258792909 shutdown_connections 2024-05-14T17:10:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 -- 172.21.15.69:0/3258792909 wait complete. 2024-05-14T17:10:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 -- start start 2024-05-14T17:10:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.380+0000 7f80cd1fb740 1 Processor -- start 2024-05-14T17:10:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd098f0 0x55d0fdd1cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd0a210 0x55d0fdd19f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80cd1fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd1a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80c41ab840 con 0x55d0fdd14ad0 2024-05-14T17:10:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80c41aa430 con 0x55d0fdd0a210 2024-05-14T17:10:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80cd1fb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80c41aaaa0 con 0x55d0fdd098f0 2024-05-14T17:10:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80cd1fb740 10 client.0 _ll_register_callbacks cb 0x7f80b80c99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c8a77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd1a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c8a77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd1a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58978/0 (socket says 172.21.15.69:58978) 2024-05-14T17:10:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c8a77640 1 -- 172.21.15.69:0/1942550863 learned_addr learned my addr 172.21.15.69:0/1942550863 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c3fff640 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd098f0 0x55d0fdd1cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c37fe640 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd0a210 0x55d0fdd19f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c8a77640 1 -- 172.21.15.69:0/1942550863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd098f0 msgr2=0x55d0fdd1cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c8a77640 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd098f0 0x55d0fdd1cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c8a77640 1 -- 172.21.15.69:0/1942550863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd0a210 msgr2=0x55d0fdd19f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c8a77640 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd0a210 0x55d0fdd19f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c8a77640 1 -- 172.21.15.69:0/1942550863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fdd1acd0 con 0x55d0fdd14ad0 2024-05-14T17:10:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c37fe640 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd0a210 0x55d0fdd19f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80c8a77640 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd1a4d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009cc0 tx=0x7f80bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c04262838e7c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:21.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80a37fe640 1 -- 172.21.15.69:0/1942550863 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc02e060 con 0x55d0fdd14ad0 2024-05-14T17:10:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80a37fe640 1 -- 172.21.15.69:0/1942550863 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7f80bc0293f0 con 0x55d0fdd14ad0 2024-05-14T17:10:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.381+0000 7f80cd1fb740 1 -- 172.21.15.69:0/1942550863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80b8110030 con 0x55d0fdd14ad0 2024-05-14T17:10:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80a37fe640 1 -- 172.21.15.69:0/1942550863 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f80bc011c90 con 0x55d0fdd14ad0 2024-05-14T17:10:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80cd1fb740 10 client.17373 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80cd1fb740 1 -- 172.21.15.69:0/1942550863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55d0fdd8e9b0 con 0x55d0fdd14ad0 2024-05-14T17:10:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80a37fe640 1 -- 172.21.15.69:0/1942550863 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b8110030 con 0x55d0fdd14ad0 2024-05-14T17:10:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80a37fe640 1 -- 172.21.15.69:0/1942550863 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f80bc028070 con 0x55d0fdd14ad0 2024-05-14T17:10:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80cd1fb740 10 client.17373 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80cd1fb740 10 client.17373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80cd1fb740 20 client.17373 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d0fdd8f160 0x55d0fdd91580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80cd1fb740 4 client.17373 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80cd1fb740 1 -- 172.21.15.69:0/1942550863 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0fdc78360 con 0x55d0fdd8f160 2024-05-14T17:10:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.382+0000 7f80c3fff640 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d0fdd8f160 0x55d0fdd91580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.383+0000 7f80a37fe640 10 client.17373 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.383+0000 7f80c3fff640 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d0fdd8f160 0x55d0fdd91580 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55106d7bfeef78e server_cookie=6269a7dc0371c6c9 in_seq=0 out_seq=0 2024-05-14T17:10:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80a37fe640 1 -- 172.21.15.69:0/1942550863 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d0fdc78360 con 0x55d0fdd8f160 2024-05-14T17:10:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80a37fe640 10 client.17373 handle_command_reply: tid=0 2024-05-14T17:10:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 client.17373 shutdown 2024-05-14T17:10:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 -- 172.21.15.69:0/1942550863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 msgr2=0x55d0fdd1a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd1a4d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009cc0 tx=0x7f80bc004ab0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 -- 172.21.15.69:0/1942550863 shutdown_connections 2024-05-14T17:10:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55d0fdd8f160 0x55d0fdd91580 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fdd14ad0 0x55d0fdd1a4d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80a37fe640 0 client.17373 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d0fdd0a210 0x55d0fdd19f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 --2- 172.21.15.69:0/1942550863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0fdd098f0 0x55d0fdd1cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 -- 172.21.15.69:0/1942550863 >> 172.21.15.69:0/1942550863 conn(0x55d0fdd05e00 msgr2=0x55d0fdd1f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 -- 172.21.15.69:0/1942550863 shutdown_connections 2024-05-14T17:10:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 1 -- 172.21.15.69:0/1942550863 wait complete. 2024-05-14T17:10:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.391+0000 7f80cd1fb740 20 client.17373 trim_cache size 0 max 16384 2024-05-14T17:10:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:10:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:10:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:10:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 -- 172.21.15.69:0/1774516223 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80ac0b2a10 msgr2=0x7f80ac0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80ac0b2a10 0x7f80ac0b4ed0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80bc011fd0 tx=0x7f80bc0119b0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 -- 172.21.15.69:0/1774516223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4104600 msgr2=0x7f80c4199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4104600 0x7f80c4199730 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005940 tx=0x7f80b80042f0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 -- 172.21.15.69:0/1774516223 shutdown_connections 2024-05-14T17:10:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f80ac0b2a10 0x7f80ac0b4ed0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80c410d020 0x7f80c419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4104600 0x7f80c4199730 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 --2- 172.21.15.69:0/1774516223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c4103ce0 0x7f80c41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 -- 172.21.15.69:0/1774516223 >> 172.21.15.69:0/1774516223 conn(0x7f80c4078ad0 msgr2=0x7f80c40fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 -- 172.21.15.69:0/1774516223 shutdown_connections 2024-05-14T17:10:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:21.392+0000 7f80ca4da640 1 -- 172.21.15.69:0/1774516223 wait complete. 2024-05-14T17:10:23.117 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:22 smithi069 ceph-mon[31507]: pgmap v617: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-14T17:10:23.149 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:24 smithi069 ceph-mon[31507]: pgmap v618: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-05-14T17:10:23.183 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:26 smithi069 ceph-mon[31507]: pgmap v619: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9.5 KiB/s wr, 5 op/s 2024-05-14T17:10:23.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:26 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:10:23.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:27 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:10:23.291 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:27 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:10:23.334 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:27 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:23.366 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:28 smithi069 ceph-mon[31507]: pgmap v620: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 7 op/s 2024-05-14T17:10:23.416 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:30 smithi069 ceph-mon[31507]: pgmap v621: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2024-05-14T17:10:23.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:32 smithi069 ceph-mon[31507]: pgmap v622: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2024-05-14T17:10:23.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:14 smithi092 ceph-mon[35763]: pgmap v644: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:10:23.516 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:14 smithi092 ceph-mon[35763]: osdmap e114: 12 total, 12 up, 12 in 2024-05-14T17:10:23.549 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:16 smithi092 ceph-mon[35763]: pgmap v646: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2024-05-14T17:10:23.598 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:18 smithi092 ceph-mon[35763]: pgmap v647: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-05-14T17:10:23.633 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:20 smithi092 ceph-mon[35763]: pgmap v648: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-05-14T17:10:23.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:22 smithi092 ceph-mon[35763]: pgmap v649: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-05-14T17:10:23.700 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:14 smithi203 ceph-mon[35787]: pgmap v644: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:10:23.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:14 smithi203 ceph-mon[35787]: osdmap e114: 12 total, 12 up, 12 in 2024-05-14T17:10:23.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:16 smithi203 ceph-mon[35787]: pgmap v646: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2024-05-14T17:10:23.800 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:18 smithi203 ceph-mon[35787]: pgmap v647: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-05-14T17:10:23.833 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:20 smithi203 ceph-mon[35787]: pgmap v648: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-05-14T17:10:23.866 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:22 smithi203 ceph-mon[35787]: pgmap v649: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-05-14T17:10:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.489+0000 7fa1e1072640 1 -- 172.21.15.69:0/47188961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc102310 msgr2=0x7fa1dc10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.489+0000 7fa1e1072640 1 --2- 172.21.15.69:0/47188961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc102310 0x7fa1dc10e820 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4009cd0 tx=0x7fa1d402a8e0 comp rx=0 tx=0).stop 2024-05-14T17:10:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.489+0000 7fa1e1072640 1 -- 172.21.15.69:0/47188961 shutdown_connections 2024-05-14T17:10:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.489+0000 7fa1e1072640 1 --2- 172.21.15.69:0/47188961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc102310 0x7fa1dc10e820 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.489+0000 7fa1e1072640 1 --2- 172.21.15.69:0/47188961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1dc101950 0x7fa1dc101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.489+0000 7fa1e1072640 1 --2- 172.21.15.69:0/47188961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1dc107960 0x7fa1dc107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.489+0000 7fa1e1072640 1 -- 172.21.15.69:0/47188961 >> 172.21.15.69:0/47188961 conn(0x7fa1dc078ad0 msgr2=0x7fa1dc078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.489+0000 7fa1e1072640 1 -- 172.21.15.69:0/47188961 shutdown_connections 2024-05-14T17:10:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.489+0000 7fa1e1072640 1 -- 172.21.15.69:0/47188961 wait complete. 2024-05-14T17:10:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.490+0000 7fa1e1072640 1 Processor -- start 2024-05-14T17:10:24.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.490+0000 7fa1e1072640 1 -- start start 2024-05-14T17:10:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.490+0000 7fa1e1072640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1dc101950 0x7fa1dc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.490+0000 7fa1e1072640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1dc102310 0x7fa1dc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.490+0000 7fa1e1072640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc107960 0x7fa1dc19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.490+0000 7fa1e1072640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc111bf0 con 0x7fa1dc107960 2024-05-14T17:10:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.490+0000 7fa1e1072640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc111a70 con 0x7fa1dc102310 2024-05-14T17:10:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.490+0000 7fa1e1072640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc111d70 con 0x7fa1dc101950 2024-05-14T17:10:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.490+0000 7fa1db59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc107960 0x7fa1dc19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1db59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc107960 0x7fa1dc19c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54258/0 (socket says 172.21.15.69:54258) 2024-05-14T17:10:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1db59e640 1 -- 172.21.15.69:0/1738388028 learned_addr learned my addr 172.21.15.69:0/1738388028 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1da59c640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1dc102310 0x7fa1dc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1dc101950 0x7fa1dc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1db59e640 1 -- 172.21.15.69:0/1738388028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1dc101950 msgr2=0x7fa1dc197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1db59e640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1dc101950 0x7fa1dc197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1db59e640 1 -- 172.21.15.69:0/1738388028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1dc102310 msgr2=0x7fa1dc198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1db59e640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1dc102310 0x7fa1dc198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1db59e640 1 -- 172.21.15.69:0/1738388028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1dc19cfc0 con 0x7fa1dc107960 2024-05-14T17:10:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1db59e640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc107960 0x7fa1dc19c890 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4005cc0 tx=0x7fa1d4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1d3fff640 1 -- 172.21.15.69:0/1738388028 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d404a070 con 0x7fa1dc107960 2024-05-14T17:10:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1e1072640 1 -- 172.21.15.69:0/1738388028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1dc19d250 con 0x7fa1dc107960 2024-05-14T17:10:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1d3fff640 1 -- 172.21.15.69:0/1738388028 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1d4031c90 con 0x7fa1dc107960 2024-05-14T17:10:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1d3fff640 1 -- 172.21.15.69:0/1738388028 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40307e0 con 0x7fa1dc107960 2024-05-14T17:10:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.491+0000 7fa1e1072640 1 -- 172.21.15.69:0/1738388028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1dc06b2a0 con 0x7fa1dc107960 2024-05-14T17:10:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.494+0000 7fa1d3fff640 1 -- 172.21.15.69:0/1738388028 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa1d40309e0 con 0x7fa1dc107960 2024-05-14T17:10:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.494+0000 7fa1e3d8c740 1 Processor -- start 2024-05-14T17:10:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.494+0000 7fa1e3d8c740 1 -- start start 2024-05-14T17:10:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.494+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9db7f0 0x562aff9dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.494+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9eaee0 0x562aff9ed2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562aff94ec30 con 0x562aff9dc110 2024-05-14T17:10:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562aff93e7d0 con 0x562aff9eaee0 2024-05-14T17:10:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562aff942810 con 0x562aff9db7f0 2024-05-14T17:10:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1d3fff640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa1c80b2a10 0x7fa1c80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1dad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1dad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54260/0 (socket says 172.21.15.69:54260) 2024-05-14T17:10:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1dad9d640 1 -- 172.21.15.69:0/1456914119 learned_addr learned my addr 172.21.15.69:0/1456914119 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1da59c640 1 --2- 172.21.15.69:0/1456914119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9eaee0 0x562aff9ed2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1db59e640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa1c80b2a10 0x7fa1c80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1da59c640 1 --2- 172.21.15.69:0/1456914119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9db7f0 0x562aff9dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1dad9d640 1 -- 172.21.15.69:0/1456914119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9db7f0 msgr2=0x562aff9dbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/1456914119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9db7f0 0x562aff9dbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1dad9d640 1 -- 172.21.15.69:0/1456914119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9eaee0 msgr2=0x562aff9ed2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/1456914119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9eaee0 0x562aff9ed2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1dad9d640 1 -- 172.21.15.69:0/1456914119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aff653210 con 0x562aff9dc110 2024-05-14T17:10:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1d3fff640 1 -- 172.21.15.69:0/1738388028 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7fa1d402e020 con 0x7fa1dc107960 2024-05-14T17:10:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/1456914119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9ea9a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00ac90 tx=0x7fa1cc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f736701a5143bae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1db59e640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa1c80b2a10 0x7fa1c80b4ed0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40cb9b0 tx=0x7fa1d4099880 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1d17fa640 1 -- 172.21.15.69:0/1456914119 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc01d070 con 0x562aff9dc110 2024-05-14T17:10:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1d17fa640 1 -- 172.21.15.69:0/1456914119 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1cc018af0 con 0x562aff9dc110 2024-05-14T17:10:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.495+0000 7fa1d17fa640 1 -- 172.21.15.69:0/1456914119 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc010a50 con 0x562aff9dc110 2024-05-14T17:10:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.496+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/1456914119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 msgr2=0x562aff9ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.496+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/1456914119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9ea9a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00ac90 tx=0x7fa1cc00f790 comp rx=0 tx=0).stop 2024-05-14T17:10:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.496+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/1456914119 shutdown_connections 2024-05-14T17:10:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.496+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/1456914119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9eaee0 0x562aff9ed2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.496+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/1456914119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9ea9a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.496+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/1456914119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9db7f0 0x562aff9dbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.496+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/1456914119 >> 172.21.15.69:0/1456914119 conn(0x562aff9d7e00 msgr2=0x562aff9d81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.496+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/1456914119 shutdown_connections 2024-05-14T17:10:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.496+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/1456914119 wait complete. 2024-05-14T17:10:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1e3d8c740 1 -- start start 2024-05-14T17:10:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1e3d8c740 1 Processor -- start 2024-05-14T17:10:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 0x562aff9ea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9dc110 0x562aff9e7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9eaee0 0x562aff9e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562aff624790 con 0x562aff9eaee0 2024-05-14T17:10:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562aff8fa890 con 0x562aff9db7f0 2024-05-14T17:10:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562aff9edbb0 con 0x562aff9dc110 2024-05-14T17:10:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1e3d8c740 10 client.0 _ll_register_callbacks cb 0x562aff8fdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9eaee0 0x562aff9e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9eaee0 0x562aff9e7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54264/0 (socket says 172.21.15.69:54264) 2024-05-14T17:10:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1da59c640 1 -- 172.21.15.69:0/3691553330 learned_addr learned my addr 172.21.15.69:0/3691553330 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1da59c640 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 0x562aff9ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9dc110 0x562aff9e7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1da59c640 1 -- 172.21.15.69:0/3691553330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9dc110 msgr2=0x562aff9e7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1da59c640 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9dc110 0x562aff9e7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1da59c640 1 -- 172.21.15.69:0/3691553330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 msgr2=0x562aff9ea790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1da59c640 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 0x562aff9ea790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.497+0000 7fa1da59c640 1 -- 172.21.15.69:0/3691553330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aff71f620 con 0x562aff9eaee0 2024-05-14T17:10:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1da59c640 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9eaee0 0x562aff9e7fc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc078f60 tx=0x7fa1c4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54713a9fd9bc2f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1c27fc640 1 -- 172.21.15.69:0/3691553330 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c40128f0 con 0x562aff9eaee0 2024-05-14T17:10:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1c27fc640 1 -- 172.21.15.69:0/3691553330 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7fa1c4012a90 con 0x562aff9eaee0 2024-05-14T17:10:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/3691553330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1cc018af0 con 0x562aff9eaee0 2024-05-14T17:10:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1c27fc640 1 -- 172.21.15.69:0/3691553330 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa1c4012c90 con 0x562aff9eaee0 2024-05-14T17:10:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1e3d8c740 10 client.17385 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/3691553330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562aff9f9630 con 0x562aff9eaee0 2024-05-14T17:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1c27fc640 1 -- 172.21.15.69:0/3691553330 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1cc018af0 con 0x562aff9eaee0 2024-05-14T17:10:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1c27fc640 1 -- 172.21.15.69:0/3691553330 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fa1c400f3f0 con 0x562aff9eaee0 2024-05-14T17:10:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1e3d8c740 10 client.17385 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1e3d8c740 10 client.17385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1e3d8c740 20 client.17385 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562aff9fab40 0x562aff9fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1e3d8c740 4 client.17385 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.498+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/3691553330 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aff950b80 con 0x562aff9fab40 2024-05-14T17:10:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.499+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562aff9fab40 0x562aff9fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.499+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562aff9fab40 0x562aff9fcfb0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f73e13c63c172e7 server_cookie=a2c2bf6fb44418df in_seq=0 out_seq=0 2024-05-14T17:10:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.499+0000 7fa1c27fc640 10 client.17385 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1c27fc640 1 -- 172.21.15.69:0/3691553330 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562aff950b80 con 0x562aff9fab40 2024-05-14T17:10:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1c27fc640 10 client.17385 handle_command_reply: tid=0 2024-05-14T17:10:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 client.17385 shutdown 2024-05-14T17:10:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/3691553330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9eaee0 msgr2=0x562aff9e7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9eaee0 0x562aff9e7fc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc078f60 tx=0x7fa1c4014040 comp rx=0 tx=0).stop 2024-05-14T17:10:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/3691553330 shutdown_connections 2024-05-14T17:10:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562aff9fab40 0x562aff9fcfb0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9eaee0 0x562aff9e7fc0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9dc110 0x562aff9e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1c27fc640 0 client.17385 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/3691553330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 0x562aff9ea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/3691553330 >> 172.21.15.69:0/3691553330 conn(0x562aff9d7e00 msgr2=0x562aff9ddc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/3691553330 shutdown_connections 2024-05-14T17:10:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/3691553330 wait complete. 2024-05-14T17:10:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.500+0000 7fa1e3d8c740 20 client.17385 trim_cache size 0 max 16384 2024-05-14T17:10:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.515+0000 7fa1e3d8c740 1 Processor -- start 2024-05-14T17:10:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.515+0000 7fa1e3d8c740 1 -- start start 2024-05-14T17:10:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9db7f0 0x562aff9ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9dc110 0x562aff9d6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9eaee0 0x562affa02880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc1ade90 con 0x562aff9db7f0 2024-05-14T17:10:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc1ae560 con 0x562aff9dc110 2024-05-14T17:10:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc1abe70 con 0x562aff9eaee0 2024-05-14T17:10:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9db7f0 0x562aff9ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9db7f0 0x562aff9ea2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54268/0 (socket says 172.21.15.69:54268) 2024-05-14T17:10:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1da59c640 1 -- 172.21.15.69:0/2106075849 learned_addr learned my addr 172.21.15.69:0/2106075849 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/2106075849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9dc110 0x562aff9d6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1da59c640 1 -- 172.21.15.69:0/2106075849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9eaee0 msgr2=0x562affa02880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1da59c640 1 --2- 172.21.15.69:0/2106075849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9eaee0 0x562affa02880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1da59c640 1 -- 172.21.15.69:0/2106075849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9dc110 msgr2=0x562aff9d6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1da59c640 1 --2- 172.21.15.69:0/2106075849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9dc110 0x562aff9d6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1da59c640 1 -- 172.21.15.69:0/2106075849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aff9d7c30 con 0x562aff9db7f0 2024-05-14T17:10:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1da59c640 1 --2- 172.21.15.69:0/2106075849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9db7f0 0x562aff9ea2c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c4005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead0d064bb06816 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1d0ff9640 1 -- 172.21.15.69:0/2106075849 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c402e070 con 0x562aff9db7f0 2024-05-14T17:10:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1d0ff9640 1 -- 172.21.15.69:0/2106075849 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c400b390 con 0x562aff9db7f0 2024-05-14T17:10:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.516+0000 7fa1d0ff9640 1 -- 172.21.15.69:0/2106075849 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400fa30 con 0x562aff9db7f0 2024-05-14T17:10:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.517+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2106075849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9db7f0 msgr2=0x562aff9ea2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.517+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/2106075849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9db7f0 0x562aff9ea2c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c4005900 comp rx=0 tx=0).stop 2024-05-14T17:10:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.517+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2106075849 shutdown_connections 2024-05-14T17:10:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.517+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/2106075849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9eaee0 0x562affa02880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.517+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/2106075849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9dc110 0x562aff9d6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.517+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/2106075849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9db7f0 0x562aff9ea2c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.517+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2106075849 >> 172.21.15.69:0/2106075849 conn(0x562aff9d7e00 msgr2=0x562aff9d6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.517+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2106075849 shutdown_connections 2024-05-14T17:10:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.517+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2106075849 wait complete. 2024-05-14T17:10:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1e3d8c740 1 -- start start 2024-05-14T17:10:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1e3d8c740 1 Processor -- start 2024-05-14T17:10:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 0x562aff9f3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9e4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1e3d8c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9eaee0 0x562aff9e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc1ab7a0 con 0x562aff9dc110 2024-05-14T17:10:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc1aa3b0 con 0x562aff9db7f0 2024-05-14T17:10:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1e3d8c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc1aaa00 con 0x562aff9eaee0 2024-05-14T17:10:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1e3d8c740 10 client.0 _ll_register_callbacks cb 0x7fa1c4004250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1dad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9e4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1dad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9e4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54276/0 (socket says 172.21.15.69:54276) 2024-05-14T17:10:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1dad9d640 1 -- 172.21.15.69:0/2446337367 learned_addr learned my addr 172.21.15.69:0/2446337367 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1da59c640 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 0x562aff9f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1da59c640 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9eaee0 0x562aff9e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1dad9d640 1 -- 172.21.15.69:0/2446337367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9eaee0 msgr2=0x562aff9e4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9eaee0 0x562aff9e4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1dad9d640 1 -- 172.21.15.69:0/2446337367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 msgr2=0x562aff9f3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 0x562aff9f3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1dad9d640 1 -- 172.21.15.69:0/2446337367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aff9dc650 con 0x562aff9dc110 2024-05-14T17:10:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1da59c640 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 0x562aff9f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1dad9d640 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9e4710 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00ac60 tx=0x7fa1cc0106f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e265c7618692766 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.518+0000 7fa1c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1c27fc640 1 -- 172.21.15.69:0/2446337367 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc02c030 con 0x562aff9dc110 2024-05-14T17:10:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1c27fc640 1 -- 172.21.15.69:0/2446337367 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7fa1cc005040 con 0x562aff9dc110 2024-05-14T17:10:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2446337367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c400b390 con 0x562aff9dc110 2024-05-14T17:10:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1c27fc640 1 -- 172.21.15.69:0/2446337367 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa1cc01d070 con 0x562aff9dc110 2024-05-14T17:10:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1e3d8c740 10 client.17391 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2446337367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562affa65380 con 0x562aff9dc110 2024-05-14T17:10:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1c27fc640 1 -- 172.21.15.69:0/2446337367 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c400b390 con 0x562aff9dc110 2024-05-14T17:10:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1c27fc640 1 -- 172.21.15.69:0/2446337367 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fa1cc010760 con 0x562aff9dc110 2024-05-14T17:10:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1e3d8c740 10 client.17391 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1e3d8c740 10 client.17391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1e3d8c740 20 client.17391 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562affa659a0 0x562affa67dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1e3d8c740 4 client.17391 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2446337367 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562aff94a360 con 0x562affa659a0 2024-05-14T17:10:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.519+0000 7fa1da59c640 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562affa659a0 0x562affa67dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.520+0000 7fa1c27fc640 10 client.17391 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.520+0000 7fa1da59c640 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562affa659a0 0x562affa67dc0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b08897e8ed3ae69f server_cookie=f64200aef5718e52 in_seq=0 out_seq=0 2024-05-14T17:10:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1c27fc640 1 -- 172.21.15.69:0/2446337367 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562aff94a360 con 0x562affa659a0 2024-05-14T17:10:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1c27fc640 10 client.17391 handle_command_reply: tid=0 2024-05-14T17:10:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 client.17391 shutdown 2024-05-14T17:10:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2446337367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 msgr2=0x562aff9e4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9e4710 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00ac60 tx=0x7fa1cc0106f0 comp rx=0 tx=0).stop 2024-05-14T17:10:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2446337367 shutdown_connections 2024-05-14T17:10:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562affa659a0 0x562affa67dc0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562aff9eaee0 0x562aff9e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aff9dc110 0x562aff9e4710 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1c27fc640 0 client.17391 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 --2- 172.21.15.69:0/2446337367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aff9db7f0 0x562aff9f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2446337367 >> 172.21.15.69:0/2446337367 conn(0x562aff9d7e00 msgr2=0x562aff9ddbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2446337367 shutdown_connections 2024-05-14T17:10:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.525+0000 7fa1e3d8c740 1 -- 172.21.15.69:0/2446337367 wait complete. 2024-05-14T17:10:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e3d8c740 20 client.17391 trim_cache size 0 max 16384 2024-05-14T17:10:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:10:31.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:10:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:10:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 -- 172.21.15.69:0/1738388028 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa1c80b2a10 msgr2=0x7fa1c80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa1c80b2a10 0x7fa1c80b4ed0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40cb9b0 tx=0x7fa1d4099880 comp rx=0 tx=0).stop 2024-05-14T17:10:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 -- 172.21.15.69:0/1738388028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc107960 msgr2=0x7fa1dc19c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc107960 0x7fa1dc19c890 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4005cc0 tx=0x7fa1d4003ba0 comp rx=0 tx=0).stop 2024-05-14T17:10:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 -- 172.21.15.69:0/1738388028 shutdown_connections 2024-05-14T17:10:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa1c80b2a10 0x7fa1c80b4ed0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc107960 0x7fa1dc19c890 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1dc102310 0x7fa1dc198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 --2- 172.21.15.69:0/1738388028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1dc101950 0x7fa1dc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 -- 172.21.15.69:0/1738388028 >> 172.21.15.69:0/1738388028 conn(0x7fa1dc078ad0 msgr2=0x7fa1dc1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 -- 172.21.15.69:0/1738388028 shutdown_connections 2024-05-14T17:10:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:34.526+0000 7fa1e1072640 1 -- 172.21.15.69:0/1738388028 wait complete. 2024-05-14T17:10:32.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:34 smithi069 ceph-mon[31507]: pgmap v623: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-14T17:10:32.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:36 smithi069 ceph-mon[31507]: pgmap v624: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-14T17:10:32.505 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:38 smithi069 ceph-mon[31507]: pgmap v625: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 KiB/s wr, 8 op/s 2024-05-14T17:10:32.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:40 smithi069 ceph-mon[31507]: pgmap v626: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 6 op/s 2024-05-14T17:10:32.572 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:42 smithi069 ceph-mon[31507]: pgmap v627: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 6 op/s 2024-05-14T17:10:32.639 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:44 smithi069 ceph-mon[31507]: pgmap v628: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 7 op/s 2024-05-14T17:10:32.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:46 smithi069 ceph-mon[31507]: pgmap v629: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 5 op/s 2024-05-14T17:10:32.707 INFO:tasks.quiescer.fs.[cephfs]:Total duration (36.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-14T17:10:32.755 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.4 2024-05-14T17:10:32.789 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:10:32.825 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:24 smithi092 ceph-mon[35763]: pgmap v650: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-05-14T17:10:32.856 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:26 smithi092 ceph-mon[35763]: pgmap v651: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-14T17:10:32.889 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:27 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:10:32.922 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:27 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:32.956 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:27 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:33.008 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:27 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:33.047 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:27 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:33.081 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:28 smithi092 ceph-mon[35763]: pgmap v652: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-05-14T17:10:33.114 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:33.147 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:33.182 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:33.239 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:33.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: pgmap v653: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-14T17:10:33.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:33.339 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:33.410 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.448 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.481 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.515 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.548 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-05-14T17:10:33.581 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-05-14T17:10:33.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.648 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.681 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.715 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.748 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.840 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.873 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:33.906 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:10:33.961 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:10:34.003 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.063 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:30 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-14T17:10:34.098 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:30 smithi092 ceph-mon[35763]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-14T17:10:34.132 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:30 smithi092 ceph-mon[35763]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-14T17:10:34.165 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:30 smithi092 ceph-mon[35763]: Unable to set osd_memory_target on smithi069 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-14T17:10:34.198 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:30 smithi092 ceph-mon[35763]: pgmap v654: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:10:34.232 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:30 smithi092 ceph-mon[35763]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-14T17:10:34.265 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:24 smithi203 ceph-mon[35787]: pgmap v650: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-05-14T17:10:34.303 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:26 smithi203 ceph-mon[35787]: pgmap v651: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-14T17:10:34.340 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:27 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:10:34.373 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:27 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.407 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:27 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.465 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:27 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:27 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.532 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:28 smithi203 ceph-mon[35787]: pgmap v652: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-05-14T17:10:34.565 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.599 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.799 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: pgmap v653: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-14T17:10:34.832 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.865 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:34.903 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:34.941 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:34.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:35.027 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:35.066 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-05-14T17:10:35.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-05-14T17:10:35.157 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:35.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:35.258 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:35.291 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:35.324 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:35.358 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:35.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:35.424 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:10:35.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:10:35.508 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:10:35.541 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:10:35.579 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:30 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-14T17:10:35.624 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:30 smithi203 ceph-mon[35787]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-14T17:10:35.658 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:30 smithi203 ceph-mon[35787]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-14T17:10:35.710 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:30 smithi203 ceph-mon[35787]: Unable to set osd_memory_target on smithi069 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-14T17:10:35.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:30 smithi203 ceph-mon[35787]: pgmap v654: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:10:35.800 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:30 smithi203 ceph-mon[35787]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-14T17:10:35.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:32 smithi092 ceph-mon[35763]: pgmap v655: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T17:10:35.867 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:34 smithi092 ceph-mon[35763]: pgmap v656: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:10:35.900 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:32 smithi203 ceph-mon[35787]: pgmap v655: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T17:10:35.933 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:34 smithi203 ceph-mon[35787]: pgmap v656: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:10:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.621+0000 7fe72916a640 1 -- 172.21.15.69:0/2571399735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe724103410 msgr2=0x7fe72410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.621+0000 7fe72916a640 1 --2- 172.21.15.69:0/2571399735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe724103410 0x7fe72410f920 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009930 tx=0x7fe71c02a420 comp rx=0 tx=0).stop 2024-05-14T17:10:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.621+0000 7fe72916a640 1 -- 172.21.15.69:0/2571399735 shutdown_connections 2024-05-14T17:10:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.621+0000 7fe72916a640 1 --2- 172.21.15.69:0/2571399735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe724103410 0x7fe72410f920 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.621+0000 7fe72916a640 1 --2- 172.21.15.69:0/2571399735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe724102a50 0x7fe724102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.621+0000 7fe72916a640 1 --2- 172.21.15.69:0/2571399735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724108a60 0x7fe724108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.621+0000 7fe72916a640 1 -- 172.21.15.69:0/2571399735 >> 172.21.15.69:0/2571399735 conn(0x7fe7240fe520 msgr2=0x7fe724100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.622+0000 7fe72916a640 1 -- 172.21.15.69:0/2571399735 shutdown_connections 2024-05-14T17:10:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.622+0000 7fe72916a640 1 -- 172.21.15.69:0/2571399735 wait complete. 2024-05-14T17:10:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.622+0000 7fe72916a640 1 Processor -- start 2024-05-14T17:10:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe72916a640 1 -- start start 2024-05-14T17:10:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe72916a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe724102a50 0x7fe724074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe72916a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724103410 0x7fe724075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe72916a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe724108a60 0x7fe7240757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe72916a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe724112d20 con 0x7fe724102a50 2024-05-14T17:10:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe72916a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe724112ba0 con 0x7fe724103410 2024-05-14T17:10:36.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe72916a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe724112ea0 con 0x7fe724108a60 2024-05-14T17:10:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe722d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe724102a50 0x7fe724074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe722d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe724102a50 0x7fe724074d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35724/0 (socket says 172.21.15.69:35724) 2024-05-14T17:10:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe722d9d640 1 -- 172.21.15.69:0/1505052833 learned_addr learned my addr 172.21.15.69:0/1505052833 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe72359e640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe724108a60 0x7fe7240757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe72259c640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724103410 0x7fe724075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe722d9d640 1 -- 172.21.15.69:0/1505052833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe724108a60 msgr2=0x7fe7240757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe722d9d640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe724108a60 0x7fe7240757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe722d9d640 1 -- 172.21.15.69:0/1505052833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724103410 msgr2=0x7fe724075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe722d9d640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724103410 0x7fe724075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe722d9d640 1 -- 172.21.15.69:0/1505052833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7241a7660 con 0x7fe724102a50 2024-05-14T17:10:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe72359e640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe724108a60 0x7fe7240757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.623+0000 7fe722d9d640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe724102a50 0x7fe724074d50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe71400ebc0 tx=0x7fe71400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.624+0000 7fe70bfff640 1 -- 172.21.15.69:0/1505052833 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714010630 con 0x7fe724102a50 2024-05-14T17:10:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.624+0000 7fe72916a640 1 -- 172.21.15.69:0/1505052833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7241a7950 con 0x7fe724102a50 2024-05-14T17:10:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.624+0000 7fe70bfff640 1 -- 172.21.15.69:0/1505052833 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe714010c80 con 0x7fe724102a50 2024-05-14T17:10:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.624+0000 7fe70bfff640 1 -- 172.21.15.69:0/1505052833 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714018b60 con 0x7fe724102a50 2024-05-14T17:10:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.624+0000 7fe72916a640 1 -- 172.21.15.69:0/1505052833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7241a7ef0 con 0x7fe724102a50 2024-05-14T17:10:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.626+0000 7fe70bfff640 1 -- 172.21.15.69:0/1505052833 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe714018d00 con 0x7fe724102a50 2024-05-14T17:10:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.626+0000 7fe72be81740 1 Processor -- start 2024-05-14T17:10:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.626+0000 7fe72be81740 1 -- start start 2024-05-14T17:10:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72be81740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34ef5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72be81740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef6210 0x55fd34f00590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72be81740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34f02ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72be81740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fd34e68c30 con 0x55fd34ef58f0 2024-05-14T17:10:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72be81740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fd34e587d0 con 0x55fd34ef6210 2024-05-14T17:10:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72be81740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fd34e5c810 con 0x55fd34f00ad0 2024-05-14T17:10:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72359e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34ef5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72359e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34ef5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35732/0 (socket says 172.21.15.69:35732) 2024-05-14T17:10:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72359e640 1 -- 172.21.15.69:0/3986932103 learned_addr learned my addr 172.21.15.69:0/3986932103 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe70bfff640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe70c0b2a10 0x7fe70c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72259c640 1 --2- 172.21.15.69:0/3986932103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34f02ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72359e640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe70c0b2a10 0x7fe70c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe70bfff640 1 -- 172.21.15.69:0/1505052833 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7fe7140e2cf0 con 0x7fe724102a50 2024-05-14T17:10:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe722d9d640 1 --2- 172.21.15.69:0/3986932103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef6210 0x55fd34f00590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72359e640 1 -- 172.21.15.69:0/3986932103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 msgr2=0x55fd34f02ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72359e640 1 --2- 172.21.15.69:0/3986932103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34f02ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72359e640 1 -- 172.21.15.69:0/3986932103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef6210 msgr2=0x55fd34f00590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72359e640 1 --2- 172.21.15.69:0/3986932103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef6210 0x55fd34f00590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.627+0000 7fe72359e640 1 -- 172.21.15.69:0/3986932103 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd34b6d210 con 0x55fd34ef58f0 2024-05-14T17:10:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe72359e640 1 --2- 172.21.15.69:0/3986932103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34ef5cd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009230 tx=0x7fe71c031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a7af12a911d644 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe7097fa640 1 -- 172.21.15.69:0/3986932103 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c035070 con 0x55fd34ef58f0 2024-05-14T17:10:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe7097fa640 1 -- 172.21.15.69:0/3986932103 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe71c0044c0 con 0x55fd34ef58f0 2024-05-14T17:10:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe7097fa640 1 -- 172.21.15.69:0/3986932103 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c004d70 con 0x55fd34ef58f0 2024-05-14T17:10:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe72be81740 1 -- 172.21.15.69:0/3986932103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 msgr2=0x55fd34ef5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe72359e640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe70c0b2a10 0x7fe70c0b4ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe71c04a000 tx=0x7fe71c048000 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe72be81740 1 --2- 172.21.15.69:0/3986932103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34ef5cd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009230 tx=0x7fe71c031730 comp rx=0 tx=0).stop 2024-05-14T17:10:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe72be81740 1 -- 172.21.15.69:0/3986932103 shutdown_connections 2024-05-14T17:10:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe72be81740 1 --2- 172.21.15.69:0/3986932103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34f02ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe72be81740 1 --2- 172.21.15.69:0/3986932103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef6210 0x55fd34f00590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.628+0000 7fe72be81740 1 --2- 172.21.15.69:0/3986932103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34ef5cd0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 -- 172.21.15.69:0/3986932103 >> 172.21.15.69:0/3986932103 conn(0x55fd34ef1e00 msgr2=0x55fd34ef21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 -- 172.21.15.69:0/3986932103 shutdown_connections 2024-05-14T17:10:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 -- 172.21.15.69:0/3986932103 wait complete. 2024-05-14T17:10:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 -- start start 2024-05-14T17:10:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 Processor -- start 2024-05-14T17:10:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f004c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef6210 0x55fd34efd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34efddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fd34b3e790 con 0x55fd34ef58f0 2024-05-14T17:10:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fd34e14890 con 0x55fd34ef6210 2024-05-14T17:10:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fd34f037a0 con 0x55fd34f00ad0 2024-05-14T17:10:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72be81740 10 client.0 _ll_register_callbacks cb 0x55fd34e17d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe722d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe722d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f004c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35742/0 (socket says 172.21.15.69:35742) 2024-05-14T17:10:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe722d9d640 1 -- 172.21.15.69:0/4177667137 learned_addr learned my addr 172.21.15.69:0/4177667137 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72259c640 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef6210 0x55fd34efd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.629+0000 7fe72359e640 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34efddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe722d9d640 1 -- 172.21.15.69:0/4177667137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 msgr2=0x55fd34efddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe722d9d640 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34efddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe722d9d640 1 -- 172.21.15.69:0/4177667137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef6210 msgr2=0x55fd34efd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe722d9d640 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef6210 0x55fd34efd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe722d9d640 1 -- 172.21.15.69:0/4177667137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd34c39620 con 0x55fd34ef58f0 2024-05-14T17:10:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe72359e640 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34efddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe722d9d640 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f004c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe7140e0490 tx=0x7fe7140aa5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4405082abca36827 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe706ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe706ff5640 1 -- 172.21.15.69:0/4177667137 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140c4060 con 0x55fd34ef58f0 2024-05-14T17:10:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe706ff5640 1 -- 172.21.15.69:0/4177667137 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7fe7140e0d90 con 0x55fd34ef58f0 2024-05-14T17:10:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe72be81740 1 -- 172.21.15.69:0/4177667137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fd34e6ab80 con 0x55fd34ef58f0 2024-05-14T17:10:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe706ff5640 1 -- 172.21.15.69:0/4177667137 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe7140a9730 con 0x55fd34ef58f0 2024-05-14T17:10:39.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe72be81740 10 client.17403 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe72be81740 1 -- 172.21.15.69:0/4177667137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55fd34f0f210 con 0x55fd34ef58f0 2024-05-14T17:10:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe706ff5640 1 -- 172.21.15.69:0/4177667137 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fd34e6ab80 con 0x55fd34ef58f0 2024-05-14T17:10:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe706ff5640 1 -- 172.21.15.69:0/4177667137 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fe7140ac030 con 0x55fd34ef58f0 2024-05-14T17:10:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe72be81740 10 client.17403 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe72be81740 10 client.17403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe72be81740 20 client.17403 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe72be81740 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55fd34f106d0 0x55fd34f12ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe72be81740 4 client.17403 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.630+0000 7fe72be81740 1 -- 172.21.15.69:0/4177667137 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd34e64360 con 0x55fd34f106d0 2024-05-14T17:10:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.631+0000 7fe72259c640 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55fd34f106d0 0x55fd34f12ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.631+0000 7fe706ff5640 10 client.17403 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.631+0000 7fe72259c640 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55fd34f106d0 0x55fd34f12ac0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a890c946d0dbfbf9 server_cookie=1863e7022c19f97 in_seq=0 out_seq=0 2024-05-14T17:10:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe706ff5640 1 -- 172.21.15.69:0/4177667137 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fd34e64360 con 0x55fd34f106d0 2024-05-14T17:10:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe706ff5640 10 client.17403 handle_command_reply: tid=0 2024-05-14T17:10:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 client.17403 shutdown 2024-05-14T17:10:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 -- 172.21.15.69:0/4177667137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 msgr2=0x55fd34f004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f004c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe7140e0490 tx=0x7fe7140aa5e0 comp rx=0 tx=0).stop 2024-05-14T17:10:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 -- 172.21.15.69:0/4177667137 shutdown_connections 2024-05-14T17:10:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55fd34f106d0 0x55fd34f12ac0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34efddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef6210 0x55fd34efd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe706ff5640 0 client.17403 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 --2- 172.21.15.69:0/4177667137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f004c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:41.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 -- 172.21.15.69:0/4177667137 >> 172.21.15.69:0/4177667137 conn(0x55fd34ef1e00 msgr2=0x55fd34ef1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 -- 172.21.15.69:0/4177667137 shutdown_connections 2024-05-14T17:10:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 1 -- 172.21.15.69:0/4177667137 wait complete. 2024-05-14T17:10:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.632+0000 7fe72be81740 20 client.17403 trim_cache size 0 max 16384 2024-05-14T17:10:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72be81740 1 Processor -- start 2024-05-14T17:10:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72be81740 1 -- start start 2024-05-14T17:10:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72be81740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef58f0 0x55fd34ef7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72be81740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef6210 0x55fd34ef8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72be81740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34effff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72be81740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241b2370 con 0x55fd34ef6210 2024-05-14T17:10:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72be81740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241b2a40 con 0x55fd34ef58f0 2024-05-14T17:10:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72be81740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241b0350 con 0x55fd34f00ad0 2024-05-14T17:10:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef6210 0x55fd34ef8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef6210 0x55fd34ef8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35756/0 (socket says 172.21.15.69:35756) 2024-05-14T17:10:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72259c640 1 -- 172.21.15.69:0/1738755353 learned_addr learned my addr 172.21.15.69:0/1738755353 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72359e640 1 --2- 172.21.15.69:0/1738755353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34effff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe722d9d640 1 --2- 172.21.15.69:0/1738755353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef58f0 0x55fd34ef7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72259c640 1 -- 172.21.15.69:0/1738755353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 msgr2=0x55fd34effff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72259c640 1 --2- 172.21.15.69:0/1738755353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34effff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72259c640 1 -- 172.21.15.69:0/1738755353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef58f0 msgr2=0x55fd34ef7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72259c640 1 --2- 172.21.15.69:0/1738755353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef58f0 0x55fd34ef7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72259c640 1 -- 172.21.15.69:0/1738755353 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd34efe130 con 0x55fd34ef6210 2024-05-14T17:10:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe722d9d640 1 --2- 172.21.15.69:0/1738755353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef58f0 0x55fd34ef7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.648+0000 7fe72259c640 1 --2- 172.21.15.69:0/1738755353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef6210 0x55fd34ef8b60 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x55fd34ef3220 tx=0x7fe718014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2e662281adf194 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe708ff9640 1 -- 172.21.15.69:0/1738755353 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718005020 con 0x55fd34ef6210 2024-05-14T17:10:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe708ff9640 1 -- 172.21.15.69:0/1738755353 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7180047a0 con 0x55fd34ef6210 2024-05-14T17:10:42.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe708ff9640 1 -- 172.21.15.69:0/1738755353 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7180126b0 con 0x55fd34ef6210 2024-05-14T17:10:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe72be81740 1 -- 172.21.15.69:0/1738755353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef6210 msgr2=0x55fd34ef8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe72be81740 1 --2- 172.21.15.69:0/1738755353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef6210 0x55fd34ef8b60 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x55fd34ef3220 tx=0x7fe718014900 comp rx=0 tx=0).stop 2024-05-14T17:10:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe72be81740 1 -- 172.21.15.69:0/1738755353 shutdown_connections 2024-05-14T17:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe72be81740 1 --2- 172.21.15.69:0/1738755353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34f00ad0 0x55fd34effff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe72be81740 1 --2- 172.21.15.69:0/1738755353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef6210 0x55fd34ef8b60 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe72be81740 1 --2- 172.21.15.69:0/1738755353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34ef58f0 0x55fd34ef7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe72be81740 1 -- 172.21.15.69:0/1738755353 >> 172.21.15.69:0/1738755353 conn(0x55fd34ef1e00 msgr2=0x55fd34ef8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe72be81740 1 -- 172.21.15.69:0/1738755353 shutdown_connections 2024-05-14T17:10:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.649+0000 7fe72be81740 1 -- 172.21.15.69:0/1738755353 wait complete. 2024-05-14T17:10:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72be81740 1 -- start start 2024-05-14T17:10:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72be81740 1 Processor -- start 2024-05-14T17:10:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72be81740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72be81740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34ef6210 0x55fd34f04450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72be81740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34f00ad0 0x55fd34f04b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72be81740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241afc80 con 0x55fd34ef58f0 2024-05-14T17:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72be81740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241ae890 con 0x55fd34f00ad0 2024-05-14T17:10:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72be81740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241aeee0 con 0x55fd34ef6210 2024-05-14T17:10:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72be81740 10 client.0 _ll_register_callbacks cb 0x7fe7140a6480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe722d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe722d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35760/0 (socket says 172.21.15.69:35760) 2024-05-14T17:10:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe722d9d640 1 -- 172.21.15.69:0/3028478485 learned_addr learned my addr 172.21.15.69:0/3028478485 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72259c640 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34ef6210 0x55fd34f04450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72359e640 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34f00ad0 0x55fd34f04b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe722d9d640 1 -- 172.21.15.69:0/3028478485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34ef6210 msgr2=0x55fd34f04450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe722d9d640 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34ef6210 0x55fd34f04450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe722d9d640 1 -- 172.21.15.69:0/3028478485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34f00ad0 msgr2=0x55fd34f04b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe722d9d640 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34f00ad0 0x55fd34f04b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe722d9d640 1 -- 172.21.15.69:0/3028478485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd34f79f40 con 0x55fd34ef58f0 2024-05-14T17:10:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe72359e640 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34f00ad0 0x55fd34f04b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe722d9d640 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f090e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe7140a6040 tx=0x7fe7140a5ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad40606a06271eb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.650+0000 7fe706ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe706ff5640 1 -- 172.21.15.69:0/3028478485 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140a9d90 con 0x55fd34ef58f0 2024-05-14T17:10:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe706ff5640 1 -- 172.21.15.69:0/3028478485 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7fe7140a8c90 con 0x55fd34ef58f0 2024-05-14T17:10:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe72be81740 1 -- 172.21.15.69:0/3028478485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7180047a0 con 0x55fd34ef58f0 2024-05-14T17:10:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe706ff5640 1 -- 172.21.15.69:0/3028478485 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe7140a5930 con 0x55fd34ef58f0 2024-05-14T17:10:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe72be81740 10 client.17409 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe72be81740 1 -- 172.21.15.69:0/3028478485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55fd34f7b060 con 0x55fd34ef58f0 2024-05-14T17:10:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe706ff5640 1 -- 172.21.15.69:0/3028478485 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7180047a0 con 0x55fd34ef58f0 2024-05-14T17:10:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe706ff5640 1 -- 172.21.15.69:0/3028478485 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fe7140ac040 con 0x55fd34ef58f0 2024-05-14T17:10:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe72be81740 10 client.17409 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe72be81740 10 client.17409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe72be81740 20 client.17409 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe72be81740 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55fd34f7b590 0x55fd34f7d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe72be81740 4 client.17409 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe72be81740 1 -- 172.21.15.69:0/3028478485 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd34dde5f0 con 0x55fd34f7b590 2024-05-14T17:10:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.651+0000 7fe72259c640 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55fd34f7b590 0x55fd34f7d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.652+0000 7fe706ff5640 10 client.17409 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.652+0000 7fe72259c640 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55fd34f7b590 0x55fd34f7d980 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e48f38f55e5731 server_cookie=5050e22a9b1e78c4 in_seq=0 out_seq=0 2024-05-14T17:10:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe706ff5640 1 -- 172.21.15.69:0/3028478485 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fd34dde5f0 con 0x55fd34f7b590 2024-05-14T17:10:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe706ff5640 10 client.17409 handle_command_reply: tid=0 2024-05-14T17:10:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 client.17409 shutdown 2024-05-14T17:10:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 -- 172.21.15.69:0/3028478485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 msgr2=0x55fd34f090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f090e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe7140a6040 tx=0x7fe7140a5ea0 comp rx=0 tx=0).stop 2024-05-14T17:10:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 -- 172.21.15.69:0/3028478485 shutdown_connections 2024-05-14T17:10:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55fd34f7b590 0x55fd34f7d980 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd34f00ad0 0x55fd34f04b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd34ef6210 0x55fd34f04450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:45.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe706ff5640 0 client.17409 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 --2- 172.21.15.69:0/3028478485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fd34ef58f0 0x55fd34f090e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 -- 172.21.15.69:0/3028478485 >> 172.21.15.69:0/3028478485 conn(0x55fd34ef1e00 msgr2=0x55fd34ef7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 -- 172.21.15.69:0/3028478485 shutdown_connections 2024-05-14T17:10:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.657+0000 7fe72be81740 1 -- 172.21.15.69:0/3028478485 wait complete. 2024-05-14T17:10:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72be81740 20 client.17409 trim_cache size 0 max 16384 2024-05-14T17:10:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:10:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:10:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:10:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 -- 172.21.15.69:0/1505052833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe70c0b2a10 msgr2=0x7fe70c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe70c0b2a10 0x7fe70c0b4ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe71c04a000 tx=0x7fe71c048000 comp rx=0 tx=0).stop 2024-05-14T17:10:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 -- 172.21.15.69:0/1505052833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe724102a50 msgr2=0x7fe724074d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe724102a50 0x7fe724074d50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe71400ebc0 tx=0x7fe71400c6a0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 -- 172.21.15.69:0/1505052833 shutdown_connections 2024-05-14T17:10:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe70c0b2a10 0x7fe70c0b4ed0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe724108a60 0x7fe7240757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724103410 0x7fe724075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 --2- 172.21.15.69:0/1505052833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe724102a50 0x7fe724074d50 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 -- 172.21.15.69:0/1505052833 >> 172.21.15.69:0/1505052833 conn(0x7fe7240fe520 msgr2=0x7fe7240fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 -- 172.21.15.69:0/1505052833 shutdown_connections 2024-05-14T17:10:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:09:47.658+0000 7fe72916a640 1 -- 172.21.15.69:0/1505052833 wait complete. 2024-05-14T17:10:46.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:48 smithi069 ceph-mon[31507]: pgmap v630: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 7 op/s 2024-05-14T17:10:46.891 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:50 smithi069 ceph-mon[31507]: pgmap v631: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:10:46.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:52 smithi069 ceph-mon[31507]: pgmap v632: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:10:46.999 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:54 smithi069 ceph-mon[31507]: pgmap v633: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-14T17:10:47.049 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:56 smithi069 ceph-mon[31507]: pgmap v634: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-05-14T17:10:47.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:09:58 smithi069 ceph-mon[31507]: pgmap v635: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-14T17:10:47.149 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:00 smithi069 ceph-mon[31507]: pgmap v636: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-05-14T17:10:47.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:00 smithi069 ceph-mon[31507]: overall HEALTH_OK 2024-05-14T17:10:47.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:36 smithi092 ceph-mon[35763]: pgmap v657: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:10:47.299 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:38 smithi092 ceph-mon[35763]: pgmap v658: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:10:47.358 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:40 smithi092 ceph-mon[35763]: pgmap v659: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:10:47.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:42 smithi092 ceph-mon[35763]: pgmap v660: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:10:47.458 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:44 smithi092 ceph-mon[35763]: pgmap v661: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:10:47.508 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:46 smithi092 ceph-mon[35763]: pgmap v662: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:10:47.558 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:36 smithi203 ceph-mon[35787]: pgmap v657: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:10:47.608 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:38 smithi203 ceph-mon[35787]: pgmap v658: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:10:47.659 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:40 smithi203 ceph-mon[35787]: pgmap v659: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:10:47.708 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:42 smithi203 ceph-mon[35787]: pgmap v660: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:10:47.758 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:44 smithi203 ceph-mon[35787]: pgmap v661: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:10:47.808 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:46 smithi203 ceph-mon[35787]: pgmap v662: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:10:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.755+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/1761118863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f18107940 msgr2=0x7f1f18107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.755+0000 7f1f1d0c4640 1 --2- 172.21.15.69:0/1761118863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f18107940 0x7f1f18107d20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009cd0 tx=0x7f1f0802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:10:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.756+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/1761118863 shutdown_connections 2024-05-14T17:10:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.756+0000 7f1f1d0c4640 1 --2- 172.21.15.69:0/1761118863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f18102340 0x7f1f1810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.756+0000 7f1f1d0c4640 1 --2- 172.21.15.69:0/1761118863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f18101980 0x7f1f18101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.756+0000 7f1f1d0c4640 1 --2- 172.21.15.69:0/1761118863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f18107940 0x7f1f18107d20 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.756+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/1761118863 >> 172.21.15.69:0/1761118863 conn(0x7f1f18078ad0 msgr2=0x7f1f18078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.756+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/1761118863 shutdown_connections 2024-05-14T17:10:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.756+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/1761118863 wait complete. 2024-05-14T17:10:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.756+0000 7f1f1d0c4640 1 Processor -- start 2024-05-14T17:10:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1d0c4640 1 -- start start 2024-05-14T17:10:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1d0c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f18101980 0x7f1f1819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1d0c4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f18102340 0x7f1f1819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1d0c4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f18107940 0x7f1f181a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1d0c4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18111c00 con 0x7f1f18107940 2024-05-14T17:10:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1d0c4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18111a80 con 0x7f1f18102340 2024-05-14T17:10:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1d0c4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18111d80 con 0x7f1f18101980 2024-05-14T17:10:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f18107940 0x7f1f181a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f18107940 0x7f1f181a5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38624/0 (socket says 172.21.15.69:38624) 2024-05-14T17:10:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1759e640 1 -- 172.21.15.69:0/2936111913 learned_addr learned my addr 172.21.15.69:0/2936111913 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1659c640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f18102340 0x7f1f1819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f18101980 0x7f1f1819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1759e640 1 -- 172.21.15.69:0/2936111913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f18101980 msgr2=0x7f1f1819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1759e640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f18101980 0x7f1f1819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1759e640 1 -- 172.21.15.69:0/2936111913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f18102340 msgr2=0x7f1f1819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1759e640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f18102340 0x7f1f1819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.757+0000 7f1f1759e640 1 -- 172.21.15.69:0/2936111913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f181a5640 con 0x7f1f18107940 2024-05-14T17:10:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.758+0000 7f1f1759e640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f18107940 0x7f1f181a5100 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e970 tx=0x7f1f1000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.758+0000 7f1efffff640 1 -- 172.21.15.69:0/2936111913 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000ccc0 con 0x7f1f18107940 2024-05-14T17:10:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.758+0000 7f1efffff640 1 -- 172.21.15.69:0/2936111913 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1000ce60 con 0x7f1f18107940 2024-05-14T17:10:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.758+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/2936111913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f181a5810 con 0x7f1f18107940 2024-05-14T17:10:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.758+0000 7f1efffff640 1 -- 172.21.15.69:0/2936111913 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10010430 con 0x7f1f18107940 2024-05-14T17:10:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.758+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/2936111913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f181a5d60 con 0x7f1f18107940 2024-05-14T17:10:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.760+0000 7f1efffff640 1 -- 172.21.15.69:0/2936111913 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1f10004f90 con 0x7f1f18107940 2024-05-14T17:10:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.760+0000 7f1f1fdd9740 1 Processor -- start 2024-05-14T17:10:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.760+0000 7f1f1fdd9740 1 -- start start 2024-05-14T17:10:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee818f0 0x5591dee81cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee82210 0x5591dee8c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee8cad0 0x5591dee8eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5591dedf4c30 con 0x5591dee82210 2024-05-14T17:10:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5591dede47d0 con 0x5591dee818f0 2024-05-14T17:10:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5591dede8810 con 0x5591dee8cad0 2024-05-14T17:10:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f16d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee82210 0x5591dee8c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f16d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee82210 0x5591dee8c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38630/0 (socket says 172.21.15.69:38630) 2024-05-14T17:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f16d9d640 1 -- 172.21.15.69:0/2314633764 learned_addr learned my addr 172.21.15.69:0/2314633764 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f1659c640 1 --2- 172.21.15.69:0/2314633764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee8cad0 0x5591dee8eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f1659c640 1 --2- 172.21.15.69:0/2314633764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee818f0 0x5591dee81cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1efffff640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1f000b2a10 0x7f1f000b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f16d9d640 1 -- 172.21.15.69:0/2314633764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee8cad0 msgr2=0x5591dee8eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/2314633764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee8cad0 0x5591dee8eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f16d9d640 1 -- 172.21.15.69:0/2314633764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee818f0 msgr2=0x5591dee81cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/2314633764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee818f0 0x5591dee81cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f16d9d640 1 -- 172.21.15.69:0/2314633764 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591deaf9210 con 0x5591dee82210 2024-05-14T17:10:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f1659c640 1 --2- 172.21.15.69:0/2314633764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee818f0 0x5591dee81cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f1759e640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1f000b2a10 0x7f1f000b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/2314633764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee82210 0x5591dee8c590 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009cd0 tx=0x7f1f08009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d622f74acaaa48d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.761+0000 7f1efffff640 1 -- 172.21.15.69:0/2936111913 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7f1f100e29b0 con 0x7f1f18107940 2024-05-14T17:10:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1efd7fa640 1 -- 172.21.15.69:0/2314633764 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08049070 con 0x5591dee82210 2024-05-14T17:10:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1efd7fa640 1 -- 172.21.15.69:0/2314633764 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f08036040 con 0x5591dee82210 2024-05-14T17:10:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1efd7fa640 1 -- 172.21.15.69:0/2314633764 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08004040 con 0x5591dee82210 2024-05-14T17:10:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2314633764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee82210 msgr2=0x5591dee8c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1f1759e640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1f000b2a10 0x7f1f000b4ed0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1f100e06f0 tx=0x7f1f100cc040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/2314633764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee82210 0x5591dee8c590 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009cd0 tx=0x7f1f08009360 comp rx=0 tx=0).stop 2024-05-14T17:10:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2314633764 shutdown_connections 2024-05-14T17:10:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/2314633764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee8cad0 0x5591dee8eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/2314633764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee82210 0x5591dee8c590 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/2314633764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee818f0 0x5591dee81cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2314633764 >> 172.21.15.69:0/2314633764 conn(0x5591dee7de00 msgr2=0x5591dee7e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2314633764 shutdown_connections 2024-05-14T17:10:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.762+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2314633764 wait complete. 2024-05-14T17:10:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1fdd9740 1 -- start start 2024-05-14T17:10:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1fdd9740 1 Processor -- start 2024-05-14T17:10:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee8c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee82210 0x5591dee897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee8cad0 0x5591dee89dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5591deaca790 con 0x5591dee818f0 2024-05-14T17:10:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5591deda0890 con 0x5591dee8cad0 2024-05-14T17:10:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5591dee8f740 con 0x5591dee82210 2024-05-14T17:10:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1fdd9740 10 client.0 _ll_register_callbacks cb 0x5591deda3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee8c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee8c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38634/0 (socket says 172.21.15.69:38634) 2024-05-14T17:10:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1659c640 1 -- 172.21.15.69:0/162569459 learned_addr learned my addr 172.21.15.69:0/162569459 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee82210 0x5591dee897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1659c640 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee8cad0 0x5591dee89dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1659c640 1 -- 172.21.15.69:0/162569459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee82210 msgr2=0x5591dee897c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1659c640 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee82210 0x5591dee897c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1659c640 1 -- 172.21.15.69:0/162569459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee8cad0 msgr2=0x5591dee89dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.763+0000 7f1f1659c640 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee8cad0 0x5591dee89dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1659c640 1 -- 172.21.15.69:0/162569459 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591debc5620 con 0x5591dee818f0 2024-05-14T17:10:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee82210 0x5591dee897c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1659c640 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee8c4d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009e40 tx=0x7f1f0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c393b0471ece42 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1efaff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1efaff5640 1 -- 172.21.15.69:0/162569459 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c01a040 con 0x5591dee818f0 2024-05-14T17:10:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1efaff5640 1 -- 172.21.15.69:0/162569459 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7f1f0c004230 con 0x5591dee818f0 2024-05-14T17:10:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/162569459 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f08036040 con 0x5591dee818f0 2024-05-14T17:10:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1efaff5640 1 -- 172.21.15.69:0/162569459 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1f0c028050 con 0x5591dee818f0 2024-05-14T17:10:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1fdd9740 10 client.17421 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/162569459 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5591dee9b1b0 con 0x5591dee818f0 2024-05-14T17:10:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1efaff5640 1 -- 172.21.15.69:0/162569459 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f08036040 con 0x5591dee818f0 2024-05-14T17:10:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1efaff5640 1 -- 172.21.15.69:0/162569459 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f1f0c0203c0 con 0x5591dee818f0 2024-05-14T17:10:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1fdd9740 10 client.17421 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1fdd9740 10 client.17421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1fdd9740 20 client.17421 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5591dee9c670 0x5591dee9ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:52.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1fdd9740 4 client.17421 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.764+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/162569459 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591dedf6b80 con 0x5591dee9c670 2024-05-14T17:10:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.765+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5591dee9c670 0x5591dee9ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.765+0000 7f1efaff5640 10 client.17421 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.765+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5591dee9c670 0x5591dee9ea60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adcae36f8f0a8c30 server_cookie=470507701a43c5d2 in_seq=0 out_seq=0 2024-05-14T17:10:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1efaff5640 1 -- 172.21.15.69:0/162569459 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5591dedf6b80 con 0x5591dee9c670 2024-05-14T17:10:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1efaff5640 10 client.17421 handle_command_reply: tid=0 2024-05-14T17:10:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 client.17421 shutdown 2024-05-14T17:10:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/162569459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 msgr2=0x5591dee8c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee8c4d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009e40 tx=0x7f1f0c004730 comp rx=0 tx=0).stop 2024-05-14T17:10:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/162569459 shutdown_connections 2024-05-14T17:10:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5591dee9c670 0x5591dee9ea60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee8cad0 0x5591dee89dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee82210 0x5591dee897c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1efaff5640 0 client.17421 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/162569459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee8c4d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/162569459 >> 172.21.15.69:0/162569459 conn(0x5591dee7de00 msgr2=0x5591dee7d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/162569459 shutdown_connections 2024-05-14T17:10:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/162569459 wait complete. 2024-05-14T17:10:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.766+0000 7f1f1fdd9740 20 client.17421 trim_cache size 0 max 16384 2024-05-14T17:10:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.782+0000 7f1f1fdd9740 1 Processor -- start 2024-05-14T17:10:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.782+0000 7f1f1fdd9740 1 -- start start 2024-05-14T17:10:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.782+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee818f0 0x5591dee98610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.782+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 0x5591dee8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.782+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee8cad0 0x5591deea4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.782+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181b0180 con 0x5591dee8cad0 2024-05-14T17:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.782+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181b0850 con 0x5591dee82210 2024-05-14T17:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.782+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181ae160 con 0x5591dee818f0 2024-05-14T17:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.782+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee8cad0 0x5591deea4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.782+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee8cad0 0x5591deea4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38640/0 (socket says 172.21.15.69:38640) 2024-05-14T17:10:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1f1659c640 1 -- 172.21.15.69:0/3583593232 learned_addr learned my addr 172.21.15.69:0/3583593232 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/3583593232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 0x5591dee8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1f1659c640 1 --2- 172.21.15.69:0/3583593232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee818f0 0x5591dee98610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1f1659c640 1 -- 172.21.15.69:0/3583593232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee818f0 msgr2=0x5591dee98610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1f1659c640 1 --2- 172.21.15.69:0/3583593232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee818f0 0x5591dee98610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1f1659c640 1 -- 172.21.15.69:0/3583593232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 msgr2=0x5591dee8c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1f1659c640 1 --2- 172.21.15.69:0/3583593232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 0x5591dee8c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1f1659c640 1 -- 172.21.15.69:0/3583593232 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591dee9f0c0 con 0x5591dee8cad0 2024-05-14T17:10:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/3583593232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 0x5591dee8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1f1659c640 1 --2- 172.21.15.69:0/3583593232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee8cad0 0x5591deea4330 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0097a0 tx=0x7f1f0c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d03336706674d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1efcff9640 1 -- 172.21.15.69:0/3583593232 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c020e20 con 0x5591dee8cad0 2024-05-14T17:10:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1efcff9640 1 -- 172.21.15.69:0/3583593232 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0c010d30 con 0x5591dee8cad0 2024-05-14T17:10:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.783+0000 7f1efcff9640 1 -- 172.21.15.69:0/3583593232 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0047a0 con 0x5591dee8cad0 2024-05-14T17:10:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/3583593232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee8cad0 msgr2=0x5591deea4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/3583593232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee8cad0 0x5591deea4330 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0097a0 tx=0x7f1f0c005f90 comp rx=0 tx=0).stop 2024-05-14T17:10:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/3583593232 shutdown_connections 2024-05-14T17:10:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/3583593232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee8cad0 0x5591deea4330 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/3583593232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 0x5591dee8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/3583593232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee818f0 0x5591dee98610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/3583593232 >> 172.21.15.69:0/3583593232 conn(0x5591dee7de00 msgr2=0x5591dee83b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/3583593232 shutdown_connections 2024-05-14T17:10:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/3583593232 wait complete. 2024-05-14T17:10:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 -- start start 2024-05-14T17:10:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 Processor -- start 2024-05-14T17:10:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee95020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 0x5591dee90390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee8cad0 0x5591dee90ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.784+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181ada90 con 0x5591dee818f0 2024-05-14T17:10:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181ac6a0 con 0x5591dee82210 2024-05-14T17:10:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1fdd9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181accf0 con 0x5591dee8cad0 2024-05-14T17:10:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1fdd9740 10 client.0 _ll_register_callbacks cb 0x7f1f0c0109e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee95020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee95020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38656/0 (socket says 172.21.15.69:38656) 2024-05-14T17:10:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1659c640 1 -- 172.21.15.69:0/2941283617 learned_addr learned my addr 172.21.15.69:0/2941283617 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 0x5591dee90390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1659c640 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee8cad0 0x5591dee90ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1659c640 1 -- 172.21.15.69:0/2941283617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee8cad0 msgr2=0x5591dee90ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:54.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1659c640 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee8cad0 0x5591dee90ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1659c640 1 -- 172.21.15.69:0/2941283617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 msgr2=0x5591dee90390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1659c640 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 0x5591dee90390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1659c640 1 -- 172.21.15.69:0/2941283617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591def05ee0 con 0x5591dee818f0 2024-05-14T17:10:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1659c640 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee95020 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009c90 tx=0x7f1f0c020980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7695e7947cd7b04f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1efaff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1efaff5640 1 -- 172.21.15.69:0/2941283617 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c005030 con 0x5591dee818f0 2024-05-14T17:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1efaff5640 1 -- 172.21.15.69:0/2941283617 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8542+0+0 (secure 0 0 0) 0x7f1f0c0051d0 con 0x5591dee818f0 2024-05-14T17:10:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2941283617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f0c010d30 con 0x5591dee818f0 2024-05-14T17:10:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1efaff5640 1 -- 172.21.15.69:0/2941283617 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1f0c0053d0 con 0x5591dee818f0 2024-05-14T17:10:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1fdd9740 10 client.17427 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.785+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2941283617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5591def07020 con 0x5591dee818f0 2024-05-14T17:10:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.786+0000 7f1efaff5640 1 -- 172.21.15.69:0/2941283617 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0c010d30 con 0x5591dee818f0 2024-05-14T17:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.786+0000 7f1efaff5640 1 -- 172.21.15.69:0/2941283617 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f1f0c038440 con 0x5591dee818f0 2024-05-14T17:10:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.786+0000 7f1f1fdd9740 10 client.17427 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.786+0000 7f1f1fdd9740 10 client.17427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.786+0000 7f1f1fdd9740 20 client.17427 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.786+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5591def07580 0x5591def09970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.786+0000 7f1f1fdd9740 4 client.17427 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.786+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2941283617 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591dedf0360 con 0x5591def07580 2024-05-14T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.786+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5591def07580 0x5591def09970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.787+0000 7f1efaff5640 10 client.17427 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.787+0000 7f1f16d9d640 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5591def07580 0x5591def09970 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258cbbe35ed4226e server_cookie=9c6b964ac5043eac in_seq=0 out_seq=0 2024-05-14T17:10:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1efaff5640 1 -- 172.21.15.69:0/2941283617 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5591dedf0360 con 0x5591def07580 2024-05-14T17:10:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1efaff5640 10 client.17427 handle_command_reply: tid=0 2024-05-14T17:10:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 client.17427 shutdown 2024-05-14T17:10:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2941283617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 msgr2=0x5591dee95020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee95020 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009c90 tx=0x7f1f0c020980 comp rx=0 tx=0).stop 2024-05-14T17:10:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2941283617 shutdown_connections 2024-05-14T17:10:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5591def07580 0x5591def09970 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591dee8cad0 0x5591dee90ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1efaff5640 0 client.17427 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5591dee82210 0x5591dee90390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 --2- 172.21.15.69:0/2941283617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591dee818f0 0x5591dee95020 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2941283617 >> 172.21.15.69:0/2941283617 conn(0x5591dee7de00 msgr2=0x5591dee843a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2941283617 shutdown_connections 2024-05-14T17:10:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 1 -- 172.21.15.69:0/2941283617 wait complete. 2024-05-14T17:10:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.800+0000 7f1f1fdd9740 20 client.17427 trim_cache size 0 max 16384 2024-05-14T17:10:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:10:55.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:10:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:10:56.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:48 smithi092 ceph-mon[35763]: pgmap v663: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-05-14T17:10:56.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:49 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1057222616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:56.073 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:50 smithi092 ceph-mon[35763]: pgmap v664: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:10:56.103 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:52 smithi092 ceph-mon[35763]: pgmap v665: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-05-14T17:10:56.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:54 smithi092 ceph-mon[35763]: pgmap v666: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:10:56.184 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:48 smithi203 ceph-mon[35787]: pgmap v663: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-05-14T17:10:56.207 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:49 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1057222616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:56.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:50 smithi203 ceph-mon[35787]: pgmap v664: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:10:56.256 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:52 smithi203 ceph-mon[35787]: pgmap v665: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-05-14T17:10:56.281 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:54 smithi203 ceph-mon[35787]: pgmap v666: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:10:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/2936111913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1f000b2a10 msgr2=0x7f1f000b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1f000b2a10 0x7f1f000b4ed0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1f100e06f0 tx=0x7f1f100cc040 comp rx=0 tx=0).stop 2024-05-14T17:10:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/2936111913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f18107940 msgr2=0x7f1f181a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f18107940 0x7f1f181a5100 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e970 tx=0x7f1f1000ed70 comp rx=0 tx=0).stop 2024-05-14T17:10:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/2936111913 shutdown_connections 2024-05-14T17:10:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1f000b2a10 0x7f1f000b4ed0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f18107940 0x7f1f181a5100 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f18102340 0x7f1f1819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 --2- 172.21.15.69:0/2936111913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f18101980 0x7f1f1819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/2936111913 >> 172.21.15.69:0/2936111913 conn(0x7f1f18078ad0 msgr2=0x7f1f18104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/2936111913 shutdown_connections 2024-05-14T17:10:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:00.801+0000 7f1f1d0c4640 1 -- 172.21.15.69:0/2936111913 wait complete. 2024-05-14T17:10:56.668 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:02 smithi069 ceph-mon[31507]: pgmap v637: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 KiB/s wr, 7 op/s 2024-05-14T17:10:56.690 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:04 smithi069 ceph-mon[31507]: pgmap v638: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-14T17:10:56.715 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:06 smithi069 ceph-mon[31507]: pgmap v639: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 51 KiB/s wr, 24 op/s 2024-05-14T17:10:56.740 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:08 smithi069 ceph-mon[31507]: pgmap v640: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 63 KiB/s wr, 28 op/s 2024-05-14T17:10:56.765 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:10 smithi069 ceph-mon[31507]: pgmap v641: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 63 KiB/s wr, 26 op/s 2024-05-14T17:10:56.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:10 smithi069 ceph-mon[31507]: osdmap e113: 12 total, 12 up, 12 in 2024-05-14T17:10:56.815 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:12 smithi069 ceph-mon[31507]: pgmap v643: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 62 KiB/s wr, 29 op/s 2024-05-14T17:10:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.902+0000 7f16d6008640 1 -- 172.21.15.69:0/1582761584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16d0101b70 msgr2=0x7f16d01076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.902+0000 7f16d6008640 1 --2- 172.21.15.69:0/1582761584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16d0101b70 0x7f16d01076a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009900 tx=0x7f16c402a410 comp rx=0 tx=0).stop 2024-05-14T17:10:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.902+0000 7f16d6008640 1 -- 172.21.15.69:0/1582761584 shutdown_connections 2024-05-14T17:10:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.902+0000 7f16d6008640 1 --2- 172.21.15.69:0/1582761584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d0107d80 0x7f16d010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.902+0000 7f16d6008640 1 --2- 172.21.15.69:0/1582761584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16d0101b70 0x7f16d01076a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.902+0000 7f16d6008640 1 --2- 172.21.15.69:0/1582761584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d01011c0 0x7f16d01015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.902+0000 7f16d6008640 1 -- 172.21.15.69:0/1582761584 >> 172.21.15.69:0/1582761584 conn(0x7f16d0077e70 msgr2=0x7f16d00ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.902+0000 7f16d6008640 1 -- 172.21.15.69:0/1582761584 shutdown_connections 2024-05-14T17:10:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.902+0000 7f16d6008640 1 -- 172.21.15.69:0/1582761584 wait complete. 2024-05-14T17:10:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.903+0000 7f16d6008640 1 Processor -- start 2024-05-14T17:10:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.903+0000 7f16d6008640 1 -- start start 2024-05-14T17:10:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.903+0000 7f16d6008640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d01011c0 0x7f16d0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.903+0000 7f16d6008640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d0107d80 0x7f16d01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16d6008640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16d019dad0 0x7f16d019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16d6008640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112ca0 con 0x7f16d019dad0 2024-05-14T17:10:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16d6008640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112b20 con 0x7f16d0107d80 2024-05-14T17:10:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16d6008640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112e20 con 0x7f16d01011c0 2024-05-14T17:10:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16d019dad0 0x7f16d019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16d019dad0 0x7f16d019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43708/0 (socket says 172.21.15.69:43708) 2024-05-14T17:10:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16cffff640 1 -- 172.21.15.69:0/268500971 learned_addr learned my addr 172.21.15.69:0/268500971 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d01011c0 0x7f16d0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16cffff640 1 -- 172.21.15.69:0/268500971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d01011c0 msgr2=0x7f16d0199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16cffff640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d01011c0 0x7f16d0199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16ceffd640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d0107d80 0x7f16d01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16cffff640 1 -- 172.21.15.69:0/268500971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d0107d80 msgr2=0x7f16d01997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16cffff640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d0107d80 0x7f16d01997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16cffff640 1 -- 172.21.15.69:0/268500971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d019e5c0 con 0x7f16d019dad0 2024-05-14T17:10:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16cffff640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16d019dad0 0x7f16d019df80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f16c800d750 tx=0x7f16c800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16ccff9640 1 -- 172.21.15.69:0/268500971 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8017070 con 0x7f16d019dad0 2024-05-14T17:10:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16d6008640 1 -- 172.21.15.69:0/268500971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d019e8b0 con 0x7f16d019dad0 2024-05-14T17:10:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16ccff9640 1 -- 172.21.15.69:0/268500971 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c80054d0 con 0x7f16d019dad0 2024-05-14T17:10:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16ccff9640 1 -- 172.21.15.69:0/268500971 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8004c60 con 0x7f16d019dad0 2024-05-14T17:10:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.904+0000 7f16d6008640 1 -- 172.21.15.69:0/268500971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d01a6050 con 0x7f16d019dad0 2024-05-14T17:10:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16ccff9640 1 -- 172.21.15.69:0/268500971 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f16c8009bd0 con 0x7f16d019dad0 2024-05-14T17:10:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16d8dfd740 1 Processor -- start 2024-05-14T17:10:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16d8dfd740 1 -- start start 2024-05-14T17:10:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01680cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b0168b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b0168dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560b015f3c30 con 0x560b0168bad0 2024-05-14T17:10:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560b015e37d0 con 0x560b01681210 2024-05-14T17:10:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560b015e7810 con 0x560b016808f0 2024-05-14T17:10:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b0168dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b0168dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43710/0 (socket says 172.21.15.69:43710) 2024-05-14T17:10:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16cf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b0168b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16ceffd640 1 -- 172.21.15.69:0/2686582050 learned_addr learned my addr 172.21.15.69:0/2686582050 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.907+0000 7f16ceffd640 1 --2- 172.21.15.69:0/2686582050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01680cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16ceffd640 1 -- 172.21.15.69:0/2686582050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 msgr2=0x560b01680cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16ceffd640 1 --2- 172.21.15.69:0/2686582050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01680cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16ceffd640 1 -- 172.21.15.69:0/2686582050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 msgr2=0x560b0168b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16ceffd640 1 --2- 172.21.15.69:0/2686582050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b0168b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16ceffd640 1 -- 172.21.15.69:0/2686582050 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b012f8210 con 0x560b0168bad0 2024-05-14T17:10:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/2686582050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b0168b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16ccff9640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f16b80b2a10 0x7f16b80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f16b80b2a10 0x7f16b80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16ceffd640 1 --2- 172.21.15.69:0/2686582050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b0168dec0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009900 tx=0x7f16c4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167ad9acdce6cbb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16ccff9640 1 -- 172.21.15.69:0/268500971 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f16c80e2030 con 0x7f16d019dad0 2024-05-14T17:10:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16b5ffb640 1 -- 172.21.15.69:0/2686582050 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4045070 con 0x560b0168bad0 2024-05-14T17:10:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16b5ffb640 1 -- 172.21.15.69:0/2686582050 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c40317e0 con 0x560b0168bad0 2024-05-14T17:10:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.908+0000 7f16b5ffb640 1 -- 172.21.15.69:0/2686582050 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4035040 con 0x560b0168bad0 2024-05-14T17:10:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2686582050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 msgr2=0x560b0168dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f16b80b2a10 0x7f16b80b4ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x560b01681e90 tx=0x7f16c00097c0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/2686582050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b0168dec0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009900 tx=0x7f16c4031400 comp rx=0 tx=0).stop 2024-05-14T17:10:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2686582050 shutdown_connections 2024-05-14T17:10:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/2686582050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b0168dec0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/2686582050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b0168b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/2686582050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01680cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2686582050 >> 172.21.15.69:0/2686582050 conn(0x560b0167ce00 msgr2=0x560b0167d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2686582050 shutdown_connections 2024-05-14T17:10:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2686582050 wait complete. 2024-05-14T17:10:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.909+0000 7f16d8dfd740 1 -- start start 2024-05-14T17:10:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16d8dfd740 1 Processor -- start 2024-05-14T17:10:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b016808f0 0x560b0168b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b016887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b0168bad0 0x560b01688dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560b012c9790 con 0x560b016808f0 2024-05-14T17:10:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560b0159f890 con 0x560b01681210 2024-05-14T17:10:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560b0168e740 con 0x560b0168bad0 2024-05-14T17:10:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16d8dfd740 10 client.0 _ll_register_callbacks cb 0x560b015a2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16cf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b016808f0 0x560b0168b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16cf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b016808f0 0x560b0168b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43724/0 (socket says 172.21.15.69:43724) 2024-05-14T17:10:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16cf7fe640 1 -- 172.21.15.69:0/390808367 learned_addr learned my addr 172.21.15.69:0/390808367 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16ceffd640 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b016887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16cffff640 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b0168bad0 0x560b01688dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16cf7fe640 1 -- 172.21.15.69:0/390808367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b0168bad0 msgr2=0x560b01688dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b0168bad0 0x560b01688dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16cf7fe640 1 -- 172.21.15.69:0/390808367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 msgr2=0x560b016887c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b016887c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16cf7fe640 1 -- 172.21.15.69:0/390808367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b013c4620 con 0x560b016808f0 2024-05-14T17:10:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.910+0000 7f16cffff640 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b0168bad0 0x560b01688dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b016808f0 0x560b0168b4d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f16c00109b0 tx=0x7f16c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1990fec096d38c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:10:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16aeffd640 1 -- 172.21.15.69:0/390808367 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c001ecc0 con 0x560b016808f0 2024-05-14T17:10:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16aeffd640 1 -- 172.21.15.69:0/390808367 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f16c001f420 con 0x560b016808f0 2024-05-14T17:10:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16d8dfd740 1 -- 172.21.15.69:0/390808367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c40317e0 con 0x560b016808f0 2024-05-14T17:10:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16aeffd640 1 -- 172.21.15.69:0/390808367 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f16c001ee60 con 0x560b016808f0 2024-05-14T17:10:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16d8dfd740 10 client.17439 fetch_fsmap learned FSMap version 61 2024-05-14T17:10:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16d8dfd740 1 -- 172.21.15.69:0/390808367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x560b0169a2d0 con 0x560b016808f0 2024-05-14T17:10:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16aeffd640 1 -- 172.21.15.69:0/390808367 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c40317e0 con 0x560b016808f0 2024-05-14T17:10:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16aeffd640 1 -- 172.21.15.69:0/390808367 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f16c000f040 con 0x560b016808f0 2024-05-14T17:10:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16d8dfd740 10 client.17439 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:10:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.911+0000 7f16d8dfd740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.912+0000 7f16d8dfd740 20 client.17439 populate_metadata read hostname 'smithi069' 2024-05-14T17:10:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.912+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560b0169b800 0x560b0169dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.912+0000 7f16d8dfd740 4 client.17439 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.912+0000 7f16d8dfd740 1 -- 172.21.15.69:0/390808367 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b015f5b80 con 0x560b0169b800 2024-05-14T17:10:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.912+0000 7f16ceffd640 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560b0169b800 0x560b0169dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.912+0000 7f16ceffd640 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560b0169b800 0x560b0169dbf0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654e258445cbd95 server_cookie=387dcc4f33ea7aba in_seq=0 out_seq=0 2024-05-14T17:10:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.912+0000 7f16aeffd640 10 client.17439 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16aeffd640 1 -- 172.21.15.69:0/390808367 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560b015f5b80 con 0x560b0169b800 2024-05-14T17:10:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16aeffd640 10 client.17439 handle_command_reply: tid=0 2024-05-14T17:10:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 client.17439 shutdown 2024-05-14T17:10:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 -- 172.21.15.69:0/390808367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b016808f0 msgr2=0x560b0168b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b016808f0 0x560b0168b4d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f16c00109b0 tx=0x7f16c0004a40 comp rx=0 tx=0).stop 2024-05-14T17:10:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 -- 172.21.15.69:0/390808367 shutdown_connections 2024-05-14T17:10:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560b0169b800 0x560b0169dbf0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b0168bad0 0x560b01688dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b016887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/390808367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b016808f0 0x560b0168b4d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16aeffd640 0 client.17439 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:10:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 -- 172.21.15.69:0/390808367 >> 172.21.15.69:0/390808367 conn(0x560b0167ce00 msgr2=0x560b0167c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 -- 172.21.15.69:0/390808367 shutdown_connections 2024-05-14T17:10:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 1 -- 172.21.15.69:0/390808367 wait complete. 2024-05-14T17:10:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.913+0000 7f16d8dfd740 20 client.17439 trim_cache size 0 max 16384 2024-05-14T17:10:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16d8dfd740 1 Processor -- start 2024-05-14T17:10:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16d8dfd740 1 -- start start 2024-05-14T17:10:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01682d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b01681210 0x560b0168b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b0168bad0 0x560b016a3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b0160 con 0x560b01681210 2024-05-14T17:10:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b0830 con 0x560b0168bad0 2024-05-14T17:10:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01ae140 con 0x560b016808f0 2024-05-14T17:10:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b01681210 0x560b0168b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:59.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b01681210 0x560b0168b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43738/0 (socket says 172.21.15.69:43738) 2024-05-14T17:10:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16ceffd640 1 -- 172.21.15.69:0/565007010 learned_addr learned my addr 172.21.15.69:0/565007010 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:10:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/565007010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01682d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16cffff640 1 --2- 172.21.15.69:0/565007010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b0168bad0 0x560b016a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16ceffd640 1 -- 172.21.15.69:0/565007010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 msgr2=0x560b01682d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16ceffd640 1 --2- 172.21.15.69:0/565007010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01682d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16ceffd640 1 -- 172.21.15.69:0/565007010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b0168bad0 msgr2=0x560b016a3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16ceffd640 1 --2- 172.21.15.69:0/565007010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b0168bad0 0x560b016a3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16ceffd640 1 -- 172.21.15.69:0/565007010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b0168e230 con 0x560b01681210 2024-05-14T17:10:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.929+0000 7f16ceffd640 1 --2- 172.21.15.69:0/565007010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b01681210 0x560b0168b000 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f16c402a450 tx=0x7f16c4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca48cb63b95e58a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16b57fa640 1 -- 172.21.15.69:0/565007010 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4035070 con 0x560b01681210 2024-05-14T17:10:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16b57fa640 1 -- 172.21.15.69:0/565007010 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c4031660 con 0x560b01681210 2024-05-14T17:10:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16b57fa640 1 -- 172.21.15.69:0/565007010 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4045040 con 0x560b01681210 2024-05-14T17:10:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16d8dfd740 1 -- 172.21.15.69:0/565007010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b01681210 msgr2=0x560b0168b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/565007010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b01681210 0x560b0168b000 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f16c402a450 tx=0x7f16c4005c00 comp rx=0 tx=0).stop 2024-05-14T17:10:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16d8dfd740 1 -- 172.21.15.69:0/565007010 shutdown_connections 2024-05-14T17:10:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/565007010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b0168bad0 0x560b016a3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/565007010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b01681210 0x560b0168b000 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/565007010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01682d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16d8dfd740 1 -- 172.21.15.69:0/565007010 >> 172.21.15.69:0/565007010 conn(0x560b0167ce00 msgr2=0x560b01683d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16d8dfd740 1 -- 172.21.15.69:0/565007010 shutdown_connections 2024-05-14T17:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16d8dfd740 1 -- 172.21.15.69:0/565007010 wait complete. 2024-05-14T17:10:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.930+0000 7f16d8dfd740 1 -- start start 2024-05-14T17:10:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16d8dfd740 1 Processor -- start 2024-05-14T17:10:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01685890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b01685dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16d8dfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b01686310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01ada70 con 0x560b0168bad0 2024-05-14T17:10:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01ac680 con 0x560b01681210 2024-05-14T17:10:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16d8dfd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01accd0 con 0x560b016808f0 2024-05-14T17:10:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16d8dfd740 10 client.0 _ll_register_callbacks cb 0x7f16c0010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b01686310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b01686310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43752/0 (socket says 172.21.15.69:43752) 2024-05-14T17:11:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16cffff640 1 -- 172.21.15.69:0/2799121746 learned_addr learned my addr 172.21.15.69:0/2799121746 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16ceffd640 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b01685dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01685890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16cffff640 1 -- 172.21.15.69:0/2799121746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 msgr2=0x560b01685890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16cffff640 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01685890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16cffff640 1 -- 172.21.15.69:0/2799121746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 msgr2=0x560b01685dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16cffff640 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b01685dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16cffff640 1 -- 172.21.15.69:0/2799121746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b0170daf0 con 0x560b0168bad0 2024-05-14T17:11:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.931+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01685890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16cffff640 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b01686310 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f16c80a9290 tx=0x7f16c8005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124b2e37426216f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16aeffd640 1 -- 172.21.15.69:0/2799121746 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8020040 con 0x560b0168bad0 2024-05-14T17:11:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16aeffd640 1 -- 172.21.15.69:0/2799121746 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f16c80a8770 con 0x560b0168bad0 2024-05-14T17:11:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2799121746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c4031660 con 0x560b0168bad0 2024-05-14T17:11:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16aeffd640 1 -- 172.21.15.69:0/2799121746 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f16c80ad6b0 con 0x560b0168bad0 2024-05-14T17:11:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16d8dfd740 10 client.17445 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2799121746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x560b0170ee20 con 0x560b0168bad0 2024-05-14T17:11:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16aeffd640 1 -- 172.21.15.69:0/2799121746 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c4031660 con 0x560b0168bad0 2024-05-14T17:11:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16aeffd640 1 -- 172.21.15.69:0/2799121746 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f16c80a7fd0 con 0x560b0168bad0 2024-05-14T17:11:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16d8dfd740 10 client.17445 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16d8dfd740 10 client.17445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16d8dfd740 20 client.17445 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560b0170f590 0x560b017119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16d8dfd740 4 client.17445 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2799121746 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b015ef360 con 0x560b0170f590 2024-05-14T17:11:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.932+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560b0170f590 0x560b017119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.933+0000 7f16aeffd640 10 client.17445 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.933+0000 7f16cf7fe640 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560b0170f590 0x560b017119b0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa7b377655260d0 server_cookie=10232f9bab88a7a7 in_seq=0 out_seq=0 2024-05-14T17:11:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16aeffd640 1 -- 172.21.15.69:0/2799121746 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560b015ef360 con 0x560b0170f590 2024-05-14T17:11:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16aeffd640 10 client.17445 handle_command_reply: tid=0 2024-05-14T17:11:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 client.17445 shutdown 2024-05-14T17:11:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2799121746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 msgr2=0x560b01686310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b01686310 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f16c80a9290 tx=0x7f16c8005680 comp rx=0 tx=0).stop 2024-05-14T17:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2799121746 shutdown_connections 2024-05-14T17:11:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x560b0170f590 0x560b017119b0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b0168bad0 0x560b01686310 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b01681210 0x560b01685dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16aeffd640 0 client.17445 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 --2- 172.21.15.69:0/2799121746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560b016808f0 0x560b01685890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2799121746 >> 172.21.15.69:0/2799121746 conn(0x560b0167ce00 msgr2=0x560b0167b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2799121746 shutdown_connections 2024-05-14T17:11:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 1 -- 172.21.15.69:0/2799121746 wait complete. 2024-05-14T17:11:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.950+0000 7f16d8dfd740 20 client.17445 trim_cache size 0 max 16384 2024-05-14T17:11:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:11:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:11:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:11:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 -- 172.21.15.69:0/268500971 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f16b80b2a10 msgr2=0x7f16b80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f16b80b2a10 0x7f16b80b4ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x560b01681e90 tx=0x7f16c00097c0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 -- 172.21.15.69:0/268500971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16d019dad0 msgr2=0x7f16d019df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16d019dad0 0x7f16d019df80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f16c800d750 tx=0x7f16c800dc10 comp rx=0 tx=0).stop 2024-05-14T17:11:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 -- 172.21.15.69:0/268500971 shutdown_connections 2024-05-14T17:11:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f16b80b2a10 0x7f16b80b4ed0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16d019dad0 0x7f16d019df80 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d0107d80 0x7f16d01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 --2- 172.21.15.69:0/268500971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d01011c0 0x7f16d0199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 -- 172.21.15.69:0/268500971 >> 172.21.15.69:0/268500971 conn(0x7f16d0077e70 msgr2=0x7f16d00ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 -- 172.21.15.69:0/268500971 shutdown_connections 2024-05-14T17:11:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:13.951+0000 7f16d6008640 1 -- 172.21.15.69:0/268500971 wait complete. 2024-05-14T17:11:00.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:14 smithi069 ceph-mon[31507]: pgmap v644: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:11:00.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:14 smithi069 ceph-mon[31507]: osdmap e114: 12 total, 12 up, 12 in 2024-05-14T17:11:00.701 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:16 smithi069 ceph-mon[31507]: pgmap v646: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2024-05-14T17:11:00.709 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:18 smithi069 ceph-mon[31507]: pgmap v647: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-05-14T17:11:00.717 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:20 smithi069 ceph-mon[31507]: pgmap v648: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-05-14T17:11:00.726 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:22 smithi069 ceph-mon[31507]: pgmap v649: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-05-14T17:11:00.739 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:24 smithi069 ceph-mon[31507]: pgmap v650: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-05-14T17:11:00.755 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:26 smithi069 ceph-mon[31507]: pgmap v651: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-14T17:11:00.765 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:56 smithi092 ceph-mon[35763]: pgmap v667: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:11:00.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:10:58 smithi092 ceph-mon[35763]: pgmap v668: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-05-14T17:11:00.781 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:56 smithi203 ceph-mon[35787]: pgmap v667: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:11:00.789 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:10:58 smithi203 ceph-mon[35787]: pgmap v668: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-05-14T17:11:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.040+0000 7fe26655f640 1 -- 172.21.15.69:0/2904923512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe260107a30 msgr2=0x7fe260107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.040+0000 7fe26655f640 1 --2- 172.21.15.69:0/2904923512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe260107a30 0x7fe260107e10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe250009930 tx=0x7fe25002a420 comp rx=0 tx=0).stop 2024-05-14T17:11:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.040+0000 7fe26655f640 1 -- 172.21.15.69:0/2904923512 shutdown_connections 2024-05-14T17:11:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.040+0000 7fe26655f640 1 --2- 172.21.15.69:0/2904923512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe260101910 0x7fe26010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.040+0000 7fe26655f640 1 --2- 172.21.15.69:0/2904923512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe260100f50 0x7fe2601013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.040+0000 7fe26655f640 1 --2- 172.21.15.69:0/2904923512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe260107a30 0x7fe260107e10 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.040+0000 7fe26655f640 1 -- 172.21.15.69:0/2904923512 >> 172.21.15.69:0/2904923512 conn(0x7fe260077ee0 msgr2=0x7fe2600ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.040+0000 7fe26655f640 1 -- 172.21.15.69:0/2904923512 shutdown_connections 2024-05-14T17:11:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.040+0000 7fe26655f640 1 -- 172.21.15.69:0/2904923512 wait complete. 2024-05-14T17:11:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.041+0000 7fe26655f640 1 Processor -- start 2024-05-14T17:11:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.041+0000 7fe26655f640 1 -- start start 2024-05-14T17:11:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.041+0000 7fe26655f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe260100f50 0x7fe26019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.041+0000 7fe26655f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe260101910 0x7fe26019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.041+0000 7fe26655f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe260107a30 0x7fe26019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.041+0000 7fe26655f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe260112df0 con 0x7fe260101910 2024-05-14T17:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.041+0000 7fe26655f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe260112c70 con 0x7fe260100f50 2024-05-14T17:11:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.041+0000 7fe26655f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe260112f70 con 0x7fe260107a30 2024-05-14T17:11:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.041+0000 7fe25f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe260101910 0x7fe26019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.041+0000 7fe25f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe260101910 0x7fe26019b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44834/0 (socket says 172.21.15.69:44834) 2024-05-14T17:11:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe260100f50 0x7fe26019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25f7fe640 1 -- 172.21.15.69:0/2034633848 learned_addr learned my addr 172.21.15.69:0/2034633848 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe264afc640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe260107a30 0x7fe26019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25ffff640 1 -- 172.21.15.69:0/2034633848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe260107a30 msgr2=0x7fe26019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25ffff640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe260107a30 0x7fe26019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25ffff640 1 -- 172.21.15.69:0/2034633848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe260101910 msgr2=0x7fe26019b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25ffff640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe260101910 0x7fe26019b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25ffff640 1 -- 172.21.15.69:0/2034633848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2601a0440 con 0x7fe260100f50 2024-05-14T17:11:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe260101910 0x7fe26019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:01.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25ffff640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe260100f50 0x7fe26019b450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe250031440 tx=0x7fe250004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25d7fa640 1 -- 172.21.15.69:0/2034633848 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25003f070 con 0x7fe260100f50 2024-05-14T17:11:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe26655f640 1 -- 172.21.15.69:0/2034633848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2601a06d0 con 0x7fe260100f50 2024-05-14T17:11:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25d7fa640 1 -- 172.21.15.69:0/2034633848 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe250003aa0 con 0x7fe260100f50 2024-05-14T17:11:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe25d7fa640 1 -- 172.21.15.69:0/2034633848 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250035070 con 0x7fe260100f50 2024-05-14T17:11:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.042+0000 7fe26655f640 1 -- 172.21.15.69:0/2034633848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2601a7fc0 con 0x7fe260100f50 2024-05-14T17:11:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.045+0000 7fe25d7fa640 1 -- 172.21.15.69:0/2034633848 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe25003b460 con 0x7fe260100f50 2024-05-14T17:11:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.045+0000 7fe25d7fa640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe24c0b2a10 0x7fe24c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.045+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe24c0b2a10 0x7fe24c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.045+0000 7fe25d7fa640 1 -- 172.21.15.69:0/2034633848 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7fe250102a40 con 0x7fe260100f50 2024-05-14T17:11:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe26927c740 1 Processor -- start 2024-05-14T17:11:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe26927c740 1 -- start start 2024-05-14T17:11:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe24c0b2a10 0x7fe24c0b4ed0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe26010fc90 tx=0x7fe248005f90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe26927c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec04cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe26927c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 0x56549ec0f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe26927c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec0fad0 0x56549ec11ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe26927c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56549eb77c30 con 0x56549ec048f0 2024-05-14T17:11:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe26927c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56549eb677d0 con 0x56549ec05210 2024-05-14T17:11:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe26927c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56549eb6b810 con 0x56549ec0fad0 2024-05-14T17:11:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec04cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec04cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44840/0 (socket says 172.21.15.69:44840) 2024-05-14T17:11:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25ffff640 1 -- 172.21.15.69:0/2857184349 learned_addr learned my addr 172.21.15.69:0/2857184349 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/2857184349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 0x56549ec0f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe264afc640 1 --2- 172.21.15.69:0/2857184349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec0fad0 0x56549ec11ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25ffff640 1 -- 172.21.15.69:0/2857184349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec0fad0 msgr2=0x56549ec11ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25ffff640 1 --2- 172.21.15.69:0/2857184349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec0fad0 0x56549ec11ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25ffff640 1 -- 172.21.15.69:0/2857184349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 msgr2=0x56549ec0f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25ffff640 1 --2- 172.21.15.69:0/2857184349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 0x56549ec0f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25ffff640 1 -- 172.21.15.69:0/2857184349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549e87c210 con 0x56549ec048f0 2024-05-14T17:11:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.046+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/2857184349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 0x56549ec0f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.047+0000 7fe25ffff640 1 --2- 172.21.15.69:0/2857184349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec04cd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe2500c5e40 tx=0x7fe2500c98d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc050c398ecb856 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.047+0000 7fe2567fc640 1 -- 172.21.15.69:0/2857184349 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250048040 con 0x56549ec048f0 2024-05-14T17:11:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.047+0000 7fe2567fc640 1 -- 172.21.15.69:0/2857184349 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2500ce0c0 con 0x56549ec048f0 2024-05-14T17:11:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.047+0000 7fe2567fc640 1 -- 172.21.15.69:0/2857184349 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500d2940 con 0x56549ec048f0 2024-05-14T17:11:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.047+0000 7fe26927c740 1 -- 172.21.15.69:0/2857184349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 msgr2=0x56549ec04cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.047+0000 7fe26927c740 1 --2- 172.21.15.69:0/2857184349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec04cd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe2500c5e40 tx=0x7fe2500c98d0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.047+0000 7fe26927c740 1 -- 172.21.15.69:0/2857184349 shutdown_connections 2024-05-14T17:11:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.048+0000 7fe26927c740 1 --2- 172.21.15.69:0/2857184349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec0fad0 0x56549ec11ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.048+0000 7fe26927c740 1 --2- 172.21.15.69:0/2857184349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 0x56549ec0f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.048+0000 7fe26927c740 1 --2- 172.21.15.69:0/2857184349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec04cd0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.048+0000 7fe26927c740 1 -- 172.21.15.69:0/2857184349 >> 172.21.15.69:0/2857184349 conn(0x56549ec00e00 msgr2=0x56549ec011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.048+0000 7fe26927c740 1 -- 172.21.15.69:0/2857184349 shutdown_connections 2024-05-14T17:11:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.048+0000 7fe26927c740 1 -- 172.21.15.69:0/2857184349 wait complete. 2024-05-14T17:11:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.048+0000 7fe26927c740 1 -- start start 2024-05-14T17:11:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.048+0000 7fe26927c740 1 Processor -- start 2024-05-14T17:11:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe26927c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec048f0 0x56549ec0f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe26927c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 0x56549ec0c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe26927c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec0fad0 0x56549ec0cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe26927c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56549e84d790 con 0x56549ec0fad0 2024-05-14T17:11:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe26927c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56549eb23890 con 0x56549ec048f0 2024-05-14T17:11:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe26927c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56549ec124c0 con 0x56549ec05210 2024-05-14T17:11:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe26927c740 10 client.0 _ll_register_callbacks cb 0x56549eb26d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe264afc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec0fad0 0x56549ec0cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe25f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 0x56549ec0c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe25ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec048f0 0x56549ec0f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe264afc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec0fad0 0x56549ec0cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44844/0 (socket says 172.21.15.69:44844) 2024-05-14T17:11:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe25f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 0x56549ec0c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.69:35934/0 (socket says 172.21.15.69:35934) 2024-05-14T17:11:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe264afc640 1 -- 172.21.15.69:0/2759715605 learned_addr learned my addr 172.21.15.69:0/2759715605 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe25ffff640 1 -- 172.21.15.69:0/2759715605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 msgr2=0x56549ec0c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe25ffff640 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 0x56549ec0c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe25ffff640 1 -- 172.21.15.69:0/2759715605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec0fad0 msgr2=0x56549ec0cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe25ffff640 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec0fad0 0x56549ec0cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe25ffff640 1 -- 172.21.15.69:0/2759715605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549e948620 con 0x56549ec048f0 2024-05-14T17:11:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe264afc640 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec0fad0 0x56549ec0cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe25ffff640 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec048f0 0x56549ec0f400 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe2500c6360 tx=0x7fe250005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f76d21c0ddd554b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe23f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-14T17:11:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe23f7fe640 1 -- 172.21.15.69:0/2759715605 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250048040 con 0x56549ec048f0 2024-05-14T17:11:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe23f7fe640 1 -- 172.21.15.69:0/2759715605 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7fe250100400 con 0x56549ec048f0 2024-05-14T17:11:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.049+0000 7fe26927c740 1 -- 172.21.15.69:0/2759715605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2500ce0c0 con 0x56549ec048f0 2024-05-14T17:11:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe23f7fe640 1 -- 172.21.15.69:0/2759715605 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe2500e43f0 con 0x56549ec048f0 2024-05-14T17:11:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe26927c740 10 client.26738 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe26927c740 1 -- 172.21.15.69:0/2759715605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56549ec1df30 con 0x56549ec048f0 2024-05-14T17:11:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe23f7fe640 1 -- 172.21.15.69:0/2759715605 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2500ce0c0 con 0x56549ec048f0 2024-05-14T17:11:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe23f7fe640 1 -- 172.21.15.69:0/2759715605 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fe25004b060 con 0x56549ec048f0 2024-05-14T17:11:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe26927c740 10 client.26738 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe26927c740 10 client.26738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe26927c740 20 client.26738 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe26927c740 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56549ec1f280 0x56549ec21670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe26927c740 4 client.26738 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe26927c740 1 -- 172.21.15.69:0/2759715605 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56549eb79b80 con 0x56549ec1f280 2024-05-14T17:11:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.050+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56549ec1f280 0x56549ec21670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe23f7fe640 10 client.26738 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56549ec1f280 0x56549ec21670 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66544520b3bd4e1a server_cookie=6e5ba71a4d02969b in_seq=0 out_seq=0 2024-05-14T17:11:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe23f7fe640 1 -- 172.21.15.69:0/2759715605 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56549eb79b80 con 0x56549ec1f280 2024-05-14T17:11:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe23f7fe640 10 client.26738 handle_command_reply: tid=0 2024-05-14T17:11:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe26927c740 1 client.26738 shutdown 2024-05-14T17:11:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe26927c740 1 -- 172.21.15.69:0/2759715605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec048f0 msgr2=0x56549ec0f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe26927c740 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec048f0 0x56549ec0f400 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe2500c6360 tx=0x7fe250005bc0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe26927c740 1 -- 172.21.15.69:0/2759715605 shutdown_connections 2024-05-14T17:11:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe26927c740 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56549ec1f280 0x56549ec21670 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe26927c740 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec0fad0 0x56549ec0cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe23f7fe640 0 client.26738 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe26927c740 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 0x56549ec0c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe26927c740 1 --2- 172.21.15.69:0/2759715605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec048f0 0x56549ec0f400 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.051+0000 7fe26927c740 1 -- 172.21.15.69:0/2759715605 >> 172.21.15.69:0/2759715605 conn(0x56549ec00e00 msgr2=0x56549ec00100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.052+0000 7fe26927c740 1 -- 172.21.15.69:0/2759715605 shutdown_connections 2024-05-14T17:11:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.052+0000 7fe26927c740 1 -- 172.21.15.69:0/2759715605 wait complete. 2024-05-14T17:11:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.052+0000 7fe26927c740 20 client.26738 trim_cache size 0 max 16384 2024-05-14T17:11:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe26927c740 1 Processor -- start 2024-05-14T17:11:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe26927c740 1 -- start start 2024-05-14T17:11:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe26927c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec0ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe26927c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 0x56549ec069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe26927c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec0fad0 0x56549ec26f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe26927c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b22f0 con 0x56549ec048f0 2024-05-14T17:11:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe26927c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b29c0 con 0x56549ec05210 2024-05-14T17:11:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe26927c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b02d0 con 0x56549ec0fad0 2024-05-14T17:11:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe25ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec0ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe25ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec0ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44848/0 (socket says 172.21.15.69:44848) 2024-05-14T17:11:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe25ffff640 1 -- 172.21.15.69:0/1799255172 learned_addr learned my addr 172.21.15.69:0/1799255172 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe264afc640 1 --2- 172.21.15.69:0/1799255172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec0fad0 0x56549ec26f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.082+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/1799255172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 0x56549ec069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.083+0000 7fe25ffff640 1 -- 172.21.15.69:0/1799255172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec0fad0 msgr2=0x56549ec26f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.083+0000 7fe25ffff640 1 --2- 172.21.15.69:0/1799255172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec0fad0 0x56549ec26f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.083+0000 7fe25ffff640 1 -- 172.21.15.69:0/1799255172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 msgr2=0x56549ec069b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.083+0000 7fe25ffff640 1 --2- 172.21.15.69:0/1799255172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 0x56549ec069b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.083+0000 7fe25ffff640 1 -- 172.21.15.69:0/1799255172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549ec219e0 con 0x56549ec048f0 2024-05-14T17:11:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.083+0000 7fe25ffff640 1 --2- 172.21.15.69:0/1799255172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec0ef30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe2500c9f80 tx=0x7fe250005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d98b788543befa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.083+0000 7fe255ffb640 1 -- 172.21.15.69:0/1799255172 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500e43f0 con 0x56549ec048f0 2024-05-14T17:11:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.083+0000 7fe255ffb640 1 -- 172.21.15.69:0/1799255172 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2500ce0c0 con 0x56549ec048f0 2024-05-14T17:11:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.083+0000 7fe255ffb640 1 -- 172.21.15.69:0/1799255172 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500d26e0 con 0x56549ec048f0 2024-05-14T17:11:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 -- 172.21.15.69:0/1799255172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 msgr2=0x56549ec0ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 --2- 172.21.15.69:0/1799255172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec0ef30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe2500c9f80 tx=0x7fe250005b50 comp rx=0 tx=0).stop 2024-05-14T17:11:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 -- 172.21.15.69:0/1799255172 shutdown_connections 2024-05-14T17:11:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 --2- 172.21.15.69:0/1799255172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec0fad0 0x56549ec26f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 --2- 172.21.15.69:0/1799255172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec05210 0x56549ec069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 --2- 172.21.15.69:0/1799255172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec0ef30 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 -- 172.21.15.69:0/1799255172 >> 172.21.15.69:0/1799255172 conn(0x56549ec00e00 msgr2=0x56549ec062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 -- 172.21.15.69:0/1799255172 shutdown_connections 2024-05-14T17:11:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 -- 172.21.15.69:0/1799255172 wait complete. 2024-05-14T17:11:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 -- start start 2024-05-14T17:11:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.084+0000 7fe26927c740 1 Processor -- start 2024-05-14T17:11:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe26927c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec13450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe26927c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 0x56549ec13990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe26927c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec0fad0 0x56549ec13ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe26927c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601afc00 con 0x56549ec048f0 2024-05-14T17:11:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe26927c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601ae810 con 0x56549ec0fad0 2024-05-14T17:11:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe26927c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601aee60 con 0x56549ec05210 2024-05-14T17:11:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe26927c740 10 client.0 _ll_register_callbacks cb 0x7fe2500c97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec13450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec13450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44864/0 (socket says 172.21.15.69:44864) 2024-05-14T17:11:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25ffff640 1 -- 172.21.15.69:0/3470462631 learned_addr learned my addr 172.21.15.69:0/3470462631 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 0x56549ec13990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe264afc640 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec0fad0 0x56549ec13ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25ffff640 1 -- 172.21.15.69:0/3470462631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 msgr2=0x56549ec13990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25ffff640 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 0x56549ec13990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25ffff640 1 -- 172.21.15.69:0/3470462631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec0fad0 msgr2=0x56549ec13ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25ffff640 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec0fad0 0x56549ec13ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25ffff640 1 -- 172.21.15.69:0/3470462631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549ec88c20 con 0x56549ec048f0 2024-05-14T17:11:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe264afc640 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec0fad0 0x56549ec13ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 0x56549ec13990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe25ffff640 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec13450 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe250100780 tx=0x7fe2500c9230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52041483cbe73f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe23f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe23f7fe640 1 -- 172.21.15.69:0/3470462631 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500e43f0 con 0x56549ec048f0 2024-05-14T17:11:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe23f7fe640 1 -- 172.21.15.69:0/3470462631 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7fe2500d26e0 con 0x56549ec048f0 2024-05-14T17:11:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe26927c740 1 -- 172.21.15.69:0/3470462631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2500ce0c0 con 0x56549ec048f0 2024-05-14T17:11:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.085+0000 7fe23f7fe640 1 -- 172.21.15.69:0/3470462631 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe2500d2b70 con 0x56549ec048f0 2024-05-14T17:11:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe26927c740 10 client.17463 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe26927c740 1 -- 172.21.15.69:0/3470462631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56549ec88c20 con 0x56549ec048f0 2024-05-14T17:11:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe23f7fe640 1 -- 172.21.15.69:0/3470462631 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2500ce0c0 con 0x56549ec048f0 2024-05-14T17:11:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe23f7fe640 1 -- 172.21.15.69:0/3470462631 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fe2500c8a90 con 0x56549ec048f0 2024-05-14T17:11:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe26927c740 10 client.17463 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe26927c740 10 client.17463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe26927c740 20 client.17463 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe26927c740 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56549ec8a160 0x56549ec8c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe26927c740 4 client.17463 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe26927c740 1 -- 172.21.15.69:0/3470462631 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56549eb73360 con 0x56549ec8a160 2024-05-14T17:11:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56549ec8a160 0x56549ec8c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe23f7fe640 10 client.17463 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.086+0000 7fe25f7fe640 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56549ec8a160 0x56549ec8c550 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe0ad245f8be70f4 server_cookie=7a5925cb7c1dd511 in_seq=0 out_seq=0 2024-05-14T17:11:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe23f7fe640 1 -- 172.21.15.69:0/3470462631 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56549eb73360 con 0x56549ec8a160 2024-05-14T17:11:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe23f7fe640 10 client.17463 handle_command_reply: tid=0 2024-05-14T17:11:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 client.17463 shutdown 2024-05-14T17:11:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 -- 172.21.15.69:0/3470462631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 msgr2=0x56549ec13450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec13450 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe250100780 tx=0x7fe2500c9230 comp rx=0 tx=0).stop 2024-05-14T17:11:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 -- 172.21.15.69:0/3470462631 shutdown_connections 2024-05-14T17:11:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x56549ec8a160 0x56549ec8c550 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56549ec0fad0 0x56549ec13ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549ec05210 0x56549ec13990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe23f7fe640 0 client.17463 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 --2- 172.21.15.69:0/3470462631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56549ec048f0 0x56549ec13450 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 -- 172.21.15.69:0/3470462631 >> 172.21.15.69:0/3470462631 conn(0x56549ec00e00 msgr2=0x56549ec00100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 -- 172.21.15.69:0/3470462631 shutdown_connections 2024-05-14T17:11:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 1 -- 172.21.15.69:0/3470462631 wait complete. 2024-05-14T17:11:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.094+0000 7fe26927c740 20 client.17463 trim_cache size 0 max 16384 2024-05-14T17:11:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:11:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:11:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:11:03.151 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:00 smithi092 ceph-mon[35763]: pgmap v669: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:11:03.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:00 smithi203 ceph-mon[35787]: pgmap v669: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:11:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 -- 172.21.15.69:0/2034633848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe24c0b2a10 msgr2=0x7fe24c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe24c0b2a10 0x7fe24c0b4ed0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe26010fc90 tx=0x7fe248005f90 comp rx=0 tx=0).stop 2024-05-14T17:11:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 -- 172.21.15.69:0/2034633848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe260100f50 msgr2=0x7fe26019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe260100f50 0x7fe26019b450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe250031440 tx=0x7fe250004270 comp rx=0 tx=0).stop 2024-05-14T17:11:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 -- 172.21.15.69:0/2034633848 shutdown_connections 2024-05-14T17:11:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fe24c0b2a10 0x7fe24c0b4ed0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe260107a30 0x7fe26019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe260101910 0x7fe26019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 --2- 172.21.15.69:0/2034633848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe260100f50 0x7fe26019b450 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 -- 172.21.15.69:0/2034633848 >> 172.21.15.69:0/2034633848 conn(0x7fe260077ee0 msgr2=0x7fe260105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 -- 172.21.15.69:0/2034633848 shutdown_connections 2024-05-14T17:11:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:27.095+0000 7fe26655f640 1 -- 172.21.15.69:0/2034633848 wait complete. 2024-05-14T17:11:03.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:27 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:03.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:27 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.299 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:27 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:27 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.329 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:27 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:28 smithi069 ceph-mon[31507]: pgmap v652: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-05-14T17:11:03.350 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.362 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.378 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.386 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.396 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: pgmap v653: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-14T17:11:03.406 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.417 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.425 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.433 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.463 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-05-14T17:11:03.472 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-05-14T17:11:03.483 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.519 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.547 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.556 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.567 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:11:03.577 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:03.586 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:03.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:03.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:30 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-14T17:11:03.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:30 smithi069 ceph-mon[31507]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-14T17:11:03.646 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:30 smithi069 ceph-mon[31507]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-14T17:11:03.653 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:30 smithi069 ceph-mon[31507]: Unable to set osd_memory_target on smithi069 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-14T17:11:03.659 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:30 smithi069 ceph-mon[31507]: pgmap v654: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:11:03.666 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:30 smithi069 ceph-mon[31507]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-14T17:11:03.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:32 smithi069 ceph-mon[31507]: pgmap v655: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T17:11:03.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:34 smithi069 ceph-mon[31507]: pgmap v656: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:11:03.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:36 smithi069 ceph-mon[31507]: pgmap v657: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:11:03.717 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:38 smithi069 ceph-mon[31507]: pgmap v658: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:11:03.727 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:02 smithi092 ceph-mon[35763]: pgmap v670: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-05-14T17:11:03.736 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:02 smithi203 ceph-mon[35787]: pgmap v670: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-05-14T17:11:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 -- 172.21.15.69:0/4170102477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8198101980 msgr2=0x7f8198101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 --2- 172.21.15.69:0/4170102477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8198101980 0x7f8198101e00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8180009930 tx=0x7f818002a420 comp rx=0 tx=0).stop 2024-05-14T17:11:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 -- 172.21.15.69:0/4170102477 shutdown_connections 2024-05-14T17:11:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 --2- 172.21.15.69:0/4170102477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8198102340 0x7f819810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 --2- 172.21.15.69:0/4170102477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8198101980 0x7f8198101e00 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 --2- 172.21.15.69:0/4170102477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8198107940 0x7f8198107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 -- 172.21.15.69:0/4170102477 >> 172.21.15.69:0/4170102477 conn(0x7f8198078ad0 msgr2=0x7f8198078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 -- 172.21.15.69:0/4170102477 shutdown_connections 2024-05-14T17:11:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 -- 172.21.15.69:0/4170102477 wait complete. 2024-05-14T17:11:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 Processor -- start 2024-05-14T17:11:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.191+0000 7f819f0f1640 1 -- start start 2024-05-14T17:11:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819f0f1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8198101980 0x7f8198199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819f0f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8198102340 0x7f8198199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819f0f1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8198107940 0x7f819819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819f0f1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8198111bd0 con 0x7f8198101980 2024-05-14T17:11:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819f0f1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8198111a50 con 0x7f8198102340 2024-05-14T17:11:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819f0f1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8198111d50 con 0x7f8198107940 2024-05-14T17:11:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819ce8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8198101980 0x7f8198199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819ce8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8198101980 0x7f8198199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54052/0 (socket says 172.21.15.69:54052) 2024-05-14T17:11:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819ce8d640 1 -- 172.21.15.69:0/972971534 learned_addr learned my addr 172.21.15.69:0/972971534 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819d68e640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8198107940 0x7f819819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f8197fff640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8198102340 0x7f8198199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819ce8d640 1 -- 172.21.15.69:0/972971534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8198107940 msgr2=0x7f819819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819ce8d640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8198107940 0x7f819819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819ce8d640 1 -- 172.21.15.69:0/972971534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8198102340 msgr2=0x7f8198199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819ce8d640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8198102340 0x7f8198199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819ce8d640 1 -- 172.21.15.69:0/972971534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819819e0d0 con 0x7f8198101980 2024-05-14T17:11:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819ce8d640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8198101980 0x7f8198199130 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f818800cce0 tx=0x7f818800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f8195ffb640 1 -- 172.21.15.69:0/972971534 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818800f650 con 0x7f8198101980 2024-05-14T17:11:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f819f0f1640 1 -- 172.21.15.69:0/972971534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819819e3c0 con 0x7f8198101980 2024-05-14T17:11:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f8195ffb640 1 -- 172.21.15.69:0/972971534 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818800fca0 con 0x7f8198101980 2024-05-14T17:11:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.192+0000 7f8195ffb640 1 -- 172.21.15.69:0/972971534 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188018b40 con 0x7f8198101980 2024-05-14T17:11:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.193+0000 7f819f0f1640 1 -- 172.21.15.69:0/972971534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81981a5c50 con 0x7f8198101980 2024-05-14T17:11:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.195+0000 7f8195ffb640 1 -- 172.21.15.69:0/972971534 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f818800f7f0 con 0x7f8198101980 2024-05-14T17:11:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.195+0000 7f81a1e0b740 1 Processor -- start 2024-05-14T17:11:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.195+0000 7f81a1e0b740 1 -- start start 2024-05-14T17:11:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 0x56281848acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818495590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 0x562818497ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5628183fdc30 con 0x56281848b210 2024-05-14T17:11:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5628183ed7d0 con 0x56281848a8f0 2024-05-14T17:11:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5628183f1810 con 0x562818495ad0 2024-05-14T17:11:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819ce8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818495590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819ce8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818495590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54062/0 (socket says 172.21.15.69:54062) 2024-05-14T17:11:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f8197fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 0x562818497ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819ce8d640 1 -- 172.21.15.69:0/1418050394 learned_addr learned my addr 172.21.15.69:0/1418050394 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f8195ffb640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81840b2a10 0x7f81840b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819d68e640 1 --2- 172.21.15.69:0/1418050394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 0x56281848acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819d68e640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81840b2a10 0x7f81840b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f8195ffb640 1 -- 172.21.15.69:0/972971534 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f81880e2a70 con 0x7f8198101980 2024-05-14T17:11:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819ce8d640 1 -- 172.21.15.69:0/1418050394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 msgr2=0x562818497ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819ce8d640 1 --2- 172.21.15.69:0/1418050394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 0x562818497ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819ce8d640 1 -- 172.21.15.69:0/1418050394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 msgr2=0x56281848acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819ce8d640 1 --2- 172.21.15.69:0/1418050394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 0x56281848acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819ce8d640 1 -- 172.21.15.69:0/1418050394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562818102210 con 0x56281848b210 2024-05-14T17:11:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819d68e640 1 --2- 172.21.15.69:0/1418050394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 0x56281848acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f8197fff640 1 --2- 172.21.15.69:0/1418050394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 0x562818497ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.196+0000 7f819ce8d640 1 --2- 172.21.15.69:0/1418050394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818495590 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f81880a9a40 tx=0x7f8188007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eccc96b92459300 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f819d68e640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81840b2a10 0x7f81840b4ed0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f81900058c0 tx=0x7f819000a250 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f818effd640 1 -- 172.21.15.69:0/1418050394 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81880a94b0 con 0x56281848b210 2024-05-14T17:11:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f818effd640 1 -- 172.21.15.69:0/1418050394 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818800fae0 con 0x56281848b210 2024-05-14T17:11:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f818effd640 1 -- 172.21.15.69:0/1418050394 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81880ae0f0 con 0x56281848b210 2024-05-14T17:11:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1418050394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 msgr2=0x562818495590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/1418050394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818495590 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f81880a9a40 tx=0x7f8188007940 comp rx=0 tx=0).stop 2024-05-14T17:11:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1418050394 shutdown_connections 2024-05-14T17:11:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/1418050394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 0x562818497ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/1418050394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818495590 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/1418050394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 0x56281848acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.197+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1418050394 >> 172.21.15.69:0/1418050394 conn(0x562818486e00 msgr2=0x5628184871e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1418050394 shutdown_connections 2024-05-14T17:11:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1418050394 wait complete. 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 1 -- start start 2024-05-14T17:11:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 1 Processor -- start 2024-05-14T17:11:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 0x562818495520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818492810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562818495ad0 0x562818492f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5628180d3790 con 0x56281848b210 2024-05-14T17:11:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5628183a9890 con 0x562818495ad0 2024-05-14T17:11:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5628184986e0 con 0x56281848a8f0 2024-05-14T17:11:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f81a1e0b740 10 client.0 _ll_register_callbacks cb 0x5628183acd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f8197fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818492810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f8197fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818492810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54068/0 (socket says 172.21.15.69:54068) 2024-05-14T17:11:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.198+0000 7f8197fff640 1 -- 172.21.15.69:0/721453937 learned_addr learned my addr 172.21.15.69:0/721453937 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f819d68e640 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562818495ad0 0x562818492f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f819ce8d640 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 0x562818495520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8197fff640 1 -- 172.21.15.69:0/721453937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 msgr2=0x562818495520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8197fff640 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 0x562818495520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8197fff640 1 -- 172.21.15.69:0/721453937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562818495ad0 msgr2=0x562818492f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8197fff640 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562818495ad0 0x562818492f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8197fff640 1 -- 172.21.15.69:0/721453937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628181ce620 con 0x56281848b210 2024-05-14T17:11:04.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f819ce8d640 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 0x562818495520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8197fff640 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818492810 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f818002a940 tx=0x7f8180031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bba16a99d7e42a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8177fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8177fff640 1 -- 172.21.15.69:0/721453937 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180035040 con 0x56281848b210 2024-05-14T17:11:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8177fff640 1 -- 172.21.15.69:0/721453937 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f8180004300 con 0x56281848b210 2024-05-14T17:11:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f81a1e0b740 1 -- 172.21.15.69:0/721453937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5628183ffb80 con 0x56281848b210 2024-05-14T17:11:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8177fff640 1 -- 172.21.15.69:0/721453937 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8180005020 con 0x56281848b210 2024-05-14T17:11:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f81a1e0b740 10 client.17475 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f81a1e0b740 1 -- 172.21.15.69:0/721453937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5628184a41e0 con 0x56281848b210 2024-05-14T17:11:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.199+0000 7f8177fff640 1 -- 172.21.15.69:0/721453937 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5628183ffb80 con 0x56281848b210 2024-05-14T17:11:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.200+0000 7f8177fff640 1 -- 172.21.15.69:0/721453937 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f81800035f0 con 0x56281848b210 2024-05-14T17:11:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.200+0000 7f81a1e0b740 10 client.17475 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.200+0000 7f81a1e0b740 10 client.17475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.200+0000 7f81a1e0b740 20 client.17475 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.200+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628184a56a0 0x5628184a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.200+0000 7f81a1e0b740 4 client.17475 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.200+0000 7f81a1e0b740 1 -- 172.21.15.69:0/721453937 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628183f9360 con 0x5628184a56a0 2024-05-14T17:11:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.200+0000 7f819ce8d640 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628184a56a0 0x5628184a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.200+0000 7f8177fff640 10 client.17475 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.200+0000 7f819ce8d640 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628184a56a0 0x5628184a7ad0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bbd7a119a3df451 server_cookie=fdabe09cc7a31ab9 in_seq=0 out_seq=0 2024-05-14T17:11:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f8177fff640 1 -- 172.21.15.69:0/721453937 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628183f9360 con 0x5628184a56a0 2024-05-14T17:11:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f8177fff640 10 client.17475 handle_command_reply: tid=0 2024-05-14T17:11:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 client.17475 shutdown 2024-05-14T17:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 -- 172.21.15.69:0/721453937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 msgr2=0x562818492810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818492810 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f818002a940 tx=0x7f8180031950 comp rx=0 tx=0).stop 2024-05-14T17:11:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 -- 172.21.15.69:0/721453937 shutdown_connections 2024-05-14T17:11:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x5628184a56a0 0x5628184a7ad0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562818495ad0 0x562818492f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818492810 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f8177fff640 0 client.17475 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/721453937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 0x562818495520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 -- 172.21.15.69:0/721453937 >> 172.21.15.69:0/721453937 conn(0x562818486e00 msgr2=0x562818486100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 -- 172.21.15.69:0/721453937 shutdown_connections 2024-05-14T17:11:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 1 -- 172.21.15.69:0/721453937 wait complete. 2024-05-14T17:11:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.201+0000 7f81a1e0b740 20 client.17475 trim_cache size 0 max 16384 2024-05-14T17:11:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.216+0000 7f81a1e0b740 1 Processor -- start 2024-05-14T17:11:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.216+0000 7f81a1e0b740 1 -- start start 2024-05-14T17:11:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.216+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 0x56281848dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.216+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848b210 0x562818495050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.216+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562818495ad0 0x562818495590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.216+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81981b0060 con 0x562818495ad0 2024-05-14T17:11:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.216+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81981b0730 con 0x56281848b210 2024-05-14T17:11:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.216+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81981ae040 con 0x56281848a8f0 2024-05-14T17:11:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f819d68e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562818495ad0 0x562818495590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f819d68e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562818495ad0 0x562818495590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54082/0 (socket says 172.21.15.69:54082) 2024-05-14T17:11:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f819d68e640 1 -- 172.21.15.69:0/9770110 learned_addr learned my addr 172.21.15.69:0/9770110 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f8197fff640 1 --2- 172.21.15.69:0/9770110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848b210 0x562818495050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f819ce8d640 1 --2- 172.21.15.69:0/9770110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 0x56281848dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f819d68e640 1 -- 172.21.15.69:0/9770110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 msgr2=0x56281848dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f819d68e640 1 --2- 172.21.15.69:0/9770110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 0x56281848dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f819d68e640 1 -- 172.21.15.69:0/9770110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848b210 msgr2=0x562818495050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f819d68e640 1 --2- 172.21.15.69:0/9770110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848b210 0x562818495050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f819d68e640 1 -- 172.21.15.69:0/9770110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56281848b000 con 0x562818495ad0 2024-05-14T17:11:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f8197fff640 1 --2- 172.21.15.69:0/9770110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848b210 0x562818495050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f819d68e640 1 --2- 172.21.15.69:0/9770110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562818495ad0 0x562818495590 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f819000f6e0 tx=0x7f8190016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c271b2926341ac24 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f818e7fc640 1 -- 172.21.15.69:0/9770110 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190017030 con 0x562818495ad0 2024-05-14T17:11:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f818e7fc640 1 -- 172.21.15.69:0/9770110 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819000fe10 con 0x562818495ad0 2024-05-14T17:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.217+0000 7f818e7fc640 1 -- 172.21.15.69:0/9770110 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819001c690 con 0x562818495ad0 2024-05-14T17:11:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 -- 172.21.15.69:0/9770110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562818495ad0 msgr2=0x562818495590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/9770110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562818495ad0 0x562818495590 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f819000f6e0 tx=0x7f8190016000 comp rx=0 tx=0).stop 2024-05-14T17:11:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 -- 172.21.15.69:0/9770110 shutdown_connections 2024-05-14T17:11:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/9770110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562818495ad0 0x562818495590 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/9770110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848b210 0x562818495050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/9770110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56281848a8f0 0x56281848dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 -- 172.21.15.69:0/9770110 >> 172.21.15.69:0/9770110 conn(0x562818486e00 msgr2=0x56281848d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 -- 172.21.15.69:0/9770110 shutdown_connections 2024-05-14T17:11:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 -- 172.21.15.69:0/9770110 wait complete. 2024-05-14T17:11:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 -- start start 2024-05-14T17:11:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.218+0000 7f81a1e0b740 1 Processor -- start 2024-05-14T17:11:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 0x5628184995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818499b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f81a1e0b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 0x56281849a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81981ad970 con 0x56281848b210 2024-05-14T17:11:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81981ac580 con 0x56281848a8f0 2024-05-14T17:11:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f81a1e0b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81981acbd0 con 0x562818495ad0 2024-05-14T17:11:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f81a1e0b740 10 client.0 _ll_register_callbacks cb 0x7f81800315e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8197fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818499b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8197fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818499b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54086/0 (socket says 172.21.15.69:54086) 2024-05-14T17:11:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8197fff640 1 -- 172.21.15.69:0/1703073968 learned_addr learned my addr 172.21.15.69:0/1703073968 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f819ce8d640 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 0x5628184995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f819d68e640 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 0x56281849a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8197fff640 1 -- 172.21.15.69:0/1703073968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 msgr2=0x56281849a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8197fff640 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 0x56281849a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8197fff640 1 -- 172.21.15.69:0/1703073968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 msgr2=0x5628184995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8197fff640 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 0x5628184995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8197fff640 1 -- 172.21.15.69:0/1703073968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628185134a0 con 0x56281848b210 2024-05-14T17:11:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f819d68e640 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 0x56281849a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f819ce8d640 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 0x5628184995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8197fff640 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818499b20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f8180009850 tx=0x7f8180003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dfc8112213e152 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8177fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8177fff640 1 -- 172.21.15.69:0/1703073968 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180052070 con 0x56281848b210 2024-05-14T17:11:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8177fff640 1 -- 172.21.15.69:0/1703073968 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f8180004300 con 0x56281848b210 2024-05-14T17:11:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1703073968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f819000fe10 con 0x56281848b210 2024-05-14T17:11:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f8177fff640 1 -- 172.21.15.69:0/1703073968 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f818005e050 con 0x56281848b210 2024-05-14T17:11:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f81a1e0b740 10 client.17481 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.219+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1703073968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562818514750 con 0x56281848b210 2024-05-14T17:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f8177fff640 1 -- 172.21.15.69:0/1703073968 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819000fe10 con 0x56281848b210 2024-05-14T17:11:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f8177fff640 1 -- 172.21.15.69:0/1703073968 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f8180004b60 con 0x56281848b210 2024-05-14T17:11:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f81a1e0b740 10 client.17481 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f81a1e0b740 10 client.17481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f81a1e0b740 20 client.17481 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562818514f00 0x562818517320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f81a1e0b740 4 client.17481 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1703073968 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628183735f0 con 0x562818514f00 2024-05-14T17:11:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f819ce8d640 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562818514f00 0x562818517320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f8177fff640 10 client.17481 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.220+0000 7f819ce8d640 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562818514f00 0x562818517320 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb6cbff953c0eb8 server_cookie=4575027f741b4533 in_seq=0 out_seq=0 2024-05-14T17:11:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f8177fff640 1 -- 172.21.15.69:0/1703073968 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5628183735f0 con 0x562818514f00 2024-05-14T17:11:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f8177fff640 10 client.17481 handle_command_reply: tid=0 2024-05-14T17:11:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 client.17481 shutdown 2024-05-14T17:11:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1703073968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 msgr2=0x562818499b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818499b20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f8180009850 tx=0x7f8180003fd0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1703073968 shutdown_connections 2024-05-14T17:11:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562818514f00 0x562818517320 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562818495ad0 0x56281849a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f8177fff640 0 client.17481 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56281848b210 0x562818499b20 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 --2- 172.21.15.69:0/1703073968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56281848a8f0 0x5628184995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1703073968 >> 172.21.15.69:0/1703073968 conn(0x562818486e00 msgr2=0x56281848dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1703073968 shutdown_connections 2024-05-14T17:11:06.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 1 -- 172.21.15.69:0/1703073968 wait complete. 2024-05-14T17:11:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.226+0000 7f81a1e0b740 20 client.17481 trim_cache size 0 max 16384 2024-05-14T17:11:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:11:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:11:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:11:06.356 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:04 smithi092 ceph-mon[35763]: pgmap v671: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-05-14T17:11:06.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:05 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2383868156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:06.390 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:04 smithi203 ceph-mon[35787]: pgmap v671: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-05-14T17:11:06.396 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:05 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2383868156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 -- 172.21.15.69:0/972971534 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81840b2a10 msgr2=0x7f81840b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81840b2a10 0x7f81840b4ed0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f81900058c0 tx=0x7f819000a250 comp rx=0 tx=0).stop 2024-05-14T17:11:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 -- 172.21.15.69:0/972971534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8198101980 msgr2=0x7f8198199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8198101980 0x7f8198199130 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f818800cce0 tx=0x7f818800a620 comp rx=0 tx=0).stop 2024-05-14T17:11:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 -- 172.21.15.69:0/972971534 shutdown_connections 2024-05-14T17:11:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f81840b2a10 0x7f81840b4ed0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8198107940 0x7f819819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8198102340 0x7f8198199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 --2- 172.21.15.69:0/972971534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8198101980 0x7f8198199130 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 -- 172.21.15.69:0/972971534 >> 172.21.15.69:0/972971534 conn(0x7f8198078ad0 msgr2=0x7f8198100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 -- 172.21.15.69:0/972971534 shutdown_connections 2024-05-14T17:11:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:40.227+0000 7f819f0f1640 1 -- 172.21.15.69:0/972971534 wait complete. 2024-05-14T17:11:06.495 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:40 smithi069 ceph-mon[31507]: pgmap v659: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:11:06.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:42 smithi069 ceph-mon[31507]: pgmap v660: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:11:06.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:44 smithi069 ceph-mon[31507]: pgmap v661: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:11:06.550 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:11:06.560 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:46 smithi069 ceph-mon[31507]: pgmap v662: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:11:06.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.883+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1724360476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a5c1022d0 msgr2=0x7f0a5c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:06.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.883+0000 7f0a62bb9700 1 --2- 172.21.15.69:0/1724360476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a5c1022d0 0x7f0a5c109320 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0a480099c0 tx=0x7f0a48009cd0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.884+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1724360476 shutdown_connections 2024-05-14T17:11:06.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.884+0000 7f0a62bb9700 1 --2- 172.21.15.69:0/1724360476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a5c109a00 0x7f0a5c110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.884+0000 7f0a62bb9700 1 --2- 172.21.15.69:0/1724360476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5c101920 0x7f0a5c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.884+0000 7f0a62bb9700 1 --2- 172.21.15.69:0/1724360476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a5c1022d0 0x7f0a5c109320 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.884+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1724360476 >> 172.21.15.69:0/1724360476 conn(0x7f0a5c078640 msgr2=0x7f0a5c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:06.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.884+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1724360476 shutdown_connections 2024-05-14T17:11:06.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.884+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1724360476 wait complete. 2024-05-14T17:11:06.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.884+0000 7f0a62bb9700 1 Processor -- start 2024-05-14T17:11:06.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.884+0000 7f0a62bb9700 1 -- start start 2024-05-14T17:11:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.884+0000 7f0a62bb9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5c101920 0x7f0a5c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:06.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a62bb9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a5c1022d0 0x7f0a5c19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:06.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a62bb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a5c109a00 0x7f0a5c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:06.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a62bb9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c113750 con 0x7f0a5c1022d0 2024-05-14T17:11:06.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a62bb9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1135d0 con 0x7f0a5c109a00 2024-05-14T17:11:06.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a62bb9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1138d0 con 0x7f0a5c101920 2024-05-14T17:11:06.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a5bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a5c1022d0 0x7f0a5c19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:06.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a5bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a5c1022d0 0x7f0a5c19fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43444/0 (socket says 172.21.15.69:43444) 2024-05-14T17:11:06.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a5bfff700 1 -- 172.21.15.69:0/1057222616 learned_addr learned my addr 172.21.15.69:0/1057222616 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:06.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a60955700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5c101920 0x7f0a5c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:06.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a61156700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a5c109a00 0x7f0a5c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:06.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a5bfff700 1 -- 172.21.15.69:0/1057222616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5c101920 msgr2=0x7f0a5c19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:06.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a5bfff700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5c101920 0x7f0a5c19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a5bfff700 1 -- 172.21.15.69:0/1057222616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a5c109a00 msgr2=0x7f0a5c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:06.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a5bfff700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a5c109a00 0x7f0a5c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a5bfff700 1 -- 172.21.15.69:0/1057222616 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c19a1d0 con 0x7f0a5c1022d0 2024-05-14T17:11:06.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a5bfff700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a5c1022d0 0x7f0a5c19fdf0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0a48006010 tx=0x7f0a480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:06.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a59ffb700 1 -- 172.21.15.69:0/1057222616 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48027070 con 0x7f0a5c1022d0 2024-05-14T17:11:06.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1057222616 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5c19a460 con 0x7f0a5c1022d0 2024-05-14T17:11:06.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a59ffb700 1 -- 172.21.15.69:0/1057222616 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a48003f20 con 0x7f0a5c1022d0 2024-05-14T17:11:06.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.885+0000 7f0a59ffb700 1 -- 172.21.15.69:0/1057222616 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48022cd0 con 0x7f0a5c1022d0 2024-05-14T17:11:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.886+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1057222616 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5c1a6780 con 0x7f0a5c1022d0 2024-05-14T17:11:06.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.887+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1057222616 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a5c102a50 con 0x7f0a5c1022d0 2024-05-14T17:11:06.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.888+0000 7f0a59ffb700 1 -- 172.21.15.69:0/1057222616 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0a4802a070 con 0x7f0a5c1022d0 2024-05-14T17:11:06.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.889+0000 7f0a59ffb700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a4c0b29f0 0x7f0a4c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:06.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.889+0000 7f0a60955700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a4c0b29f0 0x7f0a4c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:06.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.889+0000 7f0a59ffb700 1 -- 172.21.15.69:0/1057222616 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f0a480e3460 con 0x7f0a5c1022d0 2024-05-14T17:11:06.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.889+0000 7f0a60955700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a4c0b29f0 0x7f0a4c0b4eb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c19abb0 tx=0x7f0a50009190 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:06.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:48.893+0000 7f0a59ffb700 1 -- 172.21.15.69:0/1057222616 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a48013070 con 0x7f0a5c1022d0 2024-05-14T17:11:06.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.148+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1057222616 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a5c19afa0 con 0x7f0a5c1022d0 2024-05-14T17:11:07.007 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:48 smithi069 ceph-mon[31507]: pgmap v663: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-05-14T17:11:07.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.149+0000 7f0a59ffb700 1 -- 172.21.15.69:0/1057222616 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11031 (secure 0 0 0) 0x7f0a480aaec0 con 0x7f0a5c1022d0 2024-05-14T17:11:07.056 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:11:07.067 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:08:18.843541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24385":{"gid":24385,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T17:11:07.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1057222616 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a4c0b29f0 msgr2=0x7f0a4c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:07.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a4c0b29f0 0x7f0a4c0b4eb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c19abb0 tx=0x7f0a50009190 comp rx=0 tx=0).stop 2024-05-14T17:11:07.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1057222616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a5c1022d0 msgr2=0x7f0a5c19fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:07.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a5c1022d0 0x7f0a5c19fdf0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0a48006010 tx=0x7f0a480047b0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1057222616 shutdown_connections 2024-05-14T17:11:07.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a5c109a00 0x7f0a5c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5c101920 0x7f0a5c19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f0a4c0b29f0 0x7f0a4c0b4eb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 --2- 172.21.15.69:0/1057222616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a5c1022d0 0x7f0a5c19fdf0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1057222616 >> 172.21.15.69:0/1057222616 conn(0x7f0a5c078640 msgr2=0x7f0a5c1002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:07.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1057222616 shutdown_connections 2024-05-14T17:11:07.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:10:49.151+0000 7f0a62bb9700 1 -- 172.21.15.69:0/1057222616 wait complete. 2024-05-14T17:11:07.231 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 61 2024-05-14T17:11:07.241 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --cancel --all' 2024-05-14T17:11:07.249 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db --cancel --all 2024-05-14T17:11:07.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:49 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1057222616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:07.269 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:50 smithi069 ceph-mon[31507]: pgmap v664: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:11:07.279 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:52 smithi069 ceph-mon[31507]: pgmap v665: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-05-14T17:11:07.289 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:06 smithi092 ceph-mon[35763]: pgmap v672: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-05-14T17:11:07.297 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:06 smithi203 ceph-mon[35787]: pgmap v672: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-05-14T17:11:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.325+0000 7f06cdca0640 1 -- 172.21.15.69:0/136771012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c8101980 msgr2=0x7f06c8101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.325+0000 7f06cdca0640 1 --2- 172.21.15.69:0/136771012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c8101980 0x7f06c8101e00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009930 tx=0x7f06b002a420 comp rx=0 tx=0).stop 2024-05-14T17:11:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.325+0000 7f06cdca0640 1 -- 172.21.15.69:0/136771012 shutdown_connections 2024-05-14T17:11:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.325+0000 7f06cdca0640 1 --2- 172.21.15.69:0/136771012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06c8102340 0x7f06c810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.325+0000 7f06cdca0640 1 --2- 172.21.15.69:0/136771012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c8101980 0x7f06c8101e00 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.325+0000 7f06cdca0640 1 --2- 172.21.15.69:0/136771012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c8107940 0x7f06c8107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.325+0000 7f06cdca0640 1 -- 172.21.15.69:0/136771012 >> 172.21.15.69:0/136771012 conn(0x7f06c8078ad0 msgr2=0x7f06c8078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.325+0000 7f06cdca0640 1 -- 172.21.15.69:0/136771012 shutdown_connections 2024-05-14T17:11:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.325+0000 7f06cdca0640 1 -- 172.21.15.69:0/136771012 wait complete. 2024-05-14T17:11:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06cdca0640 1 Processor -- start 2024-05-14T17:11:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06cdca0640 1 -- start start 2024-05-14T17:11:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06cdca0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06c8101980 0x7f06c80fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06cdca0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c8102340 0x7f06c80fee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06cdca0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c8107940 0x7f06c80ff390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06cdca0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8111c00 con 0x7f06c8102340 2024-05-14T17:11:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06cdca0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8111a80 con 0x7f06c8101980 2024-05-14T17:11:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06cdca0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8111d80 con 0x7f06c8107940 2024-05-14T17:11:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c8102340 0x7f06c80fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c8102340 0x7f06c80fee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43452/0 (socket says 172.21.15.69:43452) 2024-05-14T17:11:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06beffd640 1 -- 172.21.15.69:0/167482495 learned_addr learned my addr 172.21.15.69:0/167482495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.326+0000 7f06c77fe640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06c8101980 0x7f06c80fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06c7fff640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c8107940 0x7f06c80ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06beffd640 1 -- 172.21.15.69:0/167482495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c8107940 msgr2=0x7f06c80ff390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06beffd640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c8107940 0x7f06c80ff390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06beffd640 1 -- 172.21.15.69:0/167482495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06c8101980 msgr2=0x7f06c80fe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06beffd640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06c8101980 0x7f06c80fe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06beffd640 1 -- 172.21.15.69:0/167482495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c81a7770 con 0x7f06c8102340 2024-05-14T17:11:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06beffd640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c8102340 0x7f06c80fee50 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f06b0004770 tx=0x7f06b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06c57fa640 1 -- 172.21.15.69:0/167482495 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b003f070 con 0x7f06c8102340 2024-05-14T17:11:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06c57fa640 1 -- 172.21.15.69:0/167482495 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b003b460 con 0x7f06c8102340 2024-05-14T17:11:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06cdca0640 1 -- 172.21.15.69:0/167482495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c81a7a00 con 0x7f06c8102340 2024-05-14T17:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06c57fa640 1 -- 172.21.15.69:0/167482495 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0035070 con 0x7f06c8102340 2024-05-14T17:11:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.327+0000 7f06cdca0640 1 -- 172.21.15.69:0/167482495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c81a7ef0 con 0x7f06c8102340 2024-05-14T17:11:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06c57fa640 1 -- 172.21.15.69:0/167482495 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f06b003b8d0 con 0x7f06c8102340 2024-05-14T17:11:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06d09c4740 1 Processor -- start 2024-05-14T17:11:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06d09c4740 1 -- start start 2024-05-14T17:11:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea85f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8600590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 0x562ea8602ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06d09c4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562ea8568c30 con 0x562ea85f6210 2024-05-14T17:11:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06d09c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562ea85587d0 con 0x562ea85f58f0 2024-05-14T17:11:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06d09c4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562ea855c810 con 0x562ea8600ad0 2024-05-14T17:11:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06c77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8600590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06c77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8600590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43460/0 (socket says 172.21.15.69:43460) 2024-05-14T17:11:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.330+0000 7f06c77fe640 1 -- 172.21.15.69:0/253577899 learned_addr learned my addr 172.21.15.69:0/253577899 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06c7fff640 1 --2- 172.21.15.69:0/253577899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea85f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:07.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06beffd640 1 --2- 172.21.15.69:0/253577899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 0x562ea8602ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:07.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06c77fe640 1 -- 172.21.15.69:0/253577899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 msgr2=0x562ea8602ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06c77fe640 1 --2- 172.21.15.69:0/253577899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 0x562ea8602ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06c77fe640 1 -- 172.21.15.69:0/253577899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 msgr2=0x562ea85f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06c77fe640 1 --2- 172.21.15.69:0/253577899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea85f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06c77fe640 1 -- 172.21.15.69:0/253577899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea826d210 con 0x562ea85f6210 2024-05-14T17:11:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06beffd640 1 --2- 172.21.15.69:0/253577899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 0x562ea8602ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06c57fa640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06b40b2a10 0x7f06b40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06c77fe640 1 --2- 172.21.15.69:0/253577899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8600590 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f06b800bc50 tx=0x7f06b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf4b1800b93a394 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06c7fff640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06b40b2a10 0x7f06b40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06c57fa640 1 -- 172.21.15.69:0/167482495 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f06b0102a00 con 0x7f06c8102340 2024-05-14T17:11:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06bdffb640 1 -- 172.21.15.69:0/253577899 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b801c480 con 0x562ea85f6210 2024-05-14T17:11:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06bdffb640 1 -- 172.21.15.69:0/253577899 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b801cad0 con 0x562ea85f6210 2024-05-14T17:11:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.331+0000 7f06bdffb640 1 -- 172.21.15.69:0/253577899 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8011b90 con 0x562ea85f6210 2024-05-14T17:11:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06d09c4740 1 -- 172.21.15.69:0/253577899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 msgr2=0x562ea8600590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06d09c4740 1 --2- 172.21.15.69:0/253577899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8600590 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f06b800bc50 tx=0x7f06b80107a0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06c7fff640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06b40b2a10 0x7f06b40b4ed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f06c0005fd0 tx=0x7f06c0009740 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06d09c4740 1 -- 172.21.15.69:0/253577899 shutdown_connections 2024-05-14T17:11:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06d09c4740 1 --2- 172.21.15.69:0/253577899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 0x562ea8602ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06d09c4740 1 --2- 172.21.15.69:0/253577899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8600590 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06d09c4740 1 --2- 172.21.15.69:0/253577899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea85f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06d09c4740 1 -- 172.21.15.69:0/253577899 >> 172.21.15.69:0/253577899 conn(0x562ea85f1e00 msgr2=0x562ea85f21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06d09c4740 1 -- 172.21.15.69:0/253577899 shutdown_connections 2024-05-14T17:11:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06d09c4740 1 -- 172.21.15.69:0/253577899 wait complete. 2024-05-14T17:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.332+0000 7f06d09c4740 1 -- start start 2024-05-14T17:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06d09c4740 1 Processor -- start 2024-05-14T17:11:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea8600440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea85f6210 0x562ea85fd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea8600ad0 0x562ea85fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06d09c4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562ea823e790 con 0x562ea8600ad0 2024-05-14T17:11:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06d09c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562ea8514890 con 0x562ea85f58f0 2024-05-14T17:11:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06d09c4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562ea86037a0 con 0x562ea85f6210 2024-05-14T17:11:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06d09c4740 10 client.0 _ll_register_callbacks cb 0x562ea8517d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06c7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea8600ad0 0x562ea85fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06c7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea8600ad0 0x562ea85fdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43474/0 (socket says 172.21.15.69:43474) 2024-05-14T17:11:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06c7fff640 1 -- 172.21.15.69:0/638678790 learned_addr learned my addr 172.21.15.69:0/638678790 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06c77fe640 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea8600440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06beffd640 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea85f6210 0x562ea85fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06c7fff640 1 -- 172.21.15.69:0/638678790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea85f6210 msgr2=0x562ea85fd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06c7fff640 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea85f6210 0x562ea85fd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06c7fff640 1 -- 172.21.15.69:0/638678790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 msgr2=0x562ea8600440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06c7fff640 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea8600440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06c7fff640 1 -- 172.21.15.69:0/638678790 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea8339620 con 0x562ea8600ad0 2024-05-14T17:11:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06c7fff640 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea8600ad0 0x562ea85fdc90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f06c001e970 tx=0x7f06c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1de835f478e9bbf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.333+0000 7f06a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06a6ffd640 1 -- 172.21.15.69:0/638678790 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0010d60 con 0x562ea8600ad0 2024-05-14T17:11:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06a6ffd640 1 -- 172.21.15.69:0/638678790 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f06c001f420 con 0x562ea8600ad0 2024-05-14T17:11:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06d09c4740 1 -- 172.21.15.69:0/638678790 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b801cad0 con 0x562ea8600ad0 2024-05-14T17:11:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06a6ffd640 1 -- 172.21.15.69:0/638678790 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f06c001f620 con 0x562ea8600ad0 2024-05-14T17:11:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06d09c4740 10 client.17499 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06d09c4740 1 -- 172.21.15.69:0/638678790 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562ea860f190 con 0x562ea8600ad0 2024-05-14T17:11:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06a6ffd640 1 -- 172.21.15.69:0/638678790 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b801cad0 con 0x562ea8600ad0 2024-05-14T17:11:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06a6ffd640 1 -- 172.21.15.69:0/638678790 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f06c000f040 con 0x562ea8600ad0 2024-05-14T17:11:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06d09c4740 10 client.17499 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06d09c4740 10 client.17499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06d09c4740 20 client.17499 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06d09c4740 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ea86106a0 0x562ea8612a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06d09c4740 4 client.17499 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.334+0000 7f06d09c4740 1 -- 172.21.15.69:0/638678790 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ea856ab80 con 0x562ea86106a0 2024-05-14T17:11:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.335+0000 7f06c77fe640 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ea86106a0 0x562ea8612a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.335+0000 7f06a6ffd640 10 client.17499 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.335+0000 7f06c77fe640 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ea86106a0 0x562ea8612a90 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a2643d283dec69 server_cookie=8cec0eb707c641fe in_seq=0 out_seq=0 2024-05-14T17:11:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06a6ffd640 1 -- 172.21.15.69:0/638678790 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562ea856ab80 con 0x562ea86106a0 2024-05-14T17:11:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06a6ffd640 10 client.17499 handle_command_reply: tid=0 2024-05-14T17:11:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 client.17499 shutdown 2024-05-14T17:11:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 -- 172.21.15.69:0/638678790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea8600ad0 msgr2=0x562ea85fdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea8600ad0 0x562ea85fdc90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f06c001e970 tx=0x7f06c0004a40 comp rx=0 tx=0).stop 2024-05-14T17:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 -- 172.21.15.69:0/638678790 shutdown_connections 2024-05-14T17:11:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ea86106a0 0x562ea8612a90 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea8600ad0 0x562ea85fdc90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea85f6210 0x562ea85fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 --2- 172.21.15.69:0/638678790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea8600440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06a6ffd640 0 client.17499 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 -- 172.21.15.69:0/638678790 >> 172.21.15.69:0/638678790 conn(0x562ea85f1e00 msgr2=0x562ea85f1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 -- 172.21.15.69:0/638678790 shutdown_connections 2024-05-14T17:11:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 1 -- 172.21.15.69:0/638678790 wait complete. 2024-05-14T17:11:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.336+0000 7f06d09c4740 20 client.17499 trim_cache size 0 max 16384 2024-05-14T17:11:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.351+0000 7f06d09c4740 1 Processor -- start 2024-05-14T17:11:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.351+0000 7f06d09c4740 1 -- start start 2024-05-14T17:11:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.351+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea85f58f0 0x562ea860c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.351+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8618380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.351+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea8600ad0 0x562ea86188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.351+0000 7f06d09c4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b2410 con 0x562ea85f6210 2024-05-14T17:11:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.351+0000 7f06d09c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b2ae0 con 0x562ea8600ad0 2024-05-14T17:11:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.351+0000 7f06d09c4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b03f0 con 0x562ea85f58f0 2024-05-14T17:11:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8618380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8618380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43490/0 (socket says 172.21.15.69:43490) 2024-05-14T17:11:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06beffd640 1 -- 172.21.15.69:0/2879841607 learned_addr learned my addr 172.21.15.69:0/2879841607 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06c7fff640 1 --2- 172.21.15.69:0/2879841607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea8600ad0 0x562ea86188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06c77fe640 1 --2- 172.21.15.69:0/2879841607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea85f58f0 0x562ea860c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06beffd640 1 -- 172.21.15.69:0/2879841607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea85f58f0 msgr2=0x562ea860c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06beffd640 1 --2- 172.21.15.69:0/2879841607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea85f58f0 0x562ea860c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06beffd640 1 -- 172.21.15.69:0/2879841607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea8600ad0 msgr2=0x562ea86188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06beffd640 1 --2- 172.21.15.69:0/2879841607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea8600ad0 0x562ea86188c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06beffd640 1 -- 172.21.15.69:0/2879841607 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea85fc6e0 con 0x562ea85f6210 2024-05-14T17:11:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06c7fff640 1 --2- 172.21.15.69:0/2879841607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea8600ad0 0x562ea86188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06beffd640 1 --2- 172.21.15.69:0/2879841607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8618380 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005c20 tx=0x7f06b002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68248a24cd69cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06bd7fa640 1 -- 172.21.15.69:0/2879841607 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0048040 con 0x562ea85f6210 2024-05-14T17:11:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06bd7fa640 1 -- 172.21.15.69:0/2879841607 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b00d2410 con 0x562ea85f6210 2024-05-14T17:11:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.352+0000 7f06bd7fa640 1 -- 172.21.15.69:0/2879841607 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b00d2ad0 con 0x562ea85f6210 2024-05-14T17:11:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 -- 172.21.15.69:0/2879841607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 msgr2=0x562ea8618380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 --2- 172.21.15.69:0/2879841607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8618380 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005c20 tx=0x7f06b002a420 comp rx=0 tx=0).stop 2024-05-14T17:11:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 -- 172.21.15.69:0/2879841607 shutdown_connections 2024-05-14T17:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 --2- 172.21.15.69:0/2879841607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea8600ad0 0x562ea86188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 --2- 172.21.15.69:0/2879841607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8618380 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 --2- 172.21.15.69:0/2879841607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea85f58f0 0x562ea860c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 -- 172.21.15.69:0/2879841607 >> 172.21.15.69:0/2879841607 conn(0x562ea85f1e00 msgr2=0x562ea85f1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 -- 172.21.15.69:0/2879841607 shutdown_connections 2024-05-14T17:11:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 -- 172.21.15.69:0/2879841607 wait complete. 2024-05-14T17:11:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 -- start start 2024-05-14T17:11:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 Processor -- start 2024-05-14T17:11:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea8606c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.353+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8607160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06d09c4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 0x562ea8679f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06d09c4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81afd20 con 0x562ea85f6210 2024-05-14T17:11:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06d09c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81ae930 con 0x562ea85f58f0 2024-05-14T17:11:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06d09c4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81aef80 con 0x562ea8600ad0 2024-05-14T17:11:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06d09c4740 10 client.0 _ll_register_callbacks cb 0x7f06c001e520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8607160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8607160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43502/0 (socket says 172.21.15.69:43502) 2024-05-14T17:11:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06beffd640 1 -- 172.21.15.69:0/2479277106 learned_addr learned my addr 172.21.15.69:0/2479277106 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06c7fff640 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 0x562ea8679f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06c77fe640 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea8606c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06beffd640 1 -- 172.21.15.69:0/2479277106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 msgr2=0x562ea8679f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06beffd640 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 0x562ea8679f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06beffd640 1 -- 172.21.15.69:0/2479277106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 msgr2=0x562ea8606c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06beffd640 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea8606c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06beffd640 1 -- 172.21.15.69:0/2479277106 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea85f6040 con 0x562ea85f6210 2024-05-14T17:11:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06beffd640 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8607160 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005940 tx=0x7f06b003b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b59659666d527c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06a6ffd640 1 -- 172.21.15.69:0/2479277106 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b00e9060 con 0x562ea85f6210 2024-05-14T17:11:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06a6ffd640 1 -- 172.21.15.69:0/2479277106 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f06b0100360 con 0x562ea85f6210 2024-05-14T17:11:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06d09c4740 1 -- 172.21.15.69:0/2479277106 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ea8564360 con 0x562ea85f6210 2024-05-14T17:11:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06a6ffd640 1 -- 172.21.15.69:0/2479277106 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f06b0048070 con 0x562ea85f6210 2024-05-14T17:11:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06d09c4740 10 client.17505 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06d09c4740 1 -- 172.21.15.69:0/2479277106 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562ea867b4f0 con 0x562ea85f6210 2024-05-14T17:11:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.354+0000 7f06a6ffd640 1 -- 172.21.15.69:0/2479277106 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562ea8564360 con 0x562ea85f6210 2024-05-14T17:11:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.355+0000 7f06a6ffd640 1 -- 172.21.15.69:0/2479277106 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f06b004b060 con 0x562ea85f6210 2024-05-14T17:11:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.355+0000 7f06d09c4740 10 client.17505 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.355+0000 7f06d09c4740 10 client.17505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.355+0000 7f06d09c4740 20 client.17505 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.355+0000 7f06d09c4740 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ea867bc70 0x562ea867e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.355+0000 7f06d09c4740 4 client.17505 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.355+0000 7f06d09c4740 1 -- 172.21.15.69:0/2479277106 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ea84de5f0 con 0x562ea867bc70 2024-05-14T17:11:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.355+0000 7f06c77fe640 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ea867bc70 0x562ea867e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.355+0000 7f06a6ffd640 10 client.17505 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.355+0000 7f06c77fe640 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ea867bc70 0x562ea867e090 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3818f410fc5674a server_cookie=efca8ce4f6a9b14c in_seq=0 out_seq=0 2024-05-14T17:11:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06a6ffd640 1 -- 172.21.15.69:0/2479277106 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562ea84de5f0 con 0x562ea867bc70 2024-05-14T17:11:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06a6ffd640 10 client.17505 handle_command_reply: tid=0 2024-05-14T17:11:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 client.17505 shutdown 2024-05-14T17:11:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 -- 172.21.15.69:0/2479277106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 msgr2=0x562ea8607160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8607160 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005940 tx=0x7f06b003b840 comp rx=0 tx=0).stop 2024-05-14T17:11:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 -- 172.21.15.69:0/2479277106 shutdown_connections 2024-05-14T17:11:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x562ea867bc70 0x562ea867e090 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ea8600ad0 0x562ea8679f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ea85f6210 0x562ea8607160 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06a6ffd640 0 client.17505 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 --2- 172.21.15.69:0/2479277106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ea85f58f0 0x562ea8606c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 -- 172.21.15.69:0/2479277106 >> 172.21.15.69:0/2479277106 conn(0x562ea85f1e00 msgr2=0x562ea860b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 -- 172.21.15.69:0/2479277106 shutdown_connections 2024-05-14T17:11:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 1 -- 172.21.15.69:0/2479277106 wait complete. 2024-05-14T17:11:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06d09c4740 20 client.17505 trim_cache size 0 max 16384 2024-05-14T17:11:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:11:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:11:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:11:09.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:08 smithi092 ceph-mon[35763]: pgmap v673: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-14T17:11:09.880 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:08 smithi203 ceph-mon[35787]: pgmap v673: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-14T17:11:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06cdca0640 1 -- 172.21.15.69:0/167482495 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06b40b2a10 msgr2=0x7f06b40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.361+0000 7f06cdca0640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06b40b2a10 0x7f06b40b4ed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f06c0005fd0 tx=0x7f06c0009740 comp rx=0 tx=0).stop 2024-05-14T17:11:09.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.362+0000 7f06cdca0640 1 -- 172.21.15.69:0/167482495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c8102340 msgr2=0x7f06c80fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.362+0000 7f06cdca0640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c8102340 0x7f06c80fee50 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f06b0004770 tx=0x7f06b00042f0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.362+0000 7f06cdca0640 1 -- 172.21.15.69:0/167482495 shutdown_connections 2024-05-14T17:11:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.362+0000 7f06cdca0640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f06b40b2a10 0x7f06b40b4ed0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.362+0000 7f06cdca0640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c8107940 0x7f06c80ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.362+0000 7f06cdca0640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c8102340 0x7f06c80fee50 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.362+0000 7f06cdca0640 1 --2- 172.21.15.69:0/167482495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06c8101980 0x7f06c80fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.362+0000 7f06cdca0640 1 -- 172.21.15.69:0/167482495 >> 172.21.15.69:0/167482495 conn(0x7f06c8078ad0 msgr2=0x7f06c806a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.362+0000 7f06cdca0640 1 -- 172.21.15.69:0/167482495 shutdown_connections 2024-05-14T17:11:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:10:53.362+0000 7f06cdca0640 1 -- 172.21.15.69:0/167482495 wait complete. 2024-05-14T17:11:10.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:54 smithi069 ceph-mon[31507]: pgmap v666: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:11:10.049 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:56 smithi069 ceph-mon[31507]: pgmap v667: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:11:10.060 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:10:58 smithi069 ceph-mon[31507]: pgmap v668: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 7 op/s 2024-05-14T17:11:10.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:00 smithi069 ceph-mon[31507]: pgmap v669: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 KiB/s wr, 5 op/s 2024-05-14T17:11:10.133 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:02 smithi069 ceph-mon[31507]: pgmap v670: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-05-14T17:11:10.144 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:11:10.155 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:04 smithi069 ceph-mon[31507]: pgmap v671: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-05-14T17:11:10.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.346+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2188353050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b806c1f0 msgr2=0x7f55b806c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:10.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.346+0000 7f55bd3ca700 1 --2- 172.21.15.69:0/2188353050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b806c1f0 0x7f55b806c670 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009990 tx=0x7f55a0009ca0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.346+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2188353050 shutdown_connections 2024-05-14T17:11:10.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.346+0000 7f55bd3ca700 1 --2- 172.21.15.69:0/2188353050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55b8105fb0 0x7f55b8106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.346+0000 7f55bd3ca700 1 --2- 172.21.15.69:0/2188353050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55b8108490 0x7f55b806bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.346+0000 7f55bd3ca700 1 --2- 172.21.15.69:0/2188353050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b806c1f0 0x7f55b806c670 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.346+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2188353050 >> 172.21.15.69:0/2188353050 conn(0x7f55b80784b0 msgr2=0x7f55b80788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:10.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.346+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2188353050 shutdown_connections 2024-05-14T17:11:10.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.346+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2188353050 wait complete. 2024-05-14T17:11:10.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55bd3ca700 1 Processor -- start 2024-05-14T17:11:10.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55bd3ca700 1 -- start start 2024-05-14T17:11:10.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55bd3ca700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b806c1f0 0x7f55b81a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:10.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55bd3ca700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55b8105fb0 0x7f55b81a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:10.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55bd3ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55b8108490 0x7f55b81a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:10.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55bd3ca700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8119cb0 con 0x7f55b806c1f0 2024-05-14T17:11:10.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55bd3ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8119b30 con 0x7f55b8108490 2024-05-14T17:11:10.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55bd3ca700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8119e30 con 0x7f55b8105fb0 2024-05-14T17:11:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55b6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b806c1f0 0x7f55b81a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:10.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55b6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b806c1f0 0x7f55b81a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58410/0 (socket says 172.21.15.69:58410) 2024-05-14T17:11:10.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55b6ffd700 1 -- 172.21.15.69:0/2383868156 learned_addr learned my addr 172.21.15.69:0/2383868156 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:10.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55b77fe700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55b8108490 0x7f55b81a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:10.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55b6ffd700 1 -- 172.21.15.69:0/2383868156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55b8105fb0 msgr2=0x7f55b81a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:10.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55affff700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55b8105fb0 0x7f55b81a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:10.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55b6ffd700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55b8105fb0 0x7f55b81a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55b6ffd700 1 -- 172.21.15.69:0/2383868156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55b8108490 msgr2=0x7f55b81a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:10.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.347+0000 7f55b6ffd700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55b8108490 0x7f55b81a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.348+0000 7f55b6ffd700 1 -- 172.21.15.69:0/2383868156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b81a52d0 con 0x7f55b806c1f0 2024-05-14T17:11:10.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.348+0000 7f55b6ffd700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b806c1f0 0x7f55b81a02c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f55a800d980 tx=0x7f55a800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:10.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.348+0000 7f55b4ff9700 1 -- 172.21.15.69:0/2383868156 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8016070 con 0x7f55b806c1f0 2024-05-14T17:11:10.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.348+0000 7f55b4ff9700 1 -- 172.21.15.69:0/2383868156 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a80044d0 con 0x7f55b806c1f0 2024-05-14T17:11:10.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.348+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2383868156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b81a55c0 con 0x7f55b806c1f0 2024-05-14T17:11:10.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.348+0000 7f55b4ff9700 1 -- 172.21.15.69:0/2383868156 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8004d00 con 0x7f55b806c1f0 2024-05-14T17:11:10.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.348+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2383868156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b81ace40 con 0x7f55b806c1f0 2024-05-14T17:11:10.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.349+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2383868156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b8102a90 con 0x7f55b806c1f0 2024-05-14T17:11:10.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.351+0000 7f55b4ff9700 1 -- 172.21.15.69:0/2383868156 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f55a8004020 con 0x7f55b806c1f0 2024-05-14T17:11:10.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.352+0000 7f55b4ff9700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f55a40b29f0 0x7f55a40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:10.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.352+0000 7f55affff700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f55a40b29f0 0x7f55a40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:10.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.352+0000 7f55b4ff9700 1 -- 172.21.15.69:0/2383868156 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f55a80e29b0 con 0x7f55b806c1f0 2024-05-14T17:11:10.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.352+0000 7f55affff700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f55a40b29f0 0x7f55a40b4eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55a000b5c0 tx=0x7f55a0005fb0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:10.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.355+0000 7f55b4ff9700 1 -- 172.21.15.69:0/2383868156 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f55a801f020 con 0x7f55b806c1f0 2024-05-14T17:11:10.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.610+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2383868156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55b806a230 con 0x7f55b806c1f0 2024-05-14T17:11:10.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.610+0000 7f55b4ff9700 1 -- 172.21.15.69:0/2383868156 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11031 (secure 0 0 0) 0x7f55a80aa380 con 0x7f55b806c1f0 2024-05-14T17:11:10.675 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:11:10.684 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:08:18.843541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24385":{"gid":24385,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T17:11:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2383868156 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f55a40b29f0 msgr2=0x7f55a40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:10.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f55a40b29f0 0x7f55a40b4eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55a000b5c0 tx=0x7f55a0005fb0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2383868156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b806c1f0 msgr2=0x7f55b81a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:10.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b806c1f0 0x7f55b81a02c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f55a800d980 tx=0x7f55a800dd40 comp rx=0 tx=0).stop 2024-05-14T17:11:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2383868156 shutdown_connections 2024-05-14T17:11:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f55a40b29f0 0x7f55a40b4eb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55b806c1f0 0x7f55b81a02c0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55b8108490 0x7f55b81a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 --2- 172.21.15.69:0/2383868156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55b8105fb0 0x7f55b81a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2383868156 >> 172.21.15.69:0/2383868156 conn(0x7f55b80784b0 msgr2=0x7f55b8104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:10.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2383868156 shutdown_connections 2024-05-14T17:11:10.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:05.613+0000 7f55bd3ca700 1 -- 172.21.15.69:0/2383868156 wait complete. 2024-05-14T17:11:10.817 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 61 2024-05-14T17:11:10.872 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:05 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2383868156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:10.880 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:09 smithi092 ceph-mon[35763]: osdmap e115: 12 total, 12 up, 12 in 2024-05-14T17:11:10.886 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:09 smithi203 ceph-mon[35787]: osdmap e115: 12 total, 12 up, 12 in 2024-05-14T17:11:10.894 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db / --timeout 90 --expiration 138.4 --await-for 45.0' 2024-05-14T17:11:10.902 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24446 quiesce db / --timeout 90 --expiration 138.4 --await-for 45.0 2024-05-14T17:11:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 -- 172.21.15.69:0/998238363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72541025f0 msgr2=0x7f725410e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 --2- 172.21.15.69:0/998238363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72541025f0 0x7f725410e7a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f724c009ca0 tx=0x7f724c02a8c0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 -- 172.21.15.69:0/998238363 shutdown_connections 2024-05-14T17:11:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 --2- 172.21.15.69:0/998238363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72541025f0 0x7f725410e7a0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 --2- 172.21.15.69:0/998238363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7254101c30 0x7f72541020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 --2- 172.21.15.69:0/998238363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7254106920 0x7f7254106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 -- 172.21.15.69:0/998238363 >> 172.21.15.69:0/998238363 conn(0x7f7254078730 msgr2=0x7f7254078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 -- 172.21.15.69:0/998238363 shutdown_connections 2024-05-14T17:11:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 -- 172.21.15.69:0/998238363 wait complete. 2024-05-14T17:11:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 Processor -- start 2024-05-14T17:11:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 -- start start 2024-05-14T17:11:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7254101c30 0x7f7254199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.455+0000 7f725bca7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72541025f0 0x7f725419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f725bca7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7254106920 0x7f72541a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f725bca7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7254111bc0 con 0x7f72541025f0 2024-05-14T17:11:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f725bca7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7254111a40 con 0x7f7254101c30 2024-05-14T17:11:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f725bca7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7254111d40 con 0x7f7254106920 2024-05-14T17:11:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f7259242640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72541025f0 0x7f725419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f7259242640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72541025f0 0x7f725419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58434/0 (socket says 172.21.15.69:58434) 2024-05-14T17:11:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f7259242640 1 -- 172.21.15.69:0/1316021537 learned_addr learned my addr 172.21.15.69:0/1316021537 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f725a244640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7254106920 0x7f72541a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f7259a43640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7254101c30 0x7f7254199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f7259242640 1 -- 172.21.15.69:0/1316021537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7254106920 msgr2=0x7f72541a4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f7259242640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7254106920 0x7f72541a4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f7259242640 1 -- 172.21.15.69:0/1316021537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7254101c30 msgr2=0x7f7254199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f7259242640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7254101c30 0x7f7254199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f7259242640 1 -- 172.21.15.69:0/1316021537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72541a5440 con 0x7f72541025f0 2024-05-14T17:11:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f7259242640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72541025f0 0x7f725419a3c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f725000c920 tx=0x7f725000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f724affd640 1 -- 172.21.15.69:0/1316021537 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250007c70 con 0x7f72541025f0 2024-05-14T17:11:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f724affd640 1 -- 172.21.15.69:0/1316021537 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7250007e10 con 0x7f72541025f0 2024-05-14T17:11:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f725bca7640 1 -- 172.21.15.69:0/1316021537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72541a5610 con 0x7f72541025f0 2024-05-14T17:11:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f724affd640 1 -- 172.21.15.69:0/1316021537 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250018820 con 0x7f72541025f0 2024-05-14T17:11:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.456+0000 7f725bca7640 1 -- 172.21.15.69:0/1316021537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72541a5b30 con 0x7f72541025f0 2024-05-14T17:11:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.459+0000 7f724affd640 1 -- 172.21.15.69:0/1316021537 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f725000f450 con 0x7f72541025f0 2024-05-14T17:11:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.459+0000 7f725e9c9740 1 Processor -- start 2024-05-14T17:11:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.459+0000 7f725e9c9740 1 -- start start 2024-05-14T17:11:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.459+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 0x55e156b96bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.459+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b97110 0x55e156ba59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.459+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156ba5ee0 0x55e156ba82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.459+0000 7f725e9c9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e156b09c30 con 0x55e156b97110 2024-05-14T17:11:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f725e9c9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e156af97d0 con 0x55e156ba5ee0 2024-05-14T17:11:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f725e9c9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e156afd810 con 0x55e156b967f0 2024-05-14T17:11:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f7259a43640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b97110 0x55e156ba59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f7259a43640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b97110 0x55e156ba59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58444/0 (socket says 172.21.15.69:58444) 2024-05-14T17:11:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f7259a43640 1 -- 172.21.15.69:0/2358026677 learned_addr learned my addr 172.21.15.69:0/2358026677 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f7259242640 1 --2- 172.21.15.69:0/2358026677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156ba5ee0 0x55e156ba82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f725a244640 1 --2- 172.21.15.69:0/2358026677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 0x55e156b96bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f7259a43640 1 -- 172.21.15.69:0/2358026677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 msgr2=0x55e156b96bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f724affd640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f72400b2a10 0x7f72400b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f7259a43640 1 --2- 172.21.15.69:0/2358026677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 0x55e156b96bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f7259a43640 1 -- 172.21.15.69:0/2358026677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156ba5ee0 msgr2=0x55e156ba82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f7259a43640 1 --2- 172.21.15.69:0/2358026677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156ba5ee0 0x55e156ba82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f7259a43640 1 -- 172.21.15.69:0/2358026677 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e15680e210 con 0x55e156b97110 2024-05-14T17:11:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f725a244640 1 --2- 172.21.15.69:0/2358026677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 0x55e156b96bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f725a244640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f72400b2a10 0x7f72400b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f7259a43640 1 --2- 172.21.15.69:0/2358026677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b97110 0x55e156ba59a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f724400bc60 tx=0x7f724400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198fa6a065caa7c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f724affd640 1 -- 172.21.15.69:0/1316021537 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f72500e2790 con 0x7f72541025f0 2024-05-14T17:11:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f723ffff640 1 -- 172.21.15.69:0/2358026677 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244015070 con 0x55e156b97110 2024-05-14T17:11:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f723ffff640 1 -- 172.21.15.69:0/2358026677 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72440118f0 con 0x55e156b97110 2024-05-14T17:11:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.460+0000 7f723ffff640 1 -- 172.21.15.69:0/2358026677 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72440107d0 con 0x55e156b97110 2024-05-14T17:11:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.461+0000 7f725e9c9740 1 -- 172.21.15.69:0/2358026677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b97110 msgr2=0x55e156ba59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.461+0000 7f725a244640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f72400b2a10 0x7f72400b4ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f724c006010 tx=0x7f724c03f040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.461+0000 7f725e9c9740 1 --2- 172.21.15.69:0/2358026677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b97110 0x55e156ba59a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f724400bc60 tx=0x7f724400f730 comp rx=0 tx=0).stop 2024-05-14T17:11:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.461+0000 7f725e9c9740 1 -- 172.21.15.69:0/2358026677 shutdown_connections 2024-05-14T17:11:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.461+0000 7f725e9c9740 1 --2- 172.21.15.69:0/2358026677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156ba5ee0 0x55e156ba82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.461+0000 7f725e9c9740 1 --2- 172.21.15.69:0/2358026677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b97110 0x55e156ba59a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.461+0000 7f725e9c9740 1 --2- 172.21.15.69:0/2358026677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 0x55e156b96bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.461+0000 7f725e9c9740 1 -- 172.21.15.69:0/2358026677 >> 172.21.15.69:0/2358026677 conn(0x55e156b92e00 msgr2=0x55e156b931e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.461+0000 7f725e9c9740 1 -- 172.21.15.69:0/2358026677 shutdown_connections 2024-05-14T17:11:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.461+0000 7f725e9c9740 1 -- 172.21.15.69:0/2358026677 wait complete. 2024-05-14T17:11:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f725e9c9740 1 -- start start 2024-05-14T17:11:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f725e9c9740 1 Processor -- start 2024-05-14T17:11:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 0x55e156ba2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156ba5ee0 0x55e156ba31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f725e9c9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e1567df790 con 0x55e156b967f0 2024-05-14T17:11:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f725e9c9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e156ab5890 con 0x55e156b97110 2024-05-14T17:11:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f725e9c9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e156ba8bb0 con 0x55e156ba5ee0 2024-05-14T17:11:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f725e9c9740 10 client.0 _ll_register_callbacks cb 0x55e156ab8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f7259a43640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f7259a43640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58452/0 (socket says 172.21.15.69:58452) 2024-05-14T17:11:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f7259a43640 1 -- 172.21.15.69:0/3666635071 learned_addr learned my addr 172.21.15.69:0/3666635071 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f7259a43640 1 -- 172.21.15.69:0/3666635071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156ba5ee0 msgr2=0x55e156ba31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f725a244640 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156ba5ee0 0x55e156ba31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f7259242640 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 0x55e156ba2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f7259a43640 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156ba5ee0 0x55e156ba31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.462+0000 7f7259a43640 1 -- 172.21.15.69:0/3666635071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 msgr2=0x55e156ba2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f7259a43640 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 0x55e156ba2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f7259a43640 1 -- 172.21.15.69:0/3666635071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1568da620 con 0x55e156b967f0 2024-05-14T17:11:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f725a244640 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156ba5ee0 0x55e156ba31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f7259242640 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 0x55e156ba2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f7259a43640 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba58e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f724400bb80 tx=0x7f724400b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec23db049b787b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f723d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f723d7fa640 1 -- 172.21.15.69:0/3666635071 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244028060 con 0x55e156b967f0 2024-05-14T17:11:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f723d7fa640 1 -- 172.21.15.69:0/3666635071 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f7244011aa0 con 0x55e156b967f0 2024-05-14T17:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f725e9c9740 1 -- 172.21.15.69:0/3666635071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72440118f0 con 0x55e156b967f0 2024-05-14T17:11:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f723d7fa640 1 -- 172.21.15.69:0/3666635071 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7244015070 con 0x55e156b967f0 2024-05-14T17:11:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f725e9c9740 10 client.17523 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f725e9c9740 1 -- 172.21.15.69:0/3666635071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e156bb46d0 con 0x55e156b967f0 2024-05-14T17:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f723d7fa640 1 -- 172.21.15.69:0/3666635071 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72440118f0 con 0x55e156b967f0 2024-05-14T17:11:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f723d7fa640 1 -- 172.21.15.69:0/3666635071 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f7244004710 con 0x55e156b967f0 2024-05-14T17:11:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f725e9c9740 10 client.17523 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f725e9c9740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f725e9c9740 20 client.17523 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f725e9c9740 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e156bb5ad0 0x55e156bb7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f725e9c9740 4 client.17523 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.463+0000 7f725e9c9740 1 -- 172.21.15.69:0/3666635071 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e156b0bb80 con 0x55e156bb5ad0 2024-05-14T17:11:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.464+0000 7f7259242640 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e156bb5ad0 0x55e156bb7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.464+0000 7f723d7fa640 10 client.17523 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.464+0000 7f7259242640 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e156bb5ad0 0x55e156bb7ec0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316eade96cfd7800 server_cookie=123a6ec32b9221b7 in_seq=0 out_seq=0 2024-05-14T17:11:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f723d7fa640 1 -- 172.21.15.69:0/3666635071 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e156b0bb80 con 0x55e156bb5ad0 2024-05-14T17:11:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f723d7fa640 10 client.17523 handle_command_reply: tid=0 2024-05-14T17:11:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 client.17523 shutdown 2024-05-14T17:11:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 -- 172.21.15.69:0/3666635071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 msgr2=0x55e156ba58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba58e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f724400bb80 tx=0x7f724400b9a0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 -- 172.21.15.69:0/3666635071 shutdown_connections 2024-05-14T17:11:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e156bb5ad0 0x55e156bb7ec0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156ba5ee0 0x55e156ba31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 0x55e156ba2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 --2- 172.21.15.69:0/3666635071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba58e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f723d7fa640 0 client.17523 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 -- 172.21.15.69:0/3666635071 >> 172.21.15.69:0/3666635071 conn(0x55e156b92e00 msgr2=0x55e156b98c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 -- 172.21.15.69:0/3666635071 shutdown_connections 2024-05-14T17:11:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 1 -- 172.21.15.69:0/3666635071 wait complete. 2024-05-14T17:11:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.465+0000 7f725e9c9740 20 client.17523 trim_cache size 0 max 16384 2024-05-14T17:11:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f725e9c9740 1 Processor -- start 2024-05-14T17:11:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f725e9c9740 1 -- start start 2024-05-14T17:11:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b97110 0x55e156ba5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156ba5ee0 0x55e156b91f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f725e9c9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f72541adec0 con 0x55e156b967f0 2024-05-14T17:11:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f725e9c9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f72541ae590 con 0x55e156ba5ee0 2024-05-14T17:11:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f725e9c9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72541abea0 con 0x55e156b97110 2024-05-14T17:11:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f7259a43640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f7259a43640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58456/0 (socket says 172.21.15.69:58456) 2024-05-14T17:11:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f7259a43640 1 -- 172.21.15.69:0/273373957 learned_addr learned my addr 172.21.15.69:0/273373957 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f7259a43640 1 -- 172.21.15.69:0/273373957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b97110 msgr2=0x55e156ba5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f7259a43640 1 --2- 172.21.15.69:0/273373957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b97110 0x55e156ba5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f7259a43640 1 -- 172.21.15.69:0/273373957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156ba5ee0 msgr2=0x55e156b91f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f7259a43640 1 --2- 172.21.15.69:0/273373957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156ba5ee0 0x55e156b91f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f7259a43640 1 -- 172.21.15.69:0/273373957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e156b939d0 con 0x55e156b967f0 2024-05-14T17:11:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.480+0000 7f7259a43640 1 --2- 172.21.15.69:0/273373957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba24b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7244004ae0 tx=0x7f7244005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852b0f70075b1a84 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f723f7fe640 1 -- 172.21.15.69:0/273373957 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244028040 con 0x55e156b967f0 2024-05-14T17:11:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f723f7fe640 1 -- 172.21.15.69:0/273373957 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f724400be00 con 0x55e156b967f0 2024-05-14T17:11:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f723f7fe640 1 -- 172.21.15.69:0/273373957 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72440109c0 con 0x55e156b967f0 2024-05-14T17:11:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 -- 172.21.15.69:0/273373957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 msgr2=0x55e156ba24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 --2- 172.21.15.69:0/273373957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba24b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7244004ae0 tx=0x7f7244005990 comp rx=0 tx=0).stop 2024-05-14T17:11:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 -- 172.21.15.69:0/273373957 shutdown_connections 2024-05-14T17:11:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 --2- 172.21.15.69:0/273373957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156ba5ee0 0x55e156b91f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 --2- 172.21.15.69:0/273373957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b97110 0x55e156ba5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 --2- 172.21.15.69:0/273373957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156b967f0 0x55e156ba24b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 -- 172.21.15.69:0/273373957 >> 172.21.15.69:0/273373957 conn(0x55e156b92e00 msgr2=0x55e156b91870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 -- 172.21.15.69:0/273373957 shutdown_connections 2024-05-14T17:11:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 -- 172.21.15.69:0/273373957 wait complete. 2024-05-14T17:11:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 -- start start 2024-05-14T17:11:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.481+0000 7f725e9c9740 1 Processor -- start 2024-05-14T17:11:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 0x55e156ba93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 0x55e156ba9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725e9c9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156ba5ee0 0x55e156c181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725e9c9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f72541ab7d0 con 0x55e156ba5ee0 2024-05-14T17:11:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725e9c9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f72541aa3e0 con 0x55e156b97110 2024-05-14T17:11:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725e9c9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72541aaa30 con 0x55e156b967f0 2024-05-14T17:11:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725e9c9740 10 client.0 _ll_register_callbacks cb 0x7f724400bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725a244640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156ba5ee0 0x55e156c181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725a244640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156ba5ee0 0x55e156c181c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58462/0 (socket says 172.21.15.69:58462) 2024-05-14T17:11:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725a244640 1 -- 172.21.15.69:0/402625179 learned_addr learned my addr 172.21.15.69:0/402625179 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f7259242640 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 0x55e156ba9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f7259a43640 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 0x55e156ba93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725a244640 1 -- 172.21.15.69:0/402625179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 msgr2=0x55e156ba93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725a244640 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 0x55e156ba93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725a244640 1 -- 172.21.15.69:0/402625179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 msgr2=0x55e156ba9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725a244640 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 0x55e156ba9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725a244640 1 -- 172.21.15.69:0/402625179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e156c18780 con 0x55e156ba5ee0 2024-05-14T17:11:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f725a244640 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156ba5ee0 0x55e156c181c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f724c005cc0 tx=0x7f724c031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47af4641e99f6535 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.482+0000 7f723d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f723d7fa640 1 -- 172.21.15.69:0/402625179 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c003e40 con 0x55e156ba5ee0 2024-05-14T17:11:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f723d7fa640 1 -- 172.21.15.69:0/402625179 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8542+0+0 (secure 0 0 0) 0x7f724c004040 con 0x55e156ba5ee0 2024-05-14T17:11:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f725e9c9740 1 -- 172.21.15.69:0/402625179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f724400be00 con 0x55e156ba5ee0 2024-05-14T17:11:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f723d7fa640 1 -- 172.21.15.69:0/402625179 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f724c004240 con 0x55e156ba5ee0 2024-05-14T17:11:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f725e9c9740 10 client.17529 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f725e9c9740 1 -- 172.21.15.69:0/402625179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e156c1fb40 con 0x55e156ba5ee0 2024-05-14T17:11:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f723d7fa640 1 -- 172.21.15.69:0/402625179 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f724400be00 con 0x55e156ba5ee0 2024-05-14T17:11:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f723d7fa640 1 -- 172.21.15.69:0/402625179 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f724c0305c0 con 0x55e156ba5ee0 2024-05-14T17:11:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f725e9c9740 10 client.17529 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f725e9c9740 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f725e9c9740 20 client.17529 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f725e9c9740 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e156c20340 0x55e156c22760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f725e9c9740 4 client.17529 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f725e9c9740 1 -- 172.21.15.69:0/402625179 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e156b05360 con 0x55e156c20340 2024-05-14T17:11:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.483+0000 7f7259a43640 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e156c20340 0x55e156c22760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.484+0000 7f723d7fa640 10 client.17529 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:13.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.484+0000 7f7259a43640 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e156c20340 0x55e156c22760 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=501195e024c84b7a server_cookie=b1fb9f1a35290f45 in_seq=0 out_seq=0 2024-05-14T17:11:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.491+0000 7f723d7fa640 1 -- 172.21.15.69:0/402625179 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e156b05360 con 0x55e156c20340 2024-05-14T17:11:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.491+0000 7f723d7fa640 10 client.17529 handle_command_reply: tid=0 2024-05-14T17:11:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.491+0000 7f725e9c9740 1 client.17529 shutdown 2024-05-14T17:11:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 1 -- 172.21.15.69:0/402625179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156ba5ee0 msgr2=0x55e156c181c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156ba5ee0 0x55e156c181c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f724c005cc0 tx=0x7f724c031820 comp rx=0 tx=0).stop 2024-05-14T17:11:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 1 -- 172.21.15.69:0/402625179 shutdown_connections 2024-05-14T17:11:13.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e156c20340 0x55e156c22760 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e156ba5ee0 0x55e156c181c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e156b97110 0x55e156ba9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 1 --2- 172.21.15.69:0/402625179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e156b967f0 0x55e156ba93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f723d7fa640 0 client.17529 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 1 -- 172.21.15.69:0/402625179 >> 172.21.15.69:0/402625179 conn(0x55e156b92e00 msgr2=0x55e156b92100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 1 -- 172.21.15.69:0/402625179 shutdown_connections 2024-05-14T17:11:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 1 -- 172.21.15.69:0/402625179 wait complete. 2024-05-14T17:11:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.492+0000 7f725e9c9740 20 client.17529 trim_cache size 0 max 16384 2024-05-14T17:11:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:11:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:11:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:11:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 -- 172.21.15.69:0/1316021537 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f72400b2a10 msgr2=0x7f72400b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f72400b2a10 0x7f72400b4ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f724c006010 tx=0x7f724c03f040 comp rx=0 tx=0).stop 2024-05-14T17:11:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 -- 172.21.15.69:0/1316021537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72541025f0 msgr2=0x7f725419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72541025f0 0x7f725419a3c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f725000c920 tx=0x7f725000cde0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 -- 172.21.15.69:0/1316021537 shutdown_connections 2024-05-14T17:11:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f72400b2a10 0x7f72400b4ed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7254106920 0x7f72541a4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72541025f0 0x7f725419a3c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 --2- 172.21.15.69:0/1316021537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7254101c30 0x7f7254199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 -- 172.21.15.69:0/1316021537 >> 172.21.15.69:0/1316021537 conn(0x7f7254078730 msgr2=0x7f72540ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 -- 172.21.15.69:0/1316021537 shutdown_connections 2024-05-14T17:11:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:06.493+0000 7f725bca7640 1 -- 172.21.15.69:0/1316021537 wait complete. 2024-05-14T17:11:13.424 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:06 smithi069 ceph-mon[31507]: pgmap v672: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-05-14T17:11:13.434 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:08 smithi069 ceph-mon[31507]: pgmap v673: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-05-14T17:11:13.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:09 smithi069 ceph-mon[31507]: osdmap e115: 12 total, 12 up, 12 in 2024-05-14T17:11:13.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:10 smithi069 ceph-mon[31507]: pgmap v675: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:11:13.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:10 smithi092 ceph-mon[35763]: pgmap v675: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:11:13.475 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:12 smithi092 ceph-mon[35763]: pgmap v676: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-05-14T17:11:13.486 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:10 smithi203 ceph-mon[35787]: pgmap v675: 97 pgs: 97 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:11:13.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:12 smithi203 ceph-mon[35787]: pgmap v676: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-05-14T17:11:13.505 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:12 smithi069 ceph-mon[31507]: pgmap v676: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-05-14T17:11:13.687 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:11:13.696 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:11:15.249 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:14 smithi092 ceph-mon[35763]: pgmap v677: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-05-14T17:11:15.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:14 smithi069 ceph-mon[31507]: pgmap v677: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-05-14T17:11:15.463 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:14 smithi203 ceph-mon[35787]: pgmap v677: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-05-14T17:11:16.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.635+0000 7fd4fe37d700 1 -- 172.21.15.69:0/1724244162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4f8102e90 msgr2=0x7fd4f810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.635+0000 7fd4fe37d700 1 --2- 172.21.15.69:0/1724244162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4f8102e90 0x7fd4f810f310 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000b0a0 tx=0x7fd4f000b3b0 comp rx=0 tx=0).stop 2024-05-14T17:11:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.635+0000 7fd4fe37d700 1 -- 172.21.15.69:0/1724244162 shutdown_connections 2024-05-14T17:11:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.635+0000 7fd4fe37d700 1 --2- 172.21.15.69:0/1724244162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4f8102e90 0x7fd4f810f310 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:16.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.635+0000 7fd4fe37d700 1 --2- 172.21.15.69:0/1724244162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4f81084d0 0x7fd4f81088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:16.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.635+0000 7fd4fe37d700 1 --2- 172.21.15.69:0/1724244162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f81024d0 0x7fd4f8102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:16.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.635+0000 7fd4fe37d700 1 -- 172.21.15.69:0/1724244162 >> 172.21.15.69:0/1724244162 conn(0x7fd4f8078490 msgr2=0x7fd4f80788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:16.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.636+0000 7fd4fe37d700 1 -- 172.21.15.69:0/1724244162 shutdown_connections 2024-05-14T17:11:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.636+0000 7fd4fe37d700 1 -- 172.21.15.69:0/1724244162 wait complete. 2024-05-14T17:11:16.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.636+0000 7fd4fe37d700 1 Processor -- start 2024-05-14T17:11:16.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.636+0000 7fd4fe37d700 1 -- start start 2024-05-14T17:11:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.636+0000 7fd4fe37d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4f81024d0 0x7fd4f819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:16.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.636+0000 7fd4fe37d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4f8102e90 0x7fd4f819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:16.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.636+0000 7fd4fe37d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f81084d0 0x7fd4f81998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:16.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4fe37d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8112600 con 0x7fd4f81024d0 2024-05-14T17:11:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4fe37d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8112480 con 0x7fd4f81084d0 2024-05-14T17:11:16.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4fe37d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8112780 con 0x7fd4f8102e90 2024-05-14T17:11:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4f81024d0 0x7fd4f819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4f81024d0 0x7fd4f819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48334/0 (socket says 172.21.15.69:48334) 2024-05-14T17:11:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f7fff700 1 -- 172.21.15.69:0/2656083455 learned_addr learned my addr 172.21.15.69:0/2656083455 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:16.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4fc91a700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f81084d0 0x7fd4f81998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:16.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4effff700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4f8102e90 0x7fd4f819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:16.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f7fff700 1 -- 172.21.15.69:0/2656083455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4f8102e90 msgr2=0x7fd4f819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:16.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4f8102e90 0x7fd4f819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:16.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f7fff700 1 -- 172.21.15.69:0/2656083455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f81084d0 msgr2=0x7fd4f81998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f81084d0 0x7fd4f81998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f7fff700 1 -- 172.21.15.69:0/2656083455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f819a040 con 0x7fd4f81024d0 2024-05-14T17:11:17.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4f81024d0 0x7fd4f819f720 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c970 tx=0x7fd4e800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f5ffb700 1 -- 172.21.15.69:0/2656083455 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80078d0 con 0x7fd4f81024d0 2024-05-14T17:11:17.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f5ffb700 1 -- 172.21.15.69:0/2656083455 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e8018480 con 0x7fd4f81024d0 2024-05-14T17:11:17.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4fe37d700 1 -- 172.21.15.69:0/2656083455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f819a330 con 0x7fd4f81024d0 2024-05-14T17:11:17.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4f5ffb700 1 -- 172.21.15.69:0/2656083455 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800f6b0 con 0x7fd4f81024d0 2024-05-14T17:11:17.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.637+0000 7fd4fe37d700 1 -- 172.21.15.69:0/2656083455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f81a65f0 con 0x7fd4f81024d0 2024-05-14T17:11:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.640+0000 7fd4f5ffb700 1 -- 172.21.15.69:0/2656083455 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd4e8018930 con 0x7fd4f81024d0 2024-05-14T17:11:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.641+0000 7fd4f5ffb700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4e40b2aa0 0x7fd4e40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.641+0000 7fd4effff700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4e40b2aa0 0x7fd4e40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.641+0000 7fd4f5ffb700 1 -- 172.21.15.69:0/2656083455 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fd4e80e26c0 con 0x7fd4f81024d0 2024-05-14T17:11:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.641+0000 7fd4effff700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4e40b2aa0 0x7fd4e40b4f60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8078a70 tx=0x7fd4e000b410 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:17.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.641+0000 7fd508c13b80 1 Processor -- start 2024-05-14T17:11:17.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.641+0000 7fd508c13b80 1 -- start start 2024-05-14T17:11:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da262e30 0x55c8da263210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da263750 0x55c8da25d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da25db60 0x55c8da25dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd508c13b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c8da1bbe20 con 0x55c8da262e30 2024-05-14T17:11:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd508c13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c8da1ab9c0 con 0x55c8da25db60 2024-05-14T17:11:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd508c13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c8da1afa00 con 0x55c8da263750 2024-05-14T17:11:17.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da262e30 0x55c8da263210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da262e30 0x55c8da263210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48340/0 (socket says 172.21.15.69:48340) 2024-05-14T17:11:17.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4f7fff700 1 -- 172.21.15.69:0/1898149118 learned_addr learned my addr 172.21.15.69:0/1898149118 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:17.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4effff700 1 --2- 172.21.15.69:0/1898149118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da263750 0x55c8da25d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4f7fff700 1 -- 172.21.15.69:0/1898149118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da263750 msgr2=0x55c8da25d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/1898149118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da263750 0x55c8da25d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4f7fff700 1 -- 172.21.15.69:0/1898149118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da25db60 msgr2=0x55c8da25dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/1898149118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da25db60 0x55c8da25dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4f7fff700 1 -- 172.21.15.69:0/1898149118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8da0fba60 con 0x55c8da262e30 2024-05-14T17:11:17.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/1898149118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da262e30 0x55c8da263210 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80a6c60 tx=0x7fd4e80a9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b09ca2c7371221 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4ee7fc700 1 -- 172.21.15.69:0/1898149118 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80a99f0 con 0x55c8da262e30 2024-05-14T17:11:17.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4ee7fc700 1 -- 172.21.15.69:0/1898149118 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e80adda0 con 0x55c8da262e30 2024-05-14T17:11:17.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.642+0000 7fd4ee7fc700 1 -- 172.21.15.69:0/1898149118 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80a9c50 con 0x55c8da262e30 2024-05-14T17:11:17.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.643+0000 7fd508c13b80 1 -- 172.21.15.69:0/1898149118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da262e30 msgr2=0x55c8da263210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.643+0000 7fd508c13b80 1 --2- 172.21.15.69:0/1898149118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da262e30 0x55c8da263210 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80a6c60 tx=0x7fd4e80a9500 comp rx=0 tx=0).stop 2024-05-14T17:11:17.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.643+0000 7fd508c13b80 1 -- 172.21.15.69:0/1898149118 shutdown_connections 2024-05-14T17:11:17.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.643+0000 7fd508c13b80 1 --2- 172.21.15.69:0/1898149118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da262e30 0x55c8da263210 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.643+0000 7fd508c13b80 1 --2- 172.21.15.69:0/1898149118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da25db60 0x55c8da25dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.643+0000 7fd508c13b80 1 --2- 172.21.15.69:0/1898149118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da263750 0x55c8da25d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.643+0000 7fd508c13b80 1 -- 172.21.15.69:0/1898149118 >> 172.21.15.69:0/1898149118 conn(0x55c8da1883e0 msgr2=0x55c8da257d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:17.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.643+0000 7fd508c13b80 1 -- 172.21.15.69:0/1898149118 shutdown_connections 2024-05-14T17:11:17.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.643+0000 7fd508c13b80 1 -- 172.21.15.69:0/1898149118 wait complete. 2024-05-14T17:11:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.643+0000 7fd508c13b80 1 -- start start 2024-05-14T17:11:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd508c13b80 1 Processor -- start 2024-05-14T17:11:17.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 0x55c8da269500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da2668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da266de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd508c13b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c8da0b2130 con 0x55c8da263750 2024-05-14T17:11:17.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd508c13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c8da14d950 con 0x55c8da262e30 2024-05-14T17:11:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd508c13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c8da26a190 con 0x55c8da25db60 2024-05-14T17:11:17.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd508c13b80 10 client.0 _ll_register_callbacks cb 0x55c8da142710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:17.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd4fc91a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da266de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd4fc91a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da266de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48350/0 (socket says 172.21.15.69:48350) 2024-05-14T17:11:17.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd4fc91a700 1 -- 172.21.15.69:0/793275092 learned_addr learned my addr 172.21.15.69:0/793275092 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:17.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 0x55c8da269500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.644+0000 7fd4effff700 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da2668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4fc91a700 1 -- 172.21.15.69:0/793275092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 msgr2=0x55c8da269500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4fc91a700 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 0x55c8da269500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4fc91a700 1 -- 172.21.15.69:0/793275092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 msgr2=0x55c8da2668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4fc91a700 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da2668a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4fc91a700 1 -- 172.21.15.69:0/793275092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8da14a820 con 0x55c8da263750 2024-05-14T17:11:17.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 0x55c8da269500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:18.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4fc91a700 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da266de0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0004c40 tx=0x7fd4f0004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be671d5bcde2a9a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:18.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:18.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4d77fe700 1 -- 172.21.15.69:0/793275092 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00078f0 con 0x55c8da263750 2024-05-14T17:11:18.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4d77fe700 1 -- 172.21.15.69:0/793275092 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fd4f0007a90 con 0x55c8da263750 2024-05-14T17:11:18.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd508c13b80 1 -- 172.21.15.69:0/793275092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e80adda0 con 0x55c8da263750 2024-05-14T17:11:18.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4d77fe700 1 -- 172.21.15.69:0/793275092 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd4f0004e40 con 0x55c8da263750 2024-05-14T17:11:18.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd508c13b80 10 client.17541 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:18.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd508c13b80 1 -- 172.21.15.69:0/793275092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55c8da274940 con 0x55c8da263750 2024-05-14T17:11:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4d77fe700 1 -- 172.21.15.69:0/793275092 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e80adda0 con 0x55c8da263750 2024-05-14T17:11:18.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd4d77fe700 1 -- 172.21.15.69:0/793275092 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fd4f0011070 con 0x55c8da263750 2024-05-14T17:11:18.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd508c13b80 10 client.17541 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:18.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd508c13b80 10 client.17541 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:11:18.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd508c13b80 20 client.17541 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:18.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd508c13b80 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c8da276fd0 0x55c8da2793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:18.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd508c13b80 4 client.17541 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.645+0000 7fd508c13b80 1 -- 172.21.15.69:0/793275092 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8da2294f0 con 0x55c8da276fd0 2024-05-14T17:11:18.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.646+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c8da276fd0 0x55c8da2793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.646+0000 7fd4d77fe700 10 client.17541 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:18.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.646+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c8da276fd0 0x55c8da2793c0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ad11b35415e9b20 server_cookie=322025d970a9f416 in_seq=0 out_seq=0 2024-05-14T17:11:18.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd4d77fe700 1 -- 172.21.15.69:0/793275092 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c8da2294f0 con 0x55c8da276fd0 2024-05-14T17:11:18.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd4d77fe700 10 client.17541 handle_command_reply: tid=0 2024-05-14T17:11:18.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 client.17541 shutdown 2024-05-14T17:11:18.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 -- 172.21.15.69:0/793275092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 msgr2=0x55c8da266de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:18.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da266de0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0004c40 tx=0x7fd4f0004c70 comp rx=0 tx=0).stop 2024-05-14T17:11:18.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 -- 172.21.15.69:0/793275092 shutdown_connections 2024-05-14T17:11:18.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c8da276fd0 0x55c8da2793c0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da266de0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 0x55c8da269500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 --2- 172.21.15.69:0/793275092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da2668a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 -- 172.21.15.69:0/793275092 >> 172.21.15.69:0/793275092 conn(0x55c8da1883e0 msgr2=0x55c8da252860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:18.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd4d77fe700 0 client.17541 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:18.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 -- 172.21.15.69:0/793275092 shutdown_connections 2024-05-14T17:11:18.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 1 -- 172.21.15.69:0/793275092 wait complete. 2024-05-14T17:11:18.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.647+0000 7fd508c13b80 20 client.17541 trim_cache size 0 max 16384 2024-05-14T17:11:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd508c13b80 1 Processor -- start 2024-05-14T17:11:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd508c13b80 1 -- start start 2024-05-14T17:11:18.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da25db60 0x55c8da266f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:18.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da265880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da263750 0x55c8da260ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:18.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd508c13b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81af820 con 0x55c8da25db60 2024-05-14T17:11:18.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd508c13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81afef0 con 0x55c8da262e30 2024-05-14T17:11:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd508c13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81ad7c0 con 0x55c8da263750 2024-05-14T17:11:18.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da25db60 0x55c8da266f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:18.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da25db60 0x55c8da266f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48362/0 (socket says 172.21.15.69:48362) 2024-05-14T17:11:18.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd4f7fff700 1 -- 172.21.15.69:0/2080361871 learned_addr learned my addr 172.21.15.69:0/2080361871 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:18.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.668+0000 7fd4effff700 1 --2- 172.21.15.69:0/2080361871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da265880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.669+0000 7fd4fc91a700 1 --2- 172.21.15.69:0/2080361871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da263750 0x55c8da260ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:18.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.669+0000 7fd4f7fff700 1 -- 172.21.15.69:0/2080361871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da263750 msgr2=0x55c8da260ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:18.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.669+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/2080361871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da263750 0x55c8da260ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.669+0000 7fd4f7fff700 1 -- 172.21.15.69:0/2080361871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 msgr2=0x55c8da265880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:18.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.669+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/2080361871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da265880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.669+0000 7fd4f7fff700 1 -- 172.21.15.69:0/2080361871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8da264e30 con 0x55c8da25db60 2024-05-14T17:11:18.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.669+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/2080361871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da25db60 0x55c8da266f30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80e0350 tx=0x7fd4e80b0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f06008daa37467 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:18.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.669+0000 7fd4edffb700 1 -- 172.21.15.69:0/2080361871 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80c4050 con 0x55c8da25db60 2024-05-14T17:11:18.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.669+0000 7fd4edffb700 1 -- 172.21.15.69:0/2080361871 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e80cc0b0 con 0x55c8da25db60 2024-05-14T17:11:18.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.669+0000 7fd4edffb700 1 -- 172.21.15.69:0/2080361871 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80a9eb0 con 0x55c8da25db60 2024-05-14T17:11:18.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 -- 172.21.15.69:0/2080361871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da25db60 msgr2=0x55c8da266f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:18.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 --2- 172.21.15.69:0/2080361871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da25db60 0x55c8da266f30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80e0350 tx=0x7fd4e80b0040 comp rx=0 tx=0).stop 2024-05-14T17:11:18.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 -- 172.21.15.69:0/2080361871 shutdown_connections 2024-05-14T17:11:18.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 --2- 172.21.15.69:0/2080361871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da25db60 0x55c8da266f30 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 --2- 172.21.15.69:0/2080361871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da263750 0x55c8da260ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 --2- 172.21.15.69:0/2080361871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da265880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 -- 172.21.15.69:0/2080361871 >> 172.21.15.69:0/2080361871 conn(0x55c8da1883e0 msgr2=0x55c8da261a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:18.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 -- 172.21.15.69:0/2080361871 shutdown_connections 2024-05-14T17:11:19.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 -- 172.21.15.69:0/2080361871 wait complete. 2024-05-14T17:11:19.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 -- start start 2024-05-14T17:11:19.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.670+0000 7fd508c13b80 1 Processor -- start 2024-05-14T17:11:19.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 0x55c8da26f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:19.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da26f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:19.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd508c13b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da26c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:19.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd508c13b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81ad0f0 con 0x55c8da263750 2024-05-14T17:11:19.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd508c13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81abd00 con 0x55c8da262e30 2024-05-14T17:11:19.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd508c13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81ac350 con 0x55c8da25db60 2024-05-14T17:11:19.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd508c13b80 10 client.0 _ll_register_callbacks cb 0x55c8da142710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:19.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4fc91a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da26c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:19.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4fc91a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da26c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48378/0 (socket says 172.21.15.69:48378) 2024-05-14T17:11:19.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4fc91a700 1 -- 172.21.15.69:0/2664474546 learned_addr learned my addr 172.21.15.69:0/2664474546 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:19.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 0x55c8da26f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:19.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4effff700 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da26f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:19.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4fc91a700 1 -- 172.21.15.69:0/2664474546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 msgr2=0x55c8da26f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:19.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4fc91a700 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 0x55c8da26f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4fc91a700 1 -- 172.21.15.69:0/2664474546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 msgr2=0x55c8da26f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:19.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4fc91a700 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da26f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4fc91a700 1 -- 172.21.15.69:0/2664474546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8da26c930 con 0x55c8da263750 2024-05-14T17:11:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4effff700 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da26f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:19.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 0x55c8da26f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:19.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4fc91a700 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da26c3f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009a20 tx=0x7fd4f0007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e8c3f4af90138f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:19.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:19.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4d77fe700 1 -- 172.21.15.69:0/2664474546 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0026790 con 0x55c8da263750 2024-05-14T17:11:19.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd4d77fe700 1 -- 172.21.15.69:0/2664474546 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fd4f0026930 con 0x55c8da263750 2024-05-14T17:11:19.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.671+0000 7fd508c13b80 1 -- 172.21.15.69:0/2664474546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e80cc0b0 con 0x55c8da263750 2024-05-14T17:11:19.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd4d77fe700 1 -- 172.21.15.69:0/2664474546 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd4f0026b50 con 0x55c8da263750 2024-05-14T17:11:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd508c13b80 10 client.17547 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:19.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd508c13b80 1 -- 172.21.15.69:0/2664474546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55c8da2e2020 con 0x55c8da263750 2024-05-14T17:11:19.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd4d77fe700 1 -- 172.21.15.69:0/2664474546 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e80cc0b0 con 0x55c8da263750 2024-05-14T17:11:19.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd4d77fe700 1 -- 172.21.15.69:0/2664474546 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fd4f0011040 con 0x55c8da263750 2024-05-14T17:11:19.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd508c13b80 10 client.17547 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:19.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd508c13b80 10 client.17547 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:11:19.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd508c13b80 20 client.17547 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:19.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd508c13b80 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c8da2e2630 0x55c8da2e4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd508c13b80 4 client.17547 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.4, "await_for": 45.0}] 2024-05-14T17:11:19.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd508c13b80 1 -- 172.21.15.69:0/2664474546 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.4, "await_for": 45.0}) -- 0x55c8da1bdd70 con 0x55c8da2e2630 2024-05-14T17:11:19.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.672+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c8da2e2630 0x55c8da2e4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:19.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.673+0000 7fd4d77fe700 10 client.17547 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:19.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.673+0000 7fd4f7fff700 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c8da2e2630 0x55c8da2e4a50 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=732b57ddf87fbe8d server_cookie=3bc06e3f3e0b1d72 in_seq=0 out_seq=0 2024-05-14T17:11:19.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.995+0000 7fd4d77fe700 1 -- 172.21.15.69:0/2664474546 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c8da1bdd70 con 0x55c8da2e2630 2024-05-14T17:11:19.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.995+0000 7fd4d77fe700 10 client.17547 handle_command_reply: tid=0 2024-05-14T17:11:19.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.995+0000 7fd508c13b80 1 client.17547 shutdown 2024-05-14T17:11:19.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.995+0000 7fd508c13b80 1 -- 172.21.15.69:0/2664474546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 msgr2=0x55c8da26c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:19.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.995+0000 7fd508c13b80 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da26c3f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009a20 tx=0x7fd4f0007db0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.995+0000 7fd508c13b80 1 -- 172.21.15.69:0/2664474546 shutdown_connections 2024-05-14T17:11:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.995+0000 7fd508c13b80 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55c8da2e2630 0x55c8da2e4a50 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.995+0000 7fd4d77fe700 0 client.17547 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.995+0000 7fd508c13b80 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c8da263750 0x55c8da26c3f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.996+0000 7fd508c13b80 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8da25db60 0x55c8da26f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.996+0000 7fd508c13b80 1 --2- 172.21.15.69:0/2664474546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8da262e30 0x55c8da26f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.996+0000 7fd508c13b80 1 -- 172.21.15.69:0/2664474546 >> 172.21.15.69:0/2664474546 conn(0x55c8da1883e0 msgr2=0x55c8da261a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:19.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.996+0000 7fd508c13b80 1 -- 172.21.15.69:0/2664474546 shutdown_connections 2024-05-14T17:11:19.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.996+0000 7fd508c13b80 1 -- 172.21.15.69:0/2664474546 wait complete. 2024-05-14T17:11:19.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.996+0000 7fd508c13b80 20 client.17547 trim_cache size 0 max 16384 2024-05-14T17:11:19.781 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:11:19.795 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 61, 2024-05-14T17:11:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T17:11:19.822 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 46, 2024-05-14T17:11:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-05-14T17:11:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "b0ead1e1": { 2024-05-14T17:11:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 46, 2024-05-14T17:11:19.883 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-05-14T17:11:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:11:19.923 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T17:11:19.951 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:11:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:19.973 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-05-14T17:11:19.987 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 138.4, 2024-05-14T17:11:20.000 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-05-14T17:11:20.014 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-05-14T17:11:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-05-14T17:11:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-05-14T17:11:20.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-05-14T17:11:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-05-14T17:11:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:20.094 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:20.122 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:20.148 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:11:20.214 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:16 smithi203 ceph-mon[35787]: pgmap v678: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-05-14T17:11:20.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:18 smithi203 ceph-mon[35787]: pgmap v679: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-05-14T17:11:20.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:18 smithi203 ceph-mon[35787]: osdmap e116: 12 total, 12 up, 12 in 2024-05-14T17:11:20.257 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:16 smithi092 ceph-mon[35763]: pgmap v678: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-05-14T17:11:20.268 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:18 smithi092 ceph-mon[35763]: pgmap v679: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-05-14T17:11:20.278 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:18 smithi092 ceph-mon[35763]: osdmap e116: 12 total, 12 up, 12 in 2024-05-14T17:11:20.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 -- 172.21.15.69:0/2656083455 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4e40b2aa0 msgr2=0x7fd4e40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:20.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4e40b2aa0 0x7fd4e40b4f60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8078a70 tx=0x7fd4e000b410 comp rx=0 tx=0).stop 2024-05-14T17:11:20.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 -- 172.21.15.69:0/2656083455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4f81024d0 msgr2=0x7fd4f819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:20.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4f81024d0 0x7fd4f819f720 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c970 tx=0x7fd4e800cc80 comp rx=0 tx=0).stop 2024-05-14T17:11:20.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 -- 172.21.15.69:0/2656083455 shutdown_connections 2024-05-14T17:11:20.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fd4e40b2aa0 0x7fd4e40b4f60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4f81024d0 0x7fd4f819f720 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f81084d0 0x7fd4f81998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 --2- 172.21.15.69:0/2656083455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4f8102e90 0x7fd4f819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 -- 172.21.15.69:0/2656083455 >> 172.21.15.69:0/2656083455 conn(0x7fd4f8078490 msgr2=0x7fd4f806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:20.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 -- 172.21.15.69:0/2656083455 shutdown_connections 2024-05-14T17:11:20.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:16.997+0000 7fd4fe37d700 1 -- 172.21.15.69:0/2656083455 wait complete. 2024-05-14T17:11:20.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:16 smithi069 ceph-mon[31507]: pgmap v678: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-05-14T17:11:20.470 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b0ead1e1', quiesce took 47.7 seconds. Will release after: -29.300000000000004 2024-05-14T17:11:20.487 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:11:20.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.974+0000 7fc57d7ac700 1 -- 172.21.15.69:0/2110534202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781084d0 msgr2=0x7fc5781088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:20.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.974+0000 7fc57d7ac700 1 --2- 172.21.15.69:0/2110534202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781084d0 0x7fc5781088b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc568009990 tx=0x7fc568009ca0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.974+0000 7fc57d7ac700 1 -- 172.21.15.69:0/2110534202 shutdown_connections 2024-05-14T17:11:20.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.974+0000 7fc57d7ac700 1 --2- 172.21.15.69:0/2110534202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781084d0 0x7fc5781088b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.974+0000 7fc57d7ac700 1 --2- 172.21.15.69:0/2110534202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc578102e90 0x7fc57810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.974+0000 7fc57d7ac700 1 --2- 172.21.15.69:0/2110534202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5781024d0 0x7fc578102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.974+0000 7fc57d7ac700 1 -- 172.21.15.69:0/2110534202 >> 172.21.15.69:0/2110534202 conn(0x7fc578078490 msgr2=0x7fc5780788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:20.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.974+0000 7fc57d7ac700 1 -- 172.21.15.69:0/2110534202 shutdown_connections 2024-05-14T17:11:20.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.974+0000 7fc57d7ac700 1 -- 172.21.15.69:0/2110534202 wait complete. 2024-05-14T17:11:20.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc57d7ac700 1 Processor -- start 2024-05-14T17:11:20.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc57d7ac700 1 -- start start 2024-05-14T17:11:20.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc57d7ac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5781024d0 0x7fc57819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc57d7ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc578102e90 0x7fc57819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:20.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc57d7ac700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781084d0 0x7fc578199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:20.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc57d7ac700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc578112660 con 0x7fc5781084d0 2024-05-14T17:11:20.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc57d7ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781124e0 con 0x7fc578102e90 2024-05-14T17:11:20.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc57d7ac700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781127e0 con 0x7fc5781024d0 2024-05-14T17:11:20.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc5777fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781084d0 0x7fc578199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:20.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc5777fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781084d0 0x7fc578199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48394/0 (socket says 172.21.15.69:48394) 2024-05-14T17:11:20.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc5777fe700 1 -- 172.21.15.69:0/3042146999 learned_addr learned my addr 172.21.15.69:0/3042146999 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:20.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc576ffd700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5781024d0 0x7fc57819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:20.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc56ffff700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc578102e90 0x7fc57819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:20.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc5777fe700 1 -- 172.21.15.69:0/3042146999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5781024d0 msgr2=0x7fc57819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc5777fe700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5781024d0 0x7fc57819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc5777fe700 1 -- 172.21.15.69:0/3042146999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc578102e90 msgr2=0x7fc57819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:20.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc5777fe700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc578102e90 0x7fc57819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc5777fe700 1 -- 172.21.15.69:0/3042146999 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57819a1c0 con 0x7fc5781084d0 2024-05-14T17:11:20.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc576ffd700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5781024d0 0x7fc57819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:20.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc56ffff700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc578102e90 0x7fc57819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:20.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.975+0000 7fc5777fe700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781084d0 0x7fc578199a60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b970 tx=0x7fc57000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:20.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.976+0000 7fc574ff9700 1 -- 172.21.15.69:0/3042146999 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570019070 con 0x7fc5781084d0 2024-05-14T17:11:20.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.976+0000 7fc574ff9700 1 -- 172.21.15.69:0/3042146999 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc570014440 con 0x7fc5781084d0 2024-05-14T17:11:20.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.976+0000 7fc57d7ac700 1 -- 172.21.15.69:0/3042146999 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57819a4b0 con 0x7fc5781084d0 2024-05-14T17:11:20.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.976+0000 7fc574ff9700 1 -- 172.21.15.69:0/3042146999 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57000ce60 con 0x7fc5781084d0 2024-05-14T17:11:21.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.976+0000 7fc57d7ac700 1 -- 172.21.15.69:0/3042146999 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5781a6770 con 0x7fc5781084d0 2024-05-14T17:11:21.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.978+0000 7fc574ff9700 1 -- 172.21.15.69:0/3042146999 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc57001c070 con 0x7fc5781084d0 2024-05-14T17:11:21.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.979+0000 7fc574ff9700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc5640b29f0 0x7fc5640b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.979+0000 7fc574ff9700 1 -- 172.21.15.69:0/3042146999 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fc5700e19d0 con 0x7fc5781084d0 2024-05-14T17:11:21.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.979+0000 7fc576ffd700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc5640b29f0 0x7fc5640b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:21.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.979+0000 7fc588042b80 1 Processor -- start 2024-05-14T17:11:21.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.979+0000 7fc576ffd700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc5640b29f0 0x7fc5640b4eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc568000c00 tx=0x7fc568005fb0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:21.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.979+0000 7fc588042b80 1 -- start start 2024-05-14T17:11:21.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc588042b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc588042b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 0x55efeb79a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc588042b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb79a9c0 0x55efeb79ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc588042b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55efeb6efde0 con 0x55efeb796200 2024-05-14T17:11:21.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc588042b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55efeb6df980 con 0x55efeb796b20 2024-05-14T17:11:21.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc588042b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55efeb6e39c0 con 0x55efeb79a9c0 2024-05-14T17:11:21.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc576ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:21.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc576ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48404/0 (socket says 172.21.15.69:48404) 2024-05-14T17:11:21.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc576ffd700 1 -- 172.21.15.69:0/3361379457 learned_addr learned my addr 172.21.15.69:0/3361379457 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:21.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc56ffff700 1 --2- 172.21.15.69:0/3361379457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 0x55efeb79a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:21.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc5777fe700 1 --2- 172.21.15.69:0/3361379457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb79a9c0 0x55efeb79ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:21.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc576ffd700 1 -- 172.21.15.69:0/3361379457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb79a9c0 msgr2=0x55efeb79ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:21.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc576ffd700 1 --2- 172.21.15.69:0/3361379457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb79a9c0 0x55efeb79ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc576ffd700 1 -- 172.21.15.69:0/3361379457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 msgr2=0x55efeb79a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:21.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc576ffd700 1 --2- 172.21.15.69:0/3361379457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 0x55efeb79a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc576ffd700 1 -- 172.21.15.69:0/3361379457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efeb689020 con 0x55efeb796200 2024-05-14T17:11:21.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc56ffff700 1 --2- 172.21.15.69:0/3361379457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 0x55efeb79a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:21.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.980+0000 7fc576ffd700 1 --2- 172.21.15.69:0/3361379457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7965e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc568003b20 tx=0x7fc56801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb8e3d19e07c8e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:21.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.981+0000 7fc56d7fa700 1 -- 172.21.15.69:0/3361379457 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568009590 con 0x55efeb796200 2024-05-14T17:11:21.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.981+0000 7fc56d7fa700 1 -- 172.21.15.69:0/3361379457 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5680054e0 con 0x55efeb796200 2024-05-14T17:11:21.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.981+0000 7fc56d7fa700 1 -- 172.21.15.69:0/3361379457 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56802e930 con 0x55efeb796200 2024-05-14T17:11:21.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.981+0000 7fc588042b80 1 -- 172.21.15.69:0/3361379457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 msgr2=0x55efeb7965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:21.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.981+0000 7fc588042b80 1 --2- 172.21.15.69:0/3361379457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7965e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc568003b20 tx=0x7fc56801e000 comp rx=0 tx=0).stop 2024-05-14T17:11:21.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.981+0000 7fc588042b80 1 -- 172.21.15.69:0/3361379457 shutdown_connections 2024-05-14T17:11:21.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.981+0000 7fc588042b80 1 --2- 172.21.15.69:0/3361379457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7965e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.981+0000 7fc588042b80 1 --2- 172.21.15.69:0/3361379457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb79a9c0 0x55efeb79ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.981+0000 7fc588042b80 1 --2- 172.21.15.69:0/3361379457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 0x55efeb79a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.981+0000 7fc588042b80 1 -- 172.21.15.69:0/3361379457 >> 172.21.15.69:0/3361379457 conn(0x55efeb6bee20 msgr2=0x55efeb78c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:21.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 1 -- 172.21.15.69:0/3361379457 shutdown_connections 2024-05-14T17:11:21.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 1 -- 172.21.15.69:0/3361379457 wait complete. 2024-05-14T17:11:21.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 1 -- start start 2024-05-14T17:11:21.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 1 Processor -- start 2024-05-14T17:11:21.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796b20 0x55efeb79f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 0x55efeb79f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55efeb5e6010 con 0x55efeb796200 2024-05-14T17:11:21.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55efeb647e10 con 0x55efeb79a9c0 2024-05-14T17:11:21.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55efeb7a2b90 con 0x55efeb796b20 2024-05-14T17:11:21.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc588042b80 10 client.0 _ll_register_callbacks cb 0x55efeb67a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:21.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc576ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:21.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc576ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48406/0 (socket says 172.21.15.69:48406) 2024-05-14T17:11:21.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc576ffd700 1 -- 172.21.15.69:0/3347899037 learned_addr learned my addr 172.21.15.69:0/3347899037 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:21.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc56ffff700 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796b20 0x55efeb79f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:21.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.982+0000 7fc5777fe700 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 0x55efeb79f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:21.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc576ffd700 1 -- 172.21.15.69:0/3347899037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796b20 msgr2=0x55efeb79f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:21.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc576ffd700 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796b20 0x55efeb79f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc576ffd700 1 -- 172.21.15.69:0/3347899037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 msgr2=0x55efeb79f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:21.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc576ffd700 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 0x55efeb79f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc576ffd700 1 -- 172.21.15.69:0/3347899037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efeb62f940 con 0x55efeb796200 2024-05-14T17:11:21.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc5777fe700 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 0x55efeb79f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:21.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc576ffd700 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7a1ec0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc56800bef0 tx=0x7fc56800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11aae5608ec7032b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:21.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc55e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:21.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc55e7fc700 1 -- 172.21.15.69:0/3347899037 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568036060 con 0x55efeb796200 2024-05-14T17:11:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc55e7fc700 1 -- 172.21.15.69:0/3347899037 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fc56802e610 con 0x55efeb796200 2024-05-14T17:11:21.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc588042b80 1 -- 172.21.15.69:0/3347899037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5680054e0 con 0x55efeb796200 2024-05-14T17:11:21.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc55e7fc700 1 -- 172.21.15.69:0/3347899037 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc56802e810 con 0x55efeb796200 2024-05-14T17:11:21.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc588042b80 10 client.17559 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:21.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc588042b80 1 -- 172.21.15.69:0/3347899037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55efeb6997d0 con 0x55efeb796200 2024-05-14T17:11:21.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc55e7fc700 1 -- 172.21.15.69:0/3347899037 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5680054e0 con 0x55efeb796200 2024-05-14T17:11:21.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.983+0000 7fc55e7fc700 1 -- 172.21.15.69:0/3347899037 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fc568020070 con 0x55efeb796200 2024-05-14T17:11:21.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.984+0000 7fc588042b80 10 client.17559 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:21.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.984+0000 7fc588042b80 10 client.17559 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:11:21.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.984+0000 7fc588042b80 20 client.17559 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:21.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.984+0000 7fc588042b80 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55efeb7af5e0 0x55efeb7b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.984+0000 7fc588042b80 4 client.17559 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:22.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.984+0000 7fc588042b80 1 -- 172.21.15.69:0/3347899037 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efeb6f1d30 con 0x55efeb7af5e0 2024-05-14T17:11:22.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.984+0000 7fc56ffff700 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55efeb7af5e0 0x55efeb7b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:22.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.984+0000 7fc55e7fc700 10 client.17559 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:22.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.984+0000 7fc56ffff700 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55efeb7af5e0 0x55efeb7b19d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3d7eaeff9da110 server_cookie=5d725b2807b53803 in_seq=0 out_seq=0 2024-05-14T17:11:22.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc55e7fc700 1 -- 172.21.15.69:0/3347899037 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55efeb6f1d30 con 0x55efeb7af5e0 2024-05-14T17:11:22.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc55e7fc700 10 client.17559 handle_command_reply: tid=0 2024-05-14T17:11:22.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 client.17559 shutdown 2024-05-14T17:11:22.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 -- 172.21.15.69:0/3347899037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 msgr2=0x55efeb7a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:22.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7a1ec0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc56800bef0 tx=0x7fc56800bf20 comp rx=0 tx=0).stop 2024-05-14T17:11:22.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 -- 172.21.15.69:0/3347899037 shutdown_connections 2024-05-14T17:11:22.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55efeb7af5e0 0x55efeb7b19d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc55e7fc700 0 client.17559 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7a1ec0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 0x55efeb79f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 --2- 172.21.15.69:0/3347899037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796b20 0x55efeb79f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 -- 172.21.15.69:0/3347899037 >> 172.21.15.69:0/3347899037 conn(0x55efeb6bee20 msgr2=0x55efeb79be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 -- 172.21.15.69:0/3347899037 shutdown_connections 2024-05-14T17:11:22.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 1 -- 172.21.15.69:0/3347899037 wait complete. 2024-05-14T17:11:22.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:17.985+0000 7fc588042b80 20 client.17559 trim_cache size 0 max 16384 2024-05-14T17:11:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc588042b80 1 Processor -- start 2024-05-14T17:11:22.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc588042b80 1 -- start start 2024-05-14T17:11:22.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc588042b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796200 0x55efeb786740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:22.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc588042b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 0x55efeb798460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:22.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc588042b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb79a9c0 0x55efeb7989a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:22.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc588042b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ad720 con 0x55efeb79a9c0 2024-05-14T17:11:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc588042b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781addf0 con 0x55efeb796b20 2024-05-14T17:11:22.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc588042b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ab6c0 con 0x55efeb796200 2024-05-14T17:11:22.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc5777fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb79a9c0 0x55efeb7989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:22.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc5777fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb79a9c0 0x55efeb7989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48408/0 (socket says 172.21.15.69:48408) 2024-05-14T17:11:22.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.005+0000 7fc5777fe700 1 -- 172.21.15.69:0/465815188 learned_addr learned my addr 172.21.15.69:0/465815188 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:22.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc56ffff700 1 --2- 172.21.15.69:0/465815188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 0x55efeb798460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:22.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc5777fe700 1 -- 172.21.15.69:0/465815188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796200 msgr2=0x55efeb786740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:22.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc576ffd700 1 --2- 172.21.15.69:0/465815188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796200 0x55efeb786740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:22.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc5777fe700 1 --2- 172.21.15.69:0/465815188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796200 0x55efeb786740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc5777fe700 1 -- 172.21.15.69:0/465815188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 msgr2=0x55efeb798460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:22.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc5777fe700 1 --2- 172.21.15.69:0/465815188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 0x55efeb798460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc5777fe700 1 -- 172.21.15.69:0/465815188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efeb7a0200 con 0x55efeb79a9c0 2024-05-14T17:11:22.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc56ffff700 1 --2- 172.21.15.69:0/465815188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 0x55efeb798460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:22.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc5777fe700 1 --2- 172.21.15.69:0/465815188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb79a9c0 0x55efeb7989a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc5700a5ff0 tx=0x7fc5700a89f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc063fae0c822d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:22.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc56cff9700 1 -- 172.21.15.69:0/465815188 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5700b1be0 con 0x55efeb79a9c0 2024-05-14T17:11:22.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc56cff9700 1 -- 172.21.15.69:0/465815188 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5700c3420 con 0x55efeb79a9c0 2024-05-14T17:11:22.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.006+0000 7fc56cff9700 1 -- 172.21.15.69:0/465815188 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5700b1750 con 0x55efeb79a9c0 2024-05-14T17:11:22.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 -- 172.21.15.69:0/465815188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb79a9c0 msgr2=0x55efeb7989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:22.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 --2- 172.21.15.69:0/465815188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb79a9c0 0x55efeb7989a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc5700a5ff0 tx=0x7fc5700a89f0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 -- 172.21.15.69:0/465815188 shutdown_connections 2024-05-14T17:11:22.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 --2- 172.21.15.69:0/465815188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb79a9c0 0x55efeb7989a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 --2- 172.21.15.69:0/465815188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796200 0x55efeb786740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 --2- 172.21.15.69:0/465815188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb796b20 0x55efeb798460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 -- 172.21.15.69:0/465815188 >> 172.21.15.69:0/465815188 conn(0x55efeb6bee20 msgr2=0x55efeb786030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:22.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 -- 172.21.15.69:0/465815188 shutdown_connections 2024-05-14T17:11:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 -- 172.21.15.69:0/465815188 wait complete. 2024-05-14T17:11:22.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 -- start start 2024-05-14T17:11:22.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 Processor -- start 2024-05-14T17:11:22.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796b20 0x55efeb7b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 0x55efeb819b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:22.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781aaff0 con 0x55efeb796200 2024-05-14T17:11:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781a9c00 con 0x55efeb79a9c0 2024-05-14T17:11:22.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781aa250 con 0x55efeb796b20 2024-05-14T17:11:22.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.007+0000 7fc588042b80 10 client.0 _ll_register_callbacks cb 0x55efeb67a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:22.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc576ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:22.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc576ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48420/0 (socket says 172.21.15.69:48420) 2024-05-14T17:11:22.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc576ffd700 1 -- 172.21.15.69:0/835823801 learned_addr learned my addr 172.21.15.69:0/835823801 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:23.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc56ffff700 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796b20 0x55efeb7b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:23.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc5777fe700 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 0x55efeb819b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:23.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc576ffd700 1 -- 172.21.15.69:0/835823801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796b20 msgr2=0x55efeb7b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:23.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc576ffd700 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796b20 0x55efeb7b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc576ffd700 1 -- 172.21.15.69:0/835823801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 msgr2=0x55efeb819b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:23.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc576ffd700 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 0x55efeb819b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc576ffd700 1 -- 172.21.15.69:0/835823801 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efeb81a0a0 con 0x55efeb796200 2024-05-14T17:11:23.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc576ffd700 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7b3d20 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc568009990 tx=0x7fc56802e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fc2228165db555 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:23.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc55e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:23.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc55e7fc700 1 -- 172.21.15.69:0/835823801 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568036040 con 0x55efeb796200 2024-05-14T17:11:23.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc55e7fc700 1 -- 172.21.15.69:0/835823801 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8586+0+0 (secure 0 0 0) 0x7fc56800f460 con 0x55efeb796200 2024-05-14T17:11:23.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc588042b80 1 -- 172.21.15.69:0/835823801 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5700c3420 con 0x55efeb796200 2024-05-14T17:11:23.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc55e7fc700 1 -- 172.21.15.69:0/835823801 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc568049050 con 0x55efeb796200 2024-05-14T17:11:23.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc588042b80 10 client.17565 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:23.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.008+0000 7fc588042b80 1 -- 172.21.15.69:0/835823801 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55efeb81b2f0 con 0x55efeb796200 2024-05-14T17:11:23.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc55e7fc700 1 -- 172.21.15.69:0/835823801 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5700c3420 con 0x55efeb796200 2024-05-14T17:11:23.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc55e7fc700 1 -- 172.21.15.69:0/835823801 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7fc568020040 con 0x55efeb796200 2024-05-14T17:11:23.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc588042b80 10 client.17565 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc588042b80 10 client.17565 resolve_mds: validated gid 24446 aka daemon mds.b 2024-05-14T17:11:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc588042b80 20 client.17565 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:23.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc588042b80 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55efeb7af5e0 0x55efeb81d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:23.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc588042b80 4 client.17565 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-14T17:11:23.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc588042b80 1 -- 172.21.15.69:0/835823801 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55efeb6eb510 con 0x55efeb7af5e0 2024-05-14T17:11:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc56ffff700 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55efeb7af5e0 0x55efeb81d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:23.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc55e7fc700 10 client.17565 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:23.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.009+0000 7fc56ffff700 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55efeb7af5e0 0x55efeb81d8a0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f09acc66c37f38e server_cookie=3cd15aa6b5e86af5 in_seq=0 out_seq=0 2024-05-14T17:11:23.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc55e7fc700 1 -- 172.21.15.69:0/835823801 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55efeb6eb510 con 0x55efeb7af5e0 2024-05-14T17:11:23.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc55e7fc700 10 client.17565 handle_command_reply: tid=0 2024-05-14T17:11:23.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 client.17565 shutdown 2024-05-14T17:11:23.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 -- 172.21.15.69:0/835823801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 msgr2=0x55efeb7b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:23.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7b3d20 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc568009990 tx=0x7fc56802e790 comp rx=0 tx=0).stop 2024-05-14T17:11:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 -- 172.21.15.69:0/835823801 shutdown_connections 2024-05-14T17:11:23.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55efeb7af5e0 0x55efeb81d8a0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efeb796200 0x55efeb7b3d20 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efeb79a9c0 0x55efeb819b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 --2- 172.21.15.69:0/835823801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efeb796b20 0x55efeb7b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 -- 172.21.15.69:0/835823801 >> 172.21.15.69:0/835823801 conn(0x55efeb6bee20 msgr2=0x55efeb79a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:23.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc55e7fc700 0 client.17565 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:23.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 -- 172.21.15.69:0/835823801 shutdown_connections 2024-05-14T17:11:23.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 1 -- 172.21.15.69:0/835823801 wait complete. 2024-05-14T17:11:23.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc588042b80 20 client.17565 trim_cache size 0 max 16384 2024-05-14T17:11:23.666 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:11:23.675 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 61, 2024-05-14T17:11:23.685 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24446, 2024-05-14T17:11:23.698 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 47, 2024-05-14T17:11:23.709 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": {} 2024-05-14T17:11:23.720 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:11:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc57d7ac700 1 -- 172.21.15.69:0/3042146999 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc5640b29f0 msgr2=0x7fc5640b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:23.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc57d7ac700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc5640b29f0 0x7fc5640b4eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc568000c00 tx=0x7fc568005fb0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc57d7ac700 1 -- 172.21.15.69:0/3042146999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781084d0 msgr2=0x7fc578199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:23.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.010+0000 7fc57d7ac700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781084d0 0x7fc578199a60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b970 tx=0x7fc57000bd30 comp rx=0 tx=0).stop 2024-05-14T17:11:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.011+0000 7fc57d7ac700 1 -- 172.21.15.69:0/3042146999 shutdown_connections 2024-05-14T17:11:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.011+0000 7fc57d7ac700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fc5640b29f0 0x7fc5640b4eb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.011+0000 7fc57d7ac700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781084d0 0x7fc578199a60 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.011+0000 7fc57d7ac700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5781024d0 0x7fc57819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.011+0000 7fc57d7ac700 1 --2- 172.21.15.69:0/3042146999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc578102e90 0x7fc57819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.011+0000 7fc57d7ac700 1 -- 172.21.15.69:0/3042146999 >> 172.21.15.69:0/3042146999 conn(0x7fc578078490 msgr2=0x7fc57806bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:23.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.011+0000 7fc57d7ac700 1 -- 172.21.15.69:0/3042146999 shutdown_connections 2024-05-14T17:11:23.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:18.011+0000 7fc57d7ac700 1 -- 172.21.15.69:0/3042146999 wait complete. 2024-05-14T17:11:23.925 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:18 smithi069 ceph-mon[31507]: pgmap v679: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-05-14T17:11:23.939 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:18 smithi069 ceph-mon[31507]: osdmap e116: 12 total, 12 up, 12 in 2024-05-14T17:11:23.955 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:20 smithi203 ceph-mon[35787]: pgmap v681: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-05-14T17:11:23.970 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:22 smithi203 ceph-mon[35787]: pgmap v682: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:23.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:20 smithi092 ceph-mon[35763]: pgmap v681: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-05-14T17:11:24.025 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:22 smithi092 ceph-mon[35763]: pgmap v682: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.591+0000 7f9d8322a640 1 -- 172.21.15.69:0/2730243298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d7c107a70 msgr2=0x7f9d7c107e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.591+0000 7f9d8322a640 1 --2- 172.21.15.69:0/2730243298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d7c107a70 0x7f9d7c107e50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009930 tx=0x7f9d7002a420 comp rx=0 tx=0).stop 2024-05-14T17:11:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 -- 172.21.15.69:0/2730243298 shutdown_connections 2024-05-14T17:11:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 --2- 172.21.15.69:0/2730243298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d7c1025f0 0x7f9d7c10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 --2- 172.21.15.69:0/2730243298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d7c101c30 0x7f9d7c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 --2- 172.21.15.69:0/2730243298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d7c107a70 0x7f9d7c107e50 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 -- 172.21.15.69:0/2730243298 >> 172.21.15.69:0/2730243298 conn(0x7f9d7c0786a0 msgr2=0x7f9d7c078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 -- 172.21.15.69:0/2730243298 shutdown_connections 2024-05-14T17:11:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 -- 172.21.15.69:0/2730243298 wait complete. 2024-05-14T17:11:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 Processor -- start 2024-05-14T17:11:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 -- start start 2024-05-14T17:11:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d7c101c30 0x7f9d7c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:24.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d7c1025f0 0x7f9d7c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d7c107a70 0x7f9d7c1a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c111bc0 con 0x7f9d7c101c30 2024-05-14T17:11:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c111a40 con 0x7f9d7c107a70 2024-05-14T17:11:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.592+0000 7f9d8322a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c111d40 con 0x7f9d7c1025f0 2024-05-14T17:11:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d80fc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d7c101c30 0x7f9d7c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d80fc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d7c101c30 0x7f9d7c199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48438/0 (socket says 172.21.15.69:48438) 2024-05-14T17:11:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d80fc6640 1 -- 172.21.15.69:0/2743723061 learned_addr learned my addr 172.21.15.69:0/2743723061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d817c7640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d7c107a70 0x7f9d7c1a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d7bfff640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d7c1025f0 0x7f9d7c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d80fc6640 1 -- 172.21.15.69:0/2743723061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d7c1025f0 msgr2=0x7f9d7c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d7c1025f0 0x7f9d7c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d80fc6640 1 -- 172.21.15.69:0/2743723061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d7c107a70 msgr2=0x7f9d7c1a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d7c107a70 0x7f9d7c1a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d80fc6640 1 -- 172.21.15.69:0/2743723061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c1a54e0 con 0x7f9d7c101c30 2024-05-14T17:11:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d7bfff640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d7c1025f0 0x7f9d7c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d7c101c30 0x7f9d7c199f20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009230 tx=0x7f9d70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d79ffb640 1 -- 172.21.15.69:0/2743723061 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7003f030 con 0x7f9d7c101c30 2024-05-14T17:11:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d79ffb640 1 -- 172.21.15.69:0/2743723061 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d70003aa0 con 0x7f9d7c101c30 2024-05-14T17:11:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d79ffb640 1 -- 172.21.15.69:0/2743723061 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70035070 con 0x7f9d7c101c30 2024-05-14T17:11:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.593+0000 7f9d8322a640 1 -- 172.21.15.69:0/2743723061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7c1a56b0 con 0x7f9d7c101c30 2024-05-14T17:11:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.594+0000 7f9d8322a640 1 -- 172.21.15.69:0/2743723061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7c1a5b20 con 0x7f9d7c101c30 2024-05-14T17:11:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.596+0000 7f9d79ffb640 1 -- 172.21.15.69:0/2743723061 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9d70042030 con 0x7f9d7c101c30 2024-05-14T17:11:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.596+0000 7f9d86025740 1 Processor -- start 2024-05-14T17:11:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.596+0000 7f9d86025740 1 -- start start 2024-05-14T17:11:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d86025740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b9298f0 0x55e61b929cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d86025740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b92a210 0x55e61b934590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d86025740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b934ad0 0x55e61b936ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d86025740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e61b89cc30 con 0x55e61b9298f0 2024-05-14T17:11:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d86025740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e61b88c7d0 con 0x55e61b92a210 2024-05-14T17:11:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d86025740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e61b890810 con 0x55e61b934ad0 2024-05-14T17:11:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d817c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b9298f0 0x55e61b929cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d817c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b9298f0 0x55e61b929cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48446/0 (socket says 172.21.15.69:48446) 2024-05-14T17:11:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d817c7640 1 -- 172.21.15.69:0/3745737777 learned_addr learned my addr 172.21.15.69:0/3745737777 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/3745737777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b92a210 0x55e61b934590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d7bfff640 1 --2- 172.21.15.69:0/3745737777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b934ad0 0x55e61b936ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d79ffb640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9d6c0b2b10 0x7f9d6c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d817c7640 1 -- 172.21.15.69:0/3745737777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b934ad0 msgr2=0x55e61b936ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d817c7640 1 --2- 172.21.15.69:0/3745737777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b934ad0 0x55e61b936ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d817c7640 1 -- 172.21.15.69:0/3745737777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b92a210 msgr2=0x55e61b934590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d817c7640 1 --2- 172.21.15.69:0/3745737777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b92a210 0x55e61b934590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d817c7640 1 -- 172.21.15.69:0/3745737777 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61b5a1210 con 0x55e61b9298f0 2024-05-14T17:11:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d817c7640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9d6c0b2b10 0x7f9d6c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d79ffb640 1 -- 172.21.15.69:0/2743723061 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f9d70102b40 con 0x7f9d7c101c30 2024-05-14T17:11:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.597+0000 7f9d817c7640 1 --2- 172.21.15.69:0/3745737777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b9298f0 0x55e61b929cd0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9d740072e0 tx=0x7f9d7400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a3abdce4448950 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d66ffd640 1 -- 172.21.15.69:0/3745737777 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74012b40 con 0x55e61b9298f0 2024-05-14T17:11:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d66ffd640 1 -- 172.21.15.69:0/3745737777 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d74012d60 con 0x55e61b9298f0 2024-05-14T17:11:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d66ffd640 1 -- 172.21.15.69:0/3745737777 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74019880 con 0x55e61b9298f0 2024-05-14T17:11:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d86025740 1 -- 172.21.15.69:0/3745737777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b9298f0 msgr2=0x55e61b929cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d817c7640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9d6c0b2b10 0x7f9d6c0b4fd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d7401e000 tx=0x7f9d74019c30 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d86025740 1 --2- 172.21.15.69:0/3745737777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b9298f0 0x55e61b929cd0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9d740072e0 tx=0x7f9d7400f850 comp rx=0 tx=0).stop 2024-05-14T17:11:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d86025740 1 -- 172.21.15.69:0/3745737777 shutdown_connections 2024-05-14T17:11:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d86025740 1 --2- 172.21.15.69:0/3745737777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b934ad0 0x55e61b936ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d86025740 1 --2- 172.21.15.69:0/3745737777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b92a210 0x55e61b934590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d86025740 1 --2- 172.21.15.69:0/3745737777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b9298f0 0x55e61b929cd0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d86025740 1 -- 172.21.15.69:0/3745737777 >> 172.21.15.69:0/3745737777 conn(0x55e61b925e00 msgr2=0x55e61b9261e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d86025740 1 -- 172.21.15.69:0/3745737777 shutdown_connections 2024-05-14T17:11:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.598+0000 7f9d86025740 1 -- 172.21.15.69:0/3745737777 wait complete. 2024-05-14T17:11:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d86025740 1 -- start start 2024-05-14T17:11:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d86025740 1 Processor -- start 2024-05-14T17:11:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d86025740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 0x55e61b9343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d86025740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b92a210 0x55e61b9316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d86025740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b931c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d86025740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e61b572790 con 0x55e61b934ad0 2024-05-14T17:11:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d86025740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e61b848890 con 0x55e61b9298f0 2024-05-14T17:11:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d86025740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e61b9377a0 con 0x55e61b92a210 2024-05-14T17:11:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d86025740 10 client.0 _ll_register_callbacks cb 0x55e61b84bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d817c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b931c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d817c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b931c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48462/0 (socket says 172.21.15.69:48462) 2024-05-14T17:11:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d817c7640 1 -- 172.21.15.69:0/2726698666 learned_addr learned my addr 172.21.15.69:0/2726698666 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.599+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 0x55e61b9343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d7bfff640 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b92a210 0x55e61b9316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d817c7640 1 -- 172.21.15.69:0/2726698666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b92a210 msgr2=0x55e61b9316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d817c7640 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b92a210 0x55e61b9316c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d817c7640 1 -- 172.21.15.69:0/2726698666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 msgr2=0x55e61b9343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d817c7640 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 0x55e61b9343d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d817c7640 1 -- 172.21.15.69:0/2726698666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61b66d620 con 0x55e61b934ad0 2024-05-14T17:11:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d817c7640 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b931c00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9d74008010 tx=0x7f9d74016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9f4061effb81c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:25.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d5bfff640 1 -- 172.21.15.69:0/2726698666 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74033060 con 0x55e61b934ad0 2024-05-14T17:11:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d5bfff640 1 -- 172.21.15.69:0/2726698666 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f9d74019540 con 0x55e61b934ad0 2024-05-14T17:11:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d86025740 1 -- 172.21.15.69:0/2726698666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d74012d60 con 0x55e61b934ad0 2024-05-14T17:11:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d5bfff640 1 -- 172.21.15.69:0/2726698666 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f9d74012940 con 0x55e61b934ad0 2024-05-14T17:11:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d86025740 10 client.17577 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.600+0000 7f9d86025740 1 -- 172.21.15.69:0/2726698666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e61b943160 con 0x55e61b934ad0 2024-05-14T17:11:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d5bfff640 1 -- 172.21.15.69:0/2726698666 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d74012d60 con 0x55e61b934ad0 2024-05-14T17:11:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d5bfff640 1 -- 172.21.15.69:0/2726698666 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f9d7401a400 con 0x55e61b934ad0 2024-05-14T17:11:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d86025740 10 client.17577 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d86025740 10 client.17577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d86025740 20 client.17577 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d86025740 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e61b9445c0 0x55e61b9469b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d86025740 4 client.17577 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d86025740 1 -- 172.21.15.69:0/2726698666 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e61b89eb80 con 0x55e61b9445c0 2024-05-14T17:11:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e61b9445c0 0x55e61b9469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e61b9445c0 0x55e61b9469b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bcc41b78eae50e5 server_cookie=326b01d4d4df3d96 in_seq=0 out_seq=0 2024-05-14T17:11:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.601+0000 7f9d5bfff640 10 client.17577 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d5bfff640 1 -- 172.21.15.69:0/2726698666 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e61b89eb80 con 0x55e61b9445c0 2024-05-14T17:11:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d5bfff640 10 client.17577 handle_command_reply: tid=0 2024-05-14T17:11:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 client.17577 shutdown 2024-05-14T17:11:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 -- 172.21.15.69:0/2726698666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 msgr2=0x55e61b931c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b931c00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9d74008010 tx=0x7f9d74016000 comp rx=0 tx=0).stop 2024-05-14T17:11:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 -- 172.21.15.69:0/2726698666 shutdown_connections 2024-05-14T17:11:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e61b9445c0 0x55e61b9469b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b931c00 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b92a210 0x55e61b9316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 --2- 172.21.15.69:0/2726698666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 0x55e61b9343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d5bfff640 0 client.17577 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 -- 172.21.15.69:0/2726698666 >> 172.21.15.69:0/2726698666 conn(0x55e61b925e00 msgr2=0x55e61b925100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 -- 172.21.15.69:0/2726698666 shutdown_connections 2024-05-14T17:11:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.602+0000 7f9d86025740 1 -- 172.21.15.69:0/2726698666 wait complete. 2024-05-14T17:11:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.603+0000 7f9d86025740 20 client.17577 trim_cache size 0 max 16384 2024-05-14T17:11:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.617+0000 7f9d86025740 1 Processor -- start 2024-05-14T17:11:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.617+0000 7f9d86025740 1 -- start start 2024-05-14T17:11:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d86025740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b9298f0 0x55e61b924870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d86025740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b92a210 0x55e61b940600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d86025740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b934ad0 0x55e61b933f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d86025740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1b0070 con 0x55e61b92a210 2024-05-14T17:11:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d86025740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1b0740 con 0x55e61b934ad0 2024-05-14T17:11:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d86025740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1ae050 con 0x55e61b9298f0 2024-05-14T17:11:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d7bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b92a210 0x55e61b940600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d7bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b92a210 0x55e61b940600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48464/0 (socket says 172.21.15.69:48464) 2024-05-14T17:11:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d7bfff640 1 -- 172.21.15.69:0/2022855289 learned_addr learned my addr 172.21.15.69:0/2022855289 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/2022855289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b9298f0 0x55e61b924870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d7bfff640 1 -- 172.21.15.69:0/2022855289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b9298f0 msgr2=0x55e61b924870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d7bfff640 1 --2- 172.21.15.69:0/2022855289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b9298f0 0x55e61b924870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d817c7640 1 --2- 172.21.15.69:0/2022855289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b934ad0 0x55e61b933f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d7bfff640 1 -- 172.21.15.69:0/2022855289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b934ad0 msgr2=0x55e61b933f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d7bfff640 1 --2- 172.21.15.69:0/2022855289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b934ad0 0x55e61b933f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d7bfff640 1 -- 172.21.15.69:0/2022855289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61b925c30 con 0x55e61b92a210 2024-05-14T17:11:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d817c7640 1 --2- 172.21.15.69:0/2022855289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b934ad0 0x55e61b933f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d7bfff640 1 --2- 172.21.15.69:0/2022855289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b92a210 0x55e61b940600 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c06bde0 tx=0x7f9d68013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b122c81e0077f58f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d667fc640 1 -- 172.21.15.69:0/2022855289 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68004020 con 0x55e61b92a210 2024-05-14T17:11:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d667fc640 1 -- 172.21.15.69:0/2022855289 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d680041c0 con 0x55e61b92a210 2024-05-14T17:11:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.618+0000 7f9d667fc640 1 -- 172.21.15.69:0/2022855289 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68011740 con 0x55e61b92a210 2024-05-14T17:11:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 -- 172.21.15.69:0/2022855289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b92a210 msgr2=0x55e61b940600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 --2- 172.21.15.69:0/2022855289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b92a210 0x55e61b940600 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c06bde0 tx=0x7f9d68013830 comp rx=0 tx=0).stop 2024-05-14T17:11:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 -- 172.21.15.69:0/2022855289 shutdown_connections 2024-05-14T17:11:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 --2- 172.21.15.69:0/2022855289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b934ad0 0x55e61b933f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 --2- 172.21.15.69:0/2022855289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b92a210 0x55e61b940600 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 --2- 172.21.15.69:0/2022855289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b9298f0 0x55e61b924870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 -- 172.21.15.69:0/2022855289 >> 172.21.15.69:0/2022855289 conn(0x55e61b925e00 msgr2=0x55e61b93fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 -- 172.21.15.69:0/2022855289 shutdown_connections 2024-05-14T17:11:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 -- 172.21.15.69:0/2022855289 wait complete. 2024-05-14T17:11:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 -- start start 2024-05-14T17:11:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.619+0000 7f9d86025740 1 Processor -- start 2024-05-14T17:11:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d86025740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 0x55e61b93a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d86025740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b92a210 0x55e61b93acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d86025740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b93b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d86025740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1ad980 con 0x55e61b934ad0 2024-05-14T17:11:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d86025740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1ac590 con 0x55e61b9298f0 2024-05-14T17:11:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d86025740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1acbe0 con 0x55e61b92a210 2024-05-14T17:11:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d86025740 10 client.0 _ll_register_callbacks cb 0x7f9d74007e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d817c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b93b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d817c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b93b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48476/0 (socket says 172.21.15.69:48476) 2024-05-14T17:11:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d817c7640 1 -- 172.21.15.69:0/1809724924 learned_addr learned my addr 172.21.15.69:0/1809724924 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d7bfff640 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b92a210 0x55e61b93acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 0x55e61b93a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d817c7640 1 -- 172.21.15.69:0/1809724924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b92a210 msgr2=0x55e61b93acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d817c7640 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b92a210 0x55e61b93acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d817c7640 1 -- 172.21.15.69:0/1809724924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 msgr2=0x55e61b93a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d817c7640 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 0x55e61b93a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d817c7640 1 -- 172.21.15.69:0/1809724924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61b9adf50 con 0x55e61b934ad0 2024-05-14T17:11:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 0x55e61b93a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:26.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d817c7640 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b93b210 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9d74019420 tx=0x7f9d7400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c66951313f22c0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.620+0000 7f9d5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-14T17:11:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d5bfff640 1 -- 172.21.15.69:0/1809724924 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74045060 con 0x55e61b934ad0 2024-05-14T17:11:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d5bfff640 1 -- 172.21.15.69:0/1809724924 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f9d74019950 con 0x55e61b934ad0 2024-05-14T17:11:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d86025740 1 -- 172.21.15.69:0/1809724924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d680041c0 con 0x55e61b934ad0 2024-05-14T17:11:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d5bfff640 1 -- 172.21.15.69:0/1809724924 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f9d74033070 con 0x55e61b934ad0 2024-05-14T17:11:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d86025740 10 client.17583 fetch_fsmap learned FSMap version 61 2024-05-14T17:11:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d86025740 1 -- 172.21.15.69:0/1809724924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e61b9af190 con 0x55e61b934ad0 2024-05-14T17:11:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d5bfff640 1 -- 172.21.15.69:0/1809724924 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d680041c0 con 0x55e61b934ad0 2024-05-14T17:11:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d5bfff640 1 -- 172.21.15.69:0/1809724924 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) ==== 7733+0+0 (secure 0 0 0) 0x7f9d7401abc0 con 0x55e61b934ad0 2024-05-14T17:11:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d86025740 10 client.17583 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T17:11:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d86025740 10 client.17583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d86025740 20 client.17583 populate_metadata read hostname 'smithi069' 2024-05-14T17:11:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d86025740 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e61b9af7f0 0x55e61b9b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d86025740 4 client.17583 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d86025740 1 -- 172.21.15.69:0/1809724924 --> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e61b898360 con 0x55e61b9af7f0 2024-05-14T17:11:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.621+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e61b9af7f0 0x55e61b9b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.622+0000 7f9d5bfff640 10 client.17583 ms_handle_connect on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.622+0000 7f9d80fc6640 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e61b9af7f0 0x55e61b9b1c10 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb31d3187ba7b61 server_cookie=9802f0d3f21880c0 in_seq=0 out_seq=0 2024-05-14T17:11:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d5bfff640 1 -- 172.21.15.69:0/1809724924 <== mds.0 v2:172.21.15.92:6838/2169416341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e61b898360 con 0x55e61b9af7f0 2024-05-14T17:11:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d5bfff640 10 client.17583 handle_command_reply: tid=0 2024-05-14T17:11:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 client.17583 shutdown 2024-05-14T17:11:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 -- 172.21.15.69:0/1809724924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 msgr2=0x55e61b93b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b93b210 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9d74019420 tx=0x7f9d7400ff60 comp rx=0 tx=0).stop 2024-05-14T17:11:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 -- 172.21.15.69:0/1809724924 shutdown_connections 2024-05-14T17:11:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.92:6838/2169416341,v1:172.21.15.92:6839/2169416341] conn(0x55e61b9af7f0 0x55e61b9b1c10 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e61b934ad0 0x55e61b93b210 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d5bfff640 0 client.17583 ms_handle_reset on v2:172.21.15.92:6838/2169416341 2024-05-14T17:11:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e61b92a210 0x55e61b93acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 --2- 172.21.15.69:0/1809724924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e61b9298f0 0x55e61b93a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 -- 172.21.15.69:0/1809724924 >> 172.21.15.69:0/1809724924 conn(0x55e61b925e00 msgr2=0x55e61b940600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 -- 172.21.15.69:0/1809724924 shutdown_connections 2024-05-14T17:11:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.630+0000 7f9d86025740 1 -- 172.21.15.69:0/1809724924 wait complete. 2024-05-14T17:11:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d86025740 20 client.17583 trim_cache size 0 max 16384 2024-05-14T17:11:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-05-14T17:11:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-05-14T17:11:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-05-14T17:11:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 -- 172.21.15.69:0/2743723061 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9d6c0b2b10 msgr2=0x7f9d6c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9d6c0b2b10 0x7f9d6c0b4fd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d7401e000 tx=0x7f9d74019c30 comp rx=0 tx=0).stop 2024-05-14T17:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 -- 172.21.15.69:0/2743723061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d7c101c30 msgr2=0x7f9d7c199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d7c101c30 0x7f9d7c199f20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009230 tx=0x7f9d70004270 comp rx=0 tx=0).stop 2024-05-14T17:11:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 -- 172.21.15.69:0/2743723061 shutdown_connections 2024-05-14T17:11:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f9d6c0b2b10 0x7f9d6c0b4fd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d7c107a70 0x7f9d7c1a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d7c1025f0 0x7f9d7c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 --2- 172.21.15.69:0/2743723061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d7c101c30 0x7f9d7c199f20 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 -- 172.21.15.69:0/2743723061 >> 172.21.15.69:0/2743723061 conn(0x7f9d7c0786a0 msgr2=0x7f9d7c100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 -- 172.21.15.69:0/2743723061 shutdown_connections 2024-05-14T17:11:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-05-14T17:11:19.631+0000 7f9d8322a640 1 -- 172.21.15.69:0/2743723061 wait complete. 2024-05-14T17:11:27.265 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:20 smithi069 ceph-mon[31507]: pgmap v681: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-05-14T17:11:27.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:22 smithi069 ceph-mon[31507]: pgmap v682: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:27.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:24 smithi069 ceph-mon[31507]: pgmap v683: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:27.297 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:24 smithi203 ceph-mon[35787]: pgmap v683: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:27.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:26 smithi069 ceph-mon[31507]: pgmap v684: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:27.321 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:24 smithi092 ceph-mon[35763]: pgmap v683: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:27.332 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:26 smithi092 ceph-mon[35763]: pgmap v684: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:27.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:26 smithi203 ceph-mon[35787]: pgmap v684: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:27.518 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:11:29.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:28 smithi069 ceph-mon[31507]: pgmap v685: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:29.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.231+0000 7f4fdf561700 1 -- 172.21.15.69:0/2187298265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fd8102e30 msgr2=0x7f4fd810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:29.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.231+0000 7f4fdf561700 1 --2- 172.21.15.69:0/2187298265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fd8102e30 0x7f4fd810f2f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000b0a0 tx=0x7f4fd000b3b0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.231+0000 7f4fdf561700 1 -- 172.21.15.69:0/2187298265 shutdown_connections 2024-05-14T17:11:29.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.231+0000 7f4fdf561700 1 --2- 172.21.15.69:0/2187298265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fd8102e30 0x7f4fd810f2f0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.231+0000 7f4fdf561700 1 --2- 172.21.15.69:0/2187298265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd81084b0 0x7f4fd8108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.231+0000 7f4fdf561700 1 --2- 172.21.15.69:0/2187298265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8102470 0x7f4fd81028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.231+0000 7f4fdf561700 1 -- 172.21.15.69:0/2187298265 >> 172.21.15.69:0/2187298265 conn(0x7f4fd80793f0 msgr2=0x7f4fd8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:29.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.231+0000 7f4fdf561700 1 -- 172.21.15.69:0/2187298265 shutdown_connections 2024-05-14T17:11:29.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.231+0000 7f4fdf561700 1 -- 172.21.15.69:0/2187298265 wait complete. 2024-05-14T17:11:29.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdf561700 1 Processor -- start 2024-05-14T17:11:29.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdf561700 1 -- start start 2024-05-14T17:11:29.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdf561700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8102470 0x7f4fd819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:29.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdf561700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fd8102e30 0x7f4fd819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:29.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdf561700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd81084b0 0x7f4fd81a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdf561700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81125f0 con 0x7f4fd8102e30 2024-05-14T17:11:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdf561700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8112470 con 0x7f4fd81084b0 2024-05-14T17:11:29.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdf561700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8112770 con 0x7f4fd8102470 2024-05-14T17:11:29.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdcafc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fd8102e30 0x7f4fd819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:29.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdcafc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fd8102e30 0x7f4fd819aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54968/0 (socket says 172.21.15.69:54968) 2024-05-14T17:11:29.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdcafc700 1 -- 172.21.15.69:0/3759710281 learned_addr learned my addr 172.21.15.69:0/3759710281 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:29.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fddafe700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd81084b0 0x7f4fd81a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:29.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdd2fd700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8102470 0x7f4fd819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:29.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdcafc700 1 -- 172.21.15.69:0/3759710281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8102470 msgr2=0x7f4fd819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:29.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdcafc700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8102470 0x7f4fd819a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdcafc700 1 -- 172.21.15.69:0/3759710281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd81084b0 msgr2=0x7f4fd81a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:29.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdcafc700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd81084b0 0x7f4fd81a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fdcafc700 1 -- 172.21.15.69:0/3759710281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd81a5f30 con 0x7f4fd8102e30 2024-05-14T17:11:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.232+0000 7f4fddafe700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd81084b0 0x7f4fd81a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:29.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.233+0000 7f4fdcafc700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fd8102e30 0x7f4fd819aeb0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800d980 tx=0x7f4fc800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:29.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.233+0000 7f4fce7fc700 1 -- 172.21.15.69:0/3759710281 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8017070 con 0x7f4fd8102e30 2024-05-14T17:11:29.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.233+0000 7f4fce7fc700 1 -- 172.21.15.69:0/3759710281 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fc8004830 con 0x7f4fd8102e30 2024-05-14T17:11:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.233+0000 7f4fdf561700 1 -- 172.21.15.69:0/3759710281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd81a61c0 con 0x7f4fd8102e30 2024-05-14T17:11:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.233+0000 7f4fce7fc700 1 -- 172.21.15.69:0/3759710281 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8005440 con 0x7f4fd8102e30 2024-05-14T17:11:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.233+0000 7f4fdf561700 1 -- 172.21.15.69:0/3759710281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81a6680 con 0x7f4fd8102e30 2024-05-14T17:11:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:28 smithi203 ceph-mon[35787]: pgmap v685: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:28 smithi092 ceph-mon[35763]: pgmap v685: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-05-14T17:11:29.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.234+0000 7f4fdf561700 1 -- 172.21.15.69:0/3759710281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd806a230 con 0x7f4fd8102e30 2024-05-14T17:11:29.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.235+0000 7f4fce7fc700 1 -- 172.21.15.69:0/3759710281 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4fc80055e0 con 0x7f4fd8102e30 2024-05-14T17:11:29.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.236+0000 7f4fce7fc700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4fc40b29f0 0x7f4fc40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:29.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.236+0000 7f4fce7fc700 1 -- 172.21.15.69:0/3759710281 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f4fc80e25b0 con 0x7f4fd8102e30 2024-05-14T17:11:29.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.236+0000 7f4fdd2fd700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4fc40b29f0 0x7f4fc40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:29.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.236+0000 7f4fdd2fd700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4fc40b29f0 0x7f4fc40b4eb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4005fd0 tx=0x7f4fd4005ce0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:29.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.240+0000 7f4fce7fc700 1 -- 172.21.15.69:0/3759710281 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4fc8020020 con 0x7f4fd8102e30 2024-05-14T17:11:29.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.493+0000 7f4fdf561700 1 -- 172.21.15.69:0/3759710281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fd802e560 con 0x7f4fd8102e30 2024-05-14T17:11:29.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.494+0000 7f4fce7fc700 1 -- 172.21.15.69:0/3759710281 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11031 (secure 0 0 0) 0x7f4fc80a9f80 con 0x7f4fd8102e30 2024-05-14T17:11:29.882 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:11:29.890 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:08:18.843541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24385":{"gid":24385,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T17:11:29.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 -- 172.21.15.69:0/3759710281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4fc40b29f0 msgr2=0x7f4fc40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:29.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4fc40b29f0 0x7f4fc40b4eb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4005fd0 tx=0x7f4fd4005ce0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 -- 172.21.15.69:0/3759710281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fd8102e30 msgr2=0x7f4fd819aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:29.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fd8102e30 0x7f4fd819aeb0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800d980 tx=0x7f4fc800dd40 comp rx=0 tx=0).stop 2024-05-14T17:11:29.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 -- 172.21.15.69:0/3759710281 shutdown_connections 2024-05-14T17:11:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd81084b0 0x7f4fd81a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8102470 0x7f4fd819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f4fc40b29f0 0x7f4fc40b4eb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 --2- 172.21.15.69:0/3759710281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fd8102e30 0x7f4fd819aeb0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 -- 172.21.15.69:0/3759710281 >> 172.21.15.69:0/3759710281 conn(0x7f4fd80793f0 msgr2=0x7f4fd8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:29.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 -- 172.21.15.69:0/3759710281 shutdown_connections 2024-05-14T17:11:30.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:29.497+0000 7f4fdf561700 1 -- 172.21.15.69:0/3759710281 wait complete. 2024-05-14T17:11:30.010 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 61 2024-05-14T17:11:30.156 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:29 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3759710281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:30.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:29 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:30.202 INFO:tasks.quiescer:done joining Quiescers 2024-05-14T17:11:30.209 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-14T17:11:30.226 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T17:11:30.243 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-14T17:11:30.259 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-14T17:11:30.266 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T17:11:30.274 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T17:11:30.280 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-14T17:11:30.287 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:29 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3759710281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:30.346 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:29 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:30.388 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-14T17:11:30.456 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-14T17:11:30.463 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:29 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3759710281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:30.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:29 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:30.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.598+0000 7f660fbaf640 1 -- 172.21.15.69:0/1952627463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6608101ce0 msgr2=0x7f6608105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.598+0000 7f660fbaf640 1 --2- 172.21.15.69:0/1952627463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6608101ce0 0x7f6608105d20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f65f800aa10 tx=0x7f65f802b720 comp rx=0 tx=0).stop 2024-05-14T17:11:30.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 -- 172.21.15.69:0/1952627463 shutdown_connections 2024-05-14T17:11:30.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 --2- 172.21.15.69:0/1952627463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6608106400 0x7f660810dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 --2- 172.21.15.69:0/1952627463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6608101ce0 0x7f6608105d20 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 --2- 172.21.15.69:0/1952627463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6608101330 0x7f6608101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 -- 172.21.15.69:0/1952627463 >> 172.21.15.69:0/1952627463 conn(0x7f6608077ee0 msgr2=0x7f66080ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:30.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 -- 172.21.15.69:0/1952627463 shutdown_connections 2024-05-14T17:11:30.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 -- 172.21.15.69:0/1952627463 wait complete. 2024-05-14T17:11:30.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 Processor -- start 2024-05-14T17:11:30.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 -- start start 2024-05-14T17:11:30.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6608101330 0x7f6608199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6608106400 0x7f6608199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f660819dad0 0x7f660819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6608110ae0 con 0x7f6608106400 2024-05-14T17:11:30.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6608110960 con 0x7f660819dad0 2024-05-14T17:11:30.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660fbaf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6608110c60 con 0x7f6608101330 2024-05-14T17:11:30.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.599+0000 7f660d14a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6608106400 0x7f6608199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660d14a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6608106400 0x7f6608199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54986/0 (socket says 172.21.15.69:54986) 2024-05-14T17:11:30.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660d14a640 1 -- 172.21.15.69:0/4284931637 learned_addr learned my addr 172.21.15.69:0/4284931637 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:30.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660d94b640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6608101330 0x7f6608199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660e14c640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f660819dad0 0x7f660819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660d14a640 1 -- 172.21.15.69:0/4284931637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6608101330 msgr2=0x7f6608199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660d14a640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6608101330 0x7f6608199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660d14a640 1 -- 172.21.15.69:0/4284931637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f660819dad0 msgr2=0x7f660819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660d14a640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f660819dad0 0x7f660819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660d14a640 1 -- 172.21.15.69:0/4284931637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660819e5c0 con 0x7f6608106400 2024-05-14T17:11:30.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660e14c640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f660819dad0 0x7f660819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:30.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660d14a640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6608106400 0x7f6608199750 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f65f8004270 tx=0x7f65f80042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:30.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f65feffd640 1 -- 172.21.15.69:0/4284931637 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f803f070 con 0x7f6608106400 2024-05-14T17:11:30.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f65feffd640 1 -- 172.21.15.69:0/4284931637 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f80044c0 con 0x7f6608106400 2024-05-14T17:11:30.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660fbaf640 1 -- 172.21.15.69:0/4284931637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660819e850 con 0x7f6608106400 2024-05-14T17:11:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f65feffd640 1 -- 172.21.15.69:0/4284931637 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8036040 con 0x7f6608106400 2024-05-14T17:11:30.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.600+0000 7f660fbaf640 1 -- 172.21.15.69:0/4284931637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66081a6010 con 0x7f6608106400 2024-05-14T17:11:30.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.601+0000 7f660fbaf640 1 -- 172.21.15.69:0/4284931637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66080507e0 con 0x7f6608106400 2024-05-14T17:11:30.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.602+0000 7f65feffd640 1 -- 172.21.15.69:0/4284931637 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f65f8042070 con 0x7f6608106400 2024-05-14T17:11:30.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.603+0000 7f65feffd640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f65f40b2a10 0x7f65f40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.603+0000 7f660d94b640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f65f40b2a10 0x7f65f40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:31.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.603+0000 7f65feffd640 1 -- 172.21.15.69:0/4284931637 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f65f8102d50 con 0x7f6608106400 2024-05-14T17:11:31.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.604+0000 7f660d94b640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f65f40b2a10 0x7f65f40b4ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6604009bd0 tx=0x7f6604005f90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:31.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.608+0000 7f65feffd640 1 -- 172.21.15.69:0/4284931637 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65f80ca6c0 con 0x7f6608106400 2024-05-14T17:11:31.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.813+0000 7f660fbaf640 1 -- 172.21.15.69:0/4284931637 --> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6608101ce0 con 0x7f65f40b2a10 2024-05-14T17:11:31.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.820+0000 7f65feffd640 1 -- 172.21.15.69:0/4284931637 <== mgr.24764 v2:172.21.15.69:6800/4217409403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f6608101ce0 con 0x7f65f40b2a10 2024-05-14T17:11:31.065 INFO:teuthology.orchestra.run.smithi069.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-14T00:00:00", "created": "2024-05-14T16:49:55", "first": "2024-05-14T16:50:00", "last": "2024-05-14T17:11:00", "last_pruned": "2024-05-14T17:11:01", "created_count": 22, "pruned_count": 15, "active": true} 2024-05-14T17:11:31.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.822+0000 7f660fbaf640 1 -- 172.21.15.69:0/4284931637 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f65f40b2a10 msgr2=0x7f65f40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:31.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.822+0000 7f660fbaf640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f65f40b2a10 0x7f65f40b4ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6604009bd0 tx=0x7f6604005f90 comp rx=0 tx=0).stop 2024-05-14T17:11:31.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.822+0000 7f660fbaf640 1 -- 172.21.15.69:0/4284931637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6608106400 msgr2=0x7f6608199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:31.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.822+0000 7f660fbaf640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6608106400 0x7f6608199750 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f65f8004270 tx=0x7f65f80042a0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.823+0000 7f660fbaf640 1 -- 172.21.15.69:0/4284931637 shutdown_connections 2024-05-14T17:11:31.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.823+0000 7f660fbaf640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f65f40b2a10 0x7f65f40b4ed0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.823+0000 7f660fbaf640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f660819dad0 0x7f660819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.823+0000 7f660fbaf640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6608106400 0x7f6608199750 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.823+0000 7f660fbaf640 1 --2- 172.21.15.69:0/4284931637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6608101330 0x7f6608199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.823+0000 7f660fbaf640 1 -- 172.21.15.69:0/4284931637 >> 172.21.15.69:0/4284931637 conn(0x7f6608077ee0 msgr2=0x7f66080ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:31.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.823+0000 7f660fbaf640 1 -- 172.21.15.69:0/4284931637 shutdown_connections 2024-05-14T17:11:31.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:30.823+0000 7f660fbaf640 1 -- 172.21.15.69:0/4284931637 wait complete. 2024-05-14T17:11:31.234 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-14T17:11:31.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.349+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/1115350887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d18108a70 msgr2=0x7f1d18108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:31.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.349+0000 7f1d1dbaf640 1 --2- 172.21.15.69:0/1115350887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d18108a70 0x7f1d18108e50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009900 tx=0x7f1d0802a410 comp rx=0 tx=0).stop 2024-05-14T17:11:31.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.349+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/1115350887 shutdown_connections 2024-05-14T17:11:31.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.349+0000 7f1d1dbaf640 1 --2- 172.21.15.69:0/1115350887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d1806bc60 0x7f1d1810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.349+0000 7f1d1dbaf640 1 --2- 172.21.15.69:0/1115350887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d1806b2c0 0x7f1d1806b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.349+0000 7f1d1dbaf640 1 --2- 172.21.15.69:0/1115350887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d18108a70 0x7f1d18108e50 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.349+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/1115350887 >> 172.21.15.69:0/1115350887 conn(0x7f1d180786a0 msgr2=0x7f1d18078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:31.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.349+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/1115350887 shutdown_connections 2024-05-14T17:11:31.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.349+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/1115350887 wait complete. 2024-05-14T17:11:31.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.349+0000 7f1d1dbaf640 1 Processor -- start 2024-05-14T17:11:31.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d1dbaf640 1 -- start start 2024-05-14T17:11:31.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d1dbaf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d1806b2c0 0x7f1d181990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d1dbaf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d1806bc60 0x7f1d18199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:31.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d1dbaf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d18108a70 0x7f1d1819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:31.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d1dbaf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18112cd0 con 0x7f1d1806bc60 2024-05-14T17:11:31.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d1dbaf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18112b50 con 0x7f1d18108a70 2024-05-14T17:11:31.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d1dbaf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18112e50 con 0x7f1d1806b2c0 2024-05-14T17:11:31.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d16ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d1806bc60 0x7f1d18199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:31.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d16ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d1806bc60 0x7f1d18199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55004/0 (socket says 172.21.15.69:55004) 2024-05-14T17:11:31.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d16ffd640 1 -- 172.21.15.69:0/3385606281 learned_addr learned my addr 172.21.15.69:0/3385606281 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:31.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d17fff640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d18108a70 0x7f1d1819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:31.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d177fe640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d1806b2c0 0x7f1d181990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:31.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d16ffd640 1 -- 172.21.15.69:0/3385606281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d1806b2c0 msgr2=0x7f1d181990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:31.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d16ffd640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d1806b2c0 0x7f1d181990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d16ffd640 1 -- 172.21.15.69:0/3385606281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d18108a70 msgr2=0x7f1d1819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:31.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d16ffd640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d18108a70 0x7f1d1819d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d16ffd640 1 -- 172.21.15.69:0/3385606281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1819e0e0 con 0x7f1d1806bc60 2024-05-14T17:11:31.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d177fe640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d1806b2c0 0x7f1d181990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:31.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d17fff640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d18108a70 0x7f1d1819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:31.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d16ffd640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d1806bc60 0x7f1d18199630 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00c920 tx=0x7f1d0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:31.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.350+0000 7f1d14ff9640 1 -- 172.21.15.69:0/3385606281 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c007c70 con 0x7f1d1806bc60 2024-05-14T17:11:31.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.351+0000 7f1d14ff9640 1 -- 172.21.15.69:0/3385606281 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0c007e10 con 0x7f1d1806bc60 2024-05-14T17:11:31.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.351+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/3385606281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1819e3d0 con 0x7f1d1806bc60 2024-05-14T17:11:31.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.351+0000 7f1d14ff9640 1 -- 172.21.15.69:0/3385606281 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c018820 con 0x7f1d1806bc60 2024-05-14T17:11:31.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.351+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/3385606281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d181a5c60 con 0x7f1d1806bc60 2024-05-14T17:11:31.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.351+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/3385606281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d18104190 con 0x7f1d1806bc60 2024-05-14T17:11:31.762 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:31 smithi092 ceph-mon[35763]: pgmap v686: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 6 op/s 2024-05-14T17:11:31.768 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:31 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:31.774 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:31 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:31.781 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:31 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:31.814 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:31 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:31.820 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:31 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:31.829 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:31 smithi203 ceph-mon[35787]: pgmap v686: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 6 op/s 2024-05-14T17:11:31.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:31 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:31.847 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:31 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:31.855 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:31 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:31.891 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:31 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:31.897 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:31 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:31.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.353+0000 7f1d14ff9640 1 -- 172.21.15.69:0/3385606281 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1d0c00f450 con 0x7f1d1806bc60 2024-05-14T17:11:31.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.353+0000 7f1d14ff9640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d000b2a10 0x7f1d000b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:31.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.354+0000 7f1d177fe640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d000b2a10 0x7f1d000b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:31.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.354+0000 7f1d14ff9640 1 -- 172.21.15.69:0/3385606281 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f1d0c0e2ac0 con 0x7f1d1806bc60 2024-05-14T17:11:31.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.354+0000 7f1d177fe640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d000b2a10 0x7f1d000b4ed0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d08031440 tx=0x7f1d08005a20 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:31.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.357+0000 7f1d14ff9640 1 -- 172.21.15.69:0/3385606281 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d0c014030 con 0x7f1d1806bc60 2024-05-14T17:11:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.563+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/3385606281 --> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1d18062e90 con 0x7f1d000b2a10 2024-05-14T17:11:31.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.570+0000 7f1d14ff9640 1 -- 172.21.15.69:0/3385606281 <== mgr.24764 v2:172.21.15.69:6800/4217409403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1d18062e90 con 0x7f1d000b2a10 2024-05-14T17:11:31.957 INFO:teuthology.orchestra.run.smithi069.stdout:/ 1m 6m3h 2024-05-14T17:11:32.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/3385606281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d000b2a10 msgr2=0x7f1d000b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:32.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d000b2a10 0x7f1d000b4ed0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d08031440 tx=0x7f1d08005a20 comp rx=0 tx=0).stop 2024-05-14T17:11:32.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/3385606281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d1806bc60 msgr2=0x7f1d18199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:32.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d1806bc60 0x7f1d18199630 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00c920 tx=0x7f1d0c00cde0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/3385606281 shutdown_connections 2024-05-14T17:11:32.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1d000b2a10 0x7f1d000b4ed0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d18108a70 0x7f1d1819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d1806bc60 0x7f1d18199630 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 --2- 172.21.15.69:0/3385606281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d1806b2c0 0x7f1d181990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/3385606281 >> 172.21.15.69:0/3385606281 conn(0x7f1d180786a0 msgr2=0x7f1d18100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:32.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/3385606281 shutdown_connections 2024-05-14T17:11:32.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:31.573+0000 7f1d1dbaf640 1 -- 172.21.15.69:0/3385606281 wait complete. 2024-05-14T17:11:32.106 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:31 smithi069 ceph-mon[31507]: pgmap v686: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 6 op/s 2024-05-14T17:11:32.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:31 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:32.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:31 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:32.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:31 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:32.142 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:31 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:32.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:31 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:11:32.179 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-14T17:11:32.207 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T17:11:32.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.317+0000 7f95d5d9e640 1 -- 172.21.15.69:0/2667341664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95d0103a60 msgr2=0x7f95d0103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:32.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.317+0000 7f95d5d9e640 1 --2- 172.21.15.69:0/2667341664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95d0103a60 0x7f95d0103e40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009900 tx=0x7f95c402a410 comp rx=0 tx=0).stop 2024-05-14T17:11:32.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.317+0000 7f95d5d9e640 1 -- 172.21.15.69:0/2667341664 shutdown_connections 2024-05-14T17:11:32.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.317+0000 7f95d5d9e640 1 --2- 172.21.15.69:0/2667341664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d010ced0 0x7f95d010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.317+0000 7f95d5d9e640 1 --2- 172.21.15.69:0/2667341664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d0104380 0x7f95d010c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.317+0000 7f95d5d9e640 1 --2- 172.21.15.69:0/2667341664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95d0103a60 0x7f95d0103e40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.317+0000 7f95d5d9e640 1 -- 172.21.15.69:0/2667341664 >> 172.21.15.69:0/2667341664 conn(0x7f95d00fd200 msgr2=0x7f95d00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:32.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.317+0000 7f95d5d9e640 1 -- 172.21.15.69:0/2667341664 shutdown_connections 2024-05-14T17:11:32.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.317+0000 7f95d5d9e640 1 -- 172.21.15.69:0/2667341664 wait complete. 2024-05-14T17:11:32.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d5d9e640 1 Processor -- start 2024-05-14T17:11:32.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d5d9e640 1 -- start start 2024-05-14T17:11:32.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d5d9e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95d0103a60 0x7f95d0198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:32.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d5d9e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d0104380 0x7f95d0199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:32.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d5d9e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d010ced0 0x7f95d019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:32.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d5d9e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0112ad0 con 0x7f95d0103a60 2024-05-14T17:11:32.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d5d9e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0112950 con 0x7f95d010ced0 2024-05-14T17:11:32.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d5d9e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0112c50 con 0x7f95d0104380 2024-05-14T17:11:32.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d4d9c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95d0103a60 0x7f95d0198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d4d9c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95d0103a60 0x7f95d0198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55020/0 (socket says 172.21.15.69:55020) 2024-05-14T17:11:32.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d4d9c640 1 -- 172.21.15.69:0/208040125 learned_addr learned my addr 172.21.15.69:0/208040125 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:32.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d559d640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d010ced0 0x7f95d019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:32.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95cffff640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d0104380 0x7f95d0199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d4d9c640 1 -- 172.21.15.69:0/208040125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d0104380 msgr2=0x7f95d0199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:32.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d4d9c640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d0104380 0x7f95d0199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d4d9c640 1 -- 172.21.15.69:0/208040125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d010ced0 msgr2=0x7f95d019d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:32.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d4d9c640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d010ced0 0x7f95d019d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d4d9c640 1 -- 172.21.15.69:0/208040125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d019df70 con 0x7f95d0103a60 2024-05-14T17:11:32.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95cffff640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d0104380 0x7f95d0199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:32.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.318+0000 7f95d4d9c640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95d0103a60 0x7f95d0198fd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f95c4006010 tx=0x7f95c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:32.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.319+0000 7f95cdffb640 1 -- 172.21.15.69:0/208040125 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c403f030 con 0x7f95d0103a60 2024-05-14T17:11:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.319+0000 7f95d5d9e640 1 -- 172.21.15.69:0/208040125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d019e200 con 0x7f95d0103a60 2024-05-14T17:11:32.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.319+0000 7f95cdffb640 1 -- 172.21.15.69:0/208040125 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95c4003800 con 0x7f95d0103a60 2024-05-14T17:11:32.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.319+0000 7f95cdffb640 1 -- 172.21.15.69:0/208040125 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4035070 con 0x7f95d0103a60 2024-05-14T17:11:32.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.319+0000 7f95d5d9e640 1 -- 172.21.15.69:0/208040125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d01a5af0 con 0x7f95d0103a60 2024-05-14T17:11:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.320+0000 7f95d5d9e640 1 -- 172.21.15.69:0/208040125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d0069be0 con 0x7f95d0103a60 2024-05-14T17:11:32.827 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:32 smithi092 ceph-mon[35763]: from='client.17595 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:11:32.839 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:32 smithi092 ceph-mon[35763]: pgmap v687: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2024-05-14T17:11:32.849 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:32 smithi092 ceph-mon[35763]: pgmap v688: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-14T17:11:32.859 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:32 smithi203 ceph-mon[35787]: from='client.17595 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:11:32.870 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:32 smithi203 ceph-mon[35787]: pgmap v687: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2024-05-14T17:11:32.880 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:32 smithi203 ceph-mon[35787]: pgmap v688: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-14T17:11:32.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.321+0000 7f95cdffb640 1 -- 172.21.15.69:0/208040125 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f95c4003cb0 con 0x7f95d0103a60 2024-05-14T17:11:32.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.322+0000 7f95cdffb640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f95c00b2960 0x7f95c00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:32.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.322+0000 7f95cffff640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f95c00b2960 0x7f95c00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:32.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.322+0000 7f95cdffb640 1 -- 172.21.15.69:0/208040125 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f95c41032a0 con 0x7f95d0103a60 2024-05-14T17:11:32.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.322+0000 7f95cffff640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f95c00b2960 0x7f95c00b4e20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f95d00ff820 tx=0x7f95bc005d90 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:32.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.326+0000 7f95cdffb640 1 -- 172.21.15.69:0/208040125 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f95c40cac70 con 0x7f95d0103a60 2024-05-14T17:11:32.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.512+0000 7f95d5d9e640 1 -- 172.21.15.69:0/208040125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95d0107f20 con 0x7f95d0103a60 2024-05-14T17:11:33.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.514+0000 7f95cdffb640 1 -- 172.21.15.69:0/208040125 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+23134 (secure 0 0 0) 0x7f95c40ca570 con 0x7f95d0103a60 2024-05-14T17:11:33.030 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:11:33.039 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":116,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T17:11:18.884679+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:46:38.843325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:46:42.395401+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":38,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4012740750":"2024-05-15T16:49:54.600513+0000","172.21.15.69:6800/4012740750":"2024-05-15T16:49:54.600513+0000","172.21.15.69:0/70559253":"2024-05-15T16:49:54.600513+0000","172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2944165482":"2024-05-15T16:49:54.600513+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/1931804604":"2024-05-15T16:49:54.600513+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/3176577502":"2024-05-15T16:49:54.600513+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":26,"length":1},{"begin":38,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:11:33.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 -- 172.21.15.69:0/208040125 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f95c00b2960 msgr2=0x7f95c00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:33.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f95c00b2960 0x7f95c00b4e20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f95d00ff820 tx=0x7f95bc005d90 comp rx=0 tx=0).stop 2024-05-14T17:11:33.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 -- 172.21.15.69:0/208040125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95d0103a60 msgr2=0x7f95d0198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:33.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95d0103a60 0x7f95d0198fd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f95c4006010 tx=0x7f95c40042f0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 -- 172.21.15.69:0/208040125 shutdown_connections 2024-05-14T17:11:33.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f95c00b2960 0x7f95c00b4e20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d010ced0 0x7f95d019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d0104380 0x7f95d0199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 --2- 172.21.15.69:0/208040125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95d0103a60 0x7f95d0198fd0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 -- 172.21.15.69:0/208040125 >> 172.21.15.69:0/208040125 conn(0x7f95d00fd200 msgr2=0x7f95d00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:33.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 -- 172.21.15.69:0/208040125 shutdown_connections 2024-05-14T17:11:33.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:32.516+0000 7f95d5d9e640 1 -- 172.21.15.69:0/208040125 wait complete. 2024-05-14T17:11:33.172 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:32 smithi069 ceph-mon[31507]: from='client.17595 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:11:33.181 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:32 smithi069 ceph-mon[31507]: pgmap v687: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2024-05-14T17:11:33.224 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:32 smithi069 ceph-mon[31507]: pgmap v688: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-14T17:11:33.234 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T16:41:53.859789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-14T16:46:38.843325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-14T16:46:42.395401+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 38, 'snap_epoch': 115, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T17:11:33.244 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T17:11:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 -- 172.21.15.69:0/3686709625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ad8107930 msgr2=0x7f1ad8107d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:33.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 --2- 172.21.15.69:0/3686709625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ad8107930 0x7f1ad8107d10 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009930 tx=0x7f1ac802a420 comp rx=0 tx=0).stop 2024-05-14T17:11:33.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 -- 172.21.15.69:0/3686709625 shutdown_connections 2024-05-14T17:11:33.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 --2- 172.21.15.69:0/3686709625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad81022e0 0x7f1ad810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 --2- 172.21.15.69:0/3686709625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad8101920 0x7f1ad8101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 --2- 172.21.15.69:0/3686709625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ad8107930 0x7f1ad8107d10 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 -- 172.21.15.69:0/3686709625 >> 172.21.15.69:0/3686709625 conn(0x7f1ad8078a50 msgr2=0x7f1ad8078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:33.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 -- 172.21.15.69:0/3686709625 shutdown_connections 2024-05-14T17:11:33.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 -- 172.21.15.69:0/3686709625 wait complete. 2024-05-14T17:11:33.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 Processor -- start 2024-05-14T17:11:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.359+0000 7f1ade245640 1 -- start start 2024-05-14T17:11:33.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ade245640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ad8101920 0x7f1ad81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:33.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ade245640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad81022e0 0x7f1ad8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ade245640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8107930 0x7f1ad819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:33.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ade245640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad8111bf0 con 0x7f1ad8101920 2024-05-14T17:11:33.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ade245640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad8111a70 con 0x7f1ad8107930 2024-05-14T17:11:33.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ade245640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad8111d70 con 0x7f1ad81022e0 2024-05-14T17:11:33.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ad8101920 0x7f1ad81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:33.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ad8101920 0x7f1ad81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55040/0 (socket says 172.21.15.69:55040) 2024-05-14T17:11:33.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad77fe640 1 -- 172.21.15.69:0/3562455280 learned_addr learned my addr 172.21.15.69:0/3562455280 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:33.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1acffff640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad81022e0 0x7f1ad8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:33.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad77fe640 1 -- 172.21.15.69:0/3562455280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad81022e0 msgr2=0x7f1ad8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:33.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad77fe640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad81022e0 0x7f1ad8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad77fe640 1 -- 172.21.15.69:0/3562455280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8107930 msgr2=0x7f1ad819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:33.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad7fff640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8107930 0x7f1ad819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:33.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad77fe640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8107930 0x7f1ad819ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad77fe640 1 -- 172.21.15.69:0/3562455280 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad819d190 con 0x7f1ad8101920 2024-05-14T17:11:33.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1acffff640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad81022e0 0x7f1ad8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad77fe640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ad8101920 0x7f1ad81981f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8004770 tx=0x7f1ac80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:33.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad57fa640 1 -- 172.21.15.69:0/3562455280 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac803f070 con 0x7f1ad8101920 2024-05-14T17:11:33.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad57fa640 1 -- 172.21.15.69:0/3562455280 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac803b460 con 0x7f1ad8101920 2024-05-14T17:11:33.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ade245640 1 -- 172.21.15.69:0/3562455280 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad819d420 con 0x7f1ad8101920 2024-05-14T17:11:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.360+0000 7f1ad57fa640 1 -- 172.21.15.69:0/3562455280 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8035070 con 0x7f1ad8101920 2024-05-14T17:11:33.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.361+0000 7f1ade245640 1 -- 172.21.15.69:0/3562455280 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad81a4d10 con 0x7f1ad8101920 2024-05-14T17:11:33.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.362+0000 7f1ade245640 1 -- 172.21.15.69:0/3562455280 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad8069870 con 0x7f1ad8101920 2024-05-14T17:11:33.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:33 smithi092 ceph-mon[35763]: from='client.17601 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:11:33.818 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:33 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/208040125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:11:33.825 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:33 smithi203 ceph-mon[35787]: from='client.17601 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:11:33.832 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:33 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/208040125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:11:33.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.363+0000 7f1ad57fa640 1 -- 172.21.15.69:0/3562455280 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1ac803b8d0 con 0x7f1ad8101920 2024-05-14T17:11:33.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.363+0000 7f1ad57fa640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ac40b29e0 0x7f1ac40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:33.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.363+0000 7f1acffff640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ac40b29e0 0x7f1ac40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:33.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.364+0000 7f1ad57fa640 1 -- 172.21.15.69:0/3562455280 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f1ac8102a20 con 0x7f1ad8101920 2024-05-14T17:11:33.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.364+0000 7f1acffff640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ac40b29e0 0x7f1ac40b4ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0006fd0 tx=0x7f1ac0008040 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:33.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.368+0000 7f1ad57fa640 1 -- 172.21.15.69:0/3562455280 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ac80ca3f0 con 0x7f1ad8101920 2024-05-14T17:11:33.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.550+0000 7f1ade245640 1 -- 172.21.15.69:0/3562455280 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ad8109e90 con 0x7f1ad8101920 2024-05-14T17:11:33.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.550+0000 7f1ad57fa640 1 -- 172.21.15.69:0/3562455280 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7f1ac80c9cf0 con 0x7f1ad8101920 2024-05-14T17:11:33.908 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-05-14T17:11:33.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 -- 172.21.15.69:0/3562455280 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ac40b29e0 msgr2=0x7f1ac40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:33.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ac40b29e0 0x7f1ac40b4ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0006fd0 tx=0x7f1ac0008040 comp rx=0 tx=0).stop 2024-05-14T17:11:33.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 -- 172.21.15.69:0/3562455280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ad8101920 msgr2=0x7f1ad81981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:33.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ad8101920 0x7f1ad81981f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8004770 tx=0x7f1ac80042f0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 -- 172.21.15.69:0/3562455280 shutdown_connections 2024-05-14T17:11:34.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1ac40b29e0 0x7f1ac40b4ea0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8107930 0x7f1ad819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad81022e0 0x7f1ad8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 --2- 172.21.15.69:0/3562455280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ad8101920 0x7f1ad81981f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 -- 172.21.15.69:0/3562455280 >> 172.21.15.69:0/3562455280 conn(0x7f1ad8078a50 msgr2=0x7f1ad8104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:34.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.552+0000 7f1ade245640 1 -- 172.21.15.69:0/3562455280 shutdown_connections 2024-05-14T17:11:34.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:33.553+0000 7f1ade245640 1 -- 172.21.15.69:0/3562455280 wait complete. 2024-05-14T17:11:34.077 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:33 smithi069 ceph-mon[31507]: from='client.17601 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:11:34.085 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:33 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/208040125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:11:34.094 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-14T17:11:34.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.206+0000 7f490d119640 1 -- 172.21.15.69:0/189144730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49081033e0 msgr2=0x7f490810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:34.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.206+0000 7f490d119640 1 --2- 172.21.15.69:0/189144730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49081033e0 0x7f490810f8f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4900009cd0 tx=0x7f490002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.206+0000 7f490d119640 1 -- 172.21.15.69:0/189144730 shutdown_connections 2024-05-14T17:11:34.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.206+0000 7f490d119640 1 --2- 172.21.15.69:0/189144730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49081033e0 0x7f490810f8f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.206+0000 7f490d119640 1 --2- 172.21.15.69:0/189144730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4908102a20 0x7f4908102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.206+0000 7f490d119640 1 --2- 172.21.15.69:0/189144730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4908108a30 0x7f4908108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.206+0000 7f490d119640 1 -- 172.21.15.69:0/189144730 >> 172.21.15.69:0/189144730 conn(0x7f4908079ea0 msgr2=0x7f490807a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:34.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.206+0000 7f490d119640 1 -- 172.21.15.69:0/189144730 shutdown_connections 2024-05-14T17:11:34.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.206+0000 7f490d119640 1 -- 172.21.15.69:0/189144730 wait complete. 2024-05-14T17:11:34.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.206+0000 7f490d119640 1 Processor -- start 2024-05-14T17:11:34.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490d119640 1 -- start start 2024-05-14T17:11:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490d119640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4908102a20 0x7f4908074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:34.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490d119640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49081033e0 0x7f4908075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490d119640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4908108a30 0x7f49080757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:34.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490d119640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4908112cc0 con 0x7f4908108a30 2024-05-14T17:11:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490d119640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4908112b40 con 0x7f4908102a20 2024-05-14T17:11:34.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490d119640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4908112e40 con 0x7f49081033e0 2024-05-14T17:11:34.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4908108a30 0x7f49080757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:34.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4908108a30 0x7f49080757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55070/0 (socket says 172.21.15.69:55070) 2024-05-14T17:11:34.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490759e640 1 -- 172.21.15.69:0/4026780867 learned_addr learned my addr 172.21.15.69:0/4026780867 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:34.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f4906d9d640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4908102a20 0x7f4908074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:34.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490659c640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49081033e0 0x7f4908075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:34.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490759e640 1 -- 172.21.15.69:0/4026780867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49081033e0 msgr2=0x7f4908075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:34.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490759e640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49081033e0 0x7f4908075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490759e640 1 -- 172.21.15.69:0/4026780867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4908102a20 msgr2=0x7f4908074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:34.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490759e640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4908102a20 0x7f4908074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490759e640 1 -- 172.21.15.69:0/4026780867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49081a7740 con 0x7f4908108a30 2024-05-14T17:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.207+0000 7f490659c640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49081033e0 0x7f4908075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:34.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.208+0000 7f490759e640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4908108a30 0x7f49080757d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4900033040 tx=0x7f4900003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.208+0000 7f48fffff640 1 -- 172.21.15.69:0/4026780867 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900004040 con 0x7f4908108a30 2024-05-14T17:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.208+0000 7f48fffff640 1 -- 172.21.15.69:0/4026780867 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4900004690 con 0x7f4908108a30 2024-05-14T17:11:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.208+0000 7f490d119640 1 -- 172.21.15.69:0/4026780867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49081a79d0 con 0x7f4908108a30 2024-05-14T17:11:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.208+0000 7f48fffff640 1 -- 172.21.15.69:0/4026780867 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900030e20 con 0x7f4908108a30 2024-05-14T17:11:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.208+0000 7f490d119640 1 -- 172.21.15.69:0/4026780867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49081a7f20 con 0x7f4908108a30 2024-05-14T17:11:34.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.209+0000 7f490d119640 1 -- 172.21.15.69:0/4026780867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4908069cc0 con 0x7f4908108a30 2024-05-14T17:11:34.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.210+0000 7f48fffff640 1 -- 172.21.15.69:0/4026780867 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f49000305a0 con 0x7f4908108a30 2024-05-14T17:11:34.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.211+0000 7f48fffff640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f48f40b2960 0x7f48f40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:34.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.211+0000 7f4906d9d640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f48f40b2960 0x7f48f40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.211+0000 7f48fffff640 1 -- 172.21.15.69:0/4026780867 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f490002e030 con 0x7f4908108a30 2024-05-14T17:11:34.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.211+0000 7f4906d9d640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f48f40b2960 0x7f48f40b4e20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f48f800bb60 tx=0x7f48f800b390 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:34.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.215+0000 7f48fffff640 1 -- 172.21.15.69:0/4026780867 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f490004b060 con 0x7f4908108a30 2024-05-14T17:11:34.776 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:34 smithi092 ceph-mon[35763]: pgmap v689: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-14T17:11:34.787 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:34 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3562455280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T17:11:34.796 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:34 smithi203 ceph-mon[35787]: pgmap v689: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-14T17:11:34.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:34 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3562455280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T17:11:34.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.397+0000 7f490d119640 1 -- 172.21.15.69:0/4026780867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f490802dba0 con 0x7f4908108a30 2024-05-14T17:11:34.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.398+0000 7f48fffff640 1 -- 172.21.15.69:0/4026780867 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) ==== 104+0+11 (secure 0 0 0) 0x7f49000cb790 con 0x7f4908108a30 2024-05-14T17:11:34.846 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-05-14T17:11:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 -- 172.21.15.69:0/4026780867 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f48f40b2960 msgr2=0x7f48f40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:34.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f48f40b2960 0x7f48f40b4e20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f48f800bb60 tx=0x7f48f800b390 comp rx=0 tx=0).stop 2024-05-14T17:11:34.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 -- 172.21.15.69:0/4026780867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4908108a30 msgr2=0x7f49080757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4908108a30 0x7f49080757d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4900033040 tx=0x7f4900003970 comp rx=0 tx=0).stop 2024-05-14T17:11:34.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 -- 172.21.15.69:0/4026780867 shutdown_connections 2024-05-14T17:11:34.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f48f40b2960 0x7f48f40b4e20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4908108a30 0x7f49080757d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49081033e0 0x7f4908075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 --2- 172.21.15.69:0/4026780867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4908102a20 0x7f4908074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 -- 172.21.15.69:0/4026780867 >> 172.21.15.69:0/4026780867 conn(0x7f4908079ea0 msgr2=0x7f4908100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:34.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 -- 172.21.15.69:0/4026780867 shutdown_connections 2024-05-14T17:11:34.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:34.400+0000 7f490d119640 1 -- 172.21.15.69:0/4026780867 wait complete. 2024-05-14T17:11:34.998 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-14T17:11:35.007 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:34 smithi069 ceph-mon[31507]: pgmap v689: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-14T17:11:35.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:34 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3562455280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T17:11:35.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 -- 172.21.15.69:0/586088795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f995c108a30 msgr2=0x7f995c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:35.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 --2- 172.21.15.69:0/586088795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f995c108a30 0x7f995c108e10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9950009930 tx=0x7f995002a420 comp rx=0 tx=0).stop 2024-05-14T17:11:35.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 -- 172.21.15.69:0/586088795 shutdown_connections 2024-05-14T17:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 --2- 172.21.15.69:0/586088795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c1033e0 0x7f995c10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 --2- 172.21.15.69:0/586088795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f995c102a20 0x7f995c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 --2- 172.21.15.69:0/586088795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f995c108a30 0x7f995c108e10 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 -- 172.21.15.69:0/586088795 >> 172.21.15.69:0/586088795 conn(0x7f995c0fe490 msgr2=0x7f995c1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:35.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 -- 172.21.15.69:0/586088795 shutdown_connections 2024-05-14T17:11:35.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 -- 172.21.15.69:0/586088795 wait complete. 2024-05-14T17:11:35.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 Processor -- start 2024-05-14T17:11:35.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 -- start start 2024-05-14T17:11:35.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.131+0000 7f99635f7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f995c102a20 0x7f995c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:35.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f99635f7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c1033e0 0x7f995c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:35.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f99635f7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f995c108a30 0x7f995c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:35.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f99635f7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f995c112cc0 con 0x7f995c102a20 2024-05-14T17:11:35.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f99635f7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f995c112b40 con 0x7f995c1033e0 2024-05-14T17:11:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f99635f7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f995c112e40 con 0x7f995c108a30 2024-05-14T17:11:35.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9961393640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f995c102a20 0x7f995c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:35.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9961393640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f995c102a20 0x7f995c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42280/0 (socket says 172.21.15.69:42280) 2024-05-14T17:11:35.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9961393640 1 -- 172.21.15.69:0/2105448558 learned_addr learned my addr 172.21.15.69:0/2105448558 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:35.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9961b94640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f995c108a30 0x7f995c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:35.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9960b92640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c1033e0 0x7f995c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:35.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9961393640 1 -- 172.21.15.69:0/2105448558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f995c108a30 msgr2=0x7f995c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9961393640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f995c108a30 0x7f995c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9961393640 1 -- 172.21.15.69:0/2105448558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c1033e0 msgr2=0x7f995c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:35.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9961393640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c1033e0 0x7f995c1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9961393640 1 -- 172.21.15.69:0/2105448558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c19e2a0 con 0x7f995c102a20 2024-05-14T17:11:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f9961393640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f995c102a20 0x7f995c1992b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9950004770 tx=0x7f99500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:35.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f994e7fc640 1 -- 172.21.15.69:0/2105448558 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995003f070 con 0x7f995c102a20 2024-05-14T17:11:35.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f994e7fc640 1 -- 172.21.15.69:0/2105448558 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995003b460 con 0x7f995c102a20 2024-05-14T17:11:35.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f994e7fc640 1 -- 172.21.15.69:0/2105448558 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950035070 con 0x7f995c102a20 2024-05-14T17:11:35.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f99635f7640 1 -- 172.21.15.69:0/2105448558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995c19e530 con 0x7f995c102a20 2024-05-14T17:11:35.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.132+0000 7f99635f7640 1 -- 172.21.15.69:0/2105448558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995c1a5e20 con 0x7f995c102a20 2024-05-14T17:11:35.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.133+0000 7f99635f7640 1 -- 172.21.15.69:0/2105448558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995c069cc0 con 0x7f995c102a20 2024-05-14T17:11:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.135+0000 7f994e7fc640 1 -- 172.21.15.69:0/2105448558 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f995003b8d0 con 0x7f995c102a20 2024-05-14T17:11:35.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.136+0000 7f994e7fc640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99480b2960 0x7f99480b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:35.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.136+0000 7f9960b92640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99480b2960 0x7f99480b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:35.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.136+0000 7f994e7fc640 1 -- 172.21.15.69:0/2105448558 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f9950102ce0 con 0x7f995c102a20 2024-05-14T17:11:35.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.136+0000 7f9960b92640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99480b2960 0x7f99480b4e20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9958005fd0 tx=0x7f9958005850 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:35.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.140+0000 7f994e7fc640 1 -- 172.21.15.69:0/2105448558 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f99500ca6b0 con 0x7f995c102a20 2024-05-14T17:11:35.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.321+0000 7f99635f7640 1 -- 172.21.15.69:0/2105448558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f995c10af90 con 0x7f995c102a20 2024-05-14T17:11:35.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.322+0000 7f994e7fc640 1 -- 172.21.15.69:0/2105448558 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) ==== 100+0+11 (secure 0 0 0) 0x7f99500c9fb0 con 0x7f995c102a20 2024-05-14T17:11:35.615 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-05-14T17:11:35.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 -- 172.21.15.69:0/2105448558 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99480b2960 msgr2=0x7f99480b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99480b2960 0x7f99480b4e20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9958005fd0 tx=0x7f9958005850 comp rx=0 tx=0).stop 2024-05-14T17:11:35.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 -- 172.21.15.69:0/2105448558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f995c102a20 msgr2=0x7f995c1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:35.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f995c102a20 0x7f995c1992b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9950004770 tx=0x7f99500042f0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 -- 172.21.15.69:0/2105448558 shutdown_connections 2024-05-14T17:11:35.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f99480b2960 0x7f99480b4e20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f995c108a30 0x7f995c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c1033e0 0x7f995c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 --2- 172.21.15.69:0/2105448558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f995c102a20 0x7f995c1992b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 -- 172.21.15.69:0/2105448558 >> 172.21.15.69:0/2105448558 conn(0x7f995c0fe490 msgr2=0x7f995c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:35.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 -- 172.21.15.69:0/2105448558 shutdown_connections 2024-05-14T17:11:35.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.324+0000 7f99635f7640 1 -- 172.21.15.69:0/2105448558 wait complete. 2024-05-14T17:11:35.724 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-14T17:11:35.733 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:35 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/4026780867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T17:11:35.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:35 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2105448558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T17:11:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:35 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/4026780867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T17:11:35.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:35 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2105448558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T17:11:35.803 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:35 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/4026780867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T17:11:35.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:35 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2105448558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T17:11:35.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.904+0000 7fea74837640 1 -- 172.21.15.69:0/1290287590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea70104600 msgr2=0x7fea7010cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:35.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.904+0000 7fea74837640 1 --2- 172.21.15.69:0/1290287590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea70104600 0x7fea7010cae0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fea58009900 tx=0x7fea5802a510 comp rx=0 tx=0).stop 2024-05-14T17:11:35.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.904+0000 7fea74837640 1 -- 172.21.15.69:0/1290287590 shutdown_connections 2024-05-14T17:11:35.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.904+0000 7fea74837640 1 --2- 172.21.15.69:0/1290287590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea7010d020 0x7fea7010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.904+0000 7fea74837640 1 --2- 172.21.15.69:0/1290287590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea70104600 0x7fea7010cae0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.904+0000 7fea74837640 1 --2- 172.21.15.69:0/1290287590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea70103ce0 0x7fea701040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.904+0000 7fea74837640 1 -- 172.21.15.69:0/1290287590 >> 172.21.15.69:0/1290287590 conn(0x7fea70078a50 msgr2=0x7fea70078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:35.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.904+0000 7fea74837640 1 -- 172.21.15.69:0/1290287590 shutdown_connections 2024-05-14T17:11:35.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.904+0000 7fea74837640 1 -- 172.21.15.69:0/1290287590 wait complete. 2024-05-14T17:11:36.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.904+0000 7fea74837640 1 Processor -- start 2024-05-14T17:11:36.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea74837640 1 -- start start 2024-05-14T17:11:36.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea74837640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea70103ce0 0x7fea701992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:36.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea74837640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea70104600 0x7fea701997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea74837640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea7010d020 0x7fea7019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:36.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea74837640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea70112cd0 con 0x7fea70103ce0 2024-05-14T17:11:36.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea74837640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fea70112b50 con 0x7fea7010d020 2024-05-14T17:11:36.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea74837640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea70112e50 con 0x7fea70104600 2024-05-14T17:11:36.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea70103ce0 0x7fea701992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:36.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea70103ce0 0x7fea701992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42294/0 (socket says 172.21.15.69:42294) 2024-05-14T17:11:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6e59c640 1 -- 172.21.15.69:0/3409620834 learned_addr learned my addr 172.21.15.69:0/3409620834 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:11:36.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6dd9b640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea70104600 0x7fea701997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:36.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6ed9d640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea7010d020 0x7fea7019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:36.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6e59c640 1 -- 172.21.15.69:0/3409620834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea70104600 msgr2=0x7fea701997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:36.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6e59c640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea70104600 0x7fea701997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:36.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6e59c640 1 -- 172.21.15.69:0/3409620834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea7010d020 msgr2=0x7fea7019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:36.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6e59c640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea7010d020 0x7fea7019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:36.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6e59c640 1 -- 172.21.15.69:0/3409620834 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7019e290 con 0x7fea70103ce0 2024-05-14T17:11:36.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6ed9d640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea7010d020 0x7fea7019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:36.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea6e59c640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea70103ce0 0x7fea701992a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fea6000ea80 tx=0x7fea6000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:36.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea677fe640 1 -- 172.21.15.69:0/3409620834 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6000ce40 con 0x7fea70103ce0 2024-05-14T17:11:36.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea74837640 1 -- 172.21.15.69:0/3409620834 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7019e580 con 0x7fea70103ce0 2024-05-14T17:11:36.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea677fe640 1 -- 172.21.15.69:0/3409620834 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea60005020 con 0x7fea70103ce0 2024-05-14T17:11:36.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea677fe640 1 -- 172.21.15.69:0/3409620834 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea600116d0 con 0x7fea70103ce0 2024-05-14T17:11:36.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.905+0000 7fea74837640 1 -- 172.21.15.69:0/3409620834 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea701a5e10 con 0x7fea70103ce0 2024-05-14T17:11:36.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.906+0000 7fea74837640 1 -- 172.21.15.69:0/3409620834 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea70069870 con 0x7fea70103ce0 2024-05-14T17:11:36.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.908+0000 7fea677fe640 1 -- 172.21.15.69:0/3409620834 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fea600054d0 con 0x7fea70103ce0 2024-05-14T17:11:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.908+0000 7fea677fe640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea5c0b2960 0x7fea5c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:36.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.908+0000 7fea6dd9b640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea5c0b2960 0x7fea5c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:36.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.908+0000 7fea677fe640 1 -- 172.21.15.69:0/3409620834 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7fea600e2ca0 con 0x7fea70103ce0 2024-05-14T17:11:36.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.909+0000 7fea6dd9b640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea5c0b2960 0x7fea5c0b4e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea58031440 tx=0x7fea58005b50 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:36.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:35.912+0000 7fea677fe640 1 -- 172.21.15.69:0/3409620834 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea60016030 con 0x7fea70103ce0 2024-05-14T17:11:36.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.140+0000 7fea74837640 1 -- 172.21.15.69:0/3409620834 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fea701081d0 con 0x7fea70103ce0 2024-05-14T17:11:36.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.148+0000 7fea677fe640 1 -- 172.21.15.69:0/3409620834 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816175 (secure 0 0 0) 0x7fea600aa670 con 0x7fea70103ce0 2024-05-14T17:11:36.388 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:11:36.419 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:11:36.428 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-05-14T17:11:36.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:36.464 INFO:teuthology.orchestra.run.smithi069.stdout: "active_gid": 24764, 2024-05-14T17:11:36.474 INFO:teuthology.orchestra.run.smithi069.stdout: "active_name": "x", 2024-05-14T17:11:36.483 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addrs": { 2024-05-14T17:11:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-05-14T17:11:36.502 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:11:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-05-14T17:11:36.521 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6800", 2024-05-14T17:11:36.530 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4217409403 2024-05-14T17:11:36.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:36.548 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:11:36.557 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-05-14T17:11:36.634 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6801", 2024-05-14T17:11:36.643 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4217409403 2024-05-14T17:11:36.652 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:36.661 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:11:36.671 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:36.678 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addr": "172.21.15.69:6801/4217409403", 2024-05-14T17:11:36.705 INFO:teuthology.orchestra.run.smithi069.stdout: "active_change": "2024-05-14T16:49:54.600787+0000", 2024-05-14T17:11:36.713 INFO:teuthology.orchestra.run.smithi069.stdout: "active_mgr_features": 4540701547738038271, 2024-05-14T17:11:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: "available": true, 2024-05-14T17:11:36.730 INFO:teuthology.orchestra.run.smithi069.stdout: "standbys": [ 2024-05-14T17:11:36.738 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:11:36.747 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 24733, 2024-05-14T17:11:36.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "z", 2024-05-14T17:11:36.765 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540701547738038271, 2024-05-14T17:11:36.774 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-05-14T17:11:36.782 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:11:36.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-05-14T17:11:36.828 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:11:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:11:36.850 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:11:36.862 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:11:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:11:36.885 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:11:36.942 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:36.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:36.965 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:11:36.975 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:36.986 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:37.018 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:37.037 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T17:11:37.046 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:37.053 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:37.060 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:37.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:37.076 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:11:37.084 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:11:37.092 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:37.101 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:37.109 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:37.118 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:37.142 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:37.175 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:37.182 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:37.192 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:11:37.200 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:11:37.208 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:11:37.220 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:11:37.228 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:11:37.235 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:11:37.243 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:37.250 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:37.259 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:37.268 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:37.276 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:37.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:37.294 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:11:37.303 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:11:37.326 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:37.354 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:37.363 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:37.371 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:37.380 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:37.389 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:37.397 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:37.405 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:37.412 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:37.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:11:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:11:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:11:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:11:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:11:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:11:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:11:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:11:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:11:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:11:37.640 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:11:37.651 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:37.660 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:37.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:37.678 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:37.687 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:37.695 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:37.703 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:37.712 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:37.720 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:37.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:37.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:37.781 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:37.790 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-05-14T17:11:37.798 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-05-14T17:11:37.808 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:37.818 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:37.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:37.862 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:37.870 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:37.879 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:37.886 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:37.894 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-05-14T17:11:37.906 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:37.915 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:37.945 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:37.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:37.963 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-05-14T17:11:37.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-05-14T17:11:37.981 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:37.989 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:37.998 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:38.007 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-05-14T17:11:38.016 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:38.051 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:38.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:38.075 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-05-14T17:11:38.085 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:38.111 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:38.121 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:38.133 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:38.142 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-05-14T17:11:38.153 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-05-14T17:11:38.164 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:38.175 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:38.185 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:38.195 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:38.206 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:38.218 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:38.229 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:38.240 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-05-14T17:11:38.282 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:38.294 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:38.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:38.314 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-05-14T17:11:38.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-05-14T17:11:38.348 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:38.359 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:38.368 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:38.378 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:38.387 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:38.395 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:38.404 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:38.413 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-05-14T17:11:38.422 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:38.430 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:38.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:38.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:38.492 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-05-14T17:11:38.501 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-05-14T17:11:38.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:38.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:38.526 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:38.534 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-05-14T17:11:38.543 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:38.552 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:38.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:38.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-05-14T17:11:38.584 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:38.592 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:38.600 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:38.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:38.616 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-05-14T17:11:38.623 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-05-14T17:11:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:38.661 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:38.670 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:38.688 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:38.697 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:38.706 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:38.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-05-14T17:11:38.723 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:38.732 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:38.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:38.750 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:38.758 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-05-14T17:11:38.766 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-05-14T17:11:38.793 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:38.817 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:38.824 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:38.851 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:38.857 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:38.863 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:38.877 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:38.884 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T17:11:38.893 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:38.900 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:38.910 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:38.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:38.926 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-05-14T17:11:38.933 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-05-14T17:11:38.940 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:38.947 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:38.964 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:38.971 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:38.990 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:39.017 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:39.032 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-05-14T17:11:39.041 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:39.052 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:39.061 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:39.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:39.080 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:11:39.089 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:11:39.098 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:39.107 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:11:39.116 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:39.191 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:11:39.201 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:39.210 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:39.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:39.227 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:39.243 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:39.293 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:39.302 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:39.309 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:39.317 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:39.328 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:39.337 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:11:39.344 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-05-14T17:11:39.351 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:11:39.391 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:11:39.398 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:11:39.405 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-05-14T17:11:39.413 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-05-14T17:11:39.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:39.430 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:39.439 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:39.447 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:39.457 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:39.483 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:39.492 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:39.501 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T17:11:39.509 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:39.518 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:39.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:39.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:39.583 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-05-14T17:11:39.594 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-05-14T17:11:39.604 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:39.614 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:39.626 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:39.638 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-05-14T17:11:39.648 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:39.660 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:39.671 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:39.691 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T17:11:39.723 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:11:39.750 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:39.772 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:39.787 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-05-14T17:11:39.797 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-05-14T17:11:39.807 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:11:39.818 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:39.829 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:39.840 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:11:39.847 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-05-14T17:11:39.856 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-05-14T17:11:39.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:39.874 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T17:11:39.908 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:11:39.917 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:39.933 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:39.961 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:39.970 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-05-14T17:11:39.977 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-05-14T17:11:39.985 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:11:39.993 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:40.003 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:40.015 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-05-14T17:11:40.025 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:11:40.035 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-05-14T17:11:40.046 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:40.056 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T17:11:40.085 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:40.096 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:40.106 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:40.119 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:40.131 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-05-14T17:11:40.141 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-05-14T17:11:40.165 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:40.174 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:40.184 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:40.194 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T17:11:40.202 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:40.210 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:40.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:40.225 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T17:11:40.233 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T17:11:40.241 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:40.268 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:40.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:40.286 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-05-14T17:11:40.294 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-05-14T17:11:40.302 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:11:40.310 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:40.342 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:40.372 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-05-14T17:11:40.379 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-05-14T17:11:40.387 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-05-14T17:11:40.394 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:40.401 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-05-14T17:11:40.407 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T17:11:40.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:40.448 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:40.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:40.460 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-05-14T17:11:40.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-05-14T17:11:40.474 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:40.481 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:40.488 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:40.494 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-05-14T17:11:40.501 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:40.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:40.540 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:40.546 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T17:11:40.552 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:11:40.558 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:40.566 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:40.575 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:40.585 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-05-14T17:11:40.606 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-05-14T17:11:40.616 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:11:40.626 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:40.634 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:40.644 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:11:40.654 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-05-14T17:11:40.663 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-05-14T17:11:40.673 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:40.682 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T17:11:40.692 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:11:40.730 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:40.740 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:40.749 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:40.760 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:11:40.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:11:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:11:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:11:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:11:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:11:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:11:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:11:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:11:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:11:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:11:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:11:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:11:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:11:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:11:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:11:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:11:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:11:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:11:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:11:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:11:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-05-14T17:11:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-05-14T17:11:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:11:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:11:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T17:11:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-05-14T17:11:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-05-14T17:11:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-05-14T17:11:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-05-14T17:11:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-05-14T17:11:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-05-14T17:11:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-05-14T17:11:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-05-14T17:11:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-05-14T17:11:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:41.910 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-05-14T17:11:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-05-14T17:11:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T17:11:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:11:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:11:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:11:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:11:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T17:11:42.169 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:42.175 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:42.181 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:42.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:42.193 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:11:42.224 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:11:42.234 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:42.243 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:11:42.253 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:42.261 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:11:42.270 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:42.295 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:42.304 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:42.341 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:42.351 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:42.359 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:42.369 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-05-14T17:11:42.375 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-05-14T17:11:42.383 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:42.394 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:42.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:42.450 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:11:42.458 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:11:42.466 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:42.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:42.491 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T17:11:42.498 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T17:11:42.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:42.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:42.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:42.531 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-05-14T17:11:42.541 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-05-14T17:11:42.549 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:11:42.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:42.568 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:42.579 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:11:42.589 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:42.641 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:42.679 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:42.692 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T17:11:42.703 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:42.713 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:42.723 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:42.733 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:42.743 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:42.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:42.763 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:11:42.773 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-05-14T17:11:42.784 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:11:42.794 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:11:42.803 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:11:42.848 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-05-14T17:11:42.858 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-05-14T17:11:42.869 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:11:42.883 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:42.935 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:42.945 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-05-14T17:11:42.954 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:42.993 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:43.019 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:43.027 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T17:11:43.035 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:43.044 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:43.052 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:43.067 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-05-14T17:11:43.075 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-05-14T17:11:43.083 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:11:43.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:43.100 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:43.111 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-05-14T17:11:43.142 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:43.151 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:43.158 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:43.201 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T17:11:43.209 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:43.218 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:43.231 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:43.238 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:43.245 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-05-14T17:11:43.254 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-05-14T17:11:43.263 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:43.272 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:43.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:43.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-05-14T17:11:43.300 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:43.309 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:43.319 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:43.380 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T17:11:43.389 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:43.397 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:43.405 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:43.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:43.423 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-05-14T17:11:43.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-05-14T17:11:43.440 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:43.447 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:43.453 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:43.461 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:43.467 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:43.472 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:43.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:43.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T17:11:43.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T17:11:43.500 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:43.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:43.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:43.559 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-05-14T17:11:43.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-05-14T17:11:43.578 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:11:43.585 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:43.591 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:43.598 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:11:43.603 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:43.609 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:43.616 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:43.622 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T17:11:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:43.636 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:43.644 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:43.653 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:43.663 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-05-14T17:11:43.691 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T17:11:43.714 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:11:43.723 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:43.731 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:43.740 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-05-14T17:11:43.748 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:43.757 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:43.766 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:43.774 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T17:11:43.784 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:43.793 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:43.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:43.813 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:43.822 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-05-14T17:11:43.831 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-05-14T17:11:43.839 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:43.848 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:43.872 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:43.880 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:43.891 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:43.902 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:43.913 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:43.922 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-05-14T17:11:43.930 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-05-14T17:11:43.938 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-05-14T17:11:43.946 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:43.954 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T17:11:43.962 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:43.970 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:43.980 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:43.988 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:43.997 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-05-14T17:11:44.007 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-05-14T17:11:44.020 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:44.043 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:44.051 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:44.060 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:44.070 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:44.080 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:44.090 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:44.100 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T17:11:44.110 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:44.120 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:44.131 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:44.140 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:44.149 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-05-14T17:11:44.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-05-14T17:11:44.192 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:44.203 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:44.213 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:44.223 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:44.256 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:44.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:44.277 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:44.287 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T17:11:44.298 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:44.354 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:44.363 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:44.373 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-05-14T17:11:44.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-05-14T17:11:44.388 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:44.396 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:44.403 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:44.410 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:44.418 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:44.426 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:44.434 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T17:11:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:44.511 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:44.519 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:44.533 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:44.555 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-05-14T17:11:44.562 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-05-14T17:11:44.569 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:44.575 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:44.581 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:44.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T17:11:44.594 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:44.599 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:44.605 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:44.613 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:11:44.620 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:44.629 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:44.655 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:44.663 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:44.673 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-05-14T17:11:44.682 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-05-14T17:11:44.691 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:44.739 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:44.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:44.756 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T17:11:44.764 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:44.773 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:44.781 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:44.789 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-05-14T17:11:44.796 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:44.805 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:44.812 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:44.819 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:44.829 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-05-14T17:11:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-05-14T17:11:44.860 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:44.866 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:44.872 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:44.878 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T17:11:44.889 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:44.896 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:44.902 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:44.908 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-05-14T17:11:44.914 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:44.920 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:44.926 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-05-14T17:11:44.944 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-05-14T17:11:44.950 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:44.959 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:44.992 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:44.998 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T17:11:45.004 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:45.013 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:45.039 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:45.049 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:11:45.058 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:45.066 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:45.075 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:45.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:45.092 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-05-14T17:11:45.100 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-05-14T17:11:45.109 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:45.119 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:45.128 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:45.139 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T17:11:45.155 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:45.189 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:45.234 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:45.245 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-05-14T17:11:45.256 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:45.266 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:45.279 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:45.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:45.300 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-05-14T17:11:45.311 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-05-14T17:11:45.322 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:45.332 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:45.343 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:45.378 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T17:11:45.388 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:45.396 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:45.403 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:45.422 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-05-14T17:11:45.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:45.462 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:45.470 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:45.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:45.486 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-05-14T17:11:45.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-05-14T17:11:45.501 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:45.508 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:45.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:45.524 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T17:11:45.534 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:45.566 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:45.574 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:45.581 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-05-14T17:11:45.590 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:45.600 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:45.614 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:45.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:45.654 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-05-14T17:11:45.665 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-05-14T17:11:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:45.695 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:45.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T17:11:45.715 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:45.747 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:45.756 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:45.766 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-05-14T17:11:45.776 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:45.788 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:45.798 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:45.806 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:45.817 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-05-14T17:11:45.824 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-05-14T17:11:45.849 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:45.856 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:45.862 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:45.869 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T17:11:45.876 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:45.883 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:45.932 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:45.939 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-05-14T17:11:45.945 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:45.950 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:45.957 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:45.962 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:45.968 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-05-14T17:11:45.974 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-05-14T17:11:45.979 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:45.985 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:46.015 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:46.046 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T17:11:46.051 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:46.057 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:46.084 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:46.091 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-05-14T17:11:46.102 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:46.109 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:46.115 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:46.123 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:46.129 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-05-14T17:11:46.138 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-05-14T17:11:46.147 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:46.155 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:46.163 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:46.173 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T17:11:46.182 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:46.191 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:46.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:46.228 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:11:46.258 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:46.266 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:46.276 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:46.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:46.287 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-05-14T17:11:46.293 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-05-14T17:11:46.299 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:46.305 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:46.311 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:46.317 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-14T17:11:46.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:46.328 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:46.336 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:46.341 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-05-14T17:11:46.365 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:46.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:46.396 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:46.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:46.431 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-05-14T17:11:46.437 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-05-14T17:11:46.444 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:46.449 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:46.456 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:46.464 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T17:11:46.472 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:46.480 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:46.488 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:46.498 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:11:46.509 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:46.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:46.530 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:46.541 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:46.581 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-05-14T17:11:46.590 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-05-14T17:11:46.601 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:46.613 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:46.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:46.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T17:11:46.644 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:46.655 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:46.666 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:46.676 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-05-14T17:11:46.721 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:46.730 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:46.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:46.751 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:46.759 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-05-14T17:11:46.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-05-14T17:11:46.791 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:46.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:46.825 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:46.835 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T17:11:46.846 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:46.863 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:46.871 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:46.879 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-05-14T17:11:46.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:46.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:46.904 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:46.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:46.921 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-05-14T17:11:46.928 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-05-14T17:11:46.935 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:46.943 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:46.951 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:46.961 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T17:11:46.990 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:47.000 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:47.020 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-05-14T17:11:47.029 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:47.039 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:47.048 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:47.058 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:47.068 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-05-14T17:11:47.077 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-05-14T17:11:47.087 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:47.099 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:47.108 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:47.117 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:47.126 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:47.135 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:47.143 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T17:11:47.187 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:47.196 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:47.204 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:47.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:47.218 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-05-14T17:11:47.224 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-05-14T17:11:47.230 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:11:47.248 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:47.254 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:47.259 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:11:47.263 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:47.268 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:47.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:47.280 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T17:11:47.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:47.339 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:47.346 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:47.352 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:47.358 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-05-14T17:11:47.364 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T17:11:47.370 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:47.375 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:47.381 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:47.386 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-05-14T17:11:47.392 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:47.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:47.406 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:47.411 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T17:11:47.417 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:47.423 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:47.429 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:47.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:47.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-05-14T17:11:47.480 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-05-14T17:11:47.486 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:47.496 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:47.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:47.514 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-05-14T17:11:47.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:47.538 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:47.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:47.555 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T17:11:47.565 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:47.574 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:47.583 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:47.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:47.600 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-05-14T17:11:47.615 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-05-14T17:11:47.625 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:11:47.648 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:47.654 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:47.662 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-05-14T17:11:47.670 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:47.678 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:47.687 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:47.695 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-05-14T17:11:47.704 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:47.711 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:47.717 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:47.725 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:47.734 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-05-14T17:11:47.742 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-05-14T17:11:47.750 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:47.785 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:47.794 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:47.809 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:47.842 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:47.854 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:47.865 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:47.876 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T17:11:47.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:47.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:47.908 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:47.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:47.931 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-05-14T17:11:47.944 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-05-14T17:11:47.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:11:47.964 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:47.974 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:47.991 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:11:48.000 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:48.009 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:48.026 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:48.055 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-05-14T17:11:48.063 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:48.072 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:48.083 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:48.092 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:48.099 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-05-14T17:11:48.108 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-05-14T17:11:48.116 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:48.142 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:48.149 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:48.158 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T17:11:48.168 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:48.176 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:48.184 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:48.190 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T17:11:48.196 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:48.209 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:48.240 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:48.253 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:48.263 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-05-14T17:11:48.275 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-05-14T17:11:48.287 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:11:48.300 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:48.359 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:48.370 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:11:48.382 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:48.408 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:48.421 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-05-14T17:11:48.433 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:48.444 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:48.456 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:48.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:48.485 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-05-14T17:11:48.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-05-14T17:11:48.531 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:48.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:48.556 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:48.567 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:48.627 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:48.639 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:48.651 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:48.662 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T17:11:48.678 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:48.691 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:48.705 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:48.715 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:48.725 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-05-14T17:11:48.737 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-05-14T17:11:48.747 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:48.760 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:48.791 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:48.804 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:48.815 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:48.825 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:48.862 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:48.873 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T17:11:48.882 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:48.892 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:48.904 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:48.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:48.923 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:11:48.934 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:11:48.945 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:48.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:48.966 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:48.976 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:48.993 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:49.023 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:49.054 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:49.064 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:11:49.074 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:11:49.086 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:11:49.098 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:11:49.107 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:11:49.117 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:11:49.126 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:49.135 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:49.147 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:49.158 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:49.168 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:49.179 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:49.190 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-05-14T17:11:49.200 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-05-14T17:11:49.222 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:49.258 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:49.275 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:49.289 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:49.307 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:49.320 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:49.331 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:49.346 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T17:11:49.361 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:49.373 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:49.388 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:49.399 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:49.413 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:11:49.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:11:49.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:49.465 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:49.478 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:49.511 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:49.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:49.557 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:49.571 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:49.581 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T17:11:49.593 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:49.604 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:49.618 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:49.630 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:49.645 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:11:49.675 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:11:49.689 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:49.705 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:49.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:49.739 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:11:49.751 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:49.767 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:49.787 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:49.821 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:11:49.833 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:11:49.843 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:11:49.854 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:11:49.865 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:11:49.902 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:11:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:49.926 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:49.935 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:49.946 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:49.958 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:49.969 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:49.980 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:11:49.991 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:11:50.000 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:50.012 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:50.029 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:50.053 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:50.062 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:50.072 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:50.080 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:50.124 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:50.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:50.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:50.153 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:50.161 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:50.170 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T17:11:50.179 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T17:11:50.187 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:50.196 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:50.204 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:50.212 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:50.221 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:50.237 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:50.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:50.316 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T17:11:50.326 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:50.340 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:50.355 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:50.370 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:50.383 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T17:11:50.398 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T17:11:50.412 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:50.428 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:50.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-05-14T17:11:50.477 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:50.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:50.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:50.521 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T17:11:50.537 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:50.595 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:50.613 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:50.627 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:50.638 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-05-14T17:11:50.651 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-05-14T17:11:50.665 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:50.678 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:50.690 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:50.701 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:11:50.716 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:50.728 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:50.741 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:50.752 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-05-14T17:11:50.763 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:50.774 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:50.787 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:50.805 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:50.860 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-05-14T17:11:50.874 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-05-14T17:11:50.890 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:50.901 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:50.914 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:50.932 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:11:50.946 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:50.957 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:50.969 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:50.979 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T17:11:50.988 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:50.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:51.006 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:51.017 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:51.026 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-05-14T17:11:51.035 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-05-14T17:11:51.055 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:51.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:51.105 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:51.118 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:51.143 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:51.156 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:51.170 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-05-14T17:11:51.181 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:51.191 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:51.202 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:51.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:51.224 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-05-14T17:11:51.234 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-05-14T17:11:51.245 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:51.254 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:51.264 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:51.297 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-05-14T17:11:51.321 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:51.331 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:51.339 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:51.349 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-05-14T17:11:51.356 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-05-14T17:11:51.364 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:51.373 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T17:11:51.381 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:51.390 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:51.398 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:51.407 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:51.416 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-05-14T17:11:51.424 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-05-14T17:11:51.447 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:51.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:51.475 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:51.497 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-05-14T17:11:51.537 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:51.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:51.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:51.583 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T17:11:51.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:51.613 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:51.630 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:51.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:51.656 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-05-14T17:11:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-05-14T17:11:51.733 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:51.746 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:51.759 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:51.775 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T17:11:51.789 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:51.806 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:51.822 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:51.882 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T17:11:51.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:51.911 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:51.979 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:51.992 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:52.005 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-05-14T17:11:52.017 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-05-14T17:11:52.033 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:52.043 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:52.053 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:52.065 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:52.077 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:52.089 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:52.101 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:52.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T17:11:52.125 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:52.141 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:52.190 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:52.238 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:52.250 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-05-14T17:11:52.259 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-05-14T17:11:52.269 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:52.281 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:52.289 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:52.303 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:52.314 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:52.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:52.335 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:52.346 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T17:11:52.356 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:52.365 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:52.375 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:52.385 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:52.398 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-05-14T17:11:52.426 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-05-14T17:11:52.439 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:52.449 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:52.459 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:52.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:52.480 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:52.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:52.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:52.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T17:11:52.519 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:52.528 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:52.538 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:52.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:52.562 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-05-14T17:11:52.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-05-14T17:11:52.579 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:52.589 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:52.622 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:52.636 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:52.647 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:52.662 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:52.677 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:52.693 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T17:11:52.709 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:52.726 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:52.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:52.759 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:52.774 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-05-14T17:11:52.788 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-05-14T17:11:52.806 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:52.820 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:52.861 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:52.877 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:52.893 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:52.925 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:52.942 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:52.959 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T17:11:52.972 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:52.983 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:52.998 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:53.010 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:53.025 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-05-14T17:11:53.039 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-05-14T17:11:53.053 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:53.063 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:53.074 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:53.094 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-05-14T17:11:53.106 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:53.119 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:53.130 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:53.148 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-05-14T17:11:53.195 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:53.206 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:53.216 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:53.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:53.238 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:11:53.249 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:11:53.260 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:53.272 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:11:53.284 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:53.294 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:11:53.305 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:53.361 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:53.372 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:53.382 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:53.394 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:53.448 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:53.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:53.519 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-05-14T17:11:53.529 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-05-14T17:11:53.539 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:53.550 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:53.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:53.571 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:53.580 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:53.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:53.599 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:53.608 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T17:11:53.618 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:53.627 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:53.638 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:53.647 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:53.700 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-05-14T17:11:53.732 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-05-14T17:11:53.765 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:53.779 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:53.792 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:53.802 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:53.815 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:53.827 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:53.839 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:53.853 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T17:11:53.865 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:53.877 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:53.892 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:53.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:53.926 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-05-14T17:11:53.942 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-05-14T17:11:53.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:53.965 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:53.982 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:54.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:54.041 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:54.057 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:54.070 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:54.087 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T17:11:54.120 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:54.139 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:54.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:54.185 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-05-14T17:11:54.197 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-05-14T17:11:54.211 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:54.222 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:54.235 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:54.248 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:54.262 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:54.279 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:54.329 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:54.339 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T17:11:54.365 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:54.377 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:54.385 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:54.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:54.407 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-05-14T17:11:54.418 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-05-14T17:11:54.427 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:54.439 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:54.449 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:54.459 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:54.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:54.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:54.491 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:54.499 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T17:11:54.516 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:54.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:54.566 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:54.577 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:54.585 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-05-14T17:11:54.596 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-05-14T17:11:54.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:54.617 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:54.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:54.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:54.646 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:54.657 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:54.670 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:54.682 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T17:11:54.695 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:54.710 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:54.752 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:54.773 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:54.836 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:54.852 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:54.868 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:11:54.886 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-05-14T17:11:54.901 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:11:54.913 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:11:54.944 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:11:54.959 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:11:54.973 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:11:54.985 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:55.000 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:55.014 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:55.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:55.041 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:55.058 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:55.073 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:55.091 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:11:55.150 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:11:55.183 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:11:55.194 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:11:55.206 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:11:55.220 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:11:55.233 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:55.248 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:55.263 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:55.277 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:55.290 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:55.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:55.316 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:11:55.331 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:11:55.342 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:55.354 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:55.366 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:55.403 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:55.454 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:55.466 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:55.476 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:55.486 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:55.496 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:55.507 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:55.516 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:55.528 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:55.537 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:11:55.550 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:11:55.561 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:55.592 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:55.604 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:55.614 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:11:55.623 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:55.636 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:55.675 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:11:55.711 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:11:55.720 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:11:55.761 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:11:55.772 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:11:55.784 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:11:55.795 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:11:55.807 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:11:55.817 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:55.827 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:55.836 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:55.846 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:55.856 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:55.865 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:11:55.875 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:11:55.886 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:55.895 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:55.918 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:55.977 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:55.991 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:56.005 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:56.018 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:56.034 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:56.048 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:56.062 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:56.076 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:56.093 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:56.107 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-05-14T17:11:56.123 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-05-14T17:11:56.171 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:11:56.186 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:56.201 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:56.213 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-05-14T17:11:56.227 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:56.242 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:56.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:56.313 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T17:11:56.327 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:56.342 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:56.353 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:56.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:56.390 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:11:56.403 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:11:56.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:56.432 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:11:56.443 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:56.454 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:11:56.467 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:56.480 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:56.493 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:56.505 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:56.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:56.546 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:56.556 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:56.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:56.590 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-05-14T17:11:56.601 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-05-14T17:11:56.611 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:11:56.621 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:56.632 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:11:56.644 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-05-14T17:11:56.653 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:56.664 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:56.676 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:56.688 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T17:11:56.700 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:56.712 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:56.722 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:56.734 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:56.785 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:11:56.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:56.803 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:11:56.811 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-05-14T17:11:56.819 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:11:56.828 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:11:56.835 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:11:56.844 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T17:11:56.852 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T17:11:56.861 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:11:56.872 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:56.883 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:56.892 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:11:56.912 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:56.923 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:56.933 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:56.951 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:56.990 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:57.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:57.023 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:57.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:57.056 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T17:11:57.070 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T17:11:57.083 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:57.096 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:57.113 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:57.155 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:57.169 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:57.184 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:57.199 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:57.215 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:57.230 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:57.245 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:57.260 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T17:11:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T17:11:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:57.557 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-05-14T17:11:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T17:11:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:11:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T17:11:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T17:11:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T17:11:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T17:11:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T17:11:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T17:11:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:58.167 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T17:11:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T17:11:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T17:11:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T17:11:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T17:11:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T17:11:58.595 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:58.604 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:58.611 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:58.618 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:58.625 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:58.632 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:58.639 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:58.645 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:58.651 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:58.658 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:58.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:58.671 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:58.677 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T17:11:58.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T17:11:58.693 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:58.702 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:58.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:58.743 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:58.752 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:58.761 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:58.771 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:58.779 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:58.788 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:58.800 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:58.809 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:58.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:58.826 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T17:11:58.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T17:11:58.879 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:58.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:58.897 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:58.906 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:58.913 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:58.927 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:58.957 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:58.967 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:58.978 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:58.989 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:59.029 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:59.040 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:59.051 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T17:11:59.062 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T17:11:59.076 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:59.088 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:59.101 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:59.114 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:59.125 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:59.135 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:59.146 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:59.179 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:59.207 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:59.264 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:59.272 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:59.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:59.290 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T17:11:59.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T17:11:59.331 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:59.339 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:59.347 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:59.356 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:59.364 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:59.375 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:59.385 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:59.396 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:59.409 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:59.425 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:59.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:59.460 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:59.515 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T17:11:59.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T17:11:59.536 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:59.546 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:59.555 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:59.563 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-05-14T17:11:59.572 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:59.580 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:59.589 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:59.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:59.606 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:59.614 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:59.635 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:59.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:59.653 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T17:11:59.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T17:11:59.677 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:11:59.703 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:59.711 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:59.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:11:59.728 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:59.738 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:59.745 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:59.753 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:59.769 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:59.778 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:59.788 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:59.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:59.803 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-05-14T17:11:59.811 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-05-14T17:11:59.818 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:59.825 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:59.833 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:59.844 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:11:59.866 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:59.872 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:59.879 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:11:59.886 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:11:59.892 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:11:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:11:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:11:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:11:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T17:11:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T17:11:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:11:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:11:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:11:59.961 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-05-14T17:11:59.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:11:59.978 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:11:59.995 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:00.021 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:00.030 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:00.046 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:00.055 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:00.065 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:00.072 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T17:12:00.078 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T17:12:00.085 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:00.091 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:00.097 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:00.103 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:00.108 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:00.115 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:00.121 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:00.127 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:00.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:00.145 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:00.208 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:00.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T17:12:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T17:12:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:00.323 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:00.332 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:00.359 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:00.371 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T17:12:00.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T17:12:00.404 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:00.414 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:00.423 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:00.434 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:00.444 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:00.454 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:00.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:00.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:00.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:00.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:00.540 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:00.551 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T17:12:00.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T17:12:00.576 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:00.587 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:00.653 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:00.664 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:00.675 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:00.686 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:00.697 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:00.708 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:00.717 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:00.725 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:00.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:00.746 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:00.757 INFO:teuthology.orchestra.run.smithi069.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T17:12:00.769 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T17:12:00.779 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:00.789 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:00.799 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:00.840 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "[]", 2024-05-14T17:12:00.854 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:00.887 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:00.898 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:00.909 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:00.919 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:00.929 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:00.940 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:00.948 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:00.958 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T17:12:00.966 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T17:12:00.982 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:00.990 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:00.998 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:01.006 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-05-14T17:12:01.014 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:01.023 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:01.031 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:01.047 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:01.077 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:01.094 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:01.102 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:01.116 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T17:12:01.125 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T17:12:01.133 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:01.183 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:01.192 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:01.200 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:01.208 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:01.217 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:01.226 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:01.235 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:01.243 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:01.250 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:01.278 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:01.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:01.352 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T17:12:01.359 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T17:12:01.366 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:01.373 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:01.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:01.386 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:01.394 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:01.401 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:01.409 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:01.419 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:01.427 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:01.436 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:01.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:01.450 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T17:12:01.473 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T17:12:01.503 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:01.577 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:01.584 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:01.591 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:01.598 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:01.605 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:01.616 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:01.623 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:01.629 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:01.635 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:01.642 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:01.649 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:01.655 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T17:12:01.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T17:12:01.668 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:01.676 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:01.690 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:01.717 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:01.736 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:01.745 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:01.754 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:01.761 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:01.767 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:01.778 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:01.788 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:01.797 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:01.806 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T17:12:01.816 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T17:12:01.825 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:01.834 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:01.843 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:01.852 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:01.862 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:01.873 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:01.911 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:01.920 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:01.929 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:01.939 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:01.949 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:01.958 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:01.968 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T17:12:01.977 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T17:12:01.988 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:01.999 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:02.009 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:02.020 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:02.030 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:02.040 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:02.050 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:02.061 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:02.096 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:02.121 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:02.133 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:02.145 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:02.156 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T17:12:02.167 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T17:12:02.179 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:02.189 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:02.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:02.222 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:02.232 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:02.243 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:02.254 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:02.267 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:02.297 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:02.307 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:02.333 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:02.360 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T17:12:02.371 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T17:12:02.381 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:02.392 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:02.402 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:02.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:02.430 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:02.440 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:02.477 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:02.490 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:02.501 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:02.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:02.523 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:02.531 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:02.539 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T17:12:02.547 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T17:12:02.557 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:02.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:02.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:02.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:02.642 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:02.651 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:02.683 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:02.691 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:02.700 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:02.708 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:02.716 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:02.725 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:02.734 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T17:12:02.741 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T17:12:02.750 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:02.758 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:02.766 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:02.774 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:02.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:02.812 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:02.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:02.871 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:02.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:02.886 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:02.893 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:02.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:02.908 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T17:12:02.916 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T17:12:02.923 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:02.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:02.937 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:02.944 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T17:12:02.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:02.958 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:02.966 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:02.992 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:02.998 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:03.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:03.040 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:03.047 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:03.054 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T17:12:03.060 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T17:12:03.066 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:03.072 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:03.079 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:03.085 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:12:03.092 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:03.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:03.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:03.114 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:03.120 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:03.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:03.182 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:03.220 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T17:12:03.227 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T17:12:03.233 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:03.239 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:03.245 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:03.253 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-05-14T17:12:03.259 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:03.267 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:03.280 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:03.289 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:03.298 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:03.307 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:03.317 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:03.326 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:03.350 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T17:12:03.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T17:12:03.421 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:03.440 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:03.449 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-05-14T17:12:03.458 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:03.467 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:03.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:03.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:03.493 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:03.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:03.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:03.540 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:03.551 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T17:12:03.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T17:12:03.574 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:03.609 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:03.638 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:03.647 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:03.658 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:03.668 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:03.701 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:03.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:03.724 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:03.735 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:03.746 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:03.757 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T17:12:03.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T17:12:03.779 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:03.790 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:03.802 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:03.813 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-05-14T17:12:03.851 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:03.916 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:03.937 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:03.944 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:03.956 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:03.967 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:03.977 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:03.988 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T17:12:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T17:12:04.009 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:04.019 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:04.029 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:04.040 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:04.049 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:04.059 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:04.117 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:04.134 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:04.166 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:04.175 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:04.186 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:04.197 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:04.208 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T17:12:04.215 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T17:12:04.224 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:04.232 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:04.240 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:04.249 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:04.257 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:04.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:04.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:04.281 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:04.287 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:04.329 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:04.343 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:04.375 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:04.382 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T17:12:04.390 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T17:12:04.397 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:04.405 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:04.412 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:04.419 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:04.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:04.436 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:04.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:04.452 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:04.460 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:04.466 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:04.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:04.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:04.506 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T17:12:04.518 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T17:12:04.540 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:04.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:04.556 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:04.564 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:04.571 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:04.578 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:04.584 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:04.590 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:04.596 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:04.602 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:04.608 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:04.614 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:04.621 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T17:12:04.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T17:12:04.649 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:04.656 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:04.668 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:04.699 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:12:04.706 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:04.711 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:04.718 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:04.724 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:04.730 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:04.743 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:04.751 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:04.761 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T17:12:04.770 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T17:12:04.781 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:04.794 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:04.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:04.825 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:12:04.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:04.851 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:04.882 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:04.894 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:04.905 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:04.915 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:04.926 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:04.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:04.956 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-05-14T17:12:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-05-14T17:12:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:04.990 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:05.002 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:05.012 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:05.076 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:05.089 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:05.103 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:05.114 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:05.154 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:05.192 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:05.202 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:05.211 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-05-14T17:12:05.248 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-05-14T17:12:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:05.279 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:05.288 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:05.297 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:05.310 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:05.319 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:05.327 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:05.336 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:05.344 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:05.352 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:05.377 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:05.415 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-05-14T17:12:05.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-05-14T17:12:05.429 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:05.446 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:05.454 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:05.462 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:05.470 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:05.477 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:05.485 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-05-14T17:12:05.493 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:05.501 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:05.510 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:05.521 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:05.536 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-05-14T17:12:05.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-05-14T17:12:05.569 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:05.630 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:05.642 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:05.654 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-05-14T17:12:05.665 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:05.688 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:05.702 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:05.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:05.722 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:05.733 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:05.742 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:05.753 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-05-14T17:12:05.762 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-05-14T17:12:05.772 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:05.782 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:05.848 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:05.870 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:05.910 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:05.919 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:05.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:05.939 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:05.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:05.955 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:05.962 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:05.970 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:05.974 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:05.981 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:05.986 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:05.993 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:06.001 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:06.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:06.044 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:06.051 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:06.098 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:06.104 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:06.109 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:06.116 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:06.123 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:06.131 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:06.146 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:06.153 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:06.160 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:06.168 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:06.173 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:06.180 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:06.187 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:06.213 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:06.220 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:06.227 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:06.236 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:06.263 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:06.269 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:06.275 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:06.282 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:06.288 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:06.294 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:06.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:06.308 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:06.319 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:06.327 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:06.336 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:06.345 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:06.358 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:06.389 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:06.400 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:06.406 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:06.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:06.483 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:06.493 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:06.502 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:06.511 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:06.525 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:06.537 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:06.548 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:06.560 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:06.572 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:06.622 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:06.635 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:06.646 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:06.654 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:06.666 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:06.677 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:06.686 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:06.704 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:06.738 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:06.748 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:06.760 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:06.770 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:06.779 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:06.788 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:06.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:06.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:06.824 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:06.833 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:06.843 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-05-14T17:12:06.852 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-05-14T17:12:06.863 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:06.872 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:06.881 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:06.890 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:06.904 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:06.950 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:06.960 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:06.967 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-05-14T17:12:07.024 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:07.034 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:07.043 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:07.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:07.083 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-05-14T17:12:07.091 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T17:12:07.098 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:07.107 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:07.116 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:07.123 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:07.131 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:07.143 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:07.155 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:07.171 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:07.240 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:07.250 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:07.263 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:07.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:07.286 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-05-14T17:12:07.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-05-14T17:12:07.304 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:07.316 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:07.326 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:07.337 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-05-14T17:12:07.350 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:07.361 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:07.378 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:07.389 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:07.402 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:07.413 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:07.430 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:07.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:07.487 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-05-14T17:12:07.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-05-14T17:12:07.514 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:07.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:07.536 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:07.544 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-05-14T17:12:07.551 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:07.557 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:07.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:07.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:07.576 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:07.583 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:07.590 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:07.597 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:07.604 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:07.613 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:07.650 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:07.657 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:07.664 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:07.671 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:07.677 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:07.686 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:07.697 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:07.705 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:07.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:07.727 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:07.732 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:07.737 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:07.744 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-05-14T17:12:07.758 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-05-14T17:12:07.764 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:07.781 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:07.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:07.832 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:07.845 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:07.855 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:07.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:07.872 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:07.884 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:07.893 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:07.903 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:07.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:07.921 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-05-14T17:12:07.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-05-14T17:12:07.938 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:07.949 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:07.958 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:07.967 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-05-14T17:12:07.981 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:08.033 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:08.043 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:08.057 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:08.070 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:08.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:08.098 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:08.108 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:08.118 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-05-14T17:12:08.135 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-05-14T17:12:08.146 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:08.156 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:08.167 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:08.177 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-05-14T17:12:08.190 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:08.200 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:08.209 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:08.223 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:08.262 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-05-14T17:12:08.272 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:08.280 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:08.287 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:08.297 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:08.305 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:08.316 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:08.324 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-05-14T17:12:08.332 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-05-14T17:12:08.341 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:08.350 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:08.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:08.369 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-05-14T17:12:08.378 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-05-14T17:12:08.389 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-05-14T17:12:08.398 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:08.477 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:08.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:08.552 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:08.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:08.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:08.577 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-05-14T17:12:08.585 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-05-14T17:12:08.593 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:08.602 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:08.611 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:08.621 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:08.631 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:08.640 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:08.649 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:08.659 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:08.666 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:08.677 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:08.712 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:08.788 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:08.804 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:08.813 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:08.822 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:08.833 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-05-14T17:12:08.842 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:08.853 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:08.862 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:08.871 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-05-14T17:12:08.881 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-05-14T17:12:08.892 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:08.906 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:08.918 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:08.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:08.939 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:08.951 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:08.973 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:09.009 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-05-14T17:12:09.020 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:09.032 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:09.041 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:09.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:09.061 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:09.069 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:09.076 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:09.087 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:09.096 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:09.108 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:09.118 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:09.130 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:09.139 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:09.149 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:09.158 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:09.173 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:09.213 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:09.221 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:09.228 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:09.235 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:09.241 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:09.248 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:09.253 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:09.259 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:09.268 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:09.275 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:09.282 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:09.289 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:09.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:09.303 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:09.318 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:09.332 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:09.368 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:09.376 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:09.383 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:09.391 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:09.397 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:09.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:09.409 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:09.416 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:09.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:09.429 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:09.435 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:09.443 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:09.455 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:09.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:09.468 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:09.478 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:09.491 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:09.522 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:09.535 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:09.545 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:09.553 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:09.564 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:09.572 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:09.584 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:09.597 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:09.610 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:09.619 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:09.629 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:09.639 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:09.648 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:09.654 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:09.664 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:09.675 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:09.702 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:09.742 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:09.753 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:09.762 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:09.771 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:09.777 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:09.786 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:09.794 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:09.800 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-05-14T17:12:09.807 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-05-14T17:12:09.819 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:12:09.830 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:09.838 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:09.848 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-05-14T17:12:09.857 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:09.866 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:09.876 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:09.912 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T17:12:09.942 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:09.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:09.962 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:09.971 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:09.981 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-05-14T17:12:09.989 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-05-14T17:12:09.997 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:10.007 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:10.015 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:10.024 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-05-14T17:12:10.033 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:10.041 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:10.049 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:10.058 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T17:12:10.064 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:10.073 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:10.091 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:10.130 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:10.141 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-05-14T17:12:10.152 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-05-14T17:12:10.161 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:12:10.169 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:10.180 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:10.194 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-05-14T17:12:10.204 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:10.215 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:10.227 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:10.239 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-05-14T17:12:10.247 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:10.256 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:10.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:10.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:10.292 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-05-14T17:12:10.316 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-05-14T17:12:10.341 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:12:10.349 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:10.359 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:10.370 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-05-14T17:12:10.378 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:10.384 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:10.394 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:10.404 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T17:12:10.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:10.426 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:10.435 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:10.447 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:10.455 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-05-14T17:12:10.463 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-05-14T17:12:10.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:10.480 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:10.491 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:10.531 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:10.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:10.546 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:10.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:10.566 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T17:12:10.573 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:10.583 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:10.592 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:10.598 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:10.603 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:12:10.611 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:12:10.617 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:12:10.623 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:10.628 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:10.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:12:10.642 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:10.658 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:10.686 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:10.692 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T17:12:10.699 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:10.705 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:10.713 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:10.720 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:10.726 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:10.733 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:10.739 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:10.745 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:10.753 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:10.759 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:10.766 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:10.772 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:10.778 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:10.785 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:10.800 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:10.869 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:10.878 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:10.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:10.893 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-05-14T17:12:10.903 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-05-14T17:12:10.912 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:12:10.921 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:10.930 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:10.940 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-05-14T17:12:10.949 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:10.957 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:10.966 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:10.974 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T17:12:10.993 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:11.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:11.012 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:11.035 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:11.100 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:11.108 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:11.115 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:11.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-05-14T17:12:11.129 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:11.135 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:11.141 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:11.147 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:11.178 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:11.186 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:11.194 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:11.203 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:11.212 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:11.221 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:11.230 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:11.238 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:11.259 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:11.287 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:11.297 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:11.308 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:11.316 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:11.335 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:11.347 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:11.368 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:11.378 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:11.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:11.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:11.410 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:11.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:11.431 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:11.441 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:11.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:11.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:11.491 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:11.502 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:11.513 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:11.525 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:11.539 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:11.549 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:11.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:11.572 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:11.582 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:11.592 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:11.601 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:11.611 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:11.620 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:11.631 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:11.641 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:11.670 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:11.706 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:11.738 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:11.745 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:11.752 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:11.760 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:11.769 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:11.778 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:11.788 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:11.796 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:11.806 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:11.813 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:11.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:11.846 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:11.856 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:11.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:11.874 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:11.883 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:11.898 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:11.928 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:11.937 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:11.946 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:11.955 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:11.964 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:11.975 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:11.984 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:11.992 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:12.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:12.058 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-05-14T17:12:12.067 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-05-14T17:12:12.077 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:12.085 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:12.095 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:12.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-05-14T17:12:12.113 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:12.131 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:12.156 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:12.189 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:12.195 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:12.205 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:12.212 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:12.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:12.227 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-05-14T17:12:12.237 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-05-14T17:12:12.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:12.255 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:12.264 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:12.273 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-05-14T17:12:12.282 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:12.294 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:12.301 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:12.312 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:12.323 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:12.361 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:12.387 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:12.396 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:12.406 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:12:12.414 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:12:12.424 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:12.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:12.453 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:12:12.465 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:12.475 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:12.486 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:12.498 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:12.509 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:12.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:12.531 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:12.557 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:12.611 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:12.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:12.629 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:12.637 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:12.645 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:12.653 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:12.661 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:12.670 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:12.678 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:12.686 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:12.694 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:12.702 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:12.710 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:12.718 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:12.726 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:12.734 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:12.760 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:12.798 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-05-14T17:12:12.806 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-05-14T17:12:12.814 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-05-14T17:12:12.822 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:12.833 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-05-14T17:12:12.842 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-05-14T17:12:12.852 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:12.859 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:12.867 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:12.876 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-05-14T17:12:12.885 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:12.895 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:12.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:12.940 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T17:12:12.950 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:12.959 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:12.974 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:13.006 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:13.016 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-05-14T17:12:13.025 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-05-14T17:12:13.033 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:13.044 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:13.052 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:13.060 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-05-14T17:12:13.069 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:13.079 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:13.111 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:13.122 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T17:12:13.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:13.144 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:13.152 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:13.163 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:13.172 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-05-14T17:12:13.190 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-05-14T17:12:13.228 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:13.238 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:13.246 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:13.255 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:13.261 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:13.268 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:13.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:13.280 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-05-14T17:12:13.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:13.307 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:13.314 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:13.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:13.329 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:12:13.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:12:13.346 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:12:13.354 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:13.366 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:13.397 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-05-14T17:12:13.408 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-05-14T17:12:13.417 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:13.426 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:13.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T17:12:13.444 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:13.453 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:13.462 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:13.482 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:13.491 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:13.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:13.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:13.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:13.528 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:13.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:13.547 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:13.566 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:13.591 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:13.600 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:13.608 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:13.617 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:13.626 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:13.636 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:13.646 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:13.670 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:13.678 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:13.686 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:13.694 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:13.703 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:13.711 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:13.719 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:13.729 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:13.737 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:13.750 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:13.777 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:13.786 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:13.794 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:13.812 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:13.821 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:13.831 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:13.840 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:13.848 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:13.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:13.867 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:13.878 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:13.887 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:13.895 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:13.903 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:13.911 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:13.919 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:13.933 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:13.979 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:13.990 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:14.016 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:14.027 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:14.037 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:14.048 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:14.058 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:14.069 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:14.081 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:14.089 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:14.098 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:14.106 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:14.114 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:14.123 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:14.132 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:14.140 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:14.150 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:14.184 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:14.192 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:14.201 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:14.212 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:14.221 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:14.229 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:14.237 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:14.243 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:14.251 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:14.260 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:14.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:14.278 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-05-14T17:12:14.286 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-05-14T17:12:14.297 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:14.304 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:14.312 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:14.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:14.349 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:14.355 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:14.365 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:14.375 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-05-14T17:12:14.385 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:14.394 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:14.404 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:14.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:14.423 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-05-14T17:12:14.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-05-14T17:12:14.441 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:14.450 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:14.459 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:14.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-05-14T17:12:14.476 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:14.486 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:14.522 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:14.546 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-05-14T17:12:14.554 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:14.564 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:14.574 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:14.583 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:14.591 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:14.599 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:14.609 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:14.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:14.628 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:14.636 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:14.645 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:14.662 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:14.677 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:14.686 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:14.701 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:14.750 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:14.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:14.767 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-05-14T17:12:14.776 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-05-14T17:12:14.784 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:14.793 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:14.802 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:14.812 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-05-14T17:12:14.855 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:14.864 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:14.871 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:14.877 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:12:14.884 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:14.890 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:14.896 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:14.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:14.910 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-05-14T17:12:14.946 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-05-14T17:12:14.952 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:14.958 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:14.964 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:14.971 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:12:14.977 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:12:14.983 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-05-14T17:12:14.990 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:14.998 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T17:12:15.020 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:15.028 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:15.036 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:15.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:15.054 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-05-14T17:12:15.062 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-05-14T17:12:15.070 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:15.085 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:15.114 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:15.125 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:15.135 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:15.146 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:15.162 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:15.171 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-05-14T17:12:15.183 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:15.194 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:15.204 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:15.214 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:15.225 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-05-14T17:12:15.233 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-05-14T17:12:15.242 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:15.251 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:15.260 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:15.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-05-14T17:12:15.295 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:15.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:15.364 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:15.372 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:12:15.380 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:15.388 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:15.396 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:15.403 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:15.413 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:15.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:15.428 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:15.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-05-14T17:12:15.444 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:15.452 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:15.460 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:15.468 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:15.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:15.485 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:15.516 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:15.524 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:15.532 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:15.540 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:15.548 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:15.559 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:15.570 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:15.581 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:15.592 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:15.608 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:15.620 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:15.630 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:15.655 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:15.664 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:15.676 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:15.689 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:15.733 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:15.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:15.776 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:15.808 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:15.816 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:15.824 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:15.833 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:15.841 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:15.850 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:15.858 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:15.866 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:15.872 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:15.880 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:15.888 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:15.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:15.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:15.908 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:15.969 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:15.991 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:15.997 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:16.003 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:16.008 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:16.015 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:16.021 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:16.046 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:16.055 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:16.064 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:16.072 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:16.082 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:16.091 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:16.099 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:16.108 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:16.148 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:16.181 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:16.190 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:16.204 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:16.212 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:16.220 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:16.228 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:16.237 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:16.245 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:16.254 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:16.273 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:16.285 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:16.291 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:16.298 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:16.306 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:16.312 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:16.317 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:16.336 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:16.360 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:16.375 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:16.382 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:16.410 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:16.435 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:16.452 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:16.458 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:16.465 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:16.471 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:16.479 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:16.487 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:16.496 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:16.505 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:16.514 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:16.523 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:16.536 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-05-14T17:12:16.573 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:16.583 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:16.594 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:16.604 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:16.615 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:16.626 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:16.636 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:16.646 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:16.656 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:16.666 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:16.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:16.687 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:16.696 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:16.707 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:16.718 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:16.729 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:16.753 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:16.790 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:16.802 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:16.811 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:16.819 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:16.827 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:16.835 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:16.844 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:16.851 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:16.863 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:16.871 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:16.879 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:16.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:16.896 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:16.904 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:16.912 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:16.922 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:16.946 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:16.974 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:16.990 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:17.001 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:17.012 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:17.023 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:17.033 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:17.043 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:17.053 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:17.062 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:17.073 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:17.084 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:17.094 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:17.105 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:17.113 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:17.122 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:17.134 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:17.146 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:17.177 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:17.185 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:17.193 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:17.201 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:17.207 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:17.213 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:17.220 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:17.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:17.232 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:17.238 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:17.245 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:17.267 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:17.274 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:17.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:17.287 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:17.293 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:17.305 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:17.332 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:17.341 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:17.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:17.369 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:17.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:17.387 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:17.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:17.405 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:17.415 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:17.452 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:17.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:17.472 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:17.480 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:17.488 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:17.499 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:17.505 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:17.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:17.543 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:17.551 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:17.558 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:17.565 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:17.571 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:17.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-05-14T17:12:17.584 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:17.602 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:17.609 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:17.615 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-05-14T17:12:17.621 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-05-14T17:12:17.627 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:17.633 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:17.638 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:17.646 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-05-14T17:12:17.653 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:17.672 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:17.695 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:17.703 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T17:12:17.711 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:17.719 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:17.728 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:17.737 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:17.777 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-05-14T17:12:17.786 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-05-14T17:12:17.797 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:17.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:17.813 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:17.822 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:12:17.832 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-05-14T17:12:17.842 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:17.855 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:17.867 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T17:12:17.907 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:17.943 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:17.953 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:17.965 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:17.976 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:17.987 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:17.997 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:18.020 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:18.030 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:18.041 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:18.051 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:18.063 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:18.074 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:18.084 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:18.096 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:18.106 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:18.116 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:18.131 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:18.187 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:18.230 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:18.238 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:18.246 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:18.254 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:18.288 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:18.314 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:18.366 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:18.421 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:18.430 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:18.438 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:18.446 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:18.454 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:18.486 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:18.506 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:18.534 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:18.555 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:18.566 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:18.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:18.620 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:18.640 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:18.774 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:18.786 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:18.791 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:18.811 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:18.818 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:18.825 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:18.884 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:18.892 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:18.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-05-14T17:12:18.927 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:18.936 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-05-14T17:12:18.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-05-14T17:12:18.982 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:18.990 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:19.006 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-05-14T17:12:19.014 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:19.045 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:19.099 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-05-14T17:12:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T17:12:19.126 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:19.135 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:19.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:19.153 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:19.163 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:19.170 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:19.184 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:19.192 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:19.198 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:19.206 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:19.263 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:19.272 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:19.311 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:19.343 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:19.365 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:19.376 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:19.396 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:19.414 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:19.467 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:19.477 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:19.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:19.529 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:19.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:19.561 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:19.570 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:19.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:19.606 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:19.614 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:19.646 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:19.676 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:19.691 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:19.713 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:19.826 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:19.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:19.869 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:19.885 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:19.900 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:19.915 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:19.946 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:20.000 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-05-14T17:12:20.010 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-05-14T17:12:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:20.082 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:20.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:20.098 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:20.137 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-05-14T17:12:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T17:12:20.151 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:20.158 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:20.166 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:20.174 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-05-14T17:12:20.182 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-05-14T17:12:20.188 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:20.194 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:20.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:20.208 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-05-14T17:12:20.215 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:20.221 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:20.249 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:20.257 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-05-14T17:12:20.264 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:20.275 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:20.302 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:20.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:20.314 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-05-14T17:12:20.320 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-05-14T17:12:20.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:20.336 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:20.346 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:20.354 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-05-14T17:12:20.364 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:20.374 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:20.398 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:20.406 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T17:12:20.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:20.422 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:20.430 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:20.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:20.450 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-05-14T17:12:20.487 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-05-14T17:12:20.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:20.506 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:20.514 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:20.524 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:20.549 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:20.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:20.567 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:20.575 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-05-14T17:12:20.584 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:20.595 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:20.603 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:20.613 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:20.621 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:20.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:20.639 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:20.688 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:20.723 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:20.732 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:20.740 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:20.756 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:20.762 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:20.777 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:20.785 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:20.794 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:20.804 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-05-14T17:12:20.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-05-14T17:12:20.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:20.836 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:20.847 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:20.857 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-05-14T17:12:20.868 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:20.912 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:20.923 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:20.933 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T17:12:20.944 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T17:12:20.954 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:20.966 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:20.977 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:20.993 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:21.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:21.014 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:21.024 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-05-14T17:12:21.064 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:21.074 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:21.112 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:21.122 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:21.132 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:21.168 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:21.198 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:21.228 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:21.236 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:21.244 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:21.280 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:21.289 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:21.297 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:21.304 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:21.312 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:21.321 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:21.329 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:21.339 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:21.347 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:21.355 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:21.365 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:21.373 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:21.406 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:21.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:21.444 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:21.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:21.461 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:21.469 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:21.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:21.485 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:21.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:21.504 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:21.512 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:21.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:21.527 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:21.534 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:21.541 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:21.546 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:21.552 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:21.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:21.595 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:21.627 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:21.634 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:21.640 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:21.646 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:21.653 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:21.659 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:21.665 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:21.672 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:21.679 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:21.686 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:21.694 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:21.700 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:21.717 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:21.722 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:21.729 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:21.743 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:21.771 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:21.805 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:21.812 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:21.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:21.829 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:21.838 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:21.846 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:21.855 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:21.866 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:21.874 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:21.882 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:21.891 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:21.899 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:21.908 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:21.917 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:21.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:21.938 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:21.981 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:21.990 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:21.999 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:22.008 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:22.016 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:22.024 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:22.033 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:22.041 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:22.049 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:22.059 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:22.069 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:22.077 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:22.086 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:22.096 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:22.105 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:22.115 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:22.136 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-05-14T17:12:22.181 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:22.190 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:22.200 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:22.209 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:22.219 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:22.230 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:22.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:22.251 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:22.260 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:22.271 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:22.281 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:22.292 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:22.303 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:22.314 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:22.324 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:22.334 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:22.345 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:22.372 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:22.383 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:22.393 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:22.404 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:22.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:22.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:22.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:22.446 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:22.457 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:22.468 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:22.479 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:22.489 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:22.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:22.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:22.536 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:22.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:22.570 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:22.593 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:22.601 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:22.609 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:22.617 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:22.625 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:22.634 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:22.642 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:22.650 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:22.659 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:22.666 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:22.674 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:22.682 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:22.690 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:22.698 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:22.719 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:22.725 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:22.736 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:22.766 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:22.774 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:22.781 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:22.789 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:22.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:22.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:22.814 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:22.822 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:22.832 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:22.840 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:22.848 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:22.857 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:22.866 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:22.895 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:22.905 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:22.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:22.920 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:22.945 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:22.952 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:22.959 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:22.965 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:22.972 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:22.978 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:22.984 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:22.992 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:22.998 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:23.004 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:23.011 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:23.018 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:23.023 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:23.053 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:23.060 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:23.065 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:23.087 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:23.118 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:23.153 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:23.174 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:23.182 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:23.191 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:23.231 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-05-14T17:12:23.283 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:23.291 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:23.354 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:23.363 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:23.371 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:23.381 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:23.392 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:23.402 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:23.413 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:23.463 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:23.474 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:23.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:23.494 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:23.504 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:23.514 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:23.525 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:23.536 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:23.583 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:23.593 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:23.608 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:23.619 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:23.630 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:23.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:23.674 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:23.686 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:23.698 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:23.726 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:23.736 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:23.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:23.757 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:23.766 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:23.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:23.784 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:23.793 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:23.822 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:23.832 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:23.849 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:23.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:23.867 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:23.875 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:23.883 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:23.892 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:23.900 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:23.935 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:23.944 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:23.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:23.962 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:23.971 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:23.979 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:23.988 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:24.046 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:24.055 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:24.066 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:24.077 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:24.085 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:24.096 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:24.105 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:24.114 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:24.122 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:24.152 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:24.161 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:24.192 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:24.201 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:24.210 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:24.218 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:24.226 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:24.235 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:24.247 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:24.255 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:24.292 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:24.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:24.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:24.313 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:24.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:24.329 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:24.337 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:24.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:24.416 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:24.431 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:24.438 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:24.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:24.451 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:24.457 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:24.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:24.472 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:24.479 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:24.485 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:24.492 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:24.517 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:24.524 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-05-14T17:12:24.532 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:24.543 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:24.551 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:24.589 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-05-14T17:12:24.598 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-05-14T17:12:24.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:24.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:24.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:24.632 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:24.640 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:24.647 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:24.656 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:24.663 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T17:12:24.689 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:24.751 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:24.760 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:24.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:24.777 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:24.785 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:24.794 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:24.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:24.830 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:24.837 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:24.849 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:24.857 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:24.866 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:24.874 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:24.882 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:24.898 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:24.906 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:24.936 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:24.945 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:24.954 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:24.961 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:24.970 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:24.978 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:24.986 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:25.016 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:25.027 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:25.038 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:25.049 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:25.059 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:25.094 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:25.105 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:25.115 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:25.127 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:25.138 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:25.151 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:25.161 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:25.173 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:25.186 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:25.197 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:25.207 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:25.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:25.246 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:25.261 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:25.272 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:25.283 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:25.294 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:25.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:25.316 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:25.328 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:25.344 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:25.352 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:25.378 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:25.385 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:25.392 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:25.401 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:25.408 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:25.415 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:25.445 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:25.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:25.476 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:25.485 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:25.495 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:25.506 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:25.516 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:25.524 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:25.533 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:25.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:25.562 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:25.594 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:25.607 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:25.616 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:25.625 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:25.633 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:25.642 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:25.651 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-05-14T17:12:25.679 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-05-14T17:12:25.687 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:25.696 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:25.706 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:25.714 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:25.724 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:25.732 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:25.741 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:25.768 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-05-14T17:12:25.777 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-05-14T17:12:25.801 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-05-14T17:12:25.811 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:25.818 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-05-14T17:12:25.824 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:25.831 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:25.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:25.844 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:25.869 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:25.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:25.882 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:25.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:25.907 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:25.917 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:25.923 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:25.929 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:25.936 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:25.945 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:25.960 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:25.967 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:25.976 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:25.983 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:25.990 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:25.997 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:26.004 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:26.034 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-05-14T17:12:26.043 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:26.071 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:26.080 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:26.088 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:26.099 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:26.112 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:26.122 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:26.153 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:26.170 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:26.185 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:26.208 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:26.216 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:26.225 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:26.233 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:26.242 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:26.251 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:26.281 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:26.290 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:26.298 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:26.306 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:26.314 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:26.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:26.330 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:26.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:26.395 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:26.417 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:26.427 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:26.436 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:26.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:26.453 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:26.461 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:26.469 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:26.477 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:26.504 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:26.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:26.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:26.529 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:26.537 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:26.545 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:26.553 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:26.564 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:26.579 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:26.609 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:26.627 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:26.637 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:26.647 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:26.659 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:26.670 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:26.682 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:26.693 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:26.724 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:26.735 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:26.747 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:26.776 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:26.786 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:26.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:26.821 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:26.832 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:26.845 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:26.861 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:26.871 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:26.881 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:26.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:26.899 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:26.908 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:26.918 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:26.951 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:26.958 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:26.964 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:26.995 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:27.003 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:27.009 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:27.015 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:27.023 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:27.032 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:27.040 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:27.053 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:27.063 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:27.079 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:27.110 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:27.119 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:27.128 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:27.138 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:27.180 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:27.198 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:27.206 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:27.215 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:27.224 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:27.232 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:27.241 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-05-14T17:12:27.250 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:27.271 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:27.280 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:27.294 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:27.304 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:27.312 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:27.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:27.332 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:27.341 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:27.350 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:27.380 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:27.391 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:27.398 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:27.407 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:27.413 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:27.419 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:27.425 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:27.431 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:27.451 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:27.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:27.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:27.480 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:27.487 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:27.494 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:27.501 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:27.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:27.527 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:27.555 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:27.563 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:27.569 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:27.575 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:27.582 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:27.589 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:27.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:27.609 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:27.635 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:27.645 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:27.657 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:27.667 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:27.677 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:27.703 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:27.714 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:27.723 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:27.733 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:27.759 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:27.766 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:27.774 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:27.784 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:27.793 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:27.801 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:27.809 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:27.818 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:27.856 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:27.865 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:27.904 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:27.912 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:27.921 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:27.929 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:27.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:27.946 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:27.955 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:27.963 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:27.991 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:28.000 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:28.008 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:28.018 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:28.027 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:28.035 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:28.043 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:28.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:28.080 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:28.100 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:28.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:28.115 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:28.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:28.132 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:28.144 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:28.155 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:28.219 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:28.230 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:28.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:28.277 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:28.286 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:28.297 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:28.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:28.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:28.349 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:28.360 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:28.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:28.436 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:28.446 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-05-14T17:12:28.455 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:28.465 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:28.476 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:28.486 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:28.540 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:28.549 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:28.583 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:28.592 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:28.599 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:28.605 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:28.612 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:28.620 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:28.631 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:28.638 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:28.650 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:28.656 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:28.664 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:28.673 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:28.681 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:28.692 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:28.700 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:28.728 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:28.737 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:28.754 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:28.764 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:28.773 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:28.785 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:28.794 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:28.802 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:28.816 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:28.824 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:28.833 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:28.842 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:28.850 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:28.858 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:28.867 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:28.875 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:28.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:28.898 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:28.951 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:28.966 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:28.974 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:28.984 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:28.994 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:29.003 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:29.012 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:29.020 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:29.028 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:29.042 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:29.049 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:29.057 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:29.063 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:29.068 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:29.074 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:29.080 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:29.110 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:29.117 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:29.123 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:29.131 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:29.138 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:29.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:29.154 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:29.162 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:29.174 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:29.184 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:29.209 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:29.218 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:29.227 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:29.254 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:29.269 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:29.277 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:29.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:29.322 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:12:29.332 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:12:29.344 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:12:29.354 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:29.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:29.372 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:12:29.382 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:29.418 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:29.442 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:29.467 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:29.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:29.484 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:29.493 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:29.502 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:29.511 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:29.519 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:29.532 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:29.553 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:29.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:29.567 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:29.575 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:29.583 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:29.607 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:29.615 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:29.624 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:29.633 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:29.656 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:29.663 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:29.671 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-05-14T17:12:29.680 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-05-14T17:12:29.687 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:12:29.696 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:29.705 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:29.730 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-05-14T17:12:29.738 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-05-14T17:12:29.768 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:29.777 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:29.786 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-05-14T17:12:29.794 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T17:12:29.805 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:29.817 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:29.853 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:29.865 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:29.893 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:29.907 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:29.917 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-05-14T17:12:29.928 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:29.941 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:29.950 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:29.960 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-05-14T17:12:29.985 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-05-14T17:12:29.995 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:30.003 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:30.013 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:30.025 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:30.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:30.043 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:30.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:30.083 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T17:12:30.093 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:30.104 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:30.137 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:30.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:30.152 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-05-14T17:12:30.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-05-14T17:12:30.164 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:30.171 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:30.198 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:30.205 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:30.211 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:30.219 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:30.228 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:30.236 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:30.245 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:30.253 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:30.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:30.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:30.324 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:30.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:30.388 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:30.397 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:30.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:30.419 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:30.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:30.486 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:30.500 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:30.515 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:30.527 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:30.539 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:30.552 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:30.566 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:30.581 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:30.626 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:30.640 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:30.682 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:30.691 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:30.706 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:30.719 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:30.733 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:30.745 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:30.760 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:30.809 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:30.820 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:30.864 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:30.873 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:30.887 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:30.896 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:30.911 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:30.920 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:30.930 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:30.942 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:30.987 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:31.000 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:31.009 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:31.021 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:31.033 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:31.040 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:31.052 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:31.126 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:31.138 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:31.148 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:31.158 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:31.169 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:31.182 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:31.195 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:31.210 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:31.222 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:31.234 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:31.292 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:31.302 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:31.314 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:31.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:31.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:31.352 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:31.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:31.426 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:31.436 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:31.445 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:31.456 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:31.494 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:31.506 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:31.518 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:31.531 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:31.543 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:31.556 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:31.583 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:31.600 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:31.614 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-05-14T17:12:31.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-05-14T17:12:31.647 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:31.659 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:31.674 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:31.740 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-05-14T17:12:31.756 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:31.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:31.783 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:31.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-05-14T17:12:31.803 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:31.814 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:31.826 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:31.843 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:31.856 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:12:31.903 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:12:31.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:12:31.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:31.944 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:31.958 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:12:31.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:32.011 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:32.070 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:32.081 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-05-14T17:12:32.092 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:32.104 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:32.115 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:32.125 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:32.138 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:32.148 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:32.188 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:32.198 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:32.269 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:32.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:32.290 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:32.300 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:32.312 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:32.323 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:32.333 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:32.386 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:32.394 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:32.408 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:32.418 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:32.428 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:32.435 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:32.444 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-05-14T17:12:32.453 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:32.468 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:32.476 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:32.531 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-05-14T17:12:32.543 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-05-14T17:12:32.557 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:32.580 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:32.592 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:32.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:32.626 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:32.687 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:32.701 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:32.717 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:32.729 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:32.741 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:32.753 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:32.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:32.780 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-05-14T17:12:32.795 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-05-14T17:12:32.807 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:32.839 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:32.852 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:32.863 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:32.877 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:32.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:32.901 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:32.913 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T17:12:32.937 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T17:12:32.950 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:32.962 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:32.976 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:32.989 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:33.002 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:33.012 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:33.022 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:33.030 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:33.093 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:33.130 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:33.139 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:33.150 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:33.162 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:33.173 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:33.183 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:33.192 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:33.216 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:33.227 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:33.237 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:33.247 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:33.256 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:33.265 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:33.277 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:33.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:33.340 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:33.353 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:33.391 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:33.401 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:33.413 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:33.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:33.438 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:33.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:33.464 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:33.496 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:33.508 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:33.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:33.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:33.550 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:33.563 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:33.578 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:33.587 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:33.609 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:33.625 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:33.694 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:33.709 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:33.723 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:33.738 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:33.749 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:33.761 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:33.771 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:33.817 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:33.828 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:33.841 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:33.851 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:33.865 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:33.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:33.890 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:33.901 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:33.952 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:33.965 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:33.993 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:34.004 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:34.016 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:34.024 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:34.034 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:34.045 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:34.056 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:34.086 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:34.099 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:34.110 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:34.123 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:34.132 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:34.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:34.155 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-05-14T17:12:34.176 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-05-14T17:12:34.190 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:34.199 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:34.227 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:34.234 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:34.243 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:34.250 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:34.258 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:34.266 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:34.277 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:34.332 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:34.342 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:34.351 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:34.378 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T17:12:34.390 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T17:12:34.401 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:34.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:34.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:34.432 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-05-14T17:12:34.443 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:34.485 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:34.496 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:34.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:34.543 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:34.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:34.588 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-05-14T17:12:34.602 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-05-14T17:12:34.614 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:12:34.643 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:34.654 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:34.668 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-05-14T17:12:34.683 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:34.724 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:34.777 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:34.789 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:34.815 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:34.831 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-05-14T17:12:34.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-05-14T17:12:34.860 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:34.893 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:34.922 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-05-14T17:12:34.937 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:34.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:34.965 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:34.978 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T17:12:34.990 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:35.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:35.018 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:35.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:35.082 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-05-14T17:12:35.092 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-05-14T17:12:35.103 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:35.114 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:35.122 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-05-14T17:12:35.172 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:35.185 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:35.194 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:35.209 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T17:12:35.223 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:35.235 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:35.248 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:35.259 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:35.274 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:35.286 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:35.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:35.338 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:35.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:35.374 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:35.386 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:35.396 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:35.407 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:35.437 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:35.448 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:35.458 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:35.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:35.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:35.487 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-05-14T17:12:35.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-05-14T17:12:35.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:35.541 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:35.571 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:35.618 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-05-14T17:12:35.664 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:35.674 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:35.692 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:35.702 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:35.734 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:35.746 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:35.806 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:35.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:35.825 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-05-14T17:12:35.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-05-14T17:12:35.844 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:35.852 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:35.862 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:35.869 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-05-14T17:12:35.883 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:35.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:35.919 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:35.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-05-14T17:12:35.934 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-05-14T17:12:35.941 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:35.953 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:35.964 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:35.973 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:36.003 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:36.019 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:36.028 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-05-14T17:12:36.037 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-05-14T17:12:36.046 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:36.054 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:36.062 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:36.072 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-05-14T17:12:36.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-05-14T17:12:36.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-05-14T17:12:36.138 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:36.150 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:36.158 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:36.166 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:36.182 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:36.195 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:36.224 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:36.236 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:36.245 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-05-14T17:12:36.256 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:36.268 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:36.276 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:36.286 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:36.299 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:36.309 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:36.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:36.358 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:36.370 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:36.379 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:36.392 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:36.404 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:36.421 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:36.434 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:36.464 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:36.478 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:36.494 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:36.511 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:36.526 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:36.540 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:36.569 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:36.586 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:36.601 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:36.633 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:36.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:36.655 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:36.670 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:36.686 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:36.725 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:36.738 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:36.800 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:36.814 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:36.828 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:36.852 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:36.864 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:36.889 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:36.914 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:36.967 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:36.994 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:37.009 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:37.021 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:37.040 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:37.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:37.066 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:37.099 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:37.111 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:37.125 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:37.137 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:37.149 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:37.161 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:37.173 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:37.187 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:37.212 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:37.226 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:37.266 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:37.281 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:37.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:37.310 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:37.324 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:37.338 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:37.353 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:37.412 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-05-14T17:12:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-05-14T17:12:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:37.619 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:12:37.628 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:37.638 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:37.650 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:37.682 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:37.690 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:37.734 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:37.744 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:37.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:37.764 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-05-14T17:12:37.773 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T17:12:37.782 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:37.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:37.799 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:37.836 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:37.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:37.851 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:37.861 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:37.869 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:37.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:37.884 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:37.908 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:37.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:37.978 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:37.989 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:38.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:38.014 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:38.025 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:38.036 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:38.048 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:38.058 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:38.093 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:38.102 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:38.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:38.125 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:38.140 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:38.153 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:38.163 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-05-14T17:12:38.208 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-05-14T17:12:38.221 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:38.281 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:38.293 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:38.308 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:38.317 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:38.329 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:38.344 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:38.389 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:38.401 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:38.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:38.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:38.474 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:38.485 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:38.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:38.506 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:38.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-05-14T17:12:38.542 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:38.589 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:38.617 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:38.629 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-05-14T17:12:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-05-14T17:12:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:38.667 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:38.713 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:38.725 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:38.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:38.781 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:38.794 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:38.807 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:38.819 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:38.832 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:38.846 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:38.911 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-05-14T17:12:38.923 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-05-14T17:12:38.937 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:38.951 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:38.965 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:38.994 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:39.011 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:39.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:39.039 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:39.073 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:39.085 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:39.100 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:39.112 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:39.127 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:39.140 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:39.156 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:39.218 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:39.235 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:39.249 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:39.266 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:39.282 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:39.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:39.317 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:39.332 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:39.356 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:39.370 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:39.416 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:39.429 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:39.443 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:39.454 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:39.470 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:39.486 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:39.499 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:39.561 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:39.574 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:39.590 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:39.601 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:39.613 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:39.625 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:39.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:39.650 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:39.675 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:39.684 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:39.710 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:39.719 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:39.728 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:39.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:39.779 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:39.806 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:39.818 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:39.828 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:39.836 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:39.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:39.851 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:39.861 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:39.869 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:39.878 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:39.886 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:39.914 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:39.922 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:39.930 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:39.939 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:39.970 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:39.978 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:39.987 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:40.043 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:40.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:40.067 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:40.078 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:40.085 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:40.091 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:40.097 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:40.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:40.138 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:40.162 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:40.215 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:40.222 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:40.227 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:40.233 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:40.240 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:40.247 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:40.254 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-05-14T17:12:40.282 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-05-14T17:12:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:40.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:40.303 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:40.311 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:40.322 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:40.328 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:40.333 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:40.341 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:40.347 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:40.381 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:40.388 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:40.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:40.400 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-05-14T17:12:40.408 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-05-14T17:12:40.414 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:40.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:40.471 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:40.480 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:40.491 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:40.539 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:40.550 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:40.560 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:40.571 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:40.582 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:40.599 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:40.610 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:40.622 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:40.633 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:40.644 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:40.655 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:40.667 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:40.677 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:40.688 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:40.718 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:40.749 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:40.757 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:40.766 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:40.774 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:40.782 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:40.792 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-05-14T17:12:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:41.909 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-05-14T17:12:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:42.179 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:42.190 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:42.204 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:42.215 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:42.227 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:42.260 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:42.271 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:42.294 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:42.305 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:42.317 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:42.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:42.339 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:42.351 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:42.360 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:42.374 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:42.383 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:42.392 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:42.401 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:42.410 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:42.421 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:42.432 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:42.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:42.500 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:42.512 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:42.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:42.534 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:42.544 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:42.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:42.566 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:42.585 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:42.596 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:42.624 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:42.648 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:42.661 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:42.672 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:42.684 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:42.695 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:42.706 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:42.736 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:42.747 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:42.759 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:42.769 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:42.781 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:42.792 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:42.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:42.815 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:42.828 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:42.845 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:42.870 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:42.881 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:42.891 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:42.901 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:42.909 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:42.918 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:42.927 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:42.980 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:42.991 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:43.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:43.012 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:43.066 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:43.076 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:43.086 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:43.095 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:43.112 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:43.122 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:43.141 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:43.151 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:43.161 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:43.170 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:43.178 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:43.188 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:43.196 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:43.227 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:43.251 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:43.257 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-05-14T17:12:43.265 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-05-14T17:12:43.273 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:43.279 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:43.286 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:43.293 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-05-14T17:12:43.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:43.319 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:43.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:43.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T17:12:43.363 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:43.372 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:43.378 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:43.385 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:43.414 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:43.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:43.446 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:43.452 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-05-14T17:12:43.459 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:43.465 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:43.473 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:43.480 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:43.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:43.516 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:43.522 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:43.533 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:43.541 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:43.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:43.556 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:43.562 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:43.568 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:43.597 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:43.620 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:43.627 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:43.633 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:43.639 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:43.645 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:43.651 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:43.657 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:43.664 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:43.680 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:43.688 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:43.700 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:43.708 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:43.717 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:43.726 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:43.734 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:43.743 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:43.781 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:43.804 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:43.854 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:43.861 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:43.870 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:43.881 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:43.889 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:43.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:43.904 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:43.929 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:43.938 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:44.002 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:44.017 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:44.028 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:44.040 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:44.051 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:44.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:44.073 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:44.106 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:44.117 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:44.128 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:44.139 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:44.151 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:44.161 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:44.172 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:44.215 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:44.255 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:44.265 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:44.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:44.358 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:44.369 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:44.396 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:44.407 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:44.440 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:44.451 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:44.515 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-05-14T17:12:44.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-05-14T17:12:44.544 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:44.555 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:44.562 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:44.573 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:44.585 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:44.596 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:44.605 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:44.667 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:44.692 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:44.698 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:44.711 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:44.720 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:44.731 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-05-14T17:12:44.739 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-05-14T17:12:44.748 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:44.765 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:44.776 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:44.795 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-05-14T17:12:44.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:44.814 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:44.823 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:44.834 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:44.843 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:44.893 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:44.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:44.918 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-05-14T17:12:44.929 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-05-14T17:12:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:44.949 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:44.958 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:44.965 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:44.974 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:44.981 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:45.013 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:45.020 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:45.026 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:45.032 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:45.039 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:45.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:45.053 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-05-14T17:12:45.078 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-05-14T17:12:45.094 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:45.101 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:45.107 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:45.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:45.120 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:45.129 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:45.134 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:45.140 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:45.147 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:45.170 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:45.179 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:45.185 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:45.191 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-05-14T17:12:45.198 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-05-14T17:12:45.204 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:12:45.212 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:45.245 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:45.283 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:45.289 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:45.296 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:45.303 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:45.310 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:45.316 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:45.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:45.332 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:45.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:45.367 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-05-14T17:12:45.374 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-05-14T17:12:45.379 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:45.389 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:45.400 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:45.411 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:45.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:45.458 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:45.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:45.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:45.530 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:45.537 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:45.546 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:45.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:45.564 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-05-14T17:12:45.574 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-05-14T17:12:45.584 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:45.653 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:45.662 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:45.687 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:45.696 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:45.705 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:45.714 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:45.723 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:45.745 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:45.762 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:45.770 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:45.780 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:45.790 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-05-14T17:12:45.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-05-14T17:12:45.812 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:45.824 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:45.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:45.881 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:45.905 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:45.915 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:45.927 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:45.939 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:45.949 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:45.959 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:45.970 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:46.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:46.013 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-05-14T17:12:46.024 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-05-14T17:12:46.035 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:46.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:46.062 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:46.074 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:46.085 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:12:46.096 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-05-14T17:12:46.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:46.164 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:46.175 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:46.187 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:46.197 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:46.209 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:46.220 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:46.238 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:46.265 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:46.276 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:46.286 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:46.296 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:46.306 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:46.316 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:46.327 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:46.335 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:46.344 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:46.353 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:46.381 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:46.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:46.410 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-05-14T17:12:46.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-05-14T17:12:46.429 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:46.438 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:46.446 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:46.477 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:46.486 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:46.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:46.501 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:46.507 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:46.515 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:46.523 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:46.529 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:46.535 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:46.545 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-05-14T17:12:46.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-05-14T17:12:46.605 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:46.614 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:46.625 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:46.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:46.644 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:46.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:46.674 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:46.683 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:46.690 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:46.696 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:46.704 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:46.710 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:46.716 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-05-14T17:12:46.723 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-05-14T17:12:46.733 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:46.741 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:46.763 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:46.772 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:46.778 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:46.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:46.793 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:46.800 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:46.807 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:46.830 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:46.839 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:46.845 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:46.853 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:46.860 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:46.866 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:46.872 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smb", 2024-05-14T17:12:46.878 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:46.901 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:46.908 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:46.924 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:46.976 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:46.983 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:46.988 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:46.995 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:47.002 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:47.036 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:47.045 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:47.053 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:47.063 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:47.072 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:47.079 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:47.087 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:47.096 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:47.117 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:47.126 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:47.151 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:47.191 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:47.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:47.211 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:47.220 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:47.229 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:47.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:47.258 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:47.267 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:47.277 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:47.284 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:47.293 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:47.304 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:47.315 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:47.334 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:47.355 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:47.390 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:47.401 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:47.413 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:47.424 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:47.435 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:47.445 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:47.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:47.485 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:47.495 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:47.505 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:47.516 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:47.526 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:47.537 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:47.557 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:47.578 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:47.587 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:47.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:47.611 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:47.622 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:47.632 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:47.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:47.656 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:47.667 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:47.700 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:47.708 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:47.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:47.731 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:47.741 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:47.752 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:47.762 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:47.772 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:47.781 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:47.792 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:47.808 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:47.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:47.826 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:47.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:47.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:47.875 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:47.883 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:47.909 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:47.919 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:47.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:47.938 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:47.946 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:47.955 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:47.964 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:47.973 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:47.985 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:47.994 INFO:teuthology.orchestra.run.smithi069.stdout: "update_orchestration": { 2024-05-14T17:12:48.020 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "update_orchestration", 2024-05-14T17:12:48.031 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:48.058 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:48.068 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:48.078 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:48.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:48.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:48.132 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:48.141 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T17:12:48.150 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:48.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:48.165 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:48.171 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:48.179 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:48.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:48.195 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:48.201 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-05-14T17:12:48.240 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:48.247 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:48.254 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:48.260 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-05-14T17:12:48.268 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-05-14T17:12:48.273 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:48.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:48.311 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:48.319 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:48.326 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:48.332 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:48.338 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:48.353 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T17:12:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:48.387 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:48.423 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-05-14T17:12:48.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-05-14T17:12:48.443 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:48.451 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:48.457 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:48.464 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:48.471 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:48.503 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:48.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:48.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-05-14T17:12:48.524 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:48.531 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:48.540 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:48.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:48.556 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:48.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:48.580 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:48.625 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:48.633 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:48.642 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:48.650 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:48.659 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:48.668 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:48.708 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:48.754 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:48.764 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:48.772 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:48.781 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:48.790 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:48.800 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:48.808 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:48.855 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:48.865 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:48.890 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:48.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:48.907 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:48.914 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:48.925 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:48.933 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:48.942 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:48.950 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:48.985 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:48.996 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:49.007 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:49.018 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:49.046 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:49.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:49.066 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:49.077 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:49.088 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:49.102 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:49.115 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:49.129 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:49.142 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:49.153 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:49.166 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:49.177 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:49.191 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:49.215 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:49.224 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:49.264 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:49.273 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:49.282 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:49.290 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:49.299 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:49.310 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:49.319 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:49.353 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:49.363 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:49.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:49.384 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:49.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:49.402 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:49.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:49.421 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:49.454 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:49.480 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:49.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:49.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:49.507 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:49.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:49.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:49.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:49.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:49.571 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:49.589 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:49.599 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:49.608 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:49.616 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:49.626 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:49.724 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:49.759 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:49.768 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:49.778 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:49.787 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:49.797 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:49.806 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:49.814 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:49.860 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:49.865 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-05-14T17:12:49.934 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:49.943 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:49.954 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:49.962 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:49.969 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:49.976 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:49.984 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:50.019 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:50.026 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:50.032 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:50.038 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:50.043 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:50.050 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:50.056 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:50.077 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:50.084 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:50.097 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:50.122 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:50.132 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:50.141 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:50.150 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:50.159 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:50.168 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:50.177 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:50.203 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:50.232 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:50.240 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:50.248 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:50.256 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:50.265 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:50.273 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:50.286 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:50.296 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:50.313 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:50.348 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:50.357 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:50.366 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:50.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:50.382 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:50.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:50.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:50.445 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:50.453 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:50.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:50.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:50.478 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:50.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:50.496 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:50.511 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:50.533 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:50.543 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:50.551 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:50.560 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:50.568 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:50.577 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:50.586 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:50.595 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:50.604 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:50.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:50.685 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:50.691 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:50.699 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:50.707 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:50.716 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:50.725 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:50.736 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:50.750 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:50.768 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:50.779 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:50.790 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:50.800 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:50.812 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:50.823 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:50.836 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:50.847 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:50.874 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:50.884 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:50.896 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:50.916 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:50.925 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:50.934 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:50.953 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:50.981 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:51.001 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:51.010 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:51.020 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:51.028 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:51.040 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:51.050 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:51.060 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-05-14T17:12:51.069 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:51.097 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:51.107 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:51.117 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:51.129 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:51.158 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:51.168 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:51.186 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:51.197 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:51.220 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:51.240 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:51.249 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:51.258 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:51.268 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:51.278 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:51.287 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:51.297 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:51.343 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:51.351 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:51.361 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:51.371 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:51.381 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:51.390 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:51.400 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:51.414 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:51.423 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:51.463 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:51.473 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:51.482 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:51.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:51.502 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:51.528 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:51.537 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:51.545 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:51.574 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:51.582 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:51.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:51.599 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:51.607 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:51.612 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:51.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:51.653 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:51.660 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:51.687 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:51.694 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:51.701 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:51.711 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:51.749 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:51.757 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:51.766 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:51.777 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:51.803 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:51.811 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:51.820 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:51.829 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:51.837 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:51.846 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:51.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:51.890 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:51.919 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:51.933 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:51.942 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:51.950 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:51.959 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:51.968 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:51.976 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:51.985 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:51.996 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:52.010 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:52.037 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:52.046 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:52.060 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:52.113 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:52.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:52.131 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:52.156 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:52.164 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:52.188 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:52.197 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:52.206 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:52.215 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:52.223 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:52.231 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:52.239 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:52.248 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:52.274 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:52.284 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:52.293 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:52.301 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:52.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-05-14T17:12:52.319 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:52.328 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:52.348 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:52.356 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-05-14T17:12:52.370 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-05-14T17:12:52.381 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:52.391 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:52.401 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:52.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T17:12:52.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:52.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:52.470 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:52.496 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:52.507 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:52.518 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:52.529 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:52.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:52.549 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:12:52.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:12:52.573 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:12:52.582 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:52.608 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:52.617 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-05-14T17:12:52.626 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:52.637 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:52.644 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:52.670 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:52.677 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:52.684 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:52.718 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:52.725 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:52.734 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:52.740 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:52.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:52.770 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:52.782 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:52.793 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:52.820 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:52.829 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:52.839 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:52.849 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:52.859 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:52.883 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:52.892 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:52.901 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:52.932 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:52.942 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:52.952 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:52.961 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:52.970 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:52.979 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:52.989 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:52.999 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:53.009 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:53.036 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:53.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:53.053 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:53.061 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:53.093 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:53.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:53.111 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:53.117 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:53.146 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:53.153 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:53.165 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:53.172 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:53.180 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:53.187 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:53.193 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:53.208 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:53.215 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:53.229 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:53.236 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:53.242 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:53.249 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:53.284 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:53.293 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:53.300 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:53.307 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:53.324 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:53.359 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:53.371 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:53.383 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:53.396 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:53.407 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:53.424 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:53.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:53.471 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:53.482 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:53.494 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:53.506 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:53.566 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:53.577 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:53.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:53.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:53.618 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:53.650 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:53.657 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:53.665 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:53.672 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:53.678 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:53.685 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:53.700 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:53.706 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:53.731 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:53.739 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:53.746 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:53.754 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:53.777 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:53.784 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:53.791 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:53.800 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:53.864 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:53.871 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:53.877 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:53.885 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:53.892 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-05-14T17:12:53.899 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:53.907 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:53.931 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:53.939 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-05-14T17:12:53.955 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-05-14T17:12:53.964 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:53.970 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:53.979 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:54.003 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:54.014 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:54.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:54.031 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:54.055 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T17:12:54.065 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:54.074 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:54.082 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:54.091 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:54.099 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-05-14T17:12:54.108 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-05-14T17:12:54.125 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:54.134 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:54.150 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:54.159 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:54.170 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:54.187 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:54.196 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:54.203 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T17:12:54.210 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:54.217 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:54.270 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:54.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:54.282 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-05-14T17:12:54.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-05-14T17:12:54.296 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:54.305 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:54.314 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:54.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:12:54.335 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:54.375 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:54.385 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:54.393 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T17:12:54.405 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:54.418 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:54.429 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:54.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:54.449 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-05-14T17:12:54.474 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-05-14T17:12:54.514 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:54.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:54.553 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:54.565 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:54.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:54.588 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:54.600 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:54.612 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T17:12:54.633 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:54.644 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:54.655 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:54.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:54.677 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-05-14T17:12:54.688 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-05-14T17:12:54.699 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:54.753 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:54.783 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:54.793 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:54.804 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:54.814 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:54.824 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:54.833 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T17:12:54.841 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:54.850 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:54.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:54.892 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:54.900 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-05-14T17:12:54.909 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-05-14T17:12:54.921 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:54.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:54.939 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:54.947 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:54.956 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:55.014 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:55.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:55.041 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:55.049 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:55.058 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:55.066 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:55.093 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:55.127 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-05-14T17:12:55.147 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-05-14T17:12:55.176 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:55.188 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:55.201 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:55.214 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:55.225 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:55.234 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:55.244 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:55.254 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:55.310 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:55.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:55.329 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:55.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:55.348 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-05-14T17:12:55.357 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-05-14T17:12:55.367 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:55.409 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:55.416 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:55.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T17:12:55.450 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:55.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:55.464 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:55.470 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:55.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:55.483 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:55.490 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:55.516 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:55.523 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-05-14T17:12:55.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-05-14T17:12:55.538 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:55.545 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:55.551 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:55.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:55.601 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:55.608 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:55.617 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:55.626 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:55.634 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:55.643 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:55.652 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:55.661 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:55.670 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:12:55.691 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:12:55.722 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:55.731 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:55.741 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:55.751 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-05-14T17:12:55.762 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-05-14T17:12:55.771 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:55.780 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:55.793 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:55.801 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:55.826 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:55.834 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:55.841 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:55.846 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-05-14T17:12:55.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-05-14T17:12:55.900 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:55.908 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:55.915 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:55.969 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-05-14T17:12:55.975 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:55.983 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:55.990 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:55.999 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:56.005 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:56.016 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:56.035 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:56.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:56.068 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-05-14T17:12:56.080 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-05-14T17:12:56.096 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:56.108 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:56.119 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:56.149 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:56.163 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:56.178 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:56.224 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:56.237 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:56.246 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:56.255 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:56.263 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:56.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:56.282 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-05-14T17:12:56.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-05-14T17:12:56.323 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:56.341 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:56.350 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:56.356 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:56.366 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:56.376 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:56.408 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:56.422 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:56.434 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:56.460 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:56.469 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:56.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:56.487 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:56.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:56.504 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:56.514 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:56.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:56.534 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:56.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:56.566 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:56.572 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:56.579 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:56.586 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:56.617 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:56.626 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:56.636 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:56.661 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:56.670 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:56.678 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:56.686 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:56.697 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:56.707 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:56.719 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:56.755 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:56.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:56.791 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:56.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:56.824 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:56.830 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:56.837 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:56.845 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:56.851 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:56.858 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:56.882 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:56.889 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:56.897 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:56.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:56.913 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:56.924 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:56.933 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:56.952 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:56.964 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:56.991 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:57.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:57.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:57.067 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:57.080 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:57.093 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:57.107 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:57.121 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:57.151 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:57.162 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:57.175 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:57.185 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:57.195 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:57.205 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:57.217 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:57.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:57.249 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-05-14T17:12:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-05-14T17:12:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-05-14T17:12:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-05-14T17:12:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:57.965 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-05-14T17:12:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-05-14T17:12:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:12:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T17:12:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-05-14T17:12:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:58.579 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:58.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:58.616 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:58.623 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:58.628 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:58.634 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:58.640 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:58.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:58.658 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:58.671 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:58.707 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:58.735 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:58.745 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:58.754 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:58.763 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:58.771 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:58.779 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:58.788 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:58.796 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:58.857 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:58.866 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:12:58.873 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:12:58.881 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:58.889 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:58.908 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:58.914 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:12:58.930 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:58.943 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:58.971 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:58.982 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:58.993 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:59.004 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:59.010 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:59.025 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:59.035 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:59.047 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:59.082 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:59.095 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:59.124 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:59.137 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:59.149 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:59.162 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:12:59.172 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:12:59.188 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:59.207 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:59.230 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:59.240 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:59.252 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:59.264 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:59.275 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:59.287 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:59.302 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:59.311 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:59.358 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:59.366 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:59.373 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:12:59.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:12:59.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:12:59.393 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:12:59.400 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:59.413 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:12:59.421 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:59.456 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:59.462 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:59.469 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:59.477 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:59.485 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:59.493 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:59.503 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:59.512 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:12:59.540 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:59.550 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:12:59.560 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-05-14T17:12:59.568 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:12:59.577 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:12:59.586 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:12:59.595 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:12:59.613 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:12:59.623 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:12:59.647 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:59.658 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:59.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:12:59.673 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:59.679 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:59.685 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:12:59.692 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:12:59.700 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:12:59.731 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:12:59.744 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:12:59.754 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:12:59.761 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:12:59.775 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:12:59.784 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:12:59.791 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:12:59.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:12:59.814 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:12:59.839 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:12:59.853 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:12:59.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:12:59.875 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:12:59.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:12:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:12:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:12:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:12:59.967 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:12:59.977 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:12:59.991 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:00.003 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:00.011 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:00.023 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:00.035 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:00.048 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:00.060 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:00.080 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:00.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:00.100 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:00.114 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:00.125 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:00.137 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:00.146 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:00.154 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:00.177 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:00.184 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:00.193 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:00.198 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:00.204 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:00.211 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:00.219 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:00.332 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:00.341 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:00.351 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:00.359 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:00.373 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:00.386 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:00.399 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:00.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:00.425 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-05-14T17:13:00.439 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-05-14T17:13:00.450 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:00.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:00.473 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:00.483 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-05-14T17:13:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:00.529 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:00.565 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T17:13:00.573 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:00.587 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:00.599 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:00.608 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:00.617 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-05-14T17:13:00.625 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-05-14T17:13:00.640 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:00.650 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:00.676 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:00.684 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:00.695 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:00.704 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:00.717 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:00.727 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-05-14T17:13:00.753 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:00.768 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:00.833 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:00.842 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:00.851 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-05-14T17:13:00.860 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-05-14T17:13:00.868 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:00.878 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:00.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:00.915 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:00.924 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:00.958 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:00.984 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:00.992 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T17:13:00.999 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:01.009 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:01.016 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:01.030 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:01.037 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-05-14T17:13:01.059 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T17:13:01.066 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:01.073 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:01.091 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:01.100 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:01.110 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:01.144 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:01.151 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T17:13:01.164 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:01.207 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:01.220 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:01.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:01.251 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:01.263 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:01.274 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:01.286 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:01.315 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:01.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:01.333 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:01.343 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:01.355 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:01.365 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:01.379 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:01.407 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:01.418 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:01.443 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:01.480 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:01.490 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:01.502 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:01.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-05-14T17:13:01.518 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:01.528 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:01.539 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:01.594 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-05-14T17:13:01.603 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-05-14T17:13:01.613 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:13:01.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:01.626 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:01.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-05-14T17:13:01.642 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:01.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:01.696 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:01.739 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:01.764 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:01.769 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:01.776 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:01.782 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:01.788 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-05-14T17:13:01.795 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-05-14T17:13:01.802 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:01.862 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:01.869 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:01.876 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:01.882 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:01.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:01.895 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:01.903 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:01.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:01.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:01.972 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:02.010 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:02.019 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:13:02.029 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:13:02.038 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:02.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:02.058 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:02.068 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:13:02.099 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:02.114 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:02.122 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:02.132 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:02.140 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:02.149 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:02.158 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:02.182 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:02.194 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:02.223 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:02.257 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:02.271 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:02.281 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:02.293 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:02.304 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:02.317 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:02.329 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:02.353 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:02.363 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:02.372 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:02.381 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:02.388 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:02.395 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:02.404 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:02.424 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:02.436 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:02.465 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:02.481 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:02.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:02.509 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:02.519 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:02.532 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:02.541 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:02.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:02.608 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:02.615 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:02.622 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:02.629 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:02.635 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:02.641 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:02.647 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:02.681 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:02.698 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:02.725 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:02.732 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:02.742 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:02.751 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:02.761 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:02.770 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:02.778 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:02.804 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:02.816 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:02.825 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:02.830 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:02.837 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:02.844 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:02.851 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:02.866 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:02.873 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:02.917 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:02.922 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:02.930 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:02.937 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:02.944 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:02.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:02.960 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:02.970 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:03.034 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:03.046 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:03.055 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:03.066 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:03.080 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:03.092 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:03.101 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:03.118 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:03.127 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:03.159 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:03.176 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:03.186 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:03.202 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:03.210 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:03.221 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:03.227 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:03.259 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:03.268 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:03.278 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:03.289 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:03.298 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:03.309 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:03.319 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:03.335 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-05-14T17:13:03.359 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-05-14T17:13:03.384 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:03.394 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:03.402 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:03.410 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:03.419 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:03.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:03.446 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:03.468 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:03.476 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:03.485 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:03.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:03.506 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-05-14T17:13:03.514 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-05-14T17:13:03.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:03.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:03.576 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:03.593 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-05-14T17:13:03.601 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:03.609 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:03.618 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:03.628 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:03.636 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:03.647 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:03.657 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:03.701 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:03.708 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-05-14T17:13:03.716 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-05-14T17:13:03.723 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:03.730 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:03.737 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:03.747 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T17:13:03.777 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:03.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:03.877 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:03.894 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:03.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:03.916 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:03.942 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:03.952 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:03.965 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:04.025 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:13:04.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:04.063 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:04.072 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 24755, 2024-05-14T17:13:04.102 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "y", 2024-05-14T17:13:04.109 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540701547738038271, 2024-05-14T17:13:04.115 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-05-14T17:13:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:04.166 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-05-14T17:13:04.202 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:04.212 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:04.220 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:04.226 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:13:04.234 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:13:04.240 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:04.247 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:04.253 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:04.279 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:13:04.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:04.322 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:04.329 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:04.335 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T17:13:04.341 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:04.348 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:04.368 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:04.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:04.391 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:04.398 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:04.404 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:04.411 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:04.418 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:04.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:04.432 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:04.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:04.467 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:04.498 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:04.510 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:04.520 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:04.535 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:04.542 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:04.549 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:04.573 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:04.586 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:04.621 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:04.635 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:04.646 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:04.656 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:04.665 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:04.677 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:04.689 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:04.702 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:04.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:04.759 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:04.775 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:04.784 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:04.795 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:04.808 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:04.819 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:04.856 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:04.865 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:04.882 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:04.892 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:04.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:04.910 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:04.918 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:04.925 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:04.931 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:04.946 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:05.002 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:05.011 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:05.019 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:05.028 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:05.037 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:05.046 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:05.056 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:05.073 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:05.082 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:05.100 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:05.109 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:05.119 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:05.134 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:05.143 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:05.152 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:05.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:05.201 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:05.231 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:05.241 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:05.250 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:05.296 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:05.305 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:05.315 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:05.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:05.360 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:05.383 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:05.391 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-05-14T17:13:05.402 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-05-14T17:13:05.412 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:05.423 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:05.434 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:05.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:05.511 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:05.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:05.561 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:05.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-05-14T17:13:05.579 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:05.590 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:05.601 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:05.623 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:05.635 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-05-14T17:13:05.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-05-14T17:13:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:05.686 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:05.697 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:05.711 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-05-14T17:13:05.722 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:05.730 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:05.738 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:05.766 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-05-14T17:13:05.776 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:05.784 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:05.792 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:05.799 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:05.807 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-05-14T17:13:05.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-05-14T17:13:05.831 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:05.839 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:05.892 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:05.903 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:05.914 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:05.924 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:05.934 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:05.942 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-05-14T17:13:05.949 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:05.954 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:05.995 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:06.003 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:06.009 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-05-14T17:13:06.017 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-05-14T17:13:06.023 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:06.030 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:06.038 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:06.048 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:06.084 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:06.101 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:06.108 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:06.115 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-05-14T17:13:06.122 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:06.129 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:06.136 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:06.146 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:06.152 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-05-14T17:13:06.212 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-05-14T17:13:06.220 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:06.225 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:06.232 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:06.239 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-05-14T17:13:06.246 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:06.254 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:06.265 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:06.308 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-05-14T17:13:06.330 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:06.338 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:06.345 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:06.354 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:06.360 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-05-14T17:13:06.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-05-14T17:13:06.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:06.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:06.412 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:06.419 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:06.425 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:06.432 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:06.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:06.445 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-05-14T17:13:06.452 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:06.471 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:06.477 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:06.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:06.508 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-05-14T17:13:06.516 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-05-14T17:13:06.526 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:06.534 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:06.543 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:06.553 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:06.563 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:06.589 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:06.598 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:06.609 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T17:13:06.638 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:06.646 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:06.655 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:06.663 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:06.679 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-05-14T17:13:06.686 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-05-14T17:13:06.719 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:06.729 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:06.740 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:06.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:06.761 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:06.772 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:06.788 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:06.798 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-05-14T17:13:06.829 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:06.840 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:06.849 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:06.859 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:06.870 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:06.880 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:06.891 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:06.904 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:06.916 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:06.946 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:06.957 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:06.969 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:06.977 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:06.987 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:06.997 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:07.009 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:07.019 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:07.081 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:07.100 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:07.110 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:07.121 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:07.132 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-05-14T17:13:07.142 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:07.153 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:07.169 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:07.180 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-05-14T17:13:07.193 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-05-14T17:13:07.204 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:07.214 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:07.225 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:07.235 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:07.246 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:07.257 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:07.268 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:07.303 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T17:13:07.311 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:07.321 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:07.332 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:07.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:07.355 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-05-14T17:13:07.364 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-05-14T17:13:07.380 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:07.386 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:07.393 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:07.400 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-05-14T17:13:07.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:07.415 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:07.423 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:07.430 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T17:13:07.448 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:13:07.455 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:07.475 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:07.482 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:07.489 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-05-14T17:13:07.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-05-14T17:13:07.502 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:13:07.511 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:07.520 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:07.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:07.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-05-14T17:13:07.561 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-05-14T17:13:07.568 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:07.577 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T17:13:07.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:13:07.630 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:07.637 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:07.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:07.651 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-05-14T17:13:07.679 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-05-14T17:13:07.686 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:13:07.692 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:07.698 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:07.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-05-14T17:13:07.714 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:13:07.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-05-14T17:13:07.755 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:07.762 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T17:13:07.778 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:07.786 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:07.793 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:07.805 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:07.814 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-05-14T17:13:07.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-05-14T17:13:07.826 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:07.833 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:07.859 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:07.866 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T17:13:07.873 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:07.880 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:07.886 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:07.892 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T17:13:07.900 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T17:13:07.914 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:07.922 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:07.936 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:07.948 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-05-14T17:13:07.956 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-05-14T17:13:07.964 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:13:07.976 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:07.982 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-05-14T17:13:07.996 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-05-14T17:13:08.030 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-05-14T17:13:08.038 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:08.047 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-05-14T17:13:08.057 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T17:13:08.065 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:08.074 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:08.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:08.097 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-05-14T17:13:08.147 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-05-14T17:13:08.161 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:08.170 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:08.178 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:08.187 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-05-14T17:13:08.195 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:08.206 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:08.217 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:08.229 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T17:13:08.263 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:13:08.274 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:08.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:08.297 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:08.308 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-05-14T17:13:08.320 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-05-14T17:13:08.367 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:13:08.381 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:08.393 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:08.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:08.423 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-05-14T17:13:08.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-05-14T17:13:08.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:08.454 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T17:13:08.464 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:13:08.474 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:08.484 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:08.511 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:08.553 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:08.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:08.577 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:08.588 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:08.599 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:08.609 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:08.668 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:08.678 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:08.715 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:08.726 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:08.736 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:08.747 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:08.780 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:08.790 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:08.801 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:08.830 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:08.840 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:08.851 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:08.861 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:08.870 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:08.879 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:08.887 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:08.906 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:08.917 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:08.946 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:08.980 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:08.989 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:08.998 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:09.008 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:09.016 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:09.025 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:09.036 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:09.093 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:09.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:09.113 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:09.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:09.131 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:09.163 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:09.173 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:09.181 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:09.188 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:09.204 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:09.211 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:09.220 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:09.227 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:09.234 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:09.241 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:09.247 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:09.253 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:09.280 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:09.287 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:09.295 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:09.302 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:09.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:09.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:09.345 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:09.357 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:09.363 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:09.388 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:09.393 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:09.400 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:09.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:09.413 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:09.419 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:09.427 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:09.433 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:09.460 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:09.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:09.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:09.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-05-14T17:13:09.502 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-05-14T17:13:09.508 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:13:09.514 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:09.528 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:09.536 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:09.546 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:09.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:09.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:09.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T17:13:09.575 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:09.581 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:09.588 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:09.594 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:09.625 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-05-14T17:13:09.631 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-05-14T17:13:09.638 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:09.646 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:09.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:09.680 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-05-14T17:13:09.689 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:09.703 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:09.711 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:09.730 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-05-14T17:13:09.739 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-05-14T17:13:09.747 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-05-14T17:13:09.758 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-05-14T17:13:09.768 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-05-14T17:13:09.779 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:09.790 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-05-14T17:13:09.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:09.812 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:09.832 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:09.866 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:09.874 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-05-14T17:13:09.885 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-05-14T17:13:09.895 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:09.906 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:09.920 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:09.931 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:09.946 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:09.957 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:09.967 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:09.978 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T17:13:09.989 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:09.998 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:10.009 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:10.019 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:10.087 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:13:10.117 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:13:10.128 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:10.138 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:10.148 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:10.161 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:13:10.172 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:10.187 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:10.198 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:10.218 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T17:13:10.228 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:10.238 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:10.249 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:10.350 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:10.358 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:10.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:10.376 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:10.440 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:10.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:10.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:10.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:10.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:10.489 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:10.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:10.536 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:10.545 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:10.575 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:10.584 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:10.593 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-05-14T17:13:10.604 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-05-14T17:13:10.613 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:10.622 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:10.631 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:10.640 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:13:10.704 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:13:10.713 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:10.721 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:10.730 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T17:13:10.739 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T17:13:10.748 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:10.779 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:10.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:10.810 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-05-14T17:13:10.837 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-05-14T17:13:10.843 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:13:10.853 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:10.863 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:10.872 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:13:10.881 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:10.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:10.896 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:10.926 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T17:13:10.932 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:10.941 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:10.948 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:10.956 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:10.963 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:10.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:10.988 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:10.997 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-05-14T17:13:11.018 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:11.026 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:11.035 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:11.043 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-05-14T17:13:11.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-05-14T17:13:11.060 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:13:11.067 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:11.076 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:11.108 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-05-14T17:13:11.118 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:11.127 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:11.136 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:11.145 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T17:13:11.174 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:11.183 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:11.223 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:11.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:11.257 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-05-14T17:13:11.265 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-05-14T17:13:11.278 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:11.285 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:11.293 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:11.303 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-05-14T17:13:11.311 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:11.318 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:11.346 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T17:13:11.370 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:11.379 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:11.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:11.397 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:11.406 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-05-14T17:13:11.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-05-14T17:13:11.430 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:11.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:11.470 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:11.478 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-05-14T17:13:11.486 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:11.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:11.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:11.511 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T17:13:11.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:11.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:11.552 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:11.560 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:11.571 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-05-14T17:13:11.582 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-05-14T17:13:11.594 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:11.602 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:11.613 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:11.622 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:11.646 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:11.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:11.662 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:11.675 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T17:13:11.688 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T17:13:11.698 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:11.707 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:11.716 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:11.741 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-05-14T17:13:11.775 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-05-14T17:13:11.783 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:11.792 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:11.803 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:11.813 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:13:11.823 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:11.839 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:11.847 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:11.872 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T17:13:11.881 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:11.890 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:11.899 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:11.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-05-14T17:13:11.926 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T17:13:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:13:11.974 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:12.011 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:12.021 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-05-14T17:13:12.029 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:12.038 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:12.047 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:12.056 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T17:13:12.072 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:12.081 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:12.097 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:12.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:12.119 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-05-14T17:13:12.131 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-05-14T17:13:12.143 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:12.154 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:12.162 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:12.173 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:12.182 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:12.223 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:12.232 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:12.241 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-05-14T17:13:12.251 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-05-14T17:13:12.259 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-05-14T17:13:12.269 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:12.281 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T17:13:12.291 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:12.314 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:12.323 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:12.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:12.340 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-05-14T17:13:12.352 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-05-14T17:13:12.360 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:12.369 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:12.377 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:12.415 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:12.453 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:12.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:12.471 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:12.482 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T17:13:12.493 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:12.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:12.523 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:12.535 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:12.559 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-05-14T17:13:12.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-05-14T17:13:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:12.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:12.600 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:12.630 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:12.662 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:12.672 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:12.680 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:12.689 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T17:13:12.699 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:12.708 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:12.718 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:12.727 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:12.737 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-05-14T17:13:12.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-05-14T17:13:12.765 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:12.796 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:12.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:12.840 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:12.879 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:12.886 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:12.894 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:12.901 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T17:13:12.909 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:12.916 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:12.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:12.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:12.938 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-05-14T17:13:12.946 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-05-14T17:13:12.952 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:12.961 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:12.973 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:12.980 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T17:13:13.011 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:13.045 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:13.121 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:13.134 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:13:13.145 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:13.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:13.169 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:13.181 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:13.193 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-05-14T17:13:13.202 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-05-14T17:13:13.211 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:13.229 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:13.238 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T17:13:13.247 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:13.257 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:13.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:13.297 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-05-14T17:13:13.326 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:13.333 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:13.343 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:13.353 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:13.365 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-05-14T17:13:13.376 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-05-14T17:13:13.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:13.398 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:13.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:13.421 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T17:13:13.432 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:13.442 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:13.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:13.463 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-05-14T17:13:13.500 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:13.514 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:13.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:13.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:13.575 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-05-14T17:13:13.581 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-05-14T17:13:13.588 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:13.595 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:13.602 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:13.608 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T17:13:13.614 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:13.620 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:13.626 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:13.632 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:13:13.639 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:13.645 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:13.653 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:13.691 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:13.701 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-05-14T17:13:13.741 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-05-14T17:13:13.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:13.758 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:13.767 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:13.775 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T17:13:13.784 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:13.792 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:13.800 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:13.809 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-05-14T17:13:13.818 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:13.824 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:13.830 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:13.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:13.844 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-05-14T17:13:13.851 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-05-14T17:13:13.891 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:13.898 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:13.926 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:13.935 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T17:13:13.944 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:13.956 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:13.968 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:13.980 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-05-14T17:13:13.992 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:14.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:14.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:14.026 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:14.036 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-05-14T17:13:14.045 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-05-14T17:13:14.053 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:14.062 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:14.070 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:14.101 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T17:13:14.123 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:14.142 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:14.167 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:14.177 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-05-14T17:13:14.187 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:14.196 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:14.206 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:14.215 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:14.226 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-05-14T17:13:14.238 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-05-14T17:13:14.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:14.258 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:14.268 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:14.278 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T17:13:14.288 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:14.299 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:14.329 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:14.340 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-05-14T17:13:14.376 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:14.388 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:14.397 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:14.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:14.415 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-05-14T17:13:14.426 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-05-14T17:13:14.434 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:14.443 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:14.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:14.458 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T17:13:14.464 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:14.469 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:14.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:14.483 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-05-14T17:13:14.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:14.526 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:14.535 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:14.603 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:14.612 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-05-14T17:13:14.621 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-05-14T17:13:14.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:14.641 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T17:13:14.662 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:14.669 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:14.676 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:14.683 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-05-14T17:13:14.693 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:14.702 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:14.710 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:14.718 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:14.752 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-05-14T17:13:14.760 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-05-14T17:13:14.796 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:14.804 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:14.812 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:14.820 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T17:13:14.829 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:14.837 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:14.845 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:14.853 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:13:14.860 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:14.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:14.877 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:14.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:14.891 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-05-14T17:13:14.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-05-14T17:13:14.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:14.946 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:14.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:15.004 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-14T17:13:15.013 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:15.025 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:15.037 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:15.049 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-05-14T17:13:15.061 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:15.071 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:15.080 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:15.090 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:15.102 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-05-14T17:13:15.114 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-05-14T17:13:15.125 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:15.136 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:15.147 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:15.160 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T17:13:15.175 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:15.203 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:15.244 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:15.253 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:13:15.262 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:15.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:15.281 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:15.293 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:15.301 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-05-14T17:13:15.309 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-05-14T17:13:15.318 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:15.330 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:15.341 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:15.352 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T17:13:15.363 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:15.372 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:15.380 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:15.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-05-14T17:13:15.429 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:15.469 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:15.475 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:15.482 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:15.490 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-05-14T17:13:15.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-05-14T17:13:15.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:15.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:15.526 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:15.534 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T17:13:15.544 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:15.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:15.561 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:15.571 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-05-14T17:13:15.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:15.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:15.625 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:15.633 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:15.646 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-05-14T17:13:15.687 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-05-14T17:13:15.693 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:15.702 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:15.708 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:15.714 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T17:13:15.721 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:15.728 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:15.734 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:15.740 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-05-14T17:13:15.747 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:15.753 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:15.761 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:15.773 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:15.788 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-05-14T17:13:15.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-05-14T17:13:15.831 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:15.867 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:15.879 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:15.889 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:15.901 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:15.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:15.923 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:15.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T17:13:15.944 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:15.953 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:15.961 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:15.973 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:15.981 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-05-14T17:13:15.990 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-05-14T17:13:15.998 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:16.045 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:16.059 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:16.068 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:13:16.162 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:16.171 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:16.183 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:16.194 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T17:13:16.206 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:16.217 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:16.229 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:16.241 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:16.259 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-05-14T17:13:16.268 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T17:13:16.298 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:16.305 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:16.313 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:16.320 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-05-14T17:13:16.341 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:16.351 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:16.360 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:16.397 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T17:13:16.405 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:16.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:16.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:16.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:16.438 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-05-14T17:13:16.481 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-05-14T17:13:16.489 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:16.496 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:16.503 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:16.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-05-14T17:13:16.525 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:16.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:16.538 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:16.544 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T17:13:16.560 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:16.566 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:16.597 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:16.603 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:16.611 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-05-14T17:13:16.621 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-05-14T17:13:16.629 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:16.636 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:16.666 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:16.672 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-05-14T17:13:16.682 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:16.692 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:16.714 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:16.724 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-05-14T17:13:16.734 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:16.743 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:16.753 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:16.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:16.777 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-05-14T17:13:16.797 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-05-14T17:13:16.808 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:16.821 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:16.830 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:16.840 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:16.873 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:16.883 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:16.896 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:16.908 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T17:13:16.920 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:16.934 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:16.946 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:16.957 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:16.968 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-05-14T17:13:16.978 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-05-14T17:13:16.994 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:17.005 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:17.032 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:17.044 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:13:17.054 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:17.065 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:17.103 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:17.114 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-05-14T17:13:17.124 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:17.135 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:17.146 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:17.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:17.183 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-05-14T17:13:17.191 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-05-14T17:13:17.200 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:17.208 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:17.217 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:17.231 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T17:13:17.241 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:17.265 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:17.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:17.283 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T17:13:17.292 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:17.327 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:17.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:17.346 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:17.355 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-05-14T17:13:17.362 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-05-14T17:13:17.393 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:17.400 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:17.405 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:17.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:13:17.418 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:17.424 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:17.432 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:17.438 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-05-14T17:13:17.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:17.462 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:17.471 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:17.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:17.519 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-05-14T17:13:17.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-05-14T17:13:17.542 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:17.550 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:17.558 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:17.586 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:17.598 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:17.607 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:17.616 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:17.625 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T17:13:17.634 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:17.649 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:17.658 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:17.673 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:17.682 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-05-14T17:13:17.690 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-05-14T17:13:17.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:17.707 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:17.743 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:17.753 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:17.763 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:17.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:17.794 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T17:13:17.812 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:17.820 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:17.829 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:17.839 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:17.849 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:17.857 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:17.866 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:17.875 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:17.892 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:17.922 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:17.930 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:17.939 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:17.947 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:17.955 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:17.986 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:18.003 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:18.013 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:18.022 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:18.034 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:18.040 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:18.075 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:18.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:18.116 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:18.159 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-05-14T17:13:18.166 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-05-14T17:13:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:18.218 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:18.227 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:18.236 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:18.253 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T17:13:18.274 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:18.309 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:18.401 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:18.414 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:18.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:18.431 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:18.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:18.449 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T17:13:18.461 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:18.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:18.594 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:18.634 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:18.669 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:18.696 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:18.789 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:18.797 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:18.865 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:18.874 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:18.882 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:18.891 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:18.900 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:18.928 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T17:13:18.967 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T17:13:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:19.014 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:19.021 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:19.029 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:19.063 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:19.070 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T17:13:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:19.101 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:19.110 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:19.134 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T17:13:19.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T17:13:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:19.223 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-05-14T17:13:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T17:13:19.299 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:19.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:19.321 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-05-14T17:13:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-05-14T17:13:19.366 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:19.374 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:19.391 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:13:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:19.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:19.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:19.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-05-14T17:13:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-05-14T17:13:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-05-14T17:13:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:19.560 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:19.566 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:19.573 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:13:19.580 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:19.587 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:19.650 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T17:13:19.664 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:19.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:19.701 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:19.709 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-05-14T17:13:19.723 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-05-14T17:13:19.730 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:19.738 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:19.751 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:19.766 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:19.772 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-05-14T17:13:19.815 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:19.844 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:19.906 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-05-14T17:13:19.918 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-05-14T17:13:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:19.947 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:19.957 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-05-14T17:13:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:19.986 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-05-14T17:13:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-05-14T17:13:20.045 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:20.058 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T17:13:20.070 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:20.124 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-05-14T17:13:20.152 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-05-14T17:13:20.166 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:20.174 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:20.185 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:20.199 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-05-14T17:13:20.211 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:20.246 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:20.257 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:20.269 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T17:13:20.284 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:20.293 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:20.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:20.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:20.343 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-05-14T17:13:20.352 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-05-14T17:13:20.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:20.396 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:20.467 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:20.476 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T17:13:20.484 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:20.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:20.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:20.514 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T17:13:20.524 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:20.532 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:20.567 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:20.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:20.585 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-05-14T17:13:20.594 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-05-14T17:13:20.604 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:20.614 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:20.623 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:20.640 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:20.649 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:20.686 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:20.696 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:20.704 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T17:13:20.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:20.722 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:20.735 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:20.747 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:20.757 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-05-14T17:13:20.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-05-14T17:13:20.803 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:20.812 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:20.821 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:20.830 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:20.838 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:20.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:20.865 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:20.905 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T17:13:20.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:20.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:20.964 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:20.975 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:20.995 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-05-14T17:13:21.006 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-05-14T17:13:21.017 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:21.028 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:21.039 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:21.051 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:21.062 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:21.070 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:21.080 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:21.119 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T17:13:21.130 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:21.141 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:21.156 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:21.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:21.183 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-05-14T17:13:21.195 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-05-14T17:13:21.228 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:21.240 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:21.247 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:21.253 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:21.262 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:21.271 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:21.282 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:21.292 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T17:13:21.302 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:21.310 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:21.353 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:21.363 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:21.389 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-05-14T17:13:21.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-05-14T17:13:21.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:21.426 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:21.435 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:21.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:21.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:21.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:21.497 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:21.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T17:13:21.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:21.524 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:21.534 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:21.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:21.561 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-05-14T17:13:21.595 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-05-14T17:13:21.606 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:21.633 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:21.643 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:21.658 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-05-14T17:13:21.670 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:21.681 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:21.722 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:21.740 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-05-14T17:13:21.751 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:21.762 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:21.774 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:21.789 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:21.803 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:21.817 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:21.833 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:21.882 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:21.893 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:21.913 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:21.932 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:21.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:21.968 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:21.984 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:22.015 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:22.030 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:22.045 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:22.059 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:22.072 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-05-14T17:13:22.082 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-05-14T17:13:22.096 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:22.108 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:22.120 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:22.133 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:22.175 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:22.187 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:22.198 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:22.216 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T17:13:22.230 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:22.268 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:22.290 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:22.302 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:22.312 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-05-14T17:13:22.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-05-14T17:13:22.333 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:22.344 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:22.354 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:22.367 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:22.379 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:22.389 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:22.398 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:22.443 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T17:13:22.455 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:22.467 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:22.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:22.494 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:22.516 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-05-14T17:13:22.560 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-05-14T17:13:22.574 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:22.589 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:22.603 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:22.619 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:22.630 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:22.642 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:22.655 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:22.666 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T17:13:22.676 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:22.705 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:22.720 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:22.733 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:22.746 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-05-14T17:13:22.763 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-05-14T17:13:22.779 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:22.817 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:22.858 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:22.874 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:22.889 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:22.905 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:22.922 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:22.938 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T17:13:22.953 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:22.971 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:22.980 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:23.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:23.067 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-05-14T17:13:23.077 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-05-14T17:13:23.088 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:23.099 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:23.109 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:23.120 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:23.152 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:23.195 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:23.207 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:23.218 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T17:13:23.228 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:23.240 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:23.260 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:23.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:23.281 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-05-14T17:13:23.291 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-05-14T17:13:23.301 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:23.337 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:23.346 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:23.353 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:23.361 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:23.375 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:23.383 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:23.409 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T17:13:23.448 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:23.458 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:23.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:23.473 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:23.485 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:23.494 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:23.505 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:23.513 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-05-14T17:13:23.522 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:23.601 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:23.609 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:23.617 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:23.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:23.638 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:23.663 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:23.670 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:23.684 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:23.713 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:23.725 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:23.734 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:23.743 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:23.749 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:23.758 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:23.770 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:23.782 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:23.795 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:23.841 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:23.856 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:23.866 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:23.879 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:23.888 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:23.910 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:23.921 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:23.936 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:23.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:23.966 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:23.980 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:23.997 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:24.010 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:24.024 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:24.051 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:24.066 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:24.093 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:24.106 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:24.117 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:24.133 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:24.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:24.163 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:24.174 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:24.211 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:24.224 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:24.236 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:24.251 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:24.263 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:24.274 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:24.285 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:24.295 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:24.326 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:24.339 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:24.352 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:24.361 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:24.370 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:24.379 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:24.390 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:24.453 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:24.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:24.493 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:24.505 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:24.516 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:24.600 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:24.612 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:24.623 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:24.632 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:24.643 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:24.654 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:24.664 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:24.696 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:24.714 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:24.729 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:24.741 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:24.752 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-05-14T17:13:24.769 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-05-14T17:13:24.780 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:24.810 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:24.831 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:24.846 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-05-14T17:13:24.857 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:24.871 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:24.886 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:24.898 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T17:13:24.912 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:24.949 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:24.964 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:24.980 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:24.994 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:25.012 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:25.024 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:25.035 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:25.081 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:25.096 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:25.118 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:25.132 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:25.144 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:25.155 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:25.166 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:25.177 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:25.191 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:25.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:25.246 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-05-14T17:13:25.257 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-05-14T17:13:25.268 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:25.280 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:25.293 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:25.304 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-05-14T17:13:25.316 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:25.340 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:25.365 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:25.388 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T17:13:25.396 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:25.404 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:25.412 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:25.420 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:25.428 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:25.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:25.463 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:25.473 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-05-14T17:13:25.484 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:25.491 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:25.500 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:25.506 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T17:13:25.514 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T17:13:25.521 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:25.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:25.575 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:25.594 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:13:25.602 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:25.609 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:25.616 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:25.625 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:25.656 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:25.665 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:25.672 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:25.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:25.692 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T17:13:25.705 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T17:13:25.711 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:25.725 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:25.734 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:25.752 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:25.769 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:25.856 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:25.874 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:25.883 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:25.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:25.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:25.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:25.959 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:25.976 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T17:13:25.987 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T17:13:25.998 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:26.012 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:26.026 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:26.043 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:26.057 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:26.071 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:26.084 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:26.103 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:26.138 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:26.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:26.175 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:26.192 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:26.205 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-05-14T17:13:26.249 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T17:13:26.262 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:26.274 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:26.287 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:26.301 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:26.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:26.324 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:26.337 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:26.348 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:26.381 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:26.392 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:26.412 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:26.423 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:26.463 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T17:13:26.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T17:13:26.485 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:26.496 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:26.506 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:26.517 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:26.528 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:26.539 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:26.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:26.606 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:26.621 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:26.629 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:26.636 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:26.645 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:26.688 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T17:13:26.708 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T17:13:26.717 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:26.749 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:26.766 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:26.782 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:26.799 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:26.874 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:26.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:26.906 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:26.920 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:26.936 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:26.967 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:26.984 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:27.001 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T17:13:27.017 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T17:13:27.032 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:27.046 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:27.062 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:27.078 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:27.108 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:27.150 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:27.163 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:27.175 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:27.189 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:27.201 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:27.215 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:27.230 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:27.278 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T17:13:27.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T17:13:27.299 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:27.311 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:27.322 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:27.333 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:27.340 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:27.354 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:27.361 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:27.391 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:27.466 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:27.475 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:27.486 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:27.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:27.505 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T17:13:27.534 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T17:13:27.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:27.564 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:27.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:27.584 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:27.591 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:27.599 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:27.608 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:27.618 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:27.637 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:27.645 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:27.684 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:27.692 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:27.701 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T17:13:27.709 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T17:13:27.719 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:27.726 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:27.752 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:27.764 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:27.774 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:27.783 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:27.795 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:27.803 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:27.816 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:27.829 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:27.843 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:27.879 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:27.925 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T17:13:27.950 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T17:13:27.964 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:27.980 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:27.994 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:28.003 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:28.018 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:28.029 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:28.038 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:28.052 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:28.066 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:28.077 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:28.090 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:28.100 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:28.119 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T17:13:28.164 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T17:13:28.187 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:28.200 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:28.241 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:28.251 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:28.259 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:28.270 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:28.281 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:28.293 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:28.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:28.312 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:28.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:28.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:28.380 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T17:13:28.390 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T17:13:28.401 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:28.411 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:28.424 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:28.458 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:28.496 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:28.525 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:28.537 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:28.547 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:28.556 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:28.565 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:28.574 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:28.583 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:28.593 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T17:13:28.603 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T17:13:28.613 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:28.698 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:28.706 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:28.719 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:28.730 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:28.739 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:28.759 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:28.781 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:28.808 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:28.824 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:28.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:28.851 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:28.865 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T17:13:28.877 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T17:13:28.891 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:28.906 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:28.921 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:28.954 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-05-14T17:13:28.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:28.985 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:28.999 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:29.012 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:29.038 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:29.052 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:29.083 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:29.119 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:29.131 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T17:13:29.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T17:13:29.160 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:29.176 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:29.192 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:29.236 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:29.253 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:29.267 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:29.279 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:29.289 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:29.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:29.330 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:29.345 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:29.358 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:29.376 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-05-14T17:13:29.384 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-05-14T17:13:29.401 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:29.409 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:29.421 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:29.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:29.436 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:29.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:29.472 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:29.483 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:29.493 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:29.505 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:29.516 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:29.541 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:29.550 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T17:13:29.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T17:13:29.569 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:29.585 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:29.594 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:29.609 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-05-14T17:13:29.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:29.625 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:29.632 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:29.669 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:29.677 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:29.684 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:29.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:29.700 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:29.708 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T17:13:29.717 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T17:13:29.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:29.758 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:29.767 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:29.776 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:29.789 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:29.800 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:29.818 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:29.831 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:29.848 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:29.878 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:29.892 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:29.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:29.919 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T17:13:29.933 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T17:13:29.947 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:29.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:29.968 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:30.000 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:30.016 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:30.031 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:30.047 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:30.069 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:30.083 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:30.101 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:30.118 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:30.134 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:30.173 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T17:13:30.190 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T17:13:30.205 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:30.218 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:30.231 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:30.242 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:30.252 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:30.264 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:30.310 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:30.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:30.327 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:30.334 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:30.352 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:30.361 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:30.393 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T17:13:30.403 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T17:13:30.449 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:30.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:30.465 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:30.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:30.482 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:30.492 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:30.500 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:30.507 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:30.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:30.542 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:30.549 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:30.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:30.569 INFO:teuthology.orchestra.run.smithi069.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T17:13:30.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T17:13:30.619 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:30.653 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:30.677 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:30.688 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "[]", 2024-05-14T17:13:30.699 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:30.708 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:30.719 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:30.731 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:30.742 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:30.787 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:30.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:30.821 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:30.836 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T17:13:30.850 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T17:13:30.865 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:30.876 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:30.894 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:30.910 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-05-14T17:13:30.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:30.967 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:30.981 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:30.998 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:31.010 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:31.020 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:31.034 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:31.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:31.088 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T17:13:31.101 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T17:13:31.115 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:31.128 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:31.142 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:31.156 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:31.170 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:31.186 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:31.199 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:31.265 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:31.279 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:31.292 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:31.307 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:31.316 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:31.325 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T17:13:31.345 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T17:13:31.356 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:31.425 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:31.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:31.448 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:31.459 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:31.472 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:31.494 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:31.511 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:31.545 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:31.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:31.572 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:31.584 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T17:13:31.595 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T17:13:31.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:31.617 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:31.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:31.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:31.680 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:31.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:31.699 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:31.708 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:31.718 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:31.727 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:31.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:31.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:31.787 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T17:13:31.803 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T17:13:31.815 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:31.826 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:31.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:31.847 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:31.859 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:31.871 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:31.886 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:31.919 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:31.932 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:31.947 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:31.962 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:31.990 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T17:13:32.002 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T17:13:32.041 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:32.055 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:32.076 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:32.087 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:32.101 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:32.118 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:32.132 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:32.145 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:32.159 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:32.172 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:32.224 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:32.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:32.256 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T17:13:32.270 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T17:13:32.285 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:32.299 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:32.354 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:32.384 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:32.394 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:32.444 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:32.451 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:32.458 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:32.466 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:32.474 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:32.482 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:32.490 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:32.563 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T17:13:32.572 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T17:13:32.580 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:32.587 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:32.596 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:32.603 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:32.630 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:32.639 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:32.687 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:32.695 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:32.712 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:32.718 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:32.726 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:32.733 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:32.741 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T17:13:32.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T17:13:32.802 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:32.814 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:32.825 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:32.837 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:32.872 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:32.884 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:32.897 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:32.908 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:32.920 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:32.937 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:32.953 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:32.970 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:32.985 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T17:13:32.997 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T17:13:33.010 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:33.027 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:33.042 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:33.056 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:33.069 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:33.150 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:33.165 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:33.177 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:33.190 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:33.202 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:33.215 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:33.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:33.238 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T17:13:33.251 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T17:13:33.277 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:33.285 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:33.295 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:33.307 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:33.329 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:33.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:33.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:33.361 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:33.393 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:33.406 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:33.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:33.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:33.435 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T17:13:33.445 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T17:13:33.459 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:33.472 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:33.483 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:33.512 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T17:13:33.526 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:33.536 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:33.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:33.567 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:33.582 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:33.594 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:33.609 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:33.654 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:33.677 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T17:13:33.690 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T17:13:33.705 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:33.716 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:33.728 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:33.740 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:13:33.767 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:33.782 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:33.825 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:33.837 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:33.850 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:33.862 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:33.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:33.910 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:33.922 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T17:13:33.933 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T17:13:33.944 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:33.960 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:33.975 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:33.988 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-05-14T17:13:34.003 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:34.014 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:34.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:34.062 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:34.070 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:34.112 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:34.135 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:34.145 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:34.156 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T17:13:34.168 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T17:13:34.186 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:34.197 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:34.209 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:34.222 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-05-14T17:13:34.234 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:34.246 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:34.257 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:34.268 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:34.284 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:34.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:34.336 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:34.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:34.388 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T17:13:34.406 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T17:13:34.421 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:34.433 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:34.446 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:34.461 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:34.476 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:34.492 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:34.574 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:34.591 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:34.605 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:34.621 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T17:13:34.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T17:13:34.684 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:34.724 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:34.738 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:34.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-05-14T17:13:34.771 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:34.783 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:34.791 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:34.813 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:34.846 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:34.854 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:34.865 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:34.876 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T17:13:34.886 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T17:13:34.919 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:34.928 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:34.968 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:34.980 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:35.005 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:35.018 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:35.041 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:35.049 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:35.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:35.079 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:35.093 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:35.103 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T17:13:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T17:13:35.126 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:35.154 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:35.167 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:35.182 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:35.194 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:35.218 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:35.304 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:35.319 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:35.330 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:35.339 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:35.371 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:35.383 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T17:13:35.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T17:13:35.406 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:35.415 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:35.426 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:35.437 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:35.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:35.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:35.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:35.502 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:35.510 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:35.542 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:35.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:35.564 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T17:13:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T17:13:35.612 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:35.623 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:35.634 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:35.645 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:35.663 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:35.672 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:35.680 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:35.689 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:35.700 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:35.713 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:35.735 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:35.771 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T17:13:35.814 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T17:13:35.883 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:35.901 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:35.913 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:35.927 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:13:35.938 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:35.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:35.969 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:35.987 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:36.000 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:36.014 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:36.027 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:36.066 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:36.078 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T17:13:36.092 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T17:13:36.128 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:36.154 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:36.191 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:13:36.207 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:36.223 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:36.242 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:36.271 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:36.287 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:36.314 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:36.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:36.336 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-05-14T17:13:36.357 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:36.384 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:36.397 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:36.427 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:36.440 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:36.498 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:36.511 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:36.523 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:36.555 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:36.569 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:36.580 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:36.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:36.604 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-05-14T17:13:36.615 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-05-14T17:13:36.627 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:36.672 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:36.687 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:36.697 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:36.709 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:36.744 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:36.754 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:36.806 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:36.841 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:36.855 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-05-14T17:13:36.902 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-05-14T17:13:36.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:36.931 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:36.952 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:36.965 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:36.975 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:36.987 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:37.000 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-05-14T17:13:37.016 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:37.028 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:37.039 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:37.126 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:37.135 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-05-14T17:13:37.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-05-14T17:13:37.178 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:37.189 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:37.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:37.209 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-05-14T17:13:37.222 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:37.240 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:37.255 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:37.269 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:37.285 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:37.299 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:37.308 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:37.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:37.336 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-05-14T17:13:37.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-05-14T17:13:37.411 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:37.645 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:37.658 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:37.697 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:37.707 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:37.718 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:37.729 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:37.741 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:37.752 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:37.765 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:37.776 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:37.786 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:37.795 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:37.808 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:37.816 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:37.824 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:37.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:37.862 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:37.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:37.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:37.939 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:37.949 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:37.963 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:37.979 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:37.991 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:38.004 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:38.018 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:38.029 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:38.040 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:38.054 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:38.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:38.086 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:38.098 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:38.114 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:38.146 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:38.185 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:38.231 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:38.247 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:38.263 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:38.278 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:38.291 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:38.303 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:38.320 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:38.339 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:38.355 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:38.369 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:38.387 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:38.400 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:38.413 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:38.440 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:38.460 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:38.473 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:38.519 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:38.543 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:38.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:38.565 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:38.577 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:38.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:38.602 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:38.614 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:38.626 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:38.638 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:38.650 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:38.662 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:38.700 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:38.714 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:38.726 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-05-14T17:13:38.802 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-05-14T17:13:38.834 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:38.869 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:38.878 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:38.887 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:38.897 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:38.905 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:38.915 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:38.922 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-05-14T17:13:38.929 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:38.940 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:38.987 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:38.998 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-05-14T17:13:39.008 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T17:13:39.041 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:39.053 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:39.079 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:39.094 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:39.124 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:39.141 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:39.153 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:39.162 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:39.174 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:39.189 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:39.199 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:39.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:39.238 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-05-14T17:13:39.249 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-05-14T17:13:39.263 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:39.275 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:39.305 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:39.318 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-05-14T17:13:39.328 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:39.346 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:39.360 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:39.410 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:39.423 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:39.433 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:39.445 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:39.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:39.472 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-05-14T17:13:39.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-05-14T17:13:39.553 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:39.567 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:39.582 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:39.616 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-05-14T17:13:39.627 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:39.641 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:39.655 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:39.666 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:39.680 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:39.692 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:39.742 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:39.754 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:39.809 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:39.820 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:39.832 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:39.841 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:39.854 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:39.863 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:39.894 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:39.903 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:39.913 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:39.922 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:39.930 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:39.939 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:39.963 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:39.974 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-05-14T17:13:40.008 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-05-14T17:13:40.054 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:40.065 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:40.075 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:40.083 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:40.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:40.128 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:40.137 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:40.147 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:40.157 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:40.166 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:40.174 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:40.181 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:40.188 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-05-14T17:13:40.197 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-05-14T17:13:40.230 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:40.240 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:40.276 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:40.298 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-05-14T17:13:40.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:40.328 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:40.344 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:40.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:40.424 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:40.442 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:40.458 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:40.476 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:40.493 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-05-14T17:13:40.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-05-14T17:13:40.529 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:40.543 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:40.559 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:40.578 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-05-14T17:13:40.632 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:40.653 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:40.667 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:40.680 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:40.694 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-05-14T17:13:40.758 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:40.774 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:40.787 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-05-14T17:13:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-05-14T17:13:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-05-14T17:13:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-05-14T17:13:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-05-14T17:13:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-05-14T17:13:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-05-14T17:13:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-05-14T17:13:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-05-14T17:13:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-05-14T17:13:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-05-14T17:13:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:42.169 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:42.175 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:42.188 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:42.195 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:42.228 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:42.235 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:42.242 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:42.266 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:42.305 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:42.316 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:42.324 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:42.334 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:42.343 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:42.352 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:42.361 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:42.371 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:42.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:42.390 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:42.400 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:42.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:42.427 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:42.462 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:42.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:42.505 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:42.517 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:42.556 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:42.566 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:42.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:42.591 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:42.602 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:42.613 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:42.624 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-05-14T17:13:42.637 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-05-14T17:13:42.649 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:42.660 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:42.671 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:42.703 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-05-14T17:13:42.714 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:42.754 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:42.770 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:42.780 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T17:13:42.794 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:42.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:42.810 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:42.819 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:42.828 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-05-14T17:13:42.836 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-05-14T17:13:42.844 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:42.851 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:42.857 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:42.863 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-05-14T17:13:42.869 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:42.878 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:42.900 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:42.957 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T17:13:42.980 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:42.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:43.005 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:43.014 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:43.023 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-05-14T17:13:43.032 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-05-14T17:13:43.042 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:43.051 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:43.068 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-05-14T17:13:43.079 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:43.087 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:43.096 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:43.104 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-05-14T17:13:43.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:43.125 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:43.159 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:43.204 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:43.217 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-05-14T17:13:43.230 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-05-14T17:13:43.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:43.256 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:43.263 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:43.273 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-05-14T17:13:43.282 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:43.291 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:43.300 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:43.309 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T17:13:43.318 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:43.327 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:43.336 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:43.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:43.361 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-05-14T17:13:43.390 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-05-14T17:13:43.421 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:43.431 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:43.440 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:43.448 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:13:43.457 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:43.465 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:43.473 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:43.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T17:13:43.498 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:43.510 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:43.521 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:43.532 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:43.543 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:13:43.554 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:13:43.565 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:43.578 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:43.587 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:43.660 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:13:43.671 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:43.680 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:43.686 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:43.694 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T17:13:43.701 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:43.708 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:43.715 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:43.722 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:43.730 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:43.737 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:43.743 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:43.753 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:43.761 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:43.769 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:43.778 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:43.815 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:43.846 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:43.853 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:43.859 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:43.867 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:43.874 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:43.881 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:43.889 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-05-14T17:13:43.899 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-05-14T17:13:43.909 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:43.918 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:43.928 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:43.937 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-05-14T17:13:43.947 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:43.955 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:43.964 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:43.979 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T17:13:44.023 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:44.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:44.068 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:44.081 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:44.093 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:44.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:44.118 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:44.130 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-05-14T17:13:44.142 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:44.153 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:44.166 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:44.178 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:44.191 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:44.235 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:44.246 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:44.259 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:44.271 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:44.283 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:44.353 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:44.363 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:44.372 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:44.382 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:44.391 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:44.400 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:44.526 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:44.534 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:44.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:44.555 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:44.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:44.577 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:44.596 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:44.606 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:44.655 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:44.664 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:44.673 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:44.682 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:44.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:44.700 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:44.725 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:44.733 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:44.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:44.811 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:44.819 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:44.828 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:44.837 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:44.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:44.879 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:44.888 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:44.927 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:44.943 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:44.952 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:44.994 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:45.013 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:45.036 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:45.046 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:45.058 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:45.067 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:45.076 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:45.085 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:45.093 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:45.103 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:45.112 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:45.129 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:45.140 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:45.174 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:45.182 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:45.191 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:45.199 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:45.246 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:45.255 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:45.281 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:45.291 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:45.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:45.311 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:45.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:45.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:45.338 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-05-14T17:13:45.346 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-05-14T17:13:45.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:45.369 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:45.381 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:45.411 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-05-14T17:13:45.423 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:45.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:45.446 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:45.492 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:45.502 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:45.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:45.530 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:45.538 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:45.547 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-05-14T17:13:45.556 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-05-14T17:13:45.566 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:45.574 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:45.581 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:45.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-05-14T17:13:45.625 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:45.633 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:45.656 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:45.683 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:45.691 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:45.714 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:45.723 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:45.731 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:45.738 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:13:45.746 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:13:45.753 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:45.759 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:45.766 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:45.773 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:13:45.779 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:45.784 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:45.792 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:45.811 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:45.820 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:45.856 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:45.900 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:45.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:45.943 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:45.954 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:45.962 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:45.970 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:45.981 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:45.993 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:46.006 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:46.021 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:46.033 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:46.046 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:46.059 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:46.071 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:46.095 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:46.106 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:46.149 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:46.258 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:46.270 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:46.283 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-05-14T17:13:46.294 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-05-14T17:13:46.305 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-05-14T17:13:46.316 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:46.327 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-05-14T17:13:46.338 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-05-14T17:13:46.348 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:46.357 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:46.365 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:46.376 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-05-14T17:13:46.387 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:46.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:46.437 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:46.486 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T17:13:46.559 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:46.566 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:46.573 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:46.581 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:46.590 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-05-14T17:13:46.599 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-05-14T17:13:46.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:46.617 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:46.625 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:46.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-05-14T17:13:46.644 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:46.653 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:46.665 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:46.674 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T17:13:46.682 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:46.695 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:46.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:46.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:46.783 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-05-14T17:13:46.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-05-14T17:13:46.800 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:46.809 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:46.817 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:46.826 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:46.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:46.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:46.850 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:46.858 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-05-14T17:13:46.864 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:46.871 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:46.877 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:46.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:46.895 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:13:46.940 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:13:46.966 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:13:46.975 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:47.012 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:47.022 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-05-14T17:13:47.032 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-05-14T17:13:47.041 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:47.051 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:47.060 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T17:13:47.069 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:47.080 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:47.089 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:47.099 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:47.111 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:47.120 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:47.132 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:47.185 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:47.200 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:47.251 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:47.260 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:47.269 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:47.278 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:47.287 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:47.299 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:47.310 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:47.321 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:47.332 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:47.342 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:47.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:47.367 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:47.378 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:47.388 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:47.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:47.424 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:47.468 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:47.491 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:47.501 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:47.510 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:47.522 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:47.533 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:47.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:47.556 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:47.566 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:47.584 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:47.594 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:47.602 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:47.611 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:47.645 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:47.663 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:47.672 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:47.691 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:47.707 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:47.716 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:47.726 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:47.736 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:47.745 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:47.755 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:47.765 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:47.774 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:47.783 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:47.793 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:47.802 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:47.812 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:47.822 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:47.856 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:47.872 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:47.882 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:47.893 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:47.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:47.941 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:47.952 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:47.963 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:47.974 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:47.985 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:47.995 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:48.003 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:48.012 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:48.023 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:48.034 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:48.121 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:48.131 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:48.140 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-05-14T17:13:48.155 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-05-14T17:13:48.165 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:48.194 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:48.259 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:48.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:48.277 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:48.285 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:48.293 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:48.300 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-05-14T17:13:48.308 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:48.316 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:48.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:48.330 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:48.336 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-05-14T17:13:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-05-14T17:13:48.372 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:48.381 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:48.403 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-05-14T17:13:48.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:48.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:48.459 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:48.466 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-05-14T17:13:48.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:48.480 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:48.486 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:48.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:48.504 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:48.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:48.524 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:48.557 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:48.568 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:48.578 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:48.588 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:48.600 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:48.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:48.634 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:48.658 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:48.671 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:48.683 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:48.695 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:48.708 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-05-14T17:13:48.720 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-05-14T17:13:48.733 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:48.808 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:48.818 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:48.830 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-05-14T17:13:48.841 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:48.852 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:48.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:48.876 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:13:48.886 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:48.896 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:48.909 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:48.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:48.965 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-05-14T17:13:48.977 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-05-14T17:13:48.988 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:48.999 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:49.010 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:49.019 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:13:49.046 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:13:49.056 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-05-14T17:13:49.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:49.069 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T17:13:49.076 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:49.083 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:49.089 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:49.099 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:49.106 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-05-14T17:13:49.117 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-05-14T17:13:49.179 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:49.189 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:49.197 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:49.206 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:49.214 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:49.223 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:49.231 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:49.278 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-05-14T17:13:49.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:49.295 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:49.304 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:49.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:49.322 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-05-14T17:13:49.331 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-05-14T17:13:49.349 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:49.358 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:49.384 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:49.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-05-14T17:13:49.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:49.432 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:49.445 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:49.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:13:49.484 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:49.499 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:49.510 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:49.522 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:49.535 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:49.546 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:49.555 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:49.565 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-05-14T17:13:49.573 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:49.602 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:49.631 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:49.651 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:49.662 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:49.672 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:49.680 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:49.716 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:49.725 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:49.734 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:49.743 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:49.754 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:49.764 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:49.775 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:49.784 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:49.794 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:49.805 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:49.815 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:49.838 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:49.967 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:49.973 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:49.979 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:49.985 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:49.993 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:50.000 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:50.009 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:50.019 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:50.032 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:50.041 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:50.053 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:50.062 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:50.071 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:50.129 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:50.143 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:50.176 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:50.186 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:50.195 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:50.205 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:50.214 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:50.223 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:50.233 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:50.242 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:50.249 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:50.260 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:50.268 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:50.281 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:50.288 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:50.297 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:50.315 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:50.355 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:50.378 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:50.445 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:50.457 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:50.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:50.481 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:50.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:50.506 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:50.524 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:50.537 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:50.547 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:50.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:50.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:50.579 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:50.587 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:50.616 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:50.639 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:50.669 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:50.697 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:50.706 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:50.715 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:50.723 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:50.732 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:50.743 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:50.753 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:50.764 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:50.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:50.784 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:50.793 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:50.803 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:50.810 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:50.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:50.853 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:50.883 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:50.937 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:50.945 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-05-14T17:13:50.952 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:50.962 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:50.969 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:50.977 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:50.983 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:50.989 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:50.997 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:51.004 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:51.011 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:51.018 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:51.024 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:51.032 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:51.056 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:51.062 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:51.096 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:51.146 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:51.155 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:51.165 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:51.173 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:51.179 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:51.188 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:51.197 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:51.206 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:51.215 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:51.225 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:51.234 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:51.246 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:51.258 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:51.298 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:51.310 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:51.345 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:51.386 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:51.395 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:51.405 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:51.415 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:51.425 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:51.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:51.443 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:51.455 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:51.466 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:51.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:51.488 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:51.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:51.510 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:51.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:51.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:51.581 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:51.616 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:51.629 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:51.641 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:51.652 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:51.674 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:51.683 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:51.693 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:51.700 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:51.706 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:51.713 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:51.720 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:51.728 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:51.734 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:51.745 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:51.772 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:51.781 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:51.818 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:51.827 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:51.839 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:51.850 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:51.860 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:51.869 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:51.879 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:51.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:51.899 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:51.908 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:51.917 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:51.927 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:51.935 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:51.963 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:51.974 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:51.988 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:52.028 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:52.045 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:52.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:52.069 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:52.081 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:52.093 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:52.104 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:52.116 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:52.129 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:52.140 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-05-14T17:13:52.152 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:52.163 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:52.176 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:52.191 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-05-14T17:13:52.201 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-05-14T17:13:52.213 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:52.246 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:52.255 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:52.270 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-05-14T17:13:52.281 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:52.291 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:52.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:52.308 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T17:13:52.319 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:52.327 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:52.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:52.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:52.359 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-05-14T17:13:52.372 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-05-14T17:13:52.409 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:52.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:52.455 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:13:52.466 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:13:52.475 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-05-14T17:13:52.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:52.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:52.521 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T17:13:52.531 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:52.540 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:52.548 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:52.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:52.561 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:52.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:52.574 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:52.584 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:52.593 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:52.625 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:52.636 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:52.676 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:52.682 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:52.690 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:52.696 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:52.737 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:52.744 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:52.750 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:52.757 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:52.772 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:52.779 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:52.786 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:52.793 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:52.799 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:52.805 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:52.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:52.849 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:52.933 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:52.945 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:52.956 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:52.966 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:52.989 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:52.999 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:53.008 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:53.016 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:53.025 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:53.034 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:53.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:53.058 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:53.070 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:53.081 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:53.093 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:53.136 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:53.146 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:53.186 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:53.197 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:53.232 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:53.244 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:53.260 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:53.271 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:53.280 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:53.289 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:53.296 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:53.307 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:53.316 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:53.324 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:53.333 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:53.341 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:53.376 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:53.383 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:53.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:53.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:53.432 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:53.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:53.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:53.470 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:53.479 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:53.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:53.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:53.507 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:53.516 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:53.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:53.534 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:53.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:53.551 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:53.576 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:53.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:53.658 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:53.666 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:53.674 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:53.684 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:53.705 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:53.714 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:53.722 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:53.731 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:53.740 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:53.750 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:53.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:53.775 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:53.788 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-05-14T17:13:53.799 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:53.811 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:53.861 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:53.871 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-05-14T17:13:53.884 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-05-14T17:13:53.897 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:53.909 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:53.922 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:53.949 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-05-14T17:13:53.960 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:53.974 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:53.986 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:53.995 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-05-14T17:13:54.005 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T17:13:54.011 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:54.016 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:54.024 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:54.031 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:54.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:54.112 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:54.119 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:54.125 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:54.132 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:54.185 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:54.191 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:54.226 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:54.235 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:54.245 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:54.254 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:54.263 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:54.273 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:54.282 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:54.291 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:54.327 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:54.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:54.366 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:54.375 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:54.382 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:54.388 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:54.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:54.428 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:54.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:54.454 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:54.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:54.473 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:54.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:54.490 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:54.501 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:54.510 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:54.533 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:54.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:54.580 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:54.591 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:54.602 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:54.610 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:54.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:54.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:54.638 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:54.645 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:54.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:54.660 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:54.667 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:54.674 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:54.680 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:54.688 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:54.695 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:54.701 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:54.726 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:54.733 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:54.738 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:54.745 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:54.753 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:54.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:54.768 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:54.783 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:54.791 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:54.815 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:54.824 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:54.836 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:54.845 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:54.855 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:54.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:54.875 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:54.907 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:54.940 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:54.947 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:54.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:54.960 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-05-14T17:13:54.967 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-05-14T17:13:54.973 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:54.980 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:55.025 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:55.033 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:55.046 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:55.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:55.061 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:55.069 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-05-14T17:13:55.079 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T17:13:55.133 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:55.192 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:55.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:55.214 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-05-14T17:13:55.226 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-05-14T17:13:55.236 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:55.245 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:55.256 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:55.264 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-05-14T17:13:55.273 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:55.313 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:55.321 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:55.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-05-14T17:13:55.449 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:55.459 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:55.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:55.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:55.489 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-05-14T17:13:55.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-05-14T17:13:55.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:55.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:55.528 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:55.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-05-14T17:13:55.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:55.560 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:55.570 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:55.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T17:13:55.645 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:55.656 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:55.679 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:55.689 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:55.700 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-05-14T17:13:55.709 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-05-14T17:13:55.718 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:55.729 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:55.741 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:55.749 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:55.758 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:55.767 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:55.775 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:55.785 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-05-14T17:13:55.793 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:55.803 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:55.846 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:55.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:55.912 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:55.946 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:55.954 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:55.965 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:55.975 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:55.988 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:55.999 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:56.008 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:56.019 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:56.032 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:56.045 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:56.055 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:56.066 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:56.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:56.094 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-05-14T17:13:56.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-05-14T17:13:56.197 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:56.208 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:56.238 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:56.249 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-05-14T17:13:56.258 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:56.265 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:56.272 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:56.280 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T17:13:56.288 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T17:13:56.294 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:56.304 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:56.313 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:56.320 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:56.337 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:56.345 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:56.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-05-14T17:13:56.374 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:56.414 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:56.421 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:56.446 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:56.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:56.460 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:56.467 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:56.475 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:56.482 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:56.492 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:56.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:56.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:56.513 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:56.520 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:56.529 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:56.536 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:56.571 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:56.579 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:56.616 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:56.623 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:56.714 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:56.724 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:56.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:56.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:56.753 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:56.763 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:56.772 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:56.823 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:56.834 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:56.843 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:56.853 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:56.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:56.903 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:56.911 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:56.923 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:56.939 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:56.947 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:56.997 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:57.006 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:57.016 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:57.022 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:57.033 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:57.040 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:57.082 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:57.090 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:57.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:57.131 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:57.140 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:57.150 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:57.158 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:57.170 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:57.180 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:57.189 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:57.198 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:57.228 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:57.238 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:57.247 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:57.256 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:57.264 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-05-14T17:13:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:58.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:58.591 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:58.603 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:58.613 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:58.624 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:58.636 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:58.648 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:58.659 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:58.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:58.681 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:58.727 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:58.760 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:58.771 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:58.781 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:58.790 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:58.805 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:58.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:58.825 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:58.837 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:13:58.848 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:13:58.857 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:13:58.864 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:13:58.873 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:58.884 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:13:58.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:58.900 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:58.909 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:58.945 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:58.970 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:58.981 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:58.992 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:59.001 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:59.011 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:13:59.021 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:59.030 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:13:59.039 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-05-14T17:13:59.048 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:13:59.061 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:13:59.070 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:13:59.080 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:13:59.090 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:13:59.099 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:59.109 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:59.118 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:59.163 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:13:59.186 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:59.196 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:59.204 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:59.214 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:59.224 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:59.234 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:59.246 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:59.258 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:59.273 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:59.285 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:59.298 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:59.312 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:59.325 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:59.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:59.359 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:59.397 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:13:59.444 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:13:59.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:59.485 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:59.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:59.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:13:59.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:59.535 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:59.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:13:59.559 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:59.570 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:59.582 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:59.593 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:59.605 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:59.616 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:13:59.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:13:59.659 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:13:59.682 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:13:59.711 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:13:59.737 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:13:59.750 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:13:59.761 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:13:59.774 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:13:59.784 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:13:59.793 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:13:59.801 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:13:59.809 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:13:59.817 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:13:59.825 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:13:59.833 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:13:59.841 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:13:59.852 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:13:59.863 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:13:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:13:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:13:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:13:59.988 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:13:59.993 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:13:59.999 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:00.004 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:00.010 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:00.016 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:00.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:00.034 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:00.041 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:00.047 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:00.054 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:00.061 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:00.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:00.106 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:00.117 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:00.125 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:00.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:00.163 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:00.190 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:00.198 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:00.210 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:00.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:00.322 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:00.334 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-05-14T17:14:00.344 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:00.370 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:00.388 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:00.400 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-05-14T17:14:00.409 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-05-14T17:14:00.418 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:00.427 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:00.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:00.446 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:00.455 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:00.464 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:00.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:00.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T17:14:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:00.523 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:00.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:00.560 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:00.566 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:00.611 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:00.619 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:00.625 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:00.634 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:00.641 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:00.649 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:00.656 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:00.663 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:00.670 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:00.677 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:00.686 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:00.697 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:00.731 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:00.740 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:00.750 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:00.760 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:00.773 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:00.810 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:00.818 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:00.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:00.837 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:00.845 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:00.854 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:00.863 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:00.871 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:00.880 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:00.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:00.899 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:00.941 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:00.950 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:00.959 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:00.972 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:00.985 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:00.998 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:01.026 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:01.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:01.064 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:01.074 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:01.098 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:01.110 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:01.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:01.135 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:01.145 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:01.178 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:01.191 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:01.203 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:01.212 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:01.222 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:01.232 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:01.241 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:01.268 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:01.291 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:01.300 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:01.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:01.318 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:01.327 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:01.335 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:01.345 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:01.355 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:01.363 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:01.394 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:01.404 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:01.413 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:01.421 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:01.428 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:01.439 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:01.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:01.498 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-05-14T17:14:01.507 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-05-14T17:14:01.518 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:01.529 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:01.537 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:01.545 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:01.555 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:01.563 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:01.593 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:01.606 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-05-14T17:14:01.615 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-05-14T17:14:01.624 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-05-14T17:14:01.634 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:01.644 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-05-14T17:14:01.665 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:01.674 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:01.727 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:01.736 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:01.746 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:01.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:01.766 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:01.775 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:01.785 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:01.828 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:01.837 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:01.846 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:01.857 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:01.868 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:01.877 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:01.888 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:01.899 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:01.932 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:01.942 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:01.960 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:01.970 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:01.980 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-05-14T17:14:01.989 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:02.029 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:02.038 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:02.047 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:02.056 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:02.067 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:02.078 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:02.085 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:02.092 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:02.099 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:02.105 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:02.112 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:02.124 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:02.130 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:02.169 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:02.176 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:02.183 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:02.225 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:02.233 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:02.241 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:02.249 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:02.257 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:02.264 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:02.271 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:02.278 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:02.285 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:02.291 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:02.301 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:02.313 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:02.335 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:02.370 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:02.419 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:02.429 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:02.439 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:02.446 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:02.453 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:02.462 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:02.470 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:02.481 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:02.493 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:02.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:02.517 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:02.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:02.542 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:02.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:02.599 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:02.642 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:02.686 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:02.698 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:02.710 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:02.721 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:02.732 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:02.744 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:02.755 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:02.766 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:02.779 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:02.790 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:02.802 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:02.839 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:02.851 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:02.863 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:02.875 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:02.927 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:02.941 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:02.952 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:02.962 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:02.974 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:02.983 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:02.993 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:03.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:03.012 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:03.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:03.038 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:03.047 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:03.088 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:03.098 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:03.106 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:03.115 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:03.150 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:03.182 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:03.204 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:03.212 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:03.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:03.228 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:03.234 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-05-14T17:14:03.241 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:03.248 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:03.254 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:03.261 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:03.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:03.302 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:03.313 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:03.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:03.344 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:03.379 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:03.391 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:03.400 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:03.411 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:03.420 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:03.439 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:03.448 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:03.456 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:03.464 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:03.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:03.483 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:03.505 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:03.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:03.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:03.535 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:03.557 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:03.590 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:03.644 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:03.656 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:03.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:03.675 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:03.700 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:03.709 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:03.720 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:03.750 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:03.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:03.767 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:03.776 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:03.788 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:03.797 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:03.833 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:03.852 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:03.860 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:03.870 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:03.878 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:03.887 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:03.897 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:03.904 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:03.912 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:03.920 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:03.954 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:03.965 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:03.976 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:03.987 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:03.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:04.024 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:04.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:04.051 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:04.058 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:04.065 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:04.072 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:04.079 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:04.085 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:04.092 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:04.098 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:04.105 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:04.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:04.141 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:04.147 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:04.165 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:04.170 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:04.210 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:04.219 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:04.267 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:04.276 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:04.285 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:04.295 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:04.308 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:04.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:04.331 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:04.344 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:04.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:04.446 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:04.480 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:04.492 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:04.504 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:04.517 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:04.530 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-05-14T17:14:04.542 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:04.555 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:04.588 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:04.600 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:04.612 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:04.622 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:04.632 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:04.650 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:04.661 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:04.671 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:04.681 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:04.692 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:04.739 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:04.753 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:04.764 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:04.777 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:04.788 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:04.799 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:04.810 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:04.857 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:04.868 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:04.876 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:04.885 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:04.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:04.935 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:04.964 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:04.989 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:04.999 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:05.008 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:05.016 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:05.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:05.034 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:05.042 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:05.050 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:05.110 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:05.151 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:05.157 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:05.181 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:05.187 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:05.195 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:05.202 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:05.212 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:05.221 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:05.232 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:05.240 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:05.246 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:05.254 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:05.282 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:05.291 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:05.316 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:05.355 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:05.391 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:05.401 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:05.412 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:05.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:05.430 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:05.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:05.449 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:05.459 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:05.469 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:05.478 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:05.487 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:05.495 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:05.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:05.526 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:05.534 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:05.550 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:05.556 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:05.599 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:05.605 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:14:05.612 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:14:05.618 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:05.627 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:05.635 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:05.642 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:14:05.650 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:05.657 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:05.663 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:05.669 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:05.683 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:05.692 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:05.699 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:05.721 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:05.752 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:05.841 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:05.849 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:05.857 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:05.863 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:05.869 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:05.876 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:05.887 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:05.903 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:05.913 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:05.921 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:05.934 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:05.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:05.955 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-05-14T17:14:05.966 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-05-14T17:14:05.978 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:14:06.003 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:06.028 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:06.062 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-05-14T17:14:06.074 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-05-14T17:14:06.088 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:06.100 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:06.109 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-05-14T17:14:06.120 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T17:14:06.131 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:06.139 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:06.147 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:06.156 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:06.166 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:06.174 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:06.185 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-05-14T17:14:06.199 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:06.212 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:06.253 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:06.288 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-05-14T17:14:06.300 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-05-14T17:14:06.312 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:06.322 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:06.338 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:06.349 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:06.358 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:06.365 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:06.374 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:06.383 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T17:14:06.393 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:06.402 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:06.412 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:06.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:06.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-05-14T17:14:06.476 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-05-14T17:14:06.525 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:06.534 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:06.542 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:06.551 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:06.559 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:06.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:06.577 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:06.586 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:06.594 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:06.602 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:06.611 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:06.620 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:06.628 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:06.636 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:06.645 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:06.660 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:06.722 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:06.731 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:06.741 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:06.747 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:06.754 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:06.761 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:06.770 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:06.782 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:06.792 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:06.801 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:06.811 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:06.821 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:06.830 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:06.840 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:06.850 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:06.860 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:06.886 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:06.931 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:06.943 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:06.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:06.967 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:06.978 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:06.989 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:07.001 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:07.010 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:07.019 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:07.027 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:07.036 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:07.047 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:07.055 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:07.064 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:07.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:07.105 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:07.139 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:07.187 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:07.196 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:07.204 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:07.213 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:07.222 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:07.232 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:07.241 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:07.252 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:07.260 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:07.267 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:07.276 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:07.283 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:07.292 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:07.298 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:07.306 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:07.319 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:07.364 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:07.407 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:07.414 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:07.421 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:07.428 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:07.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:07.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:07.451 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:07.460 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:07.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:07.472 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:07.478 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:07.486 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:07.495 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:07.503 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:07.512 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-05-14T17:14:07.596 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-05-14T17:14:07.605 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:07.632 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:07.642 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:07.651 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-05-14T17:14:07.661 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:07.667 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:07.674 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:07.681 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-05-14T17:14:07.690 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:07.699 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:07.709 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:07.718 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:07.729 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:14:07.742 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:14:07.752 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:07.763 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:07.803 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:07.822 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:14:07.858 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:07.869 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:07.881 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:07.892 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-05-14T17:14:07.904 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:07.919 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:07.931 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:07.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:07.957 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:07.970 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:08.019 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:08.033 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:08.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:08.056 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:08.067 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:08.078 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:08.090 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:08.125 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:08.137 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:08.148 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:08.158 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:08.172 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:08.182 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:08.191 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:08.200 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:08.211 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-05-14T17:14:08.220 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:08.260 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:08.272 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:08.281 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-05-14T17:14:08.290 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-05-14T17:14:08.298 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:08.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:08.332 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:08.366 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:08.376 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:08.385 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:08.397 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:08.412 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:08.446 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:08.453 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:08.460 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:08.470 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:08.479 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-05-14T17:14:08.487 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-05-14T17:14:08.496 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:08.506 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:08.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:08.525 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:08.542 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:08.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:08.576 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:08.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T17:14:08.635 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T17:14:08.671 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:08.681 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:08.690 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:08.699 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:08.709 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:08.718 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:08.730 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:08.739 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:08.749 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:08.759 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:08.768 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:08.776 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:08.816 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:08.825 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:08.851 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:08.887 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:08.915 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:08.924 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:08.933 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:08.942 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:08.951 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:08.960 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:08.969 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:08.978 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:08.986 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:08.995 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:09.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:09.011 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:09.019 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:09.034 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:09.044 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:09.075 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:09.085 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:09.118 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:09.125 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:09.131 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:09.138 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:09.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:09.149 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:09.157 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:09.164 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:09.171 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:09.177 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:09.183 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:09.190 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:09.198 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:09.209 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:09.215 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:09.263 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:09.269 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:09.287 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:09.301 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:09.318 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:09.327 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:09.336 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:09.347 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:09.356 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:09.365 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:09.374 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:09.386 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:09.392 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:09.399 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:09.410 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:09.418 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:09.452 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:09.475 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:09.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:09.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:09.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:09.528 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:09.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:09.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:09.563 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-05-14T17:14:09.576 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-05-14T17:14:09.590 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:09.604 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:09.616 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:09.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:09.649 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:09.661 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:09.673 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:09.709 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:09.751 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:09.761 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:09.777 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:09.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:09.806 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T17:14:09.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T17:14:09.837 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:09.845 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:09.854 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:09.862 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-05-14T17:14:09.870 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:09.878 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:09.890 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:09.924 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:09.952 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:09.963 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:09.993 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:10.001 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:10.010 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-05-14T17:14:10.018 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-05-14T17:14:10.027 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:14:10.035 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:10.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:10.053 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-05-14T17:14:10.063 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:10.072 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:10.081 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:10.091 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:10.126 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:10.136 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:10.160 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:10.177 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:10.189 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-05-14T17:14:10.209 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-05-14T17:14:10.224 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:10.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:10.250 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:10.259 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-05-14T17:14:10.274 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:10.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:10.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:10.312 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T17:14:10.322 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:10.333 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:10.379 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:10.387 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:10.438 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-05-14T17:14:10.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-05-14T17:14:10.469 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:10.482 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:10.518 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:10.533 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-05-14T17:14:10.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:10.562 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:10.573 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:10.585 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T17:14:10.595 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:10.604 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:10.614 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:10.621 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:10.649 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:10.690 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:10.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:10.706 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:10.711 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:10.736 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:10.745 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:10.776 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:10.783 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:10.791 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:10.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:10.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:10.813 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:10.820 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:10.826 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-05-14T17:14:10.832 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-05-14T17:14:10.863 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:10.897 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:10.904 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:10.910 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-05-14T17:14:10.916 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:10.923 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:10.964 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:10.969 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:11.022 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:11.034 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:11.048 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:11.056 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:11.066 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-05-14T17:14:11.111 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-05-14T17:14:11.123 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:11.165 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:11.177 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:11.189 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-05-14T17:14:11.201 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:11.213 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:11.228 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:11.240 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-05-14T17:14:11.248 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-05-14T17:14:11.287 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:11.297 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:11.348 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:11.363 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:11.373 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:11.402 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:11.414 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-05-14T17:14:11.428 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-05-14T17:14:11.441 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:11.455 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:11.467 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:11.480 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-05-14T17:14:11.489 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-05-14T17:14:11.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-05-14T17:14:11.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:11.521 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:11.532 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:11.561 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:11.569 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:11.629 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:11.639 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:11.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:11.655 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:11.662 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-05-14T17:14:11.671 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:11.680 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:11.687 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:11.696 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:11.706 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:11.716 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:11.726 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:11.736 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:11.747 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:11.758 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:11.787 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:11.807 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:11.848 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:11.878 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:11.891 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:11.904 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:11.917 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:11.932 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:11.944 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:11.956 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:11.968 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:11.980 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:11.992 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:12.006 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:12.021 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:12.033 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:12.045 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:12.062 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:12.104 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:12.144 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:12.152 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:12.167 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:12.179 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:12.190 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:12.202 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:12.219 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:12.237 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:12.248 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:12.262 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:12.274 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:12.285 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:12.292 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:12.299 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:12.305 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:12.338 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:12.345 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:12.415 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:12.423 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:12.430 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:12.438 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:12.444 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:12.454 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:12.466 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:12.478 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:12.487 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:12.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:12.505 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:12.516 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:12.526 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:12.536 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:12.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:12.587 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:12.616 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:12.624 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:12.632 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:12.641 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:12.647 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:12.654 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:12.661 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:12.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:12.676 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:12.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:12.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-05-14T17:14:12.704 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-05-14T17:14:12.717 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:12.725 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:12.757 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:12.780 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:14:12.790 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:12.825 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:12.836 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:12.845 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:12.857 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:12.867 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:12.877 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:12.886 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:12.899 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-05-14T17:14:12.908 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T17:14:12.921 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:12.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:12.940 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:12.951 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:12.959 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:12.992 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:13.009 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:13.017 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:13.067 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:13.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:13.096 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:13.108 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:13.123 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:13.137 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:13.148 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:13.159 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:13.168 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:13.179 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:13.190 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:13.199 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:13.206 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:13.234 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:13.241 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:13.263 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:13.270 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:13.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:13.315 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-05-14T17:14:13.323 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-05-14T17:14:13.329 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:13.336 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:13.345 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:13.351 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:13.358 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:13.366 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:13.377 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:13.388 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:13.398 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:13.408 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:13.445 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:13.457 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:13.472 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:13.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:13.528 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:13.540 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-05-14T17:14:13.551 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:13.564 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:13.576 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:13.591 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-05-14T17:14:13.610 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-05-14T17:14:13.625 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:13.640 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:13.652 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:13.666 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:13.679 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:13.710 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:13.722 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:13.734 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:13.750 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:13.785 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:13.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:13.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:13.841 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-05-14T17:14:13.856 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-05-14T17:14:13.867 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:13.878 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:13.889 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:13.905 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:13.920 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:13.933 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:13.963 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:13.977 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:13.985 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:13.993 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:14.002 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:14.035 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:14.044 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:14.085 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:14.096 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:14.106 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:14.127 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:14.136 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:14.145 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:14.154 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:14.162 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:14.189 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:14.199 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:14.206 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:14.212 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:14.221 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:14.227 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:14.245 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:14.265 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:14.271 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:14.291 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:14.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:14.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:14.310 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:14.317 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:14.327 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:14.340 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:14.377 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:14.392 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:14.410 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:14.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:14.431 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:14.440 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:14.462 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:14.472 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:14.484 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:14.491 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:14.517 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:14.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:14.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:14.542 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:14.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:14.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:14.566 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:14.601 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:14.608 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:14.617 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:14.624 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:14.631 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:14.637 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:14.681 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:14.687 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:14.709 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:14.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:14.735 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:14.741 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:14.747 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:14.754 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:14.761 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:14.767 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:14.775 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:14.804 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:14.812 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:14.820 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:14.826 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:14.836 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:14.845 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:14.877 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:14.892 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:14.909 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:14.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:14.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:14.956 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-05-14T17:14:14.971 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-05-14T17:14:14.984 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:14.998 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:15.009 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:15.048 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:15.059 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:15.074 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:15.089 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:15.100 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:15.109 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:15.118 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:15.129 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:15.155 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:15.167 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-05-14T17:14:15.179 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-05-14T17:14:15.201 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:15.214 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:15.231 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:15.241 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:15.254 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:15.265 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:15.301 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:15.311 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:15.318 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:15.329 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:15.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:15.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:15.357 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:15.397 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:15.408 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:15.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:15.431 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:15.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:15.453 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:15.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:15.471 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:15.480 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:15.489 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:15.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:15.540 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:15.550 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:15.561 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:15.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:15.581 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:15.592 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-05-14T17:14:15.628 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:15.638 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:15.651 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:15.665 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:15.685 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:15.699 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:15.716 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:15.739 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:15.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:15.760 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:15.800 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:15.814 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:15.824 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:15.837 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:15.851 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:15.861 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:15.872 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:15.880 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:15.937 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:15.962 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:15.976 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:16.017 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:16.026 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:16.042 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:16.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:16.059 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:16.104 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:16.113 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:16.120 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:16.127 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:16.135 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:16.143 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:16.159 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:16.199 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:16.205 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:16.212 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:16.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:16.235 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:16.241 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:16.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:16.254 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:16.260 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:16.267 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:16.307 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:16.313 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:16.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:16.328 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:16.334 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:16.340 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:16.349 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:16.395 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:16.406 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:16.419 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:16.431 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:16.447 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:16.460 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:16.476 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:16.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:16.493 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:16.502 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:16.555 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:16.570 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:16.580 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:16.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:16.597 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:16.608 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:16.621 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:16.667 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:16.678 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:16.695 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:16.704 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:16.732 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:16.743 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:16.751 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:16.767 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:16.776 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:16.789 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:16.831 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:16.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:16.854 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:16.862 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:16.872 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:16.882 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:16.893 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:16.927 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:16.938 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:16.948 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:16.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:16.973 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:16.982 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-05-14T17:14:16.988 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:16.994 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:17.001 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:17.008 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:17.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:17.061 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:17.071 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:17.082 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:17.094 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:17.105 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:17.113 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:17.153 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:17.162 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:17.172 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:17.182 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:17.217 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:17.235 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:17.249 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:17.262 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:17.277 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:17.294 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:17.355 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:17.369 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:17.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:17.397 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:17.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:17.423 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:17.432 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:17.474 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:17.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:17.496 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:17.513 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:17.548 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:17.563 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:17.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:17.618 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:17.635 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:17.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:17.659 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:17.665 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:17.672 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:17.681 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:17.689 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:17.697 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:17.706 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:17.757 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:17.768 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:17.811 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:17.820 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:17.857 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:17.871 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:17.890 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:17.901 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:17.911 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:17.922 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:17.929 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:17.935 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:17.942 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:17.947 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:17.960 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:17.986 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:17.992 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:18.001 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:18.009 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:18.036 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:18.075 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:18.085 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:18.098 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:18.109 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:18.121 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:18.133 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:18.149 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:18.160 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:18.191 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:18.220 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:18.230 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:18.276 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:18.307 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:18.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:18.388 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-05-14T17:14:18.395 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-05-14T17:14:18.404 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:18.414 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:18.423 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:18.434 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-05-14T17:14:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:18.449 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:18.510 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T17:14:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:18.552 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:18.608 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:18.617 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:18.625 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:18.636 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-05-14T17:14:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:18.662 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:18.670 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:18.678 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:18.695 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:18.749 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:18.817 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:18.823 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:18.863 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:18.885 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:18.902 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:18.964 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:18.993 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:19.062 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:19.090 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:19.118 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:19.130 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:19.157 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:19.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:19.189 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:19.234 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:19.261 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:19.286 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:19.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:19.360 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:19.368 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:19.377 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:19.395 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:19.404 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:19.412 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:19.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:19.433 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:19.471 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:19.503 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:19.512 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:19.556 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:19.593 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:19.646 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:19.655 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:19.667 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:19.678 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-05-14T17:14:19.740 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-05-14T17:14:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:19.770 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:19.779 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:19.834 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:19.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:19.876 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:19.889 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:19.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-05-14T17:14:19.942 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-05-14T17:14:19.955 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:19.990 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:20.001 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-05-14T17:14:20.016 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:20.056 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:20.089 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:20.118 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:20.129 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:20.150 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-05-14T17:14:20.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-05-14T17:14:20.167 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:20.179 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:20.190 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:20.201 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:20.233 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:20.248 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:20.259 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:20.273 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:20.287 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:20.303 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:20.335 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:20.358 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:20.367 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-05-14T17:14:20.375 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-05-14T17:14:20.385 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:20.392 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:20.401 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:20.409 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:20.419 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:20.431 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:20.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:20.498 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:20.510 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:20.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:20.533 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:20.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:20.592 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-05-14T17:14:20.601 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-05-14T17:14:20.617 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:14:20.627 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:20.637 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:20.650 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:20.663 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:20.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:20.691 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:20.702 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:20.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:20.727 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:20.786 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:20.800 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:20.813 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-05-14T17:14:20.827 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-05-14T17:14:20.876 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:20.904 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:20.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:20.942 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:20.953 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:20.965 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:20.978 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:20.991 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:21.003 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:21.015 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:21.027 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:21.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:21.080 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-05-14T17:14:21.093 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-05-14T17:14:21.109 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:21.125 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:21.163 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:21.174 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:21.187 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:21.198 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:21.228 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:21.239 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:21.250 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:21.261 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:21.272 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:21.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:21.294 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-05-14T17:14:21.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-05-14T17:14:21.363 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:21.373 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:21.384 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:21.396 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:21.406 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:21.442 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:21.457 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:21.471 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:21.483 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:21.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:21.520 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:21.534 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:21.549 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-05-14T17:14:21.558 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-05-14T17:14:21.567 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:21.580 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:21.622 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:21.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:21.651 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:14:21.665 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-05-14T17:14:21.702 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:21.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:21.724 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:21.735 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:21.773 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:21.788 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:21.816 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:21.825 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:21.836 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:21.847 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:21.883 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:21.892 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:21.902 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:21.914 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:21.923 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:21.954 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:21.968 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:21.980 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:21.992 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:22.001 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:22.014 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-05-14T17:14:22.030 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-05-14T17:14:22.042 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:22.074 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:22.085 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:22.095 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:22.127 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:22.136 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:22.149 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:22.181 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:22.191 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:22.209 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:22.223 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:22.234 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:22.249 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-05-14T17:14:22.263 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-05-14T17:14:22.277 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:22.289 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:22.314 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:22.327 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:22.349 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:22.360 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:22.381 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:22.417 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:22.432 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:22.447 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:22.487 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:22.500 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:22.515 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-05-14T17:14:22.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-05-14T17:14:22.540 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:22.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:22.563 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:22.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:22.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:22.597 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:22.606 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:22.639 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:22.646 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:22.683 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:22.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:22.698 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:22.727 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:22.735 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:22.743 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:22.751 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smb", 2024-05-14T17:14:22.759 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:22.767 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:22.775 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:22.784 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:22.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:22.797 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:22.876 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:22.885 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:22.950 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:22.978 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:22.994 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:23.010 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:23.024 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:23.040 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:23.057 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:23.069 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:23.081 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:23.097 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:23.111 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:23.144 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:23.158 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:23.174 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:23.187 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:23.198 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:23.225 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:23.273 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:23.289 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:23.306 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:23.319 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:23.333 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:23.351 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:23.363 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:23.379 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:23.396 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:23.410 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:23.419 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:23.428 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:23.437 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:23.446 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:23.468 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:23.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:23.545 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:23.553 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:23.624 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:23.635 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:23.647 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:23.662 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:23.671 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:23.684 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:23.693 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:23.704 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:23.715 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:23.725 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:23.735 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:23.745 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:23.754 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:23.765 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:23.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:23.856 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:23.873 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:23.887 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:23.902 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:23.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:23.929 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:23.939 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:23.951 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:23.960 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:23.972 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:23.982 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:23.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:24.011 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:24.024 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:24.038 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:24.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:24.073 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:24.120 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:24.129 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:24.141 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:24.153 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:24.165 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:24.177 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:24.189 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:24.203 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:24.215 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:24.227 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:24.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:24.254 INFO:teuthology.orchestra.run.smithi069.stdout: "update_orchestration": { 2024-05-14T17:14:24.266 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "update_orchestration", 2024-05-14T17:14:24.277 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:24.288 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:24.311 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:24.336 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:24.368 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:24.384 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:24.402 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:24.417 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T17:14:24.431 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:24.442 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:24.453 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:24.464 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:24.475 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:24.482 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:24.491 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:24.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-05-14T17:14:24.507 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:24.516 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:24.525 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:24.536 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-05-14T17:14:24.565 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-05-14T17:14:24.597 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:24.608 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:24.617 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:24.626 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:24.635 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:24.642 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:24.652 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:24.660 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T17:14:24.669 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:24.678 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:24.686 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:24.698 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:24.706 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-05-14T17:14:24.715 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-05-14T17:14:24.726 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:24.736 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:24.882 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:24.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:24.901 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:24.912 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:24.922 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-05-14T17:14:24.931 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:24.941 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:24.953 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:24.966 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:24.978 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:24.992 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:25.005 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:25.020 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:25.033 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:25.064 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:25.086 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:25.129 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:25.150 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:25.166 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:25.180 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:25.197 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:25.212 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:25.224 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:25.235 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:25.245 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:25.258 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:25.269 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:25.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:25.295 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:25.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:25.317 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:25.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:25.411 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:25.426 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:25.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:25.456 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:25.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:25.480 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:25.492 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:25.501 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:25.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:25.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:25.532 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:25.547 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:25.558 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:25.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:25.583 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:25.612 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:25.652 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:25.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:25.674 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:25.694 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:25.705 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:25.720 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:25.730 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:25.740 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:25.749 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:25.759 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:25.769 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:25.781 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:25.791 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:25.801 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:25.851 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:25.861 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:25.872 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:25.895 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:25.906 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:25.914 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:25.927 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:25.937 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:25.949 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:25.961 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:25.970 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:25.983 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:25.992 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:26.004 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:26.016 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:26.029 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:26.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:26.064 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:26.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:26.121 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:26.134 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:26.148 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:26.157 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:26.178 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:26.194 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:26.205 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:26.219 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:26.235 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:26.248 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:26.259 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:26.273 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:26.290 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:26.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:26.318 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:26.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-05-14T17:14:26.364 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:26.397 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:26.408 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:26.420 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:26.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:26.450 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:26.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:26.467 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:26.476 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:26.485 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:26.498 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:26.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:26.523 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:26.534 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:26.544 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:26.559 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:26.606 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:26.622 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:26.655 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:26.665 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:26.675 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:26.684 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:26.704 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:26.713 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:26.721 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:26.730 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:26.740 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:26.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:26.755 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:26.767 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:26.776 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:26.788 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:26.813 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:26.845 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:26.861 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:26.900 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:26.913 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:26.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:26.947 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:26.990 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:27.001 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:27.012 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:27.027 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:27.039 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:27.052 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:27.064 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:27.076 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:27.126 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:27.141 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:27.154 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:27.182 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:27.192 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:27.205 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:27.220 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:27.261 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:27.274 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:27.288 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:27.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:27.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:27.325 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:27.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:27.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:27.369 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:27.396 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:27.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:27.423 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:27.465 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:27.480 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:27.491 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:27.504 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:27.519 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:27.543 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:27.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:27.571 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:27.584 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:27.596 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:27.609 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:27.621 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:27.657 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:27.676 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:27.721 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:27.734 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:27.743 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:27.754 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:27.764 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:27.776 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:27.786 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:27.797 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:27.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:27.830 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:27.847 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-05-14T17:14:27.856 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:27.867 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:27.878 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:27.906 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:27.919 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:27.930 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:27.978 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:27.987 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:27.999 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:28.011 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:28.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:28.036 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:28.050 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:28.065 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:28.078 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:28.086 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:28.148 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:28.191 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:28.206 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:28.220 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:28.270 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:28.282 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:28.299 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:28.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:28.326 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:28.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:28.348 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:28.361 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:28.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:28.390 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:28.403 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:28.446 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:28.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:28.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:28.529 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:28.538 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:28.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:28.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:28.563 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:28.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:28.581 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:28.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:28.600 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:28.609 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:28.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:28.627 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:28.636 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:28.646 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:28.654 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:28.691 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:28.749 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:28.789 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:28.799 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:28.807 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:28.815 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:28.823 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:28.830 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:28.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:28.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:28.855 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:28.864 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:28.872 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:28.882 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:28.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:28.897 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:28.907 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:28.949 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:29.017 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:29.029 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:29.038 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:29.047 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:29.057 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:29.069 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:29.078 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:29.087 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:29.096 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:29.105 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:29.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:29.122 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:29.129 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:29.143 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:29.163 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:29.189 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:29.203 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:29.233 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:29.243 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:29.252 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:29.260 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:29.269 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:29.277 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-05-14T17:14:29.286 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:29.294 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:29.302 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:29.310 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-05-14T17:14:29.320 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-05-14T17:14:29.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:29.338 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:29.347 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:29.363 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T17:14:29.389 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:29.408 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:29.447 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:29.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:29.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:29.484 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:29.496 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:29.509 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:29.521 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:14:29.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:14:29.542 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:29.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:29.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:29.569 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-05-14T17:14:29.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:29.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:29.601 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:29.631 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:29.650 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:29.699 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:29.711 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:29.722 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:29.733 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:29.744 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:29.755 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:29.767 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:29.778 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:29.789 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:29.801 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:29.814 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:29.825 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:29.836 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:29.848 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:29.858 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:29.928 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:29.973 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:29.982 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:29.992 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:30.001 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:30.010 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:30.020 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:30.028 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:30.036 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:30.046 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:30.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:30.062 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:30.073 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:30.080 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:30.088 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:30.095 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:30.123 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:30.156 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:30.198 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:30.206 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:30.213 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:30.222 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:30.229 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:30.236 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:30.244 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:30.252 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:30.261 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:30.269 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:30.278 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:30.285 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:30.292 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:30.300 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:30.308 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:30.383 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:30.422 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:30.456 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:30.468 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:30.478 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:30.488 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:30.500 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:30.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:30.524 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:30.538 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:30.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:30.562 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:30.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:30.582 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:30.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:30.600 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:30.608 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:30.642 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:30.678 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:30.704 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:30.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:30.726 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:30.734 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:30.742 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:30.751 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:30.760 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:30.769 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:30.779 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:30.790 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:30.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:30.812 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:30.821 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:30.830 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:30.838 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:30.881 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:30.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:30.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:30.922 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:30.932 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:30.941 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:30.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:30.958 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:30.967 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-05-14T17:14:30.976 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:30.985 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:30.994 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:31.003 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-05-14T17:14:31.012 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-05-14T17:14:31.022 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:31.031 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:31.069 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:31.103 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:31.113 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:31.130 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:31.140 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:31.150 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T17:14:31.158 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:31.173 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:31.182 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:31.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:31.221 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-05-14T17:14:31.230 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-05-14T17:14:31.239 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:31.250 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:31.262 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:31.273 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:31.284 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:31.329 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:31.368 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:31.382 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T17:14:31.397 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:31.408 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:31.417 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:31.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:31.441 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-05-14T17:14:31.450 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-05-14T17:14:31.463 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:31.475 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:31.487 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:31.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:31.510 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:31.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:31.543 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:31.554 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T17:14:31.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:31.596 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:31.607 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:31.630 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:31.641 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-05-14T17:14:31.653 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-05-14T17:14:31.663 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:31.675 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:31.686 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:31.697 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:31.708 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:31.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:31.731 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:31.737 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T17:14:31.743 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:31.769 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:31.809 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:31.816 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:31.824 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-05-14T17:14:31.833 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-05-14T17:14:31.855 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:31.862 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:31.870 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:31.878 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:31.887 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:31.896 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:31.903 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:31.909 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T17:14:31.916 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:31.922 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:31.929 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:31.936 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:31.971 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-05-14T17:14:32.010 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-05-14T17:14:32.018 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:32.027 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:32.033 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:32.047 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:32.055 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:32.064 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:32.074 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:32.083 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:32.092 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:32.101 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:32.113 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:32.126 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:32.158 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-05-14T17:14:32.169 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-05-14T17:14:32.177 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:32.223 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:32.231 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:32.241 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:32.251 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:32.263 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:32.297 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:32.308 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:32.320 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:32.333 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:32.344 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:32.355 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:32.368 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-05-14T17:14:32.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-05-14T17:14:32.407 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:32.418 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:32.427 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:32.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T17:14:32.477 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:32.486 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:32.494 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:32.503 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:32.513 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:32.526 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:32.535 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:32.546 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:32.557 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-05-14T17:14:32.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-05-14T17:14:32.579 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:32.588 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:32.622 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:32.631 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:32.639 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:32.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:32.694 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:32.702 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:32.710 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:32.720 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:32.730 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:32.739 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:32.781 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:14:32.790 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:14:32.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:32.853 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:32.862 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:32.870 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-05-14T17:14:32.880 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-05-14T17:14:32.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:32.920 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:32.928 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:32.936 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:32.944 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:32.950 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:32.958 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:32.964 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-05-14T17:14:32.975 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-05-14T17:14:32.982 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:33.005 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:33.045 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:33.057 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-05-14T17:14:33.066 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:33.105 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:33.114 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:33.123 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:33.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:33.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:33.153 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:33.164 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:33.177 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-05-14T17:14:33.186 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-05-14T17:14:33.196 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:33.208 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:33.242 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:33.258 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:33.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:33.340 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:33.352 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:33.363 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:33.378 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:33.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:33.400 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:33.411 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:33.423 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-05-14T17:14:33.435 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-05-14T17:14:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:33.457 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:33.485 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:33.493 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:33.502 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:33.511 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:33.519 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:33.548 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:33.556 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:33.562 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:33.570 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:33.577 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:33.584 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:33.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:33.597 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:33.603 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:33.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:33.618 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:33.627 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:33.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:33.663 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:33.671 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:33.693 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:33.702 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:33.759 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:33.766 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:33.772 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:33.779 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:33.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:33.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:33.804 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:33.810 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:33.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:33.823 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:33.830 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:33.836 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:33.842 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:33.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:33.895 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:33.917 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:33.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:33.973 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:33.985 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:33.996 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:34.008 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:34.020 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:34.031 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:34.041 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:34.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:34.064 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:34.074 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:34.083 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:34.092 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:34.115 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:34.127 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:34.139 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:34.151 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:34.166 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:34.242 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:34.254 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:34.264 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:34.276 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:34.286 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:34.298 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:34.309 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:34.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:34.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:34.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:34.377 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:34.390 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:34.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:34.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:34.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:34.430 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:34.438 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:34.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 -- 172.21.15.69:0/3409620834 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea5c0b2960 msgr2=0x7fea5c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:34.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea5c0b2960 0x7fea5c0b4e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea58031440 tx=0x7fea58005b50 comp rx=0 tx=0).stop 2024-05-14T17:14:34.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 -- 172.21.15.69:0/3409620834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea70103ce0 msgr2=0x7fea701992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:34.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea70103ce0 0x7fea701992a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fea6000ea80 tx=0x7fea6000ef40 comp rx=0 tx=0).stop 2024-05-14T17:14:34.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 -- 172.21.15.69:0/3409620834 shutdown_connections 2024-05-14T17:14:34.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fea5c0b2960 0x7fea5c0b4e20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea7010d020 0x7fea7019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea70104600 0x7fea701997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 --2- 172.21.15.69:0/3409620834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea70103ce0 0x7fea701992a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 -- 172.21.15.69:0/3409620834 >> 172.21.15.69:0/3409620834 conn(0x7fea70078a50 msgr2=0x7fea700feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:34.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 -- 172.21.15.69:0/3409620834 shutdown_connections 2024-05-14T17:14:34.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:11:36.154+0000 7fea74837640 1 -- 172.21.15.69:0/3409620834 wait complete. 2024-05-14T17:14:34.626 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2024-05-14T17:14:34.633 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:34.652 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:34.659 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:34.716 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-05-14T17:14:34.722 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-05-14T17:14:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:34.735 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:34.743 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:34.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:34.757 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:34.774 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:34.785 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:34.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:34.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:34.808 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:34.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:34.846 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:34.875 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-05-14T17:14:34.884 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-05-14T17:14:34.893 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:34.902 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:34.912 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:34.921 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:34.929 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:34.938 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:34.947 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:34.959 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:34.978 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:34.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:35.006 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:35.015 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:35.025 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:35.044 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:35.084 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:35.093 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:35.102 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:35.110 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:35.118 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:35.127 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:35.136 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:35.155 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:35.163 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:35.192 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:35.201 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-05-14T17:14:35.213 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-05-14T17:14:35.224 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:35.288 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T17:14:35.300 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:35.307 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:35.314 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:35.322 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:35.330 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:35.351 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:35.360 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:35.369 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:35.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:35.409 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:35.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-05-14T17:14:35.430 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:35.439 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:35.458 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:35.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:35.547 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:35.557 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:35.567 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:35.578 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:35.586 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:35.606 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:35.616 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:35.635 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:35.644 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:35.664 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:35.672 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:35.708 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:35.717 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:35.733 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:35.744 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:35.759 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:35.772 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:35.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:35.809 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:35.819 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:35.830 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:35.860 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:35.871 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:35.910 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:35.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:35.946 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:35.958 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:35.971 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:35.983 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:35.995 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:36.009 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:36.022 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:36.033 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:36.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:36.055 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:36.065 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:36.076 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:36.094 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:36.102 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:36.116 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:36.152 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:36.163 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:36.185 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:36.198 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:36.217 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:36.227 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:36.251 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:36.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:36.275 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:36.282 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:36.288 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:36.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:36.316 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:36.332 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:36.361 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:36.368 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:36.375 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:36.382 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:36.388 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:36.400 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:36.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:36.434 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:36.441 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:36.448 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:36.456 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:36.464 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:36.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:36.477 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:36.503 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:36.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:36.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:36.552 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:36.562 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:36.572 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:36.581 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:36.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:36.600 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:36.618 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-05-14T17:14:36.627 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:36.636 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:36.646 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:36.656 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:36.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:36.676 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:36.701 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:36.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:36.763 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:36.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:36.784 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:36.795 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:36.805 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:36.817 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:36.843 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:36.853 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:36.862 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:36.886 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:36.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:36.908 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:36.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:36.939 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:36.948 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:36.976 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:36.986 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:36.997 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:37.009 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:37.020 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:37.031 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:37.041 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:37.060 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:37.071 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:37.081 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:37.090 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:37.100 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:37.108 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:37.117 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:37.154 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:37.163 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:37.195 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:37.206 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:37.216 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:37.226 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:37.235 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:37.245 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:37.254 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:37.263 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:37.282 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:37.291 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:37.301 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:37.310 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:37.319 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:37.328 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:37.354 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:37.363 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:37.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:37.413 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:37.619 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:37.648 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-05-14T17:14:37.664 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-05-14T17:14:37.673 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:37.685 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:37.697 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:37.708 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-05-14T17:14:37.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:37.729 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:37.738 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:37.751 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T17:14:37.761 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:37.782 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:37.821 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:37.830 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:37.840 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-05-14T17:14:37.849 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-05-14T17:14:37.857 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:37.891 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:37.903 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:37.913 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:37.922 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:37.955 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:37.989 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:38.000 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-05-14T17:14:38.012 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:38.023 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:38.059 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:38.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:38.094 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-05-14T17:14:38.114 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-05-14T17:14:38.125 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:38.135 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:38.144 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:38.152 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:38.162 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:38.170 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:38.180 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:38.188 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T17:14:38.198 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:38.207 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:38.216 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:38.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:38.233 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-05-14T17:14:38.274 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T17:14:38.281 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:38.341 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:38.349 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:38.356 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:38.363 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:38.370 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:38.377 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:38.385 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T17:14:38.393 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:38.401 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:38.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:38.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:38.425 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:38.434 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:38.444 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:38.465 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:38.487 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:38.497 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:38.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:38.535 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:38.548 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:38.557 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:38.567 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:38.577 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:38.590 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:38.600 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:38.611 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:38.623 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:38.632 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-05-14T17:14:38.657 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:38.700 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:38.746 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-05-14T17:14:38.793 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-05-14T17:14:38.866 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:14:38.878 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:38.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:38.900 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-05-14T17:14:38.948 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:38.960 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:38.971 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:38.993 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:39.029 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:39.039 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:39.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:39.058 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-05-14T17:14:39.066 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-05-14T17:14:39.076 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:39.089 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:39.098 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:39.124 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:39.135 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:39.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:39.152 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:39.189 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:39.198 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:39.205 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:39.212 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:39.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:39.277 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:14:39.285 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:14:39.294 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:39.300 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:39.307 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:39.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:14:39.326 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:39.335 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:39.358 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:39.368 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:39.377 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:39.386 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:39.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:39.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:39.439 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:39.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:39.464 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:39.505 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:39.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:39.528 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:39.543 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:39.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:39.567 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:39.579 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:39.591 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:39.608 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:39.620 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:39.636 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:39.649 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:39.680 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:39.691 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:39.702 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:39.712 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:39.720 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:39.729 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:39.757 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:39.780 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:39.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:39.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:39.838 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:39.848 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:39.866 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:39.878 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:39.886 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:39.913 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:39.920 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:39.926 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:39.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:39.942 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:39.951 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:39.995 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:40.003 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:40.013 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:40.021 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:40.029 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:40.037 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:40.046 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:40.054 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:40.066 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:40.076 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:40.085 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:40.105 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:40.115 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:40.124 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:40.133 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:40.142 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:40.152 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:40.197 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:40.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:40.213 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:40.221 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:40.230 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:40.239 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:40.246 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:40.256 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:40.267 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:40.274 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:40.343 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:40.351 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:40.359 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:40.366 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:40.372 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:40.377 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:40.408 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:40.415 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:40.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:40.429 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:40.436 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:40.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:40.449 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:40.459 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:40.476 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:40.482 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:40.491 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:40.519 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:40.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:40.537 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:40.546 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-05-14T17:14:40.555 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-05-14T17:14:40.564 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:40.594 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:40.604 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:40.613 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:40.622 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:40.632 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:40.642 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:40.651 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:40.661 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:40.686 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:40.698 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:40.707 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:40.736 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-05-14T17:14:40.745 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-05-14T17:14:40.754 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:40.764 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:40.788 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-05-14T17:14:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-05-14T17:14:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-05-14T17:14:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T17:14:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:14:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "modules": [ 2024-05-14T17:14:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-05-14T17:14:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard", 2024-05-14T17:14:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "iostat", 2024-05-14T17:14:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "nfs", 2024-05-14T17:14:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "restful", 2024-05-14T17:14:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "snap_schedule" 2024-05-14T17:14:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-05-14T17:14:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-05-14T17:14:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:14:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:14:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:14:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T17:14:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:41.684 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:42.170 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:42.177 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:42.217 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:42.230 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:42.240 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:42.248 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:42.261 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:42.271 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:42.293 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:42.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:42.326 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:42.337 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:42.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:42.384 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:42.395 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:42.406 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:42.418 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:42.429 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:42.458 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:42.469 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:42.480 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:42.491 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:42.499 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:42.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:42.517 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:42.527 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-05-14T17:14:42.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-05-14T17:14:42.552 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:42.562 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:42.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:42.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:42.620 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:42.630 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:42.636 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:42.643 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-05-14T17:14:42.716 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:42.724 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:42.730 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:42.737 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:42.745 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-05-14T17:14:42.753 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-05-14T17:14:42.761 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:42.768 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:42.778 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:42.800 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-05-14T17:14:42.809 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:42.818 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:42.840 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:42.849 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-05-14T17:14:42.860 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:42.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:42.879 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:42.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:42.961 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-05-14T17:14:42.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-05-14T17:14:42.985 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:42.994 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:43.003 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:43.013 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:43.024 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:43.038 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:43.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:43.066 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-05-14T17:14:43.078 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:43.108 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:43.124 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:43.136 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:43.147 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-05-14T17:14:43.156 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-05-14T17:14:43.188 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:43.199 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:43.208 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:43.218 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:43.227 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:43.237 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:43.246 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:43.258 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-05-14T17:14:43.270 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:43.298 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:43.308 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:43.317 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:43.353 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-05-14T17:14:43.364 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-05-14T17:14:43.375 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:43.386 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:43.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:43.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-05-14T17:14:43.466 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:43.477 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:43.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:43.500 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-05-14T17:14:43.511 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:43.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:43.542 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:43.550 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:43.564 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-05-14T17:14:43.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-05-14T17:14:43.583 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:43.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:43.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:43.636 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:43.646 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:43.688 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:43.700 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:43.709 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-05-14T17:14:43.719 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:43.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:43.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:43.743 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:43.751 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-05-14T17:14:43.801 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-05-14T17:14:43.809 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:43.831 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:43.839 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:43.873 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:43.881 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:43.889 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:43.897 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:43.938 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T17:14:43.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:43.957 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:43.966 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:43.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:43.982 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-05-14T17:14:43.993 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-05-14T17:14:44.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:44.014 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:44.026 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:44.035 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:44.048 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:44.057 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:44.098 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:44.110 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-05-14T17:14:44.123 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:44.135 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:44.178 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:44.191 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:44.203 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:44.214 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:44.225 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:44.237 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:44.249 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:44.261 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:44.272 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:44.283 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:44.295 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:44.355 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:44.366 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:44.378 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:44.389 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:44.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:44.450 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-05-14T17:14:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-05-14T17:14:44.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-05-14T17:14:44.517 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:44.526 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:44.535 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:44.565 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:44.571 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:44.612 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:44.619 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:44.628 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T17:14:44.637 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:44.676 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:44.683 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:44.693 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:44.702 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-05-14T17:14:44.710 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-05-14T17:14:44.719 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:44.728 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:44.736 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:44.745 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-05-14T17:14:44.753 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:44.764 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:44.782 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:44.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T17:14:44.818 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:14:44.827 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:44.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:44.846 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:44.879 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-05-14T17:14:44.890 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-05-14T17:14:44.899 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:14:44.908 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:44.919 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:44.930 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:44.943 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-05-14T17:14:44.955 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-05-14T17:14:44.970 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:44.993 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T17:14:45.002 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:14:45.015 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:45.025 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:45.033 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:45.044 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-05-14T17:14:45.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-05-14T17:14:45.134 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:14:45.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:45.157 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:45.165 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-05-14T17:14:45.177 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:14:45.186 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-05-14T17:14:45.194 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:45.203 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T17:14:45.214 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:45.225 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:45.235 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:45.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:45.257 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-05-14T17:14:45.266 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-05-14T17:14:45.303 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:45.314 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:45.325 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:45.399 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T17:14:45.411 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:45.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:45.433 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:45.443 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T17:14:45.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T17:14:45.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:45.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:45.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:45.489 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-05-14T17:14:45.503 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-05-14T17:14:45.510 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:14:45.524 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:45.531 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:45.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-05-14T17:14:45.567 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-05-14T17:14:45.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-05-14T17:14:45.620 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:45.629 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-05-14T17:14:45.637 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T17:14:45.645 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:45.653 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:45.658 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:45.665 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-05-14T17:14:45.672 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-05-14T17:14:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:45.707 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:45.715 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-05-14T17:14:45.724 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:45.731 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:45.763 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:45.770 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T17:14:45.778 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:14:45.823 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:45.830 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:45.836 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:45.844 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-05-14T17:14:45.851 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-05-14T17:14:45.859 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:14:45.868 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:45.880 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:45.896 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:45.906 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-05-14T17:14:45.920 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-05-14T17:14:45.932 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:45.958 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T17:14:45.972 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:14:45.997 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:46.009 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:46.021 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:46.072 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:46.084 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:46.095 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:46.107 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:46.117 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:46.128 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:46.138 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:46.149 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:46.160 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:46.169 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:46.180 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:46.191 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:46.208 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:46.217 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:46.239 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:46.259 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:46.267 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:46.308 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:46.319 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:46.327 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:46.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:46.347 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:46.357 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:46.367 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:46.376 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:46.386 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:46.395 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:46.404 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:46.416 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:46.433 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:46.443 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:46.486 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:46.494 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:46.502 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:46.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:46.559 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:46.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:46.580 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:46.592 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:46.601 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:46.611 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:46.621 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:46.629 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:46.639 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:46.675 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:46.685 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:46.720 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:46.749 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:46.760 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:46.789 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:46.799 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:46.809 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:46.818 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:46.827 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:46.838 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:46.850 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:46.862 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:46.870 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:46.882 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:46.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:46.906 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:46.925 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:46.938 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:46.961 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:46.997 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:47.010 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:47.060 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:47.074 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:47.085 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:47.094 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:47.104 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-05-14T17:14:47.115 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-05-14T17:14:47.126 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:14:47.136 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:47.147 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:47.158 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:47.169 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:47.201 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:47.212 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T17:14:47.235 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:47.257 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:47.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:47.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:47.317 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-05-14T17:14:47.332 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-05-14T17:14:47.347 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:47.357 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:47.368 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:47.377 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-05-14T17:14:47.386 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:47.396 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:47.407 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:47.419 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-05-14T17:14:47.432 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-05-14T17:14:47.442 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-05-14T17:14:47.451 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-05-14T17:14:47.479 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-05-14T17:14:47.519 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:47.531 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-05-14T17:14:47.566 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:47.577 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:47.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:47.596 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:47.606 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-05-14T17:14:47.617 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-05-14T17:14:47.628 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:47.640 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:47.651 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:47.661 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:47.670 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:47.681 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:47.693 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:47.701 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T17:14:47.719 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:47.766 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:47.775 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:47.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:47.825 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:14:47.837 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:14:47.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:47.853 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:47.869 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:47.886 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:14:47.900 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:47.917 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:47.930 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:47.946 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T17:14:47.958 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:47.989 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:47.998 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:48.024 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:48.107 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:14:48.121 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:14:48.132 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:48.143 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:14:48.154 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:48.165 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:14:48.176 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:48.186 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:48.200 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:48.214 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:48.224 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:48.236 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:48.257 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:48.270 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:48.318 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-05-14T17:14:48.329 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-05-14T17:14:48.378 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:48.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:48.433 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:48.443 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:14:48.452 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:14:48.461 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:48.472 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:48.481 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T17:14:48.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T17:14:48.500 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:48.510 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:48.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:48.531 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-05-14T17:14:48.541 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-05-14T17:14:48.551 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:14:48.561 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:48.570 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:48.589 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:14:48.636 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:48.646 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:48.655 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:48.664 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T17:14:48.679 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:48.690 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:48.701 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:48.718 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:48.729 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:14:48.745 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:48.758 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:14:48.770 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-05-14T17:14:48.795 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:14:48.825 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:14:48.837 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:14:48.848 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-05-14T17:14:48.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-05-14T17:14:48.901 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:14:48.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:48.933 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:48.949 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-05-14T17:14:48.962 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:48.975 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:48.983 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:48.996 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T17:14:49.006 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:49.017 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:49.028 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:49.039 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:49.052 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-05-14T17:14:49.067 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-05-14T17:14:49.079 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:49.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:49.126 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:49.160 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-05-14T17:14:49.176 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:49.189 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:49.199 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:49.218 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T17:14:49.234 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:49.249 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:49.260 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:49.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:49.289 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-05-14T17:14:49.299 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-05-14T17:14:49.309 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:49.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:49.332 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:49.353 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-05-14T17:14:49.366 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:49.384 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:49.412 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:49.437 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T17:14:49.450 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:49.457 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:49.465 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:49.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:49.484 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-05-14T17:14:49.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-05-14T17:14:49.500 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:49.507 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:49.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:49.525 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:49.533 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:49.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:49.561 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:49.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T17:14:49.593 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T17:14:49.619 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:49.655 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:49.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:49.673 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-05-14T17:14:49.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-05-14T17:14:49.691 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:49.701 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:49.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:49.719 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:14:49.728 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:49.737 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:49.748 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:49.758 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T17:14:49.767 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:49.790 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:49.797 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:49.810 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:49.844 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-05-14T17:14:49.854 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T17:14:49.863 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:14:49.875 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-05-14T17:14:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:49.933 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:49.945 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T17:14:49.957 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:49.968 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:49.980 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:49.993 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:50.006 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-05-14T17:14:50.017 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-05-14T17:14:50.035 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:50.081 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:50.092 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:50.105 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:50.121 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:50.136 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:50.150 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:50.161 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-05-14T17:14:50.176 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-05-14T17:14:50.191 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-05-14T17:14:50.208 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:50.222 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T17:14:50.238 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:50.254 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:50.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:50.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:50.299 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-05-14T17:14:50.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-05-14T17:14:50.364 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:50.378 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:50.389 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:50.402 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:50.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:50.458 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T17:14:50.472 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:50.486 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:50.500 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:50.513 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:50.530 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-05-14T17:14:50.540 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-05-14T17:14:50.553 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:50.565 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:50.585 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:50.619 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:50.654 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:50.670 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:50.686 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:50.700 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T17:14:50.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:50.726 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:50.737 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:50.747 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:50.756 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-05-14T17:14:50.767 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-05-14T17:14:50.776 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:50.785 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:50.795 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:50.807 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:50.817 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:50.855 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:50.867 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:50.878 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T17:14:50.888 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:50.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:50.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:50.928 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-05-14T17:14:50.937 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-05-14T17:14:50.949 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:50.962 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:50.971 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:50.981 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T17:14:50.991 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:51.000 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:51.026 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:51.037 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:14:51.071 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:51.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:51.093 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:51.101 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:51.111 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-05-14T17:14:51.121 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-05-14T17:14:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:51.142 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:51.155 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:51.168 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T17:14:51.181 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:51.194 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:51.207 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:51.217 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-05-14T17:14:51.228 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:51.242 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:51.254 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:51.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:51.304 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-05-14T17:14:51.319 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-05-14T17:14:51.330 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:51.341 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:51.350 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:51.361 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T17:14:51.373 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:51.382 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:51.391 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:51.399 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-05-14T17:14:51.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:51.421 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:51.434 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:51.444 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:51.464 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-05-14T17:14:51.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-05-14T17:14:51.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:51.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:51.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:51.544 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T17:14:51.554 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:51.568 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:51.580 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:51.591 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:14:51.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:51.610 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:51.618 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:51.626 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:51.635 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-05-14T17:14:51.647 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-05-14T17:14:51.659 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:51.697 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:51.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:51.758 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T17:14:51.769 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:51.782 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:51.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:51.809 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-05-14T17:14:51.821 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:51.833 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:51.845 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:51.854 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:51.866 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-05-14T17:14:51.878 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-05-14T17:14:51.890 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:51.901 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:51.912 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:51.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T17:14:51.939 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:51.951 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:51.992 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:52.004 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-05-14T17:14:52.016 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:52.028 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:52.040 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:52.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:52.062 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-05-14T17:14:52.073 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-05-14T17:14:52.084 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:52.095 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:52.106 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:52.117 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T17:14:52.130 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:52.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:52.161 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:52.174 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-05-14T17:14:52.185 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:52.216 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:52.229 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:52.240 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:52.251 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-05-14T17:14:52.263 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-05-14T17:14:52.275 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:52.291 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:52.300 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:52.313 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T17:14:52.329 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:52.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:52.359 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:52.371 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-05-14T17:14:52.385 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:52.398 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:52.410 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:52.423 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:52.470 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-05-14T17:14:52.480 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-05-14T17:14:52.492 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:52.501 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:52.513 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:52.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T17:14:52.532 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:52.546 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:52.557 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:52.568 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-05-14T17:14:52.581 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:52.594 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:52.605 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:52.619 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:52.629 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-05-14T17:14:52.649 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-05-14T17:14:52.661 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:52.694 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:52.708 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:52.721 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T17:14:52.733 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:52.750 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:52.775 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-05-14T17:14:52.787 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:52.799 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:52.808 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:52.821 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:52.830 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-05-14T17:14:52.841 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-05-14T17:14:52.852 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:52.860 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:52.875 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:52.891 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T17:14:52.924 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:52.934 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:52.945 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:52.956 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:14:52.967 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:52.977 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:52.988 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:52.999 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:53.011 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-05-14T17:14:53.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-05-14T17:14:53.033 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:53.043 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:53.055 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-14T17:14:53.078 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:53.085 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:53.121 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:53.173 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-05-14T17:14:53.183 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:53.192 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:53.202 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:53.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:53.221 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-05-14T17:14:53.230 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-05-14T17:14:53.238 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:53.250 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:53.258 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:53.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T17:14:53.283 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:53.295 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:53.307 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:53.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-05-14T17:14:53.328 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:53.373 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:53.410 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:53.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:53.437 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-05-14T17:14:53.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-05-14T17:14:53.465 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:53.478 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:53.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:53.507 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T17:14:53.520 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:53.534 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:53.545 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:53.560 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-05-14T17:14:53.574 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:53.588 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:53.605 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:53.619 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:53.702 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-05-14T17:14:53.747 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-05-14T17:14:53.763 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:53.777 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:53.793 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:53.806 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T17:14:53.823 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:53.838 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:53.855 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:53.867 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-05-14T17:14:53.879 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:53.893 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:53.906 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:53.922 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:53.933 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-05-14T17:14:53.946 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-05-14T17:14:53.986 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:54.058 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:54.099 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:54.112 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T17:14:54.125 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:54.138 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:54.150 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:54.161 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-05-14T17:14:54.173 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:54.184 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:54.195 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:54.206 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:54.217 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-05-14T17:14:54.228 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-05-14T17:14:54.238 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:54.248 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:54.257 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:54.292 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:54.343 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:54.413 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:54.423 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:54.433 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T17:14:54.444 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:54.454 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:54.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:54.476 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:54.485 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-05-14T17:14:54.495 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-05-14T17:14:54.504 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:54.511 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:54.518 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:54.525 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:14:54.535 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:54.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:54.562 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:54.597 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T17:14:54.651 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:54.662 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:54.675 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:54.686 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:54.695 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-05-14T17:14:54.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T17:14:54.715 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:54.725 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:54.734 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:54.744 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-05-14T17:14:54.753 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:54.778 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:54.790 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:54.801 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T17:14:54.813 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:54.841 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:54.849 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:54.881 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:54.891 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-05-14T17:14:54.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-05-14T17:14:54.910 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:54.922 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:54.933 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:54.944 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-05-14T17:14:54.958 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:54.972 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:54.982 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:55.000 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T17:14:55.013 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:55.027 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:55.038 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:55.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:55.096 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-05-14T17:14:55.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-05-14T17:14:55.160 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:55.174 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:55.190 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:55.204 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-05-14T17:14:55.215 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:55.228 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:55.242 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:55.255 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-05-14T17:14:55.268 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:55.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:55.291 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:55.302 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:55.315 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-05-14T17:14:55.329 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-05-14T17:14:55.377 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:55.396 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:55.429 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:55.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:55.463 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:55.473 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:55.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:55.500 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T17:14:55.511 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:55.525 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:55.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:55.546 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:55.557 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-05-14T17:14:55.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-05-14T17:14:55.578 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:55.589 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:55.600 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:55.631 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:14:55.640 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:55.666 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:55.694 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:55.704 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-05-14T17:14:55.716 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:55.726 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:55.737 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:55.749 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:55.758 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-05-14T17:14:55.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-05-14T17:14:55.782 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:55.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:55.800 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:55.815 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T17:14:55.826 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:55.839 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:55.880 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:55.893 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T17:14:55.917 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:55.954 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:55.971 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:55.982 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:55.994 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-05-14T17:14:56.005 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-05-14T17:14:56.018 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:14:56.031 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:56.042 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:56.057 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:14:56.069 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:56.081 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:56.092 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:56.103 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-05-14T17:14:56.132 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:56.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:56.171 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:56.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:56.224 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-05-14T17:14:56.241 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-05-14T17:14:56.254 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:56.270 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:56.283 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:56.296 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:56.307 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:56.321 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:56.332 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:56.347 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T17:14:56.361 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:56.375 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:56.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:56.422 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:56.435 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-05-14T17:14:56.454 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-05-14T17:14:56.485 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:56.522 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:56.535 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:56.547 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:56.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:56.568 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:56.578 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:56.591 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T17:14:56.602 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:56.613 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:56.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:56.638 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:56.650 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:14:56.685 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:14:56.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:56.712 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:56.724 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:56.756 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:56.783 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:56.791 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:56.802 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:56.811 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:56.821 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:56.830 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:56.841 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:56.851 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:56.864 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:56.872 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:56.881 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:56.894 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:56.904 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:56.942 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:56.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:56.970 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-05-14T17:14:57.006 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-05-14T17:14:57.024 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:57.037 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:57.048 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:57.060 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:57.071 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:57.087 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:57.095 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:57.108 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T17:14:57.114 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:57.123 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:57.131 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:57.136 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:57.171 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:14:57.179 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:14:57.187 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:57.220 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:57.259 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:14:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T17:14:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:14:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:14:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:14:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:14:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:14:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:14:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:14:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:14:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:14:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:14:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:14:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:14:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:14:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T17:14:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T17:14:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T17:14:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T17:14:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T17:14:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-05-14T17:14:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T17:14:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-05-14T17:14:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-05-14T17:14:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:14:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-05-14T17:14:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:58.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:58.585 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-05-14T17:14:58.595 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-05-14T17:14:58.603 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:58.613 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:58.626 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:58.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:14:58.645 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:58.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:58.699 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:58.708 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T17:14:58.720 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:58.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:58.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:58.781 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:58.794 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-05-14T17:14:58.817 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-05-14T17:14:58.837 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:14:58.843 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:58.848 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:58.855 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:14:58.863 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:58.869 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:58.875 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:58.881 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-05-14T17:14:58.908 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:58.917 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:58.924 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:58.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:58.940 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-05-14T17:14:58.947 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-05-14T17:14:58.972 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:58.987 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:59.012 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:59.029 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-05-14T17:14:59.036 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:59.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:59.061 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:14:59.071 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-05-14T17:14:59.078 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-05-14T17:14:59.088 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:14:59.100 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T17:14:59.110 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:59.181 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:59.191 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:59.202 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:59.215 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-05-14T17:14:59.224 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-05-14T17:14:59.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:59.259 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:59.288 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:59.305 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-05-14T17:14:59.320 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:59.331 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:59.352 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:59.366 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T17:14:59.377 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:59.386 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:59.396 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:59.443 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-05-14T17:14:59.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-05-14T17:14:59.460 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:59.476 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:59.483 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:59.490 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T17:14:59.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:59.565 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:59.597 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:59.608 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T17:14:59.618 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:59.628 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:59.636 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:59.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:59.654 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-05-14T17:14:59.663 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-05-14T17:14:59.709 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:14:59.720 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:59.729 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:59.752 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:14:59.762 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:14:59.772 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:14:59.782 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:14:59.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T17:14:59.829 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:14:59.839 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:14:59.848 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:14:59.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:14:59.871 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-05-14T17:14:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-05-14T17:14:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:14:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:14:59.977 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:14:59.994 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:00.009 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:00.024 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:00.039 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:00.054 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T17:15:00.076 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:00.088 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:00.103 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:00.130 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:00.142 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-05-14T17:15:00.154 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-05-14T17:15:00.182 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:00.195 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:00.207 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:00.219 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T17:15:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:00.335 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:00.342 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-05-14T17:15:00.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-05-14T17:15:00.372 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:00.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:00.416 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:00.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:00.431 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:00.438 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:00.446 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:00.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T17:15:00.485 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:00.503 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:00.511 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:00.519 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-05-14T17:15:00.528 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-05-14T17:15:00.576 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:00.592 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:00.613 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:00.647 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:00.679 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:00.688 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:00.698 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:00.707 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T17:15:00.720 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:00.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:00.749 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:00.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:00.772 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-05-14T17:15:00.784 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-05-14T17:15:00.794 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:00.803 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:00.811 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:00.834 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-05-14T17:15:00.853 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:00.908 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:00.922 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:00.953 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-05-14T17:15:00.964 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:00.973 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:00.980 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:00.991 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:00.999 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:01.025 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:01.034 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:01.044 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:01.053 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:01.061 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:01.071 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:01.111 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:01.130 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:01.137 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:01.147 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:01.187 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:01.198 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:01.204 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-05-14T17:15:01.211 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-05-14T17:15:01.218 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:01.225 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:01.255 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:01.263 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:01.271 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:01.279 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:01.288 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:01.294 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T17:15:01.300 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:01.306 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:01.330 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:01.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:01.366 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-05-14T17:15:01.407 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-05-14T17:15:01.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:01.424 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:01.434 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:01.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:01.508 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:01.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:01.538 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:01.554 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T17:15:01.562 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:01.606 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:01.620 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:01.636 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:01.647 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-05-14T17:15:01.664 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-05-14T17:15:01.672 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:01.696 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:01.728 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:01.742 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:01.756 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:01.768 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:01.818 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:01.832 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T17:15:01.846 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:01.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:01.868 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:01.876 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:01.940 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-05-14T17:15:01.950 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-05-14T17:15:01.959 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:01.972 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:01.998 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:02.038 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:02.089 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:02.099 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:02.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:02.115 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T17:15:02.140 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:02.149 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:02.158 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:02.167 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:02.176 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-05-14T17:15:02.188 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-05-14T17:15:02.197 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:02.248 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:02.258 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:02.268 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:02.278 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:02.298 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:02.341 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T17:15:02.387 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:02.396 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:02.434 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:02.448 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:02.460 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-05-14T17:15:02.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-05-14T17:15:02.487 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:02.501 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:02.525 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:02.537 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:02.551 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:02.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:02.586 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:02.599 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T17:15:02.633 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:02.642 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:02.659 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:02.723 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:02.733 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:02.741 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-05-14T17:15:02.754 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:02.763 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:02.772 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:02.800 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:02.807 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:02.815 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:02.822 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:02.829 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:02.836 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:02.873 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:02.881 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:02.944 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:02.992 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:03.000 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:03.006 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:03.013 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:03.020 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:03.026 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:03.052 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:03.058 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:03.065 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:03.071 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:03.077 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:03.084 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:03.091 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:03.115 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:03.124 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:03.131 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:03.142 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:03.204 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:03.216 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:03.225 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:03.234 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:03.247 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:03.269 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:03.279 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:03.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:03.299 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:03.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:03.323 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:03.359 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:03.371 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:03.399 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:03.411 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:03.446 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:03.539 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:03.554 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:03.563 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:03.571 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:03.581 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:03.592 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:03.620 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:03.630 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:03.640 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:03.649 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:03.657 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:03.668 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:03.717 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:03.726 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:03.740 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:03.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:03.788 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:03.821 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:03.829 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:03.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:03.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:03.852 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:03.860 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:03.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:03.896 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:03.903 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:03.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:03.921 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-05-14T17:15:03.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-05-14T17:15:03.976 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:03.984 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:04.009 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-05-14T17:15:04.052 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:04.065 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:04.076 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:04.088 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T17:15:04.100 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:04.128 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:04.140 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:04.148 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:04.181 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:04.192 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:04.199 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:04.213 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:04.239 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:04.252 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:04.267 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:04.276 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:04.318 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:04.340 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:04.354 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:04.368 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:04.382 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:04.404 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-05-14T17:15:04.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-05-14T17:15:04.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:04.444 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:04.454 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:04.464 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-05-14T17:15:04.474 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:04.519 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:04.527 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:04.534 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T17:15:04.589 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:04.645 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:04.687 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:04.699 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:04.710 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:04.718 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:04.728 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-05-14T17:15:04.743 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:04.752 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:04.761 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:04.771 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T17:15:04.780 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T17:15:04.789 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:04.850 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:04.862 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:04.871 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:15:04.892 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:04.916 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:04.935 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:04.960 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:04.987 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:04.996 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:05.006 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:05.018 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T17:15:05.034 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T17:15:05.046 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:05.056 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:05.066 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:05.073 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:05.108 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:05.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:05.134 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:05.142 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:05.179 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:05.194 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:05.210 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:05.227 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T17:15:05.238 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T17:15:05.246 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:05.282 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:05.308 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:05.317 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:05.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:05.364 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:05.373 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:05.384 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:05.399 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:05.411 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-05-14T17:15:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T17:15:05.446 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:05.455 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:05.465 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:05.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:05.480 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:05.487 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:05.494 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:05.499 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:05.507 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:05.514 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:05.548 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:05.557 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:05.563 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T17:15:05.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T17:15:05.591 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:05.603 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:05.689 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:05.699 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:05.708 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:05.717 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:05.726 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:05.739 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:05.748 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:05.760 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:05.770 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T17:15:05.818 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T17:15:05.830 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:05.841 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:05.853 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:05.875 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:05.895 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:05.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:05.926 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:05.937 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:05.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:05.959 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:05.971 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:05.982 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:05.993 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T17:15:06.005 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T17:15:06.016 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:06.027 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:06.060 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:06.070 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:06.081 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:06.093 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:06.109 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:06.128 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:06.146 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:06.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:06.168 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:06.180 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:06.191 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T17:15:06.200 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T17:15:06.208 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:06.218 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:06.226 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:06.239 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:06.248 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:06.292 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:06.302 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:06.311 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:06.327 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:06.384 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:06.404 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:06.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:06.445 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T17:15:06.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T17:15:06.460 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:06.466 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:06.472 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:06.479 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:06.490 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:06.500 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:06.509 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:06.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:06.550 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:06.559 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:06.570 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:06.606 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:06.643 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T17:15:06.680 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T17:15:06.731 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:06.743 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:06.757 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:06.767 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:06.776 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:06.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:06.798 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:06.808 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:06.817 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:06.826 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:06.836 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:06.878 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:06.888 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T17:15:06.898 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T17:15:06.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:06.947 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:06.962 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:06.980 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:06.989 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:06.998 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:07.007 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:07.022 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:07.032 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:07.041 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:07.049 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:07.058 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:07.066 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T17:15:07.075 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T17:15:07.083 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:07.123 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:07.135 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:07.153 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:07.177 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:07.196 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:07.240 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:07.252 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:07.263 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:07.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:07.279 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:07.287 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:07.295 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T17:15:07.304 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T17:15:07.312 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:07.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:07.329 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:07.372 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:07.380 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:07.388 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:07.425 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:07.433 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:07.446 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:07.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:07.472 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:07.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:07.489 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T17:15:07.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T17:15:07.505 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:07.513 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:07.522 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:07.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:07.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:07.546 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:07.594 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:07.601 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:07.612 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:07.629 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:07.638 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:07.655 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:07.663 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T17:15:07.673 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T17:15:07.686 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:07.698 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:07.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:07.722 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-05-14T17:15:07.735 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:07.746 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:07.759 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:07.770 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:07.806 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:07.817 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:07.827 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:07.838 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:07.862 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T17:15:07.888 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T17:15:07.905 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:07.951 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:07.964 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:07.975 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:07.996 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:08.008 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:08.020 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:08.032 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:08.043 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:08.076 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:08.087 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:08.097 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-05-14T17:15:08.108 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-05-14T17:15:08.119 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:08.138 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:08.163 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:08.180 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:08.241 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:08.252 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:08.261 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:08.268 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:08.277 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:08.285 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:08.294 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:08.302 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:08.311 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T17:15:08.338 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T17:15:08.346 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:08.355 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:08.368 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:08.376 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-05-14T17:15:08.388 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:08.414 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:08.430 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:08.444 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:08.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:08.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:08.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:08.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:08.492 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T17:15:08.501 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T17:15:08.510 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:08.517 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:08.553 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:08.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:08.567 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:08.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:08.581 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:08.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:08.622 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:08.630 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:08.639 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:08.672 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:08.681 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T17:15:08.689 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T17:15:08.697 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:08.705 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:08.715 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:08.724 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:08.733 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:08.764 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:08.774 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:08.783 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:08.794 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:08.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:08.822 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:08.864 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:08.873 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T17:15:08.887 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T17:15:08.897 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:08.905 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:08.916 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:08.928 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:08.939 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:08.950 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:08.958 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:08.998 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:09.006 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:09.017 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:09.025 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:09.035 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:09.044 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T17:15:09.056 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T17:15:09.064 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:09.121 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:09.137 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:09.146 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:09.155 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:09.164 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:09.173 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:09.182 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:09.191 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:09.224 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:09.232 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:09.241 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:09.249 INFO:teuthology.orchestra.run.smithi069.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T17:15:09.258 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T17:15:09.266 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:09.276 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:09.289 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:09.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "[]", 2024-05-14T17:15:09.339 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:09.364 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:09.372 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:09.381 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:09.390 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:09.399 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:09.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:09.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:09.458 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T17:15:09.468 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T17:15:09.476 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:09.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:09.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:09.502 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-05-14T17:15:09.512 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:09.523 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:09.540 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:09.557 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:09.590 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:09.597 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:09.604 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:09.610 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:09.617 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T17:15:09.625 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T17:15:09.631 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:09.664 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:09.673 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:09.682 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:09.690 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:09.699 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:09.711 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:09.723 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:09.739 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:09.784 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:09.799 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:09.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:09.826 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T17:15:09.836 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T17:15:09.848 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:09.860 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:09.871 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:09.884 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:09.917 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:09.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:09.940 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:09.951 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:09.965 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:09.976 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:09.987 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:09.998 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:10.034 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T17:15:10.049 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T17:15:10.073 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:10.082 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:10.090 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:10.101 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:10.112 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:10.123 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:10.132 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:10.175 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:10.182 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:10.189 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:10.196 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:10.203 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:10.210 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T17:15:10.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T17:15:10.245 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:10.253 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:10.318 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:10.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:10.333 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:10.339 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:10.346 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:10.352 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:10.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:10.365 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:10.410 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:10.417 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:10.423 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T17:15:10.429 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T17:15:10.438 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:10.446 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:10.456 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:10.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:10.484 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:10.500 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:10.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:10.533 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:10.542 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:10.555 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:10.567 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:10.579 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:10.589 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T17:15:10.620 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T17:15:10.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:10.642 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:10.651 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:10.661 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:10.673 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:10.684 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:10.699 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:10.718 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:10.734 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:10.744 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:10.772 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:10.782 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:10.792 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T17:15:10.802 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T17:15:10.811 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:10.823 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:10.831 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:10.899 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:10.908 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:10.916 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:10.926 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:10.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:10.959 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:10.975 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:11.001 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:11.017 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:11.052 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T17:15:11.062 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T17:15:11.073 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:11.084 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:11.095 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:11.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:11.116 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:11.147 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:11.156 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:11.165 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:11.176 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:11.185 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:11.194 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:11.203 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:11.216 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T17:15:11.243 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T17:15:11.259 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:11.284 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:11.294 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:11.303 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:11.311 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:11.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:11.332 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:11.341 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:11.374 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:11.382 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:11.394 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:11.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:11.410 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T17:15:11.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T17:15:11.426 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:11.442 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:11.449 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:11.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:11.501 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:11.511 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:11.521 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:11.530 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:11.540 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:11.550 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:11.560 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:11.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:11.620 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T17:15:11.632 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T17:15:11.642 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:11.652 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:11.663 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:11.674 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T17:15:11.692 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:11.719 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:11.736 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:11.762 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:11.774 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:11.785 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:11.797 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:11.808 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:11.819 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T17:15:11.862 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T17:15:11.873 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:11.883 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:11.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:11.898 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:15:11.905 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:11.912 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:11.919 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:11.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:11.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:11.975 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:12.000 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:12.007 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:12.016 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T17:15:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T17:15:12.033 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:12.061 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:12.069 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:12.077 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-05-14T17:15:12.084 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:12.090 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:12.097 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:12.103 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:12.110 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:12.117 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:12.126 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:12.133 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:12.214 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T17:15:12.256 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T17:15:12.265 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:12.277 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:12.289 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:12.298 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-05-14T17:15:12.344 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:12.356 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:12.367 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:12.379 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:12.392 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:12.404 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:12.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:12.428 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:12.439 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T17:15:12.459 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T17:15:12.498 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:12.539 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:12.573 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:12.595 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:12.606 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:12.670 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:12.681 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:12.691 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:12.702 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:12.711 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:12.721 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:12.732 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T17:15:12.742 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T17:15:12.750 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:12.758 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:12.790 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:12.799 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-05-14T17:15:12.820 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:12.854 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:12.861 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:12.867 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:12.873 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:12.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:12.926 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:12.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:12.943 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T17:15:12.951 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T17:15:12.960 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:12.968 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:12.977 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:12.986 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:12.994 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:13.013 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:13.045 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:13.073 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:13.097 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:13.106 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:13.118 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:13.129 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:13.174 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T17:15:13.182 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T17:15:13.192 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:13.201 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:13.209 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:13.217 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:13.225 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:13.235 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:13.244 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:13.253 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:13.271 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:13.298 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:13.313 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:13.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:13.354 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T17:15:13.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T17:15:13.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:13.410 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:13.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:13.429 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:13.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:13.449 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:13.458 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:13.468 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:13.478 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:13.486 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:13.496 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:13.507 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:13.567 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T17:15:13.605 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T17:15:13.657 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:13.669 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:13.680 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:13.691 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:13.701 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:13.712 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:13.723 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:13.731 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:13.740 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:13.749 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:13.758 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:13.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:13.781 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T17:15:13.799 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T17:15:13.883 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:13.899 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:13.914 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:13.923 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:15:13.933 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:13.942 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:13.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:13.963 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:13.994 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:14.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:14.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:14.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:14.058 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T17:15:14.068 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T17:15:14.077 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:14.085 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:14.095 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:14.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:15:14.125 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:14.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:14.153 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:14.161 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:14.170 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:14.181 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:14.191 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:14.216 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:14.226 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-05-14T17:15:14.249 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-05-14T17:15:14.259 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:14.288 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:14.298 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:14.306 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:14.315 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:14.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:14.341 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:14.351 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:14.386 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:14.398 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:14.410 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:14.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:14.428 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-05-14T17:15:14.437 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-05-14T17:15:14.468 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:14.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:14.488 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:14.518 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:14.528 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:14.537 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:14.547 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:14.558 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:14.569 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:14.583 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:14.593 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:14.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:14.617 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-05-14T17:15:14.627 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-05-14T17:15:14.640 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:14.663 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:14.683 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:14.695 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:14.714 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:14.752 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:14.762 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-05-14T17:15:14.769 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:14.777 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:14.786 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:14.794 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:14.816 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-05-14T17:15:14.823 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-05-14T17:15:14.832 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:14.838 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:14.846 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:14.853 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-05-14T17:15:14.860 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:14.867 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:14.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:14.899 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:14.923 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:14.953 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:14.962 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:14.970 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:14.977 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-05-14T17:15:14.983 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-05-14T17:15:14.991 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:15.000 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:15.007 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:15.026 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:15.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:15.041 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:15.048 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:15.055 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:15.063 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:15.072 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:15.142 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:15.167 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:15.212 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:15.222 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:15.232 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:15.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:15.253 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:15.264 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:15.276 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:15.288 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:15.325 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:15.337 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:15.377 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:15.390 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:15.406 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:15.417 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:15.428 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:15.439 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:15.451 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:15.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:15.507 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:15.520 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:15.528 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:15.537 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:15.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:15.567 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:15.576 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:15.609 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:15.618 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:15.643 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:15.655 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:15.684 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:15.695 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:15.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:15.717 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:15.729 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:15.740 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:15.751 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:15.762 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:15.771 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:15.782 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:15.791 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:15.809 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:15.818 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:15.832 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:15.878 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:15.887 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:15.935 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:15.944 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:15.953 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:15.962 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:15.971 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:15.980 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:15.989 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:15.997 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:16.006 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:16.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:16.027 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:16.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:16.059 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:16.083 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:16.098 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:16.106 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:16.145 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:16.152 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:16.159 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:16.167 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:16.175 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:16.182 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:16.190 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:16.198 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:16.208 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-05-14T17:15:16.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-05-14T17:15:16.226 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:16.237 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:16.251 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:16.262 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:16.274 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:16.307 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:16.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:16.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-05-14T17:15:16.368 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:16.379 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:16.391 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:16.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:16.415 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-05-14T17:15:16.426 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T17:15:16.438 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:16.449 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:16.461 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:16.472 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:16.482 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:16.493 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:16.502 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:16.525 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:16.558 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:16.567 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:16.629 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:16.639 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:16.650 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-05-14T17:15:16.658 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-05-14T17:15:16.670 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:16.682 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:16.690 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:16.701 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-05-14T17:15:16.709 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:16.716 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:16.723 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:16.731 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:16.743 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:16.753 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:16.777 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:16.800 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:16.809 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-05-14T17:15:16.839 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-05-14T17:15:16.850 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:16.860 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:16.870 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:16.879 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-05-14T17:15:16.888 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:16.896 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:16.905 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:16.914 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:16.922 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:16.933 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:16.943 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:16.960 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:16.970 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:16.993 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:17.008 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:17.018 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:17.065 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:17.071 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:17.079 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:17.085 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:17.091 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:17.098 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:17.106 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:17.115 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:17.127 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:17.139 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:17.148 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-05-14T17:15:17.157 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-05-14T17:15:17.168 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:17.179 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:17.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:17.230 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:17.239 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:17.255 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:17.278 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:17.290 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:17.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:17.314 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:17.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:17.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:17.350 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-05-14T17:15:17.361 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-05-14T17:15:17.372 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:17.384 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:17.395 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:17.410 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-05-14T17:15:17.423 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:17.459 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:17.481 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:17.505 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:17.513 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:17.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:17.531 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:17.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:17.551 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-05-14T17:15:17.562 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-05-14T17:15:17.573 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:17.584 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:17.593 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:17.602 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-05-14T17:15:17.609 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:17.616 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:17.628 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:17.636 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:17.673 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-05-14T17:15:17.687 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:17.693 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:17.702 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:17.714 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:17.720 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:17.729 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:17.739 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-05-14T17:15:17.747 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-05-14T17:15:17.756 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:17.764 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:17.774 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:17.781 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-05-14T17:15:17.789 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-05-14T17:15:17.800 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-05-14T17:15:17.811 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:17.821 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:17.846 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:17.890 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:17.903 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:17.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:17.922 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-05-14T17:15:17.931 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-05-14T17:15:17.940 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:17.950 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:17.962 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:17.974 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:17.987 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:17.998 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:18.010 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:18.026 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:18.037 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:18.053 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:18.064 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:18.099 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:18.132 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:18.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:18.168 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-05-14T17:15:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:18.186 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:18.208 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-05-14T17:15:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-05-14T17:15:18.229 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:18.250 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:18.259 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:18.297 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-05-14T17:15:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:18.420 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:18.432 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:18.451 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:18.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:18.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:18.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:18.502 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:18.530 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:18.596 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:18.628 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:18.639 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:18.656 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:18.682 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:18.690 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:18.706 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:18.902 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:18.923 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:18.968 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:18.980 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:18.992 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:19.003 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:19.027 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:19.038 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:19.066 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:19.106 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:19.162 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:19.185 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:19.206 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:19.238 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:19.293 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:19.308 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:19.390 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:19.398 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:19.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:19.423 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:19.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:19.442 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-05-14T17:15:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-05-14T17:15:19.458 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:19.466 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:19.485 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-05-14T17:15:19.494 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:19.502 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:19.516 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:19.557 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T17:15:19.577 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:19.586 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:19.595 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-05-14T17:15:19.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-05-14T17:15:19.631 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:19.640 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:19.650 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-05-14T17:15:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:19.678 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T17:15:19.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-05-14T17:15:19.816 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-05-14T17:15:19.824 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:19.833 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:19.843 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-05-14T17:15:19.862 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:19.870 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:19.890 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-05-14T17:15:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:19.916 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:19.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-05-14T17:15:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-05-14T17:15:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:20.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:20.074 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-05-14T17:15:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:20.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T17:15:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:20.155 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:20.166 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:20.179 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-05-14T17:15:20.190 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-05-14T17:15:20.201 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:20.217 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:20.267 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:20.295 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:20.306 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:20.314 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:20.323 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:20.331 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T17:15:20.342 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:20.350 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:20.358 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:20.365 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:20.372 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:15:20.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:15:20.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:20.395 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:20.403 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:20.410 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:15:20.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:20.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:20.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:20.491 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T17:15:20.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:20.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:20.513 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:20.521 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:20.529 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:20.536 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:20.542 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:20.549 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:20.556 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:20.563 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:20.571 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:20.580 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:20.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:20.601 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:20.643 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:20.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:20.684 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:20.696 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:20.710 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-05-14T17:15:20.721 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-05-14T17:15:20.736 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:20.749 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:20.761 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:20.773 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-05-14T17:15:20.782 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:20.791 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:20.801 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:20.813 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T17:15:20.854 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:20.865 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:20.900 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:20.910 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:20.958 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:20.969 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:20.979 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:20.990 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-05-14T17:15:20.999 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:21.010 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:21.021 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:21.031 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:21.061 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:21.073 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:21.081 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:21.090 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:21.100 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:21.109 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:21.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:21.154 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:21.165 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:21.176 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:21.186 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:21.195 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:21.205 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:21.215 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:21.225 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:21.233 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:21.264 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:21.275 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:21.284 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:21.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:21.300 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:21.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:21.319 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:21.349 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:21.383 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:21.391 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:21.409 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:21.419 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:21.428 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:21.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:21.474 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:21.482 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:21.491 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:21.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:21.510 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:21.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:21.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:21.531 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:21.538 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:21.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:21.572 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:21.584 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:21.596 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:21.623 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:21.635 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:21.647 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:21.659 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:21.704 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:21.717 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:21.729 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:21.740 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:21.752 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:21.762 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:21.774 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:21.784 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:21.797 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:21.811 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:21.823 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:21.857 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:21.868 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:21.883 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:21.911 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:21.946 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:21.957 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:21.968 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:21.976 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:21.984 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:21.996 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:22.004 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:22.012 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-05-14T17:15:22.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-05-14T17:15:22.028 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:22.048 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:22.055 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:22.062 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-05-14T17:15:22.082 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:22.090 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:22.106 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:22.114 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:22.152 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:22.161 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:22.169 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:22.175 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:22.182 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-05-14T17:15:22.189 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-05-14T17:15:22.199 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:22.207 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:22.233 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:22.242 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-05-14T17:15:22.252 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:22.264 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:22.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:22.290 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:22.300 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:22.323 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:22.356 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:22.368 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:22.381 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:15:22.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:15:22.407 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:22.416 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:22.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:22.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-05-14T17:15:22.479 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:22.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:22.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:22.509 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:22.518 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:22.528 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:22.550 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:22.560 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:22.609 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:22.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:22.631 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:22.640 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:22.649 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:22.672 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:22.681 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:22.690 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:22.698 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:22.734 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:22.743 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:22.752 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:22.762 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:22.772 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:22.784 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:22.797 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:22.808 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:22.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-05-14T17:15:22.854 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-05-14T17:15:22.865 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-05-14T17:15:22.876 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:22.887 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-05-14T17:15:22.897 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-05-14T17:15:22.906 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:22.915 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:22.963 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:22.972 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-05-14T17:15:22.981 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:22.989 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:22.998 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:23.008 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T17:15:23.019 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:23.046 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:23.055 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:23.086 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:23.094 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-05-14T17:15:23.102 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-05-14T17:15:23.109 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:23.116 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:23.133 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-05-14T17:15:23.141 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:23.190 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:23.198 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:23.206 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T17:15:23.215 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:23.224 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:23.232 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:23.238 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:23.260 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-05-14T17:15:23.267 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-05-14T17:15:23.304 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:23.314 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:23.323 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:23.332 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:23.343 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:23.356 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:23.368 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:23.379 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-05-14T17:15:23.402 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:23.413 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:23.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:23.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:23.448 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:15:23.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:15:23.471 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:23.514 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:23.529 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:23.549 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-05-14T17:15:23.562 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-05-14T17:15:23.571 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:23.580 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:23.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T17:15:23.599 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:23.610 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:23.641 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:23.649 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:23.658 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:23.669 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:23.677 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:23.685 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:23.695 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:23.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:23.727 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:23.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:23.787 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:23.797 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:23.806 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:23.815 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:23.827 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:23.835 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:23.878 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:23.885 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:23.896 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:23.906 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:23.915 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:23.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:23.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:23.941 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:23.950 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:23.969 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:24.009 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:24.064 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:24.074 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:24.084 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:24.093 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:24.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:24.110 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:24.160 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:24.169 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:24.177 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:24.187 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:24.195 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:24.205 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:24.492 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:24.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:24.561 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:24.569 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:24.591 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:24.604 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:24.614 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:24.623 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:24.635 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:24.644 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:24.654 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:24.664 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:24.694 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:24.706 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:24.718 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:24.731 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:24.743 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:24.755 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:24.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:24.776 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:24.794 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:24.834 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:24.863 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:24.874 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:24.885 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:24.896 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:24.908 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:24.919 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:24.961 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:24.972 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:24.981 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:24.991 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:24.999 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:25.007 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-05-14T17:15:25.015 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-05-14T17:15:25.022 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:25.030 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:25.061 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:25.069 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:25.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:25.098 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:25.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:25.116 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-05-14T17:15:25.122 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:25.129 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:25.153 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:25.161 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:25.169 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-05-14T17:15:25.177 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-05-14T17:15:25.185 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:25.191 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:25.198 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:25.205 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-05-14T17:15:25.243 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:25.253 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:25.260 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:25.285 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-05-14T17:15:25.291 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:25.298 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:25.305 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:25.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:25.319 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:25.353 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:25.366 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:25.377 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:25.389 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:25.400 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:25.411 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:25.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:25.432 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:25.475 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:25.487 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:25.497 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:25.541 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:25.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:25.564 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-05-14T17:15:25.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-05-14T17:15:25.581 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:25.602 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:25.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:25.620 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-05-14T17:15:25.629 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:25.642 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:25.649 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:25.656 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:15:25.663 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:25.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:25.706 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:25.720 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:25.728 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-05-14T17:15:25.741 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-05-14T17:15:25.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:25.758 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:25.765 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:25.786 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:15:25.798 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:15:25.808 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-05-14T17:15:25.816 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:25.827 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T17:15:25.837 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:25.846 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:25.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:25.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:25.910 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-05-14T17:15:25.918 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-05-14T17:15:25.929 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:25.938 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:25.958 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:25.967 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:26.004 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:26.013 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:26.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:26.033 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-05-14T17:15:26.045 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:26.057 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:26.069 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:26.080 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:26.109 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-05-14T17:15:26.119 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-05-14T17:15:26.133 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:26.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:26.156 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:26.204 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-05-14T17:15:26.218 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:26.264 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:26.273 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:26.282 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:15:26.290 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:26.300 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:26.308 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:26.317 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:26.326 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:26.363 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:26.372 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:26.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-05-14T17:15:26.391 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:26.401 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:26.413 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:26.423 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:26.452 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:26.476 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:26.497 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:26.508 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:26.519 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:26.531 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:26.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:26.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:26.587 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:26.596 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:26.606 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:26.615 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:26.624 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:26.632 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:26.642 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:26.650 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:26.660 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:26.679 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:26.687 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:26.705 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:26.713 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:26.720 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:26.728 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:26.734 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:26.741 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:26.779 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:26.786 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:26.794 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:26.803 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:26.817 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:26.826 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:26.833 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:26.841 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:26.855 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:26.895 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:26.914 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:26.923 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:26.934 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:26.944 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:26.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:26.962 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:26.999 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:27.010 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:27.020 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:27.030 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:27.041 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:27.052 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:27.064 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:27.074 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:27.086 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:27.099 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:27.139 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:27.168 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:27.178 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:27.189 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:27.198 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:27.207 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:27.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:27.251 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:27.262 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:27.271 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:27.280 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:27.288 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:27.297 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:27.305 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:27.313 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:27.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:27.330 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:27.376 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:27.397 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:27.406 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:27.415 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:27.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:27.431 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:27.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:27.471 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:27.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:27.487 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:27.499 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:27.507 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:27.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:27.524 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:27.534 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:27.544 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:27.555 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:27.583 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-05-14T17:15:27.595 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:27.603 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:27.613 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:27.623 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:27.653 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:27.662 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:27.672 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:27.681 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:27.691 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:27.700 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:27.710 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:27.719 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:27.728 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:27.736 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:27.745 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:27.764 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:27.794 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:27.820 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:27.828 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:27.834 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:27.841 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:27.880 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:27.887 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:27.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:27.911 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:27.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:27.930 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:27.939 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:27.949 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:27.962 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:27.973 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:27.982 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:28.053 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:28.066 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:28.080 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:28.090 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:28.101 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:28.111 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:28.142 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:28.153 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:28.166 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:28.179 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:28.189 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:28.200 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:28.211 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:28.222 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:28.233 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:28.244 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:28.255 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:28.304 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:28.315 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:28.341 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:28.352 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:28.368 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:28.379 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:28.449 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:28.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:28.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:28.470 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:28.476 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:28.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:28.491 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:28.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:28.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:28.512 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:28.519 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:28.562 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:28.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:28.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:28.605 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:28.614 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:28.622 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:28.657 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:28.665 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:28.673 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:28.680 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:28.688 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:28.696 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:28.703 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:28.710 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:28.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:28.729 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:28.738 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:28.771 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:28.779 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:28.796 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:28.805 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:28.814 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:28.823 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:28.861 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:28.872 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-05-14T17:15:28.883 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:28.894 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:28.908 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:28.920 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-05-14T17:15:28.933 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-05-14T17:15:28.945 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:28.954 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:28.964 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:28.973 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-05-14T17:15:29.003 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:29.011 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:29.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:29.035 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T17:15:29.047 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:29.074 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:29.086 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:29.098 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:29.110 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-05-14T17:15:29.118 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-05-14T17:15:29.127 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:29.136 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:29.146 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:29.155 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:15:29.164 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-05-14T17:15:29.173 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:29.182 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:29.230 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T17:15:29.239 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:29.283 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:29.294 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:29.330 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:29.339 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:29.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:29.357 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:29.368 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:29.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:29.390 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:29.397 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:29.407 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:29.415 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:29.423 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:29.432 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:29.441 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:29.485 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:29.494 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:29.515 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:29.524 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:29.595 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:29.605 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:29.614 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:29.623 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:29.631 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:29.640 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:29.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:29.657 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:29.665 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:29.674 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:29.682 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:29.696 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:29.718 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:29.750 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:29.762 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:29.790 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:29.815 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:29.823 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:29.831 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:29.842 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:29.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:29.866 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:29.877 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:29.889 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:29.902 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:29.914 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:29.926 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:29.935 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:29.965 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:29.977 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:29.992 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:30.032 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:30.048 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:30.059 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:30.072 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:30.084 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:30.097 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:30.108 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:30.118 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:30.129 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:30.140 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:30.151 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:30.163 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:30.189 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:30.201 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:30.235 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:30.246 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:30.271 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:30.285 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:30.302 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:30.312 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:30.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:30.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:30.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:30.352 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:30.361 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:30.370 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:30.409 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:30.416 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:30.423 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:30.430 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:30.458 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:30.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:30.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:30.493 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:30.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:30.524 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:30.534 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:30.543 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:30.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:30.559 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:30.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-05-14T17:15:30.573 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:30.580 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:30.617 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:30.625 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-05-14T17:15:30.633 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-05-14T17:15:30.662 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:30.673 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:30.681 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:30.689 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-05-14T17:15:30.701 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:30.708 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:30.724 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:30.734 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-05-14T17:15:30.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T17:15:30.754 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:30.765 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:30.784 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:30.793 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:30.804 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:30.858 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:30.868 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:30.878 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:30.918 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:30.931 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:30.943 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:30.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:30.968 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:30.981 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:31.008 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:31.017 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:31.026 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:31.035 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:31.044 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:31.053 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:31.061 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:31.102 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:31.111 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:31.120 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:31.151 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:31.160 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:31.171 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:31.182 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:31.192 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:31.203 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:31.214 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:31.232 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:31.241 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:31.252 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:31.260 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:31.267 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:31.277 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:31.284 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:31.305 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:31.342 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:31.381 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:31.391 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:31.398 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:31.407 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:31.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:31.425 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:31.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:31.461 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:31.513 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:31.524 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:31.535 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:31.547 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:31.589 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:31.631 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:31.642 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:31.651 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:31.660 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:31.670 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:31.679 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:31.688 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:31.696 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:31.705 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:31.714 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:31.743 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:31.757 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:31.775 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:31.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:31.821 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:31.834 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:31.846 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:31.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:31.870 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:31.886 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:31.898 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-05-14T17:15:31.907 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-05-14T17:15:31.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:31.925 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:31.937 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:31.949 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:31.960 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:31.978 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:31.990 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:32.032 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-05-14T17:15:32.044 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T17:15:32.054 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:32.068 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:32.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:32.091 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-05-14T17:15:32.102 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-05-14T17:15:32.113 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:32.124 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:32.133 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:32.142 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-05-14T17:15:32.153 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:32.162 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:32.172 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:32.182 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-05-14T17:15:32.207 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:32.254 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:32.283 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:32.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:32.302 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-05-14T17:15:32.311 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-05-14T17:15:32.321 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:32.330 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:32.342 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:32.351 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-05-14T17:15:32.360 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:32.368 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:32.375 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:32.382 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T17:15:32.390 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:32.396 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:32.403 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:32.430 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:32.488 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-05-14T17:15:32.530 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-05-14T17:15:32.538 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:32.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:32.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:32.557 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:32.565 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:32.573 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:32.582 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:32.591 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-05-14T17:15:32.601 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:32.611 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:32.623 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:32.632 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:32.642 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:32.651 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:32.704 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:32.748 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:32.772 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:32.784 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:32.796 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:32.806 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:32.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:32.830 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:32.841 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:32.852 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:32.866 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:32.877 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:32.914 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-05-14T17:15:32.923 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-05-14T17:15:32.934 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:32.944 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:32.977 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:33.012 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-05-14T17:15:33.025 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:33.038 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:33.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:33.063 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T17:15:33.073 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T17:15:33.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:33.094 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:33.105 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:33.137 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:33.146 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:33.154 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:33.196 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-05-14T17:15:33.206 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:33.216 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:33.226 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:33.232 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:33.246 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:33.253 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:33.270 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:33.277 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:33.285 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:33.293 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:33.300 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:33.309 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:33.316 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:33.349 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:33.356 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:33.362 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:33.403 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:33.410 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:33.417 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:33.422 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:33.431 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:33.488 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:33.495 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:33.505 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:33.514 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:33.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:33.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:33.543 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:33.553 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:33.566 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:33.640 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:33.652 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:33.665 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:33.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:33.727 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:33.740 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:33.752 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:33.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:33.774 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:33.786 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:33.802 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:33.814 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:33.825 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:33.835 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:33.846 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:33.856 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:33.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:33.895 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:33.905 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:33.916 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:33.956 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:33.964 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:33.975 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:33.986 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:33.995 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:34.005 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:34.016 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:34.053 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:34.063 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:34.071 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:34.081 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:34.090 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:34.098 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:34.134 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:34.141 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:34.150 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:34.158 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:34.213 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:34.225 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:34.234 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:34.242 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:34.251 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:34.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:34.290 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:34.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:34.358 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:34.368 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:34.377 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:34.426 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:34.436 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:34.445 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:34.453 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:34.487 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:34.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:34.513 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:34.540 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:34.556 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:34.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-05-14T17:15:34.598 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:34.612 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:34.624 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:34.655 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:34.663 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:34.673 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:34.682 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:34.717 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:34.726 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:34.746 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:34.755 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:34.764 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:34.774 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:34.783 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:34.796 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:34.815 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:34.826 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:34.870 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:34.885 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:34.896 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:34.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:34.929 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:34.954 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:34.965 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:34.979 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:34.991 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:35.002 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:35.013 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:35.024 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:35.065 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:35.075 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:35.120 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:35.133 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:35.141 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:35.151 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:35.157 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:35.165 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:35.172 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:35.239 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:35.246 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:35.255 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:35.262 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:35.270 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:35.317 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:35.324 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:35.344 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:35.351 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:35.386 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:35.393 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:35.399 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:35.406 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:35.414 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:35.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:35.456 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:35.464 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:35.473 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:35.479 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:35.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:35.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:35.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:35.520 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:35.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:35.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:35.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:35.612 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:35.623 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:35.634 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:35.646 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:35.680 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:35.693 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:35.706 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:35.715 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:35.725 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:35.734 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:35.744 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:35.776 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:35.798 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:35.834 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:35.846 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:35.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:35.865 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:35.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-05-14T17:15:35.920 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:35.929 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:35.938 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:35.949 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:35.957 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:35.968 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:35.979 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:35.990 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:36.001 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:36.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:36.041 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:36.073 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:36.083 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:36.092 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:36.101 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:36.111 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:36.120 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:36.145 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:36.154 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:36.163 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:36.171 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:36.181 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:36.190 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:36.208 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:36.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:36.243 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:36.252 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:36.289 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:36.309 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:36.317 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:36.334 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:36.362 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:36.371 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:36.381 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:36.390 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:36.399 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:36.406 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:36.413 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:36.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:36.427 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:36.443 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:36.453 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:36.495 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:36.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:36.516 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:36.529 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:36.541 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:36.606 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:36.618 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:36.630 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:36.643 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:36.655 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:36.667 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:36.717 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:36.729 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:36.740 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:36.788 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:36.890 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:36.951 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:36.962 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:36.973 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:36.985 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:36.999 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.007 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.013 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.056 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.062 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.069 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.076 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.084 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.092 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.107 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.113 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.165 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.173 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.178 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.187 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.195 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.202 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.209 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.216 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.266 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.273 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-05-14T17:15:37.280 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.286 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.293 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.318 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-05-14T17:15:37.326 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-05-14T17:15:37.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.362 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.403 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.414 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.415 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.417 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.418 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-05-14T17:15:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.422 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.423 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.424 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.426 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.428 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.429 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.430 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.431 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.431 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.431 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.431 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.431 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.431 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.431 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.431 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.432 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.433 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.434 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.435 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.437 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.438 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.439 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.440 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-05-14T17:15:37.441 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.442 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.444 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.445 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.446 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.447 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.448 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-05-14T17:15:37.449 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.451 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-05-14T17:15:37.452 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-05-14T17:15:37.453 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.454 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.455 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.457 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.459 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-05-14T17:15:37.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.461 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T17:15:37.462 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.463 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.465 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.466 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.467 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.468 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.470 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.471 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.472 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.474 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.476 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.477 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.479 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.480 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.482 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-05-14T17:15:37.483 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.485 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-05-14T17:15:37.486 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.487 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.488 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.490 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.492 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.493 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.494 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.495 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.496 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.497 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-05-14T17:15:37.498 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-05-14T17:15:37.499 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.500 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.501 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.502 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.503 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.505 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.506 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.506 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.506 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.506 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.506 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.506 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.506 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-05-14T17:15:37.506 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.507 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.509 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.510 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-05-14T17:15:37.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.512 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-05-14T17:15:37.513 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.515 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.516 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.517 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smb", 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.518 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.519 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.521 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.522 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.523 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "update_orchestration": { 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "update_orchestration", 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.524 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.526 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.529 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.530 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.531 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.534 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.535 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.536 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.537 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.538 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-05-14T17:15:37.539 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.540 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.541 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.542 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.543 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.544 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.546 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.548 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.550 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.552 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.553 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.554 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.555 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-05-14T17:15:37.556 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-05-14T17:15:37.557 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.559 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T17:15:37.560 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.561 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.562 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.563 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.566 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.567 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.568 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.569 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-05-14T17:15:37.570 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.571 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T17:15:37.573 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.574 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.575 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.576 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.577 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.578 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.579 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.580 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.581 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.582 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.583 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.584 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.585 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-05-14T17:15:37.586 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-05-14T17:15:37.587 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T17:15:37.588 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.589 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.590 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.591 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.592 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.593 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.594 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.595 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.596 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.597 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:37.598 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-05-14T17:15:37.599 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-05-14T17:15:37.600 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-05-14T17:15:37.601 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.602 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "services": { 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard": "https://172.21.15.69:8443/" 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "always_on_modules": { 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "octopus": [ 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-05-14T17:15:37.603 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "pacific": [ 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.604 INFO:teuthology.orchestra.run.smithi069.stdout: "quincy": [ 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "reef": [ 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-05-14T17:15:37.605 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "squid": [ 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-05-14T17:15:37.606 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: "last_failure_osd_epoch": 81, 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: "active_clients": [ 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4155567826 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:15:37.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "libcephsqlite", 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1429809741 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-05-14T17:15:37.608 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1299669703 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1270661239 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-05-14T17:15:37.609 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2576160514 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 3333248468 2024-05-14T17:15:37.610 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.611 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:15:37.611 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:37.611 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-14T17:15:37.611 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:15:37.618 DEBUG:tasks.check_counter:Getting stats from a 2024-05-14T17:15:37.618 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-14T17:15:37.651 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:36 smithi092 ceph-mon[35763]: pgmap v690: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-14T17:15:37.651 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:36 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3409620834' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:38 smithi092 ceph-mon[35763]: pgmap v691: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:40 smithi092 ceph-mon[35763]: pgmap v692: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:42 smithi092 ceph-mon[35763]: pgmap v693: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:44 smithi092 ceph-mon[35763]: pgmap v694: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:46 smithi092 ceph-mon[35763]: pgmap v695: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:48 smithi092 ceph-mon[35763]: pgmap v696: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:49 smithi092 ceph-mon[35763]: pgmap v697: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:52 smithi092 ceph-mon[35763]: pgmap v698: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:54 smithi092 ceph-mon[35763]: pgmap v699: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:56 smithi092 ceph-mon[35763]: pgmap v700: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:11:58 smithi092 ceph-mon[35763]: pgmap v701: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:00 smithi092 ceph-mon[35763]: pgmap v702: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:02 smithi092 ceph-mon[35763]: pgmap v703: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:04 smithi092 ceph-mon[35763]: pgmap v704: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:06 smithi092 ceph-mon[35763]: pgmap v705: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.652 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:08 smithi092 ceph-mon[35763]: pgmap v706: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:10 smithi092 ceph-mon[35763]: pgmap v707: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:10 smithi092 ceph-mon[35763]: osdmap e117: 12 total, 12 up, 12 in 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:12 smithi092 ceph-mon[35763]: pgmap v709: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:14 smithi092 ceph-mon[35763]: pgmap v710: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:14 smithi092 ceph-mon[35763]: osdmap e118: 12 total, 12 up, 12 in 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:16 smithi092 ceph-mon[35763]: pgmap v712: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:18 smithi092 ceph-mon[35763]: pgmap v713: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:20 smithi092 ceph-mon[35763]: pgmap v714: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:22 smithi092 ceph-mon[35763]: pgmap v715: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:24 smithi092 ceph-mon[35763]: pgmap v716: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:26 smithi092 ceph-mon[35763]: pgmap v717: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:28 smithi092 ceph-mon[35763]: pgmap v718: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:30 smithi092 ceph-mon[35763]: pgmap v719: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:32 smithi092 ceph-mon[35763]: pgmap v720: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:32 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.653 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:32 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:32 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:32 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:33 smithi092 ceph-mon[35763]: pgmap v721: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:33 smithi092 ceph-mon[35763]: pgmap v722: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:35 smithi092 ceph-mon[35763]: pgmap v723: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:37 smithi092 ceph-mon[35763]: pgmap v724: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:39 smithi092 ceph-mon[35763]: pgmap v725: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:41 smithi092 ceph-mon[35763]: pgmap v726: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:43 smithi092 ceph-mon[35763]: pgmap v727: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:45 smithi092 ceph-mon[35763]: pgmap v728: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:47 smithi092 ceph-mon[35763]: pgmap v729: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:49 smithi092 ceph-mon[35763]: pgmap v730: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:51 smithi092 ceph-mon[35763]: pgmap v731: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:53 smithi092 ceph-mon[35763]: pgmap v732: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.654 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:55 smithi092 ceph-mon[35763]: pgmap v733: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:57 smithi092 ceph-mon[35763]: pgmap v734: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:12:59 smithi092 ceph-mon[35763]: pgmap v735: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:01 smithi092 ceph-mon[35763]: pgmap v736: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:03 smithi092 ceph-mon[35763]: pgmap v737: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:05 smithi092 ceph-mon[35763]: pgmap v738: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:07 smithi092 ceph-mon[35763]: pgmap v739: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:09 smithi092 ceph-mon[35763]: pgmap v740: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:10 smithi092 ceph-mon[35763]: osdmap e119: 12 total, 12 up, 12 in 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:11 smithi092 ceph-mon[35763]: pgmap v742: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:13 smithi092 ceph-mon[35763]: pgmap v743: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:15 smithi092 ceph-mon[35763]: pgmap v744: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:17 smithi092 ceph-mon[35763]: pgmap v745: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:19 smithi092 ceph-mon[35763]: pgmap v746: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:19 smithi092 ceph-mon[35763]: osdmap e120: 12 total, 12 up, 12 in 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:21 smithi092 ceph-mon[35763]: pgmap v748: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:23 smithi092 ceph-mon[35763]: pgmap v749: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:25 smithi092 ceph-mon[35763]: pgmap v750: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:27 smithi092 ceph-mon[35763]: pgmap v751: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:29 smithi092 ceph-mon[35763]: pgmap v752: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:31 smithi092 ceph-mon[35763]: pgmap v753: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:32 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:33 smithi092 ceph-mon[35763]: pgmap v754: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:33 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:33 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:33 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:34 smithi092 ceph-mon[35763]: pgmap v755: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:34 smithi092 ceph-mon[35763]: pgmap v756: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:36 smithi092 ceph-mon[35763]: pgmap v757: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:38 smithi092 ceph-mon[35763]: pgmap v758: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:40 smithi092 ceph-mon[35763]: pgmap v759: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:42 smithi092 ceph-mon[35763]: pgmap v760: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:44 smithi092 ceph-mon[35763]: pgmap v761: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:46 smithi092 ceph-mon[35763]: pgmap v762: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.656 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:48 smithi092 ceph-mon[35763]: pgmap v763: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:50 smithi092 ceph-mon[35763]: pgmap v764: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:52 smithi092 ceph-mon[35763]: pgmap v765: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:54 smithi092 ceph-mon[35763]: pgmap v766: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:56 smithi092 ceph-mon[35763]: pgmap v767: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:13:58 smithi092 ceph-mon[35763]: pgmap v768: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:00 smithi092 ceph-mon[35763]: pgmap v769: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:02 smithi092 ceph-mon[35763]: pgmap v770: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:04 smithi092 ceph-mon[35763]: pgmap v771: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:06 smithi092 ceph-mon[35763]: pgmap v772: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:08 smithi092 ceph-mon[35763]: pgmap v773: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:10 smithi092 ceph-mon[35763]: pgmap v774: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:10 smithi092 ceph-mon[35763]: osdmap e121: 12 total, 12 up, 12 in 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:12 smithi092 ceph-mon[35763]: pgmap v776: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:14 smithi092 ceph-mon[35763]: pgmap v777: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.657 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:14 smithi092 ceph-mon[35763]: osdmap e122: 12 total, 12 up, 12 in 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:16 smithi092 ceph-mon[35763]: pgmap v779: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:18 smithi092 ceph-mon[35763]: pgmap v780: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:20 smithi092 ceph-mon[35763]: pgmap v781: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:22 smithi092 ceph-mon[35763]: pgmap v782: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:24 smithi092 ceph-mon[35763]: pgmap v783: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:26 smithi092 ceph-mon[35763]: pgmap v784: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:28 smithi092 ceph-mon[35763]: pgmap v785: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:30 smithi092 ceph-mon[35763]: pgmap v786: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:32 smithi092 ceph-mon[35763]: pgmap v787: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:33 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:34 smithi092 ceph-mon[35763]: pgmap v788: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:34 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:34 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:37.658 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:34 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:35 smithi092 ceph-mon[35763]: pgmap v789: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:35 smithi092 ceph-mon[35763]: pgmap v790: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:37 smithi092 ceph-mon[35763]: pgmap v791: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:39 smithi092 ceph-mon[35763]: pgmap v792: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:41 smithi092 ceph-mon[35763]: pgmap v793: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:43 smithi092 ceph-mon[35763]: pgmap v794: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:45 smithi092 ceph-mon[35763]: pgmap v795: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:47 smithi092 ceph-mon[35763]: pgmap v796: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:49 smithi092 ceph-mon[35763]: pgmap v797: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:51 smithi092 ceph-mon[35763]: pgmap v798: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:53 smithi092 ceph-mon[35763]: pgmap v799: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:55 smithi092 ceph-mon[35763]: pgmap v800: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:15:37.659 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:57 smithi092 ceph-mon[35763]: pgmap v801: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:14:59 smithi092 ceph-mon[35763]: pgmap v802: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:01 smithi092 ceph-mon[35763]: pgmap v803: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:03 smithi092 ceph-mon[35763]: pgmap v804: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:05 smithi092 ceph-mon[35763]: pgmap v805: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:07 smithi092 ceph-mon[35763]: pgmap v806: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:09 smithi092 ceph-mon[35763]: pgmap v807: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:10 smithi092 ceph-mon[35763]: osdmap e123: 12 total, 12 up, 12 in 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:11 smithi092 ceph-mon[35763]: pgmap v809: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:13 smithi092 ceph-mon[35763]: pgmap v810: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:15 smithi092 ceph-mon[35763]: pgmap v811: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:17 smithi092 ceph-mon[35763]: pgmap v812: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:19 smithi092 ceph-mon[35763]: pgmap v813: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:19 smithi092 ceph-mon[35763]: osdmap e124: 12 total, 12 up, 12 in 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:21 smithi092 ceph-mon[35763]: pgmap v815: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:23 smithi092 ceph-mon[35763]: pgmap v816: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.660 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:25 smithi092 ceph-mon[35763]: pgmap v817: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:27 smithi092 ceph-mon[35763]: pgmap v818: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:29 smithi092 ceph-mon[35763]: pgmap v819: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:31 smithi092 ceph-mon[35763]: pgmap v820: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:33 smithi092 ceph-mon[35763]: pgmap v821: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:34 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:35 smithi092 ceph-mon[35763]: pgmap v822: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:36 smithi092 ceph-mon[35763]: pgmap v823: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:36 smithi203 ceph-mon[35787]: pgmap v690: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:36 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3409620834' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:38 smithi203 ceph-mon[35787]: pgmap v691: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:40 smithi203 ceph-mon[35787]: pgmap v692: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:42 smithi203 ceph-mon[35787]: pgmap v693: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:44 smithi203 ceph-mon[35787]: pgmap v694: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-14T17:15:37.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:46 smithi203 ceph-mon[35787]: pgmap v695: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:48 smithi203 ceph-mon[35787]: pgmap v696: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:49 smithi203 ceph-mon[35787]: pgmap v697: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:52 smithi203 ceph-mon[35787]: pgmap v698: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:54 smithi203 ceph-mon[35787]: pgmap v699: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:56 smithi203 ceph-mon[35787]: pgmap v700: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:11:58 smithi203 ceph-mon[35787]: pgmap v701: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:00 smithi203 ceph-mon[35787]: pgmap v702: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:02 smithi203 ceph-mon[35787]: pgmap v703: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:04 smithi203 ceph-mon[35787]: pgmap v704: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:06 smithi203 ceph-mon[35787]: pgmap v705: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:08 smithi203 ceph-mon[35787]: pgmap v706: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:10 smithi203 ceph-mon[35787]: pgmap v707: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:10 smithi203 ceph-mon[35787]: osdmap e117: 12 total, 12 up, 12 in 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:12 smithi203 ceph-mon[35787]: pgmap v709: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-05-14T17:15:37.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:14 smithi203 ceph-mon[35787]: pgmap v710: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:14 smithi203 ceph-mon[35787]: osdmap e118: 12 total, 12 up, 12 in 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:16 smithi203 ceph-mon[35787]: pgmap v712: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:18 smithi203 ceph-mon[35787]: pgmap v713: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:20 smithi203 ceph-mon[35787]: pgmap v714: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:22 smithi203 ceph-mon[35787]: pgmap v715: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:24 smithi203 ceph-mon[35787]: pgmap v716: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:26 smithi203 ceph-mon[35787]: pgmap v717: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:28 smithi203 ceph-mon[35787]: pgmap v718: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:30 smithi203 ceph-mon[35787]: pgmap v719: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:32 smithi203 ceph-mon[35787]: pgmap v720: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:32 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:32 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:32 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:32 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:33 smithi203 ceph-mon[35787]: pgmap v721: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:15:37.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:33 smithi203 ceph-mon[35787]: pgmap v722: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:35 smithi203 ceph-mon[35787]: pgmap v723: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:37 smithi203 ceph-mon[35787]: pgmap v724: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:39 smithi203 ceph-mon[35787]: pgmap v725: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:41 smithi203 ceph-mon[35787]: pgmap v726: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:43 smithi203 ceph-mon[35787]: pgmap v727: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:45 smithi203 ceph-mon[35787]: pgmap v728: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:47 smithi203 ceph-mon[35787]: pgmap v729: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:49 smithi203 ceph-mon[35787]: pgmap v730: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:51 smithi203 ceph-mon[35787]: pgmap v731: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:53 smithi203 ceph-mon[35787]: pgmap v732: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:55 smithi203 ceph-mon[35787]: pgmap v733: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:57 smithi203 ceph-mon[35787]: pgmap v734: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:12:59 smithi203 ceph-mon[35787]: pgmap v735: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:01 smithi203 ceph-mon[35787]: pgmap v736: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:03 smithi203 ceph-mon[35787]: pgmap v737: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:05 smithi203 ceph-mon[35787]: pgmap v738: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:07 smithi203 ceph-mon[35787]: pgmap v739: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:09 smithi203 ceph-mon[35787]: pgmap v740: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:10 smithi203 ceph-mon[35787]: osdmap e119: 12 total, 12 up, 12 in 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:11 smithi203 ceph-mon[35787]: pgmap v742: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:13 smithi203 ceph-mon[35787]: pgmap v743: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:15 smithi203 ceph-mon[35787]: pgmap v744: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:17 smithi203 ceph-mon[35787]: pgmap v745: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:19 smithi203 ceph-mon[35787]: pgmap v746: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:19 smithi203 ceph-mon[35787]: osdmap e120: 12 total, 12 up, 12 in 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:21 smithi203 ceph-mon[35787]: pgmap v748: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:23 smithi203 ceph-mon[35787]: pgmap v749: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:25 smithi203 ceph-mon[35787]: pgmap v750: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:27 smithi203 ceph-mon[35787]: pgmap v751: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.665 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:29 smithi203 ceph-mon[35787]: pgmap v752: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:31 smithi203 ceph-mon[35787]: pgmap v753: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:32 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:33 smithi203 ceph-mon[35787]: pgmap v754: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:33 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:33 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:33 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:34 smithi203 ceph-mon[35787]: pgmap v755: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:34 smithi203 ceph-mon[35787]: pgmap v756: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:36 smithi203 ceph-mon[35787]: pgmap v757: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:38 smithi203 ceph-mon[35787]: pgmap v758: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:40 smithi203 ceph-mon[35787]: pgmap v759: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:42 smithi203 ceph-mon[35787]: pgmap v760: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:44 smithi203 ceph-mon[35787]: pgmap v761: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:46 smithi203 ceph-mon[35787]: pgmap v762: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:48 smithi203 ceph-mon[35787]: pgmap v763: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.666 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:50 smithi203 ceph-mon[35787]: pgmap v764: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:52 smithi203 ceph-mon[35787]: pgmap v765: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:54 smithi203 ceph-mon[35787]: pgmap v766: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:56 smithi203 ceph-mon[35787]: pgmap v767: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:13:58 smithi203 ceph-mon[35787]: pgmap v768: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:00 smithi203 ceph-mon[35787]: pgmap v769: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:02 smithi203 ceph-mon[35787]: pgmap v770: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:04 smithi203 ceph-mon[35787]: pgmap v771: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:06 smithi203 ceph-mon[35787]: pgmap v772: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:08 smithi203 ceph-mon[35787]: pgmap v773: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:10 smithi203 ceph-mon[35787]: pgmap v774: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:10 smithi203 ceph-mon[35787]: osdmap e121: 12 total, 12 up, 12 in 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:12 smithi203 ceph-mon[35787]: pgmap v776: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:14 smithi203 ceph-mon[35787]: pgmap v777: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:14 smithi203 ceph-mon[35787]: osdmap e122: 12 total, 12 up, 12 in 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:16 smithi203 ceph-mon[35787]: pgmap v779: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:18 smithi203 ceph-mon[35787]: pgmap v780: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:15:37.667 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:20 smithi203 ceph-mon[35787]: pgmap v781: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:22 smithi203 ceph-mon[35787]: pgmap v782: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:24 smithi203 ceph-mon[35787]: pgmap v783: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:26 smithi203 ceph-mon[35787]: pgmap v784: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:28 smithi203 ceph-mon[35787]: pgmap v785: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:30 smithi203 ceph-mon[35787]: pgmap v786: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:32 smithi203 ceph-mon[35787]: pgmap v787: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:33 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:34 smithi203 ceph-mon[35787]: pgmap v788: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:34 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:34 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:34 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:35 smithi203 ceph-mon[35787]: pgmap v789: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:35 smithi203 ceph-mon[35787]: pgmap v790: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:37 smithi203 ceph-mon[35787]: pgmap v791: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:39 smithi203 ceph-mon[35787]: pgmap v792: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:41 smithi203 ceph-mon[35787]: pgmap v793: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:43 smithi203 ceph-mon[35787]: pgmap v794: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:45 smithi203 ceph-mon[35787]: pgmap v795: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:47 smithi203 ceph-mon[35787]: pgmap v796: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:49 smithi203 ceph-mon[35787]: pgmap v797: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:51 smithi203 ceph-mon[35787]: pgmap v798: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:53 smithi203 ceph-mon[35787]: pgmap v799: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:55 smithi203 ceph-mon[35787]: pgmap v800: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:57 smithi203 ceph-mon[35787]: pgmap v801: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:14:59 smithi203 ceph-mon[35787]: pgmap v802: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:01 smithi203 ceph-mon[35787]: pgmap v803: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:03 smithi203 ceph-mon[35787]: pgmap v804: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:05 smithi203 ceph-mon[35787]: pgmap v805: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:07 smithi203 ceph-mon[35787]: pgmap v806: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:09 smithi203 ceph-mon[35787]: pgmap v807: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-14T17:15:37.669 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:10 smithi203 ceph-mon[35787]: osdmap e123: 12 total, 12 up, 12 in 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:11 smithi203 ceph-mon[35787]: pgmap v809: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:13 smithi203 ceph-mon[35787]: pgmap v810: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:15 smithi203 ceph-mon[35787]: pgmap v811: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:17 smithi203 ceph-mon[35787]: pgmap v812: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:19 smithi203 ceph-mon[35787]: pgmap v813: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:19 smithi203 ceph-mon[35787]: osdmap e124: 12 total, 12 up, 12 in 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:21 smithi203 ceph-mon[35787]: pgmap v815: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:23 smithi203 ceph-mon[35787]: pgmap v816: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:25 smithi203 ceph-mon[35787]: pgmap v817: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:27 smithi203 ceph-mon[35787]: pgmap v818: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:29 smithi203 ceph-mon[35787]: pgmap v819: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:31 smithi203 ceph-mon[35787]: pgmap v820: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:33 smithi203 ceph-mon[35787]: pgmap v821: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:34 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:35 smithi203 ceph-mon[35787]: pgmap v822: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:15:37.671 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:36 smithi203 ceph-mon[35787]: pgmap v823: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.671 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:36 smithi069 ceph-mon[31507]: pgmap v690: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-14T17:15:37.671 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:36 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3409620834' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T17:15:37.671 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:38 smithi069 ceph-mon[31507]: pgmap v691: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-14T17:15:37.671 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:40 smithi069 ceph-mon[31507]: pgmap v692: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-14T17:15:37.671 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:42 smithi069 ceph-mon[31507]: pgmap v693: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-14T17:15:37.671 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:44 smithi069 ceph-mon[31507]: pgmap v694: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-14T17:15:37.671 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:46 smithi069 ceph-mon[31507]: pgmap v695: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-14T17:15:37.671 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:48 smithi069 ceph-mon[31507]: pgmap v696: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-14T17:15:37.671 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:49 smithi069 ceph-mon[31507]: pgmap v697: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:52 smithi069 ceph-mon[31507]: pgmap v698: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:54 smithi069 ceph-mon[31507]: pgmap v699: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:56 smithi069 ceph-mon[31507]: pgmap v700: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:11:58 smithi069 ceph-mon[31507]: pgmap v701: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:00 smithi069 ceph-mon[31507]: pgmap v702: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:02 smithi069 ceph-mon[31507]: pgmap v703: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:04 smithi069 ceph-mon[31507]: pgmap v704: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:06 smithi069 ceph-mon[31507]: pgmap v705: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:08 smithi069 ceph-mon[31507]: pgmap v706: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:10 smithi069 ceph-mon[31507]: pgmap v707: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:10 smithi069 ceph-mon[31507]: osdmap e117: 12 total, 12 up, 12 in 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:12 smithi069 ceph-mon[31507]: pgmap v709: 97 pgs: 97 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:14 smithi069 ceph-mon[31507]: pgmap v710: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:14 smithi069 ceph-mon[31507]: osdmap e118: 12 total, 12 up, 12 in 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:16 smithi069 ceph-mon[31507]: pgmap v712: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-05-14T17:15:37.672 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:18 smithi069 ceph-mon[31507]: pgmap v713: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:20 smithi069 ceph-mon[31507]: pgmap v714: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:22 smithi069 ceph-mon[31507]: pgmap v715: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:24 smithi069 ceph-mon[31507]: pgmap v716: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:26 smithi069 ceph-mon[31507]: pgmap v717: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:28 smithi069 ceph-mon[31507]: pgmap v718: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:30 smithi069 ceph-mon[31507]: pgmap v719: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:32 smithi069 ceph-mon[31507]: pgmap v720: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:32 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:32 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:32 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:32 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:33 smithi069 ceph-mon[31507]: pgmap v721: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:33 smithi069 ceph-mon[31507]: pgmap v722: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:35 smithi069 ceph-mon[31507]: pgmap v723: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:37 smithi069 ceph-mon[31507]: pgmap v724: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:39 smithi069 ceph-mon[31507]: pgmap v725: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.673 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:41 smithi069 ceph-mon[31507]: pgmap v726: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:43 smithi069 ceph-mon[31507]: pgmap v727: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:45 smithi069 ceph-mon[31507]: pgmap v728: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:47 smithi069 ceph-mon[31507]: pgmap v729: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:49 smithi069 ceph-mon[31507]: pgmap v730: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:51 smithi069 ceph-mon[31507]: pgmap v731: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:53 smithi069 ceph-mon[31507]: pgmap v732: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:55 smithi069 ceph-mon[31507]: pgmap v733: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:57 smithi069 ceph-mon[31507]: pgmap v734: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:12:59 smithi069 ceph-mon[31507]: pgmap v735: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:01 smithi069 ceph-mon[31507]: pgmap v736: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:03 smithi069 ceph-mon[31507]: pgmap v737: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:05 smithi069 ceph-mon[31507]: pgmap v738: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:07 smithi069 ceph-mon[31507]: pgmap v739: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:09 smithi069 ceph-mon[31507]: pgmap v740: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:10 smithi069 ceph-mon[31507]: osdmap e119: 12 total, 12 up, 12 in 2024-05-14T17:15:37.674 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:11 smithi069 ceph-mon[31507]: pgmap v742: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:13 smithi069 ceph-mon[31507]: pgmap v743: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:15 smithi069 ceph-mon[31507]: pgmap v744: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:17 smithi069 ceph-mon[31507]: pgmap v745: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:19 smithi069 ceph-mon[31507]: pgmap v746: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:19 smithi069 ceph-mon[31507]: osdmap e120: 12 total, 12 up, 12 in 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:21 smithi069 ceph-mon[31507]: pgmap v748: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:23 smithi069 ceph-mon[31507]: pgmap v749: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:25 smithi069 ceph-mon[31507]: pgmap v750: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:27 smithi069 ceph-mon[31507]: pgmap v751: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:29 smithi069 ceph-mon[31507]: pgmap v752: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:31 smithi069 ceph-mon[31507]: pgmap v753: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:32 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:33 smithi069 ceph-mon[31507]: pgmap v754: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:33 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:33 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:33 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:37.675 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:34 smithi069 ceph-mon[31507]: pgmap v755: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:34 smithi069 ceph-mon[31507]: pgmap v756: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:36 smithi069 ceph-mon[31507]: pgmap v757: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:38 smithi069 ceph-mon[31507]: pgmap v758: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:40 smithi069 ceph-mon[31507]: pgmap v759: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:42 smithi069 ceph-mon[31507]: pgmap v760: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:44 smithi069 ceph-mon[31507]: pgmap v761: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:46 smithi069 ceph-mon[31507]: pgmap v762: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:48 smithi069 ceph-mon[31507]: pgmap v763: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:50 smithi069 ceph-mon[31507]: pgmap v764: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:52 smithi069 ceph-mon[31507]: pgmap v765: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:54 smithi069 ceph-mon[31507]: pgmap v766: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:56 smithi069 ceph-mon[31507]: pgmap v767: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:13:58 smithi069 ceph-mon[31507]: pgmap v768: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:00 smithi069 ceph-mon[31507]: pgmap v769: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:02 smithi069 ceph-mon[31507]: pgmap v770: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:04 smithi069 ceph-mon[31507]: pgmap v771: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:06 smithi069 ceph-mon[31507]: pgmap v772: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:08 smithi069 ceph-mon[31507]: pgmap v773: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:10 smithi069 ceph-mon[31507]: pgmap v774: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:10 smithi069 ceph-mon[31507]: osdmap e121: 12 total, 12 up, 12 in 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:12 smithi069 ceph-mon[31507]: pgmap v776: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:14 smithi069 ceph-mon[31507]: pgmap v777: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:14 smithi069 ceph-mon[31507]: osdmap e122: 12 total, 12 up, 12 in 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:16 smithi069 ceph-mon[31507]: pgmap v779: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:18 smithi069 ceph-mon[31507]: pgmap v780: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:20 smithi069 ceph-mon[31507]: pgmap v781: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:22 smithi069 ceph-mon[31507]: pgmap v782: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:24 smithi069 ceph-mon[31507]: pgmap v783: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:26 smithi069 ceph-mon[31507]: pgmap v784: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 95 B/s wr, 5 op/s 2024-05-14T17:15:37.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:28 smithi069 ceph-mon[31507]: pgmap v785: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:30 smithi069 ceph-mon[31507]: pgmap v786: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:32 smithi069 ceph-mon[31507]: pgmap v787: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:33 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:34 smithi069 ceph-mon[31507]: pgmap v788: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:34 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:34 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:34 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:35 smithi069 ceph-mon[31507]: pgmap v789: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:35 smithi069 ceph-mon[31507]: pgmap v790: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:37 smithi069 ceph-mon[31507]: pgmap v791: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:39 smithi069 ceph-mon[31507]: pgmap v792: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:41 smithi069 ceph-mon[31507]: pgmap v793: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:43 smithi069 ceph-mon[31507]: pgmap v794: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:45 smithi069 ceph-mon[31507]: pgmap v795: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:47 smithi069 ceph-mon[31507]: pgmap v796: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:49 smithi069 ceph-mon[31507]: pgmap v797: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:51 smithi069 ceph-mon[31507]: pgmap v798: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:53 smithi069 ceph-mon[31507]: pgmap v799: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:55 smithi069 ceph-mon[31507]: pgmap v800: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:57 smithi069 ceph-mon[31507]: pgmap v801: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:14:59 smithi069 ceph-mon[31507]: pgmap v802: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:01 smithi069 ceph-mon[31507]: pgmap v803: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:03 smithi069 ceph-mon[31507]: pgmap v804: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:05 smithi069 ceph-mon[31507]: pgmap v805: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:07 smithi069 ceph-mon[31507]: pgmap v806: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:09 smithi069 ceph-mon[31507]: pgmap v807: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:10 smithi069 ceph-mon[31507]: osdmap e123: 12 total, 12 up, 12 in 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:11 smithi069 ceph-mon[31507]: pgmap v809: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:13 smithi069 ceph-mon[31507]: pgmap v810: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-14T17:15:37.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:15 smithi069 ceph-mon[31507]: pgmap v811: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:17 smithi069 ceph-mon[31507]: pgmap v812: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:19 smithi069 ceph-mon[31507]: pgmap v813: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:19 smithi069 ceph-mon[31507]: osdmap e124: 12 total, 12 up, 12 in 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:21 smithi069 ceph-mon[31507]: pgmap v815: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:23 smithi069 ceph-mon[31507]: pgmap v816: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:25 smithi069 ceph-mon[31507]: pgmap v817: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:27 smithi069 ceph-mon[31507]: pgmap v818: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:29 smithi069 ceph-mon[31507]: pgmap v819: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:31 smithi069 ceph-mon[31507]: pgmap v820: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:33 smithi069 ceph-mon[31507]: pgmap v821: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:34 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:35 smithi069 ceph-mon[31507]: pgmap v822: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:15:37.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:36 smithi069 ceph-mon[31507]: pgmap v823: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:37.928 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:38 smithi069 ceph-mon[31507]: pgmap v824: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:38 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:38 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:38 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:38 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:38 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:38.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:38 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:38.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:38 smithi069 ceph-mon[31507]: pgmap v825: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:38.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:38 smithi069 ceph-mon[31507]: pgmap v826: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:38.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:38 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:38 smithi203 ceph-mon[35787]: pgmap v824: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:38 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:38 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:38 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:38 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:38 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:38 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:38 smithi203 ceph-mon[35787]: pgmap v825: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:38 smithi203 ceph-mon[35787]: pgmap v826: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:38.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:38 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:38.999 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:38 smithi092 ceph-mon[35763]: pgmap v824: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:38.999 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:38 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:38 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:38 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:38 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:38 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:38 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:38 smithi092 ceph-mon[35763]: pgmap v825: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:38 smithi092 ceph-mon[35763]: pgmap v826: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:39.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:38 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:15:39.760 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:15:39.760 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:15:39.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-05-14T17:15:39.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-05-14T17:15:39.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:15:39.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-05-14T17:15:39.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-05-14T17:15:39.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-05-14T17:15:39.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001593183, 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000134922, 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 480, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 467, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1786400, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 288240, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.081184947, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.029849426, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.046624035, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.010151522, 2024-05-14T17:15:39.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 467, 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.027640692, 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000059187 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1786400, 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 288240 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:15:39.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 356, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 298864, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.025982504, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.019729445, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002021920, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 353, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.032126767, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000091010 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 298864 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-14T17:15:39.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-14T17:15:39.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-14T17:15:39.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148692, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 30, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-05-14T17:15:39.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-14T17:15:39.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 479, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1716184, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 479, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 479, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1716184, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:39.770 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:39.771 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:39.771 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:15:40.412 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-14T17:15:40.412 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-14T17:15:40.412 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-14T17:15:40.412 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-14T17:15:40.412 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-14T17:15:40.412 DEBUG:tasks.check_counter:Getting stats from d 2024-05-14T17:15:40.412 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-14T17:15:40.653 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:41.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:41 smithi069 ceph-mon[31507]: pgmap v827: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:41.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:41 smithi203 ceph-mon[35787]: pgmap v827: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:41.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:41 smithi092 ceph-mon[35763]: pgmap v827: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001185372, 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:15:42.278 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000197344, 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.279 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 361, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 299744, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.027079966, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.020825813, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002146276, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 358, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.026814153, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000074899 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.280 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 299744 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 486, 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 473, 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1788032, 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 292080, 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-05-14T17:15:42.281 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.077940574, 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.030053358, 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.043433268, 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.010129555, 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 473, 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.023660052, 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000050021 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:42.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1788032, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 292080 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:15:42.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:15:42.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:15:42.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148754, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 31, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:15:42.286 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-14T17:15:42.287 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 485, 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1716964, 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 485, 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 485, 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1716964, 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:42.288 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:15:42.999 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-14T17:15:42.999 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-14T17:15:42.999 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-14T17:15:42.999 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-14T17:15:42.999 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-14T17:15:43.000 DEBUG:tasks.check_counter:Getting stats from g 2024-05-14T17:15:43.000 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-14T17:15:43.241 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:43.362 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:43 smithi069 ceph-mon[31507]: pgmap v828: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:43.458 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:43 smithi203 ceph-mon[35787]: pgmap v828: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:43.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:43 smithi092 ceph-mon[35763]: pgmap v828: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:44.930 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:15:44.930 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:15:44.930 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-05-14T17:15:44.930 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001285665, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000328358, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:44.931 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 478, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 467, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1784000, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 289072, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-05-14T17:15:44.932 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.077254531, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.030571749, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.041699955, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.010138426, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 466, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.024597926, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000052785 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1784000, 2024-05-14T17:15:44.933 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 289072 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 7, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 359, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 3424, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 299488, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.026773889, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.019940527, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002684185, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:44.934 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 357, 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.042913791, 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000120206 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 299488 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-14T17:15:44.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:15:44.936 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:15:44.937 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148714, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 30, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-14T17:15:44.938 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-14T17:15:44.939 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 481, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1716444, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 481, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 481, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1716444, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:44.940 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:44.941 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:44.941 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:44.941 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:44.941 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:15:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:45 smithi069 ceph-mon[31507]: pgmap v829: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:45.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:45 smithi203 ceph-mon[35787]: pgmap v829: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:45.481 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-14T17:15:45.490 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-14T17:15:45.517 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-14T17:15:45.528 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-14T17:15:45.540 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-14T17:15:45.551 DEBUG:tasks.check_counter:Getting stats from j 2024-05-14T17:15:45.563 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-14T17:15:45.575 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:45 smithi092 ceph-mon[35763]: pgmap v829: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:47.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:47 smithi069 ceph-mon[31507]: pgmap v830: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T17:15:47.446 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-14T17:15:47.474 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:15:47.486 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-05-14T17:15:47.497 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-05-14T17:15:47.508 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:15:47.520 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-05-14T17:15:47.531 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-05-14T17:15:47.542 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-05-14T17:15:47.553 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001192609, 2024-05-14T17:15:47.564 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:15:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000223240, 2024-05-14T17:15:47.590 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:15:47.599 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:47.610 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:47.620 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:47.630 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:47.640 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:47.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:47.710 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:47.717 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:47.725 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:47.734 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:47.742 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:15:47.750 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:15:47.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:47.773 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:15:47.780 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3, 2024-05-14T17:15:47.802 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 363, 2024-05-14T17:15:47.809 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:15:47.816 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 300096, 2024-05-14T17:15:47.824 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2024-05-14T17:15:47.830 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-05-14T17:15:47.838 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.027934391, 2024-05-14T17:15:47.847 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.021373724, 2024-05-14T17:15:47.857 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.001987410, 2024-05-14T17:15:47.896 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:15:47.912 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:47.919 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 360, 2024-05-14T17:15:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.029723286, 2024-05-14T17:15:47.935 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000082564 2024-05-14T17:15:47.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:47.949 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:47.960 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:47.971 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:47.999 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:48.012 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:48.023 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:15:48.037 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 300096 2024-05-14T17:15:48.050 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:48.062 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:15:48.073 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 488, 2024-05-14T17:15:48.090 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 475, 2024-05-14T17:15:48.137 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1788576, 2024-05-14T17:15:48.149 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 293360, 2024-05-14T17:15:48.161 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-05-14T17:15:48.171 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-05-14T17:15:48.181 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.081412151, 2024-05-14T17:15:48.191 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.029426110, 2024-05-14T17:15:48.200 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.046191945, 2024-05-14T17:15:48.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.011451670, 2024-05-14T17:15:48.218 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:48.227 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 475, 2024-05-14T17:15:48.238 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.022707323, 2024-05-14T17:15:48.246 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000047804 2024-05-14T17:15:48.255 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:48.264 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:48.279 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:48.287 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:48.307 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:48.316 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:48.355 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1788576, 2024-05-14T17:15:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 293360 2024-05-14T17:15:48.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:48.383 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-14T17:15:48.392 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-05-14T17:15:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-14T17:15:48.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:48.425 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-14T17:15:48.455 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:15:48.475 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-14T17:15:48.485 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:15:48.494 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:15:48.503 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:15:48.511 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:15:48.524 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:15:48.549 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:15:48.595 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:15:48.604 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:15:48.613 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:15:48.622 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:15:48.631 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:15:48.641 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:15:48.650 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:15:48.660 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-14T17:15:48.703 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:15:48.713 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-14T17:15:48.721 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:15:48.730 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:15:48.742 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:15:48.754 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:15:48.762 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:15:48.770 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:15:48.848 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:15:48.859 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-14T17:15:48.882 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:15:48.893 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:15:48.904 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:15:48.931 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-14T17:15:48.940 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-14T17:15:48.949 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-14T17:15:48.958 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:15:48.970 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:15:48.987 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:15:48.996 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:15:49.006 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148721, 2024-05-14T17:15:49.014 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 30, 2024-05-14T17:15:49.024 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:15:49.037 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-14T17:15:49.049 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-14T17:15:49.097 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-14T17:15:49.109 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:15:49.121 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-14T17:15:49.130 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:15:49.139 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-14T17:15:49.175 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-05-14T17:15:49.184 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-05-14T17:15:49.205 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:15:49.217 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:15:49.233 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-14T17:15:49.242 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-14T17:15:49.253 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-05-14T17:15:49.272 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-05-14T17:15:49.280 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-14T17:15:49.320 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-14T17:15:49.354 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-14T17:15:49.366 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-14T17:15:49.388 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-14T17:15:49.397 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:15:49.446 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-14T17:15:49.454 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-14T17:15:49.463 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-14T17:15:49.473 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 487, 2024-05-14T17:15:49.482 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1717224, 2024-05-14T17:15:49.493 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-14T17:15:49.502 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 487, 2024-05-14T17:15:49.510 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-14T17:15:49.522 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-14T17:15:49.532 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 487, 2024-05-14T17:15:49.541 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1717224, 2024-05-14T17:15:49.549 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-14T17:15:49.559 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-14T17:15:49.616 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-14T17:15:49.631 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-14T17:15:49.650 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:49.660 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-14T17:15:49.698 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-14T17:15:49.706 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:47 smithi092 ceph-mon[35763]: pgmap v830: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T17:15:49.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:49 smithi092 ceph-mon[35763]: pgmap v831: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:49.720 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:47 smithi203 ceph-mon[35787]: pgmap v830: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T17:15:49.726 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:49 smithi203 ceph-mon[35787]: pgmap v831: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:49.734 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-14T17:15:49.742 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-14T17:15:49.748 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-14T17:15:49.755 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-14T17:15:49.762 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-14T17:15:49.768 DEBUG:tasks.check_counter:Getting stats from b 2024-05-14T17:15:49.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-14T17:15:49.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:49 smithi069 ceph-mon[31507]: pgmap v831: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:15:50.073 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:15:51.340 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:51 smithi092 ceph-mon[35763]: pgmap v832: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:51.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:51 smithi069 ceph-mon[31507]: pgmap v832: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:51.464 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:51 smithi203 ceph-mon[35787]: pgmap v832: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:15:51.754 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-14T17:15:51.778 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:15:51.792 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 7019, 2024-05-14T17:15:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 9124, 2024-05-14T17:15:51.811 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 4593096, 2024-05-14T17:15:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 11370026, 2024-05-14T17:15:51.831 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 249, 2024-05-14T17:15:51.839 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-05-14T17:15:51.849 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.626258462, 2024-05-14T17:15:51.867 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.203048121, 2024-05-14T17:15:51.879 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.326593931, 2024-05-14T17:15:51.890 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.091768788, 2024-05-14T17:15:51.901 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:51.914 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 9127, 2024-05-14T17:15:51.924 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.888847322, 2024-05-14T17:15:51.971 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000097386 2024-05-14T17:15:52.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.019 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:52.029 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 11007, 2024-05-14T17:15:52.038 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002346537, 2024-05-14T17:15:52.048 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000213 2024-05-14T17:15:52.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2059600, 2024-05-14T17:15:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 315008 2024-05-14T17:15:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:15:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 9136, 2024-05-14T17:15:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 11362, 2024-05-14T17:15:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 3377968, 2024-05-14T17:15:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 21988902, 2024-05-14T17:15:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 21, 2024-05-14T17:15:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-05-14T17:15:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.772698606, 2024-05-14T17:15:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.280083394, 2024-05-14T17:15:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.420595429, 2024-05-14T17:15:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.157078685, 2024-05-14T17:15:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 11359, 2024-05-14T17:15:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.246625762, 2024-05-14T17:15:52.295 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000109747 2024-05-14T17:15:52.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.316 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:52.326 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 14859, 2024-05-14T17:15:52.370 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002248475, 2024-05-14T17:15:52.381 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000151 2024-05-14T17:15:52.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2767760, 2024-05-14T17:15:52.409 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 5675344 2024-05-14T17:15:52.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.450 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:15:52.464 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 7449, 2024-05-14T17:15:52.476 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 6515, 2024-05-14T17:15:52.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2840563, 2024-05-14T17:15:52.497 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 18885190, 2024-05-14T17:15:52.510 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 10, 2024-05-14T17:15:52.522 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 8, 2024-05-14T17:15:52.533 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.720416961, 2024-05-14T17:15:52.544 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.254157542, 2024-05-14T17:15:52.556 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.259302547, 2024-05-14T17:15:52.568 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.201736730, 2024-05-14T17:15:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:15:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 6510, 2024-05-14T17:15:52.628 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.094872149, 2024-05-14T17:15:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000168183 2024-05-14T17:15:52.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:15:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 8046, 2024-05-14T17:15:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001616643, 2024-05-14T17:15:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000200 2024-05-14T17:15:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.724 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 836576, 2024-05-14T17:15:52.733 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 284352 2024-05-14T17:15:52.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.748 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-14T17:15:52.755 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-05-14T17:15:52.761 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-14T17:15:52.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.808 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-05-14T17:15:52.815 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-14T17:15:52.858 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-14T17:15:52.867 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5217, 2024-05-14T17:15:52.876 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 3.262178285, 2024-05-14T17:15:52.884 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000625297 2024-05-14T17:15:52.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:15:52.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.935 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-05-14T17:15:52.942 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-14T17:15:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-14T17:15:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 650, 2024-05-14T17:15:52.964 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.025724539, 2024-05-14T17:15:52.971 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000039576 2024-05-14T17:15:52.980 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:15:52.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:52.994 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-14T17:15:53.043 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 3007, 2024-05-14T17:15:53.052 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 2702, 2024-05-14T17:15:53.062 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-05-14T17:15:53.072 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2702, 2024-05-14T17:15:53.082 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 202.911109437, 2024-05-14T17:15:53.092 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.075096635 2024-05-14T17:15:53.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:53.131 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-05-14T17:15:53.144 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 305, 2024-05-14T17:15:53.156 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 5, 2024-05-14T17:15:53.167 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-05-14T17:15:53.179 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 939, 2024-05-14T17:15:53.188 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 3, 2024-05-14T17:15:53.199 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-05-14T17:15:53.214 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 122, 2024-05-14T17:15:53.225 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 46, 2024-05-14T17:15:53.237 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 66, 2024-05-14T17:15:53.294 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 10, 2024-05-14T17:15:53.305 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 27, 2024-05-14T17:15:53.316 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 1785, 2024-05-14T17:15:53.329 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 9, 2024-05-14T17:15:53.340 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 9, 2024-05-14T17:15:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 21, 2024-05-14T17:15:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 4243, 2024-05-14T17:15:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 3229, 2024-05-14T17:15:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 302, 2024-05-14T17:15:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 3, 2024-05-14T17:15:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 2, 2024-05-14T17:15:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-05-14T17:15:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 69, 2024-05-14T17:15:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 45, 2024-05-14T17:15:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-05-14T17:15:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 50, 2024-05-14T17:15:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 1728, 2024-05-14T17:15:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 31, 2024-05-14T17:15:53.522 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 671, 2024-05-14T17:15:53.530 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-05-14T17:15:53.537 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:15:53.548 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-05-14T17:15:53.584 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 1026, 2024-05-14T17:15:53.618 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 1697912602, 2024-05-14T17:15:53.627 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 7, 2024-05-14T17:15:53.637 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:15:53.646 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-05-14T17:15:53.656 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:15:53.666 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:15:53.676 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:15:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:15:53.712 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:15:53.723 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-05-14T17:15:53.732 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 87, 2024-05-14T17:15:53.742 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 970, 2024-05-14T17:15:53.750 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 2020, 2024-05-14T17:15:53.759 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:15:53.768 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:15:53.806 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:15:53.820 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 2375, 2024-05-14T17:15:53.855 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 1195, 2024-05-14T17:15:53.866 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 133, 2024-05-14T17:15:53.878 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 630 2024-05-14T17:15:53.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:53.902 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-05-14T17:15:53.913 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-05-14T17:15:53.923 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-05-14T17:15:53.957 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-05-14T17:15:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 3, 2024-05-14T17:15:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 106, 2024-05-14T17:15:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-05-14T17:15:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-05-14T17:15:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:15:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 184, 2024-05-14T17:15:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 158, 2024-05-14T17:15:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 26, 2024-05-14T17:15:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-05-14T17:15:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-05-14T17:15:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:15:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:15:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-05-14T17:15:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-05-14T17:15:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 12, 2024-05-14T17:15:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 1378, 2024-05-14T17:15:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:15:54.174 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 50, 2024-05-14T17:15:54.179 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-05-14T17:15:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 3, 2024-05-14T17:15:54.192 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-05-14T17:15:54.220 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-05-14T17:15:54.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:54.236 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-05-14T17:15:54.242 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-05-14T17:15:54.250 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 4960, 2024-05-14T17:15:54.278 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 4923, 2024-05-14T17:15:54.285 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 4923, 2024-05-14T17:15:54.294 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 37, 2024-05-14T17:15:54.302 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-05-14T17:15:54.316 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 2, 2024-05-14T17:15:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 105, 2024-05-14T17:15:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 104, 2024-05-14T17:15:54.374 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 104, 2024-05-14T17:15:54.384 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 1, 2024-05-14T17:15:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 1, 2024-05-14T17:15:54.402 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-05-14T17:15:54.411 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 1, 2024-05-14T17:15:54.457 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 34556683, 2024-05-14T17:15:54.465 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 34688098, 2024-05-14T17:15:54.476 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-05-14T17:15:54.484 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-05-14T17:15:54.493 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2557, 2024-05-14T17:15:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 14.417375381, 2024-05-14T17:15:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005638394 2024-05-14T17:15:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-05-14T17:15:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-05-14T17:15:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 120, 2024-05-14T17:15:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 1981, 2024-05-14T17:15:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 1861, 2024-05-14T17:15:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 61, 2024-05-14T17:15:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 113, 2024-05-14T17:15:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 52, 2024-05-14T17:15:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 122, 2024-05-14T17:15:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 2354, 2024-05-14T17:15:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 2232, 2024-05-14T17:15:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 9, 2024-05-14T17:15:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 1389, 2024-05-14T17:15:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 1380, 2024-05-14T17:15:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 73508, 2024-05-14T17:15:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 231800 2024-05-14T17:15:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-05-14T17:15:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 4141, 2024-05-14T17:15:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 75, 2024-05-14T17:15:54.806 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 3007, 2024-05-14T17:15:54.814 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 325, 2024-05-14T17:15:54.821 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 403, 2024-05-14T17:15:54.828 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-05-14T17:15:54.903 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1158, 2024-05-14T17:15:54.910 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 181.250006155, 2024-05-14T17:15:54.917 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.156519867 2024-05-14T17:15:54.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:54.929 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-05-14T17:15:54.936 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 324, 2024-05-14T17:15:54.954 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 7.547825190, 2024-05-14T17:15:54.960 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.023295756 2024-05-14T17:15:54.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:54.979 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-05-14T17:15:54.987 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:54.995 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:55.004 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:55.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.042 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-05-14T17:15:55.052 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 26, 2024-05-14T17:15:55.060 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.046906117, 2024-05-14T17:15:55.101 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001804081 2024-05-14T17:15:55.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.121 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-05-14T17:15:55.128 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 293, 2024-05-14T17:15:55.136 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 11.225976210, 2024-05-14T17:15:55.143 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.038313911 2024-05-14T17:15:55.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.166 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-05-14T17:15:55.171 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:55.178 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:55.183 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:55.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.201 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-05-14T17:15:55.233 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-05-14T17:15:55.242 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000233816, 2024-05-14T17:15:55.250 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000233816 2024-05-14T17:15:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.288 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-05-14T17:15:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-05-14T17:15:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:55.364 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:55.374 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:55.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.392 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-05-14T17:15:55.397 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:55.403 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:55.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:53 smithi203 ceph-mon[35787]: pgmap v833: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:55.457 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:53 smithi069 ceph-mon[31507]: pgmap v833: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:15:55.500 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-05-14T17:15:55.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 26, 2024-05-14T17:15:55.530 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.004703312, 2024-05-14T17:15:55.538 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000180896 2024-05-14T17:15:55.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.558 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-05-14T17:15:55.576 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 37, 2024-05-14T17:15:55.587 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.328718512, 2024-05-14T17:15:55.597 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.008884284 2024-05-14T17:15:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-05-14T17:15:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-05-14T17:15:55.719 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 26, 2024-05-14T17:15:55.733 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.201492374, 2024-05-14T17:15:55.742 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.007749706 2024-05-14T17:15:55.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.763 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-05-14T17:15:55.774 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 515, 2024-05-14T17:15:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.158663908, 2024-05-14T17:15:55.796 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000308085 2024-05-14T17:15:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-05-14T17:15:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 45, 2024-05-14T17:15:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.096934224, 2024-05-14T17:15:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002154093 2024-05-14T17:15:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-05-14T17:15:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 182, 2024-05-14T17:15:55.904 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.828476122, 2024-05-14T17:15:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.010046572 2024-05-14T17:15:55.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.939 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-05-14T17:15:55.948 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:55.958 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:55.967 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:55.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:55.996 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-05-14T17:15:56.004 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:56.009 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:56.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:56.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.034 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-05-14T17:15:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-05-14T17:15:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.010124737, 2024-05-14T17:15:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005062368 2024-05-14T17:15:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-05-14T17:15:56.128 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 19, 2024-05-14T17:15:56.137 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.089457294, 2024-05-14T17:15:56.145 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004708278 2024-05-14T17:15:56.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.161 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-05-14T17:15:56.170 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:56.196 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:56.204 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:56.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.226 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-05-14T17:15:56.234 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-05-14T17:15:56.243 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001550336, 2024-05-14T17:15:56.254 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001550336 2024-05-14T17:15:56.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.299 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-05-14T17:15:56.309 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:56.349 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:56.358 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.372 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-05-14T17:15:56.380 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:56.386 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:56.395 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-05-14T17:15:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.488 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-05-14T17:15:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 7, 2024-05-14T17:15:56.545 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.019202321, 2024-05-14T17:15:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002743188 2024-05-14T17:15:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-05-14T17:15:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-05-14T17:15:56.682 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 40, 2024-05-14T17:15:56.689 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.100838809, 2024-05-14T17:15:56.697 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002520970 2024-05-14T17:15:56.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.713 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:15:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:15:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-05-14T17:15:56.772 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:56.817 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:56.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:15:56.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.839 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-05-14T17:15:56.847 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-05-14T17:15:56.867 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 5, 2024-05-14T17:15:56.877 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 3, 2024-05-14T17:15:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-05-14T17:15:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-05-14T17:15:56.926 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 20, 2024-05-14T17:15:56.934 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 10, 2024-05-14T17:15:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1552, 2024-05-14T17:15:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-05-14T17:15:56.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:56.981 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-14T17:15:57.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 360, 2024-05-14T17:15:57.052 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 360, 2024-05-14T17:15:57.069 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:15:57.077 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:15:57.087 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:15:57.094 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:15:57.102 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:15:57.111 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:15:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:15:57.127 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:15:57.151 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:15:57.164 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:15:57.172 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:15:57.181 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:15:57.193 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:15:57.201 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-14T17:15:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:15:57.213 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-14T17:15:57.238 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:15:57.264 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:15:57.286 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:15:57.293 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:15:57.303 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:15:57.315 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:15:57.327 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:15:57.337 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-14T17:15:57.345 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:15:57.358 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:15:57.366 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:15:57.372 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-14T17:15:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-14T17:15:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-14T17:15:57.389 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:15:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:15:57.411 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:15:57.450 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:15:57.460 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 291225, 2024-05-14T17:15:57.498 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 193, 2024-05-14T17:15:57.507 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 88, 2024-05-14T17:15:57.516 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 1, 2024-05-14T17:15:57.525 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-14T17:15:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-14T17:15:57.540 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:15:57.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-14T17:15:57.580 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:15:57.590 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-14T17:15:57.601 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8128, 2024-05-14T17:15:57.609 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 169, 2024-05-14T17:15:57.619 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:15:57.664 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:15:57.671 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-14T17:15:57.678 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-14T17:15:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 542748, 2024-05-14T17:15:57.698 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 5398, 2024-05-14T17:15:57.757 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-14T17:15:57.763 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-14T17:15:57.770 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-14T17:15:57.778 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-14T17:15:57.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:57.792 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-05-14T17:15:57.799 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-05-14T17:15:57.809 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-05-14T17:15:57.817 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 6359, 2024-05-14T17:15:57.828 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 35936496, 2024-05-14T17:15:57.837 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 37, 2024-05-14T17:15:57.866 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 6359, 2024-05-14T17:15:57.874 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-05-14T17:15:57.920 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 6359, 2024-05-14T17:15:57.939 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 87.901562603, 2024-05-14T17:15:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.013823173 2024-05-14T17:15:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:57.968 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-05-14T17:15:57.974 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-05-14T17:15:57.980 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 6359, 2024-05-14T17:15:57.987 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 10836 2024-05-14T17:15:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 6359, 2024-05-14T17:15:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 88, 2024-05-14T17:15:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 6271, 2024-05-14T17:15:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-05-14T17:15:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-05-14T17:15:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 868, 2024-05-14T17:15:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 1223, 2024-05-14T17:15:58.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 83, 2024-05-14T17:15:58.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 2640, 2024-05-14T17:15:58.145 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 431, 2024-05-14T17:15:58.153 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-05-14T17:15:58.161 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-05-14T17:15:58.166 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-05-14T17:15:58.175 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 3, 2024-05-14T17:15:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 698, 2024-05-14T17:15:58.193 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-05-14T17:15:58.203 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-05-14T17:15:58.214 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-05-14T17:15:58.226 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 4, 2024-05-14T17:15:58.235 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 2446, 2024-05-14T17:15:58.245 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:15:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-05-14T17:15:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:15:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-05-14T17:15:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-05-14T17:15:58.357 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-05-14T17:15:58.378 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:15:58.386 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-05-14T17:15:58.395 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:15:58.406 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-05-14T17:15:58.414 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-05-14T17:15:58.423 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-05-14T17:15:58.435 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-05-14T17:15:58.444 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-05-14T17:15:58.454 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-05-14T17:15:58.463 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-05-14T17:15:58.486 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-05-14T17:15:58.495 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-05-14T17:15:58.542 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-05-14T17:15:58.551 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-05-14T17:15:58.562 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-05-14T17:15:58.605 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-05-14T17:15:58.616 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-05-14T17:15:58.646 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-05-14T17:15:58.660 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-05-14T17:15:58.672 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-05-14T17:15:58.685 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 124, 2024-05-14T17:15:58.696 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-05-14T17:15:58.704 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 46, 2024-05-14T17:15:58.712 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-05-14T17:15:58.720 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-05-14T17:15:58.726 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-05-14T17:15:58.733 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-05-14T17:15:58.744 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 2251, 2024-05-14T17:15:58.750 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 10, 2024-05-14T17:15:58.785 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 179 2024-05-14T17:15:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-05-14T17:15:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-05-14T17:15:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:15:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-05-14T17:15:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-05-14T17:15:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-05-14T17:15:58.900 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 1343 2024-05-14T17:15:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-05-14T17:15:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-05-14T17:15:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 17, 2024-05-14T17:15:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-05-14T17:15:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 10, 2024-05-14T17:15:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 172, 2024-05-14T17:15:59.010 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 158, 2024-05-14T17:15:59.020 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-05-14T17:15:59.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:59.065 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:15:59.076 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:15:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-14T17:15:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:15:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 23600, 2024-05-14T17:15:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 8205420, 2024-05-14T17:15:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:15:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 23600, 2024-05-14T17:15:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-14T17:15:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-14T17:15:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 23600, 2024-05-14T17:15:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 8205420, 2024-05-14T17:15:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:15:59.197 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:59.248 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:59.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:15:59.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:59.298 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-05-14T17:15:59.313 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:15:59.319 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-14T17:15:59.329 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:15:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-14T17:15:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-14T17:15:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:15:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-14T17:15:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 6359, 2024-05-14T17:15:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 35952886, 2024-05-14T17:15:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 6359, 2024-05-14T17:15:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 35952886, 2024-05-14T17:15:59.412 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:15:59.418 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:15:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-05-14T17:15:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:15:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-05-14T17:15:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:15:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-14T17:15:59.588 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-14T17:15:59.602 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:15:59.614 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-14T17:15:59.624 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 6359, 2024-05-14T17:15:59.638 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 6359, 2024-05-14T17:15:59.657 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 6359, 2024-05-14T17:15:59.668 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 6359, 2024-05-14T17:15:59.724 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:15:59.741 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:59.761 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:59.770 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:15:59.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:15:59.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:15:59.797 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:15:59.806 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:15:59.813 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-14T17:15:59.819 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:15:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 158, 2024-05-14T17:15:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 16470, 2024-05-14T17:15:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:15:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 158, 2024-05-14T17:15:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-14T17:15:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-14T17:15:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 83, 2024-05-14T17:15:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 16470, 2024-05-14T17:15:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:15:59.961 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:15:59.970 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:15:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x556ff749a500": { 2024-05-14T17:16:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-14T17:16:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 4960, 2024-05-14T17:16:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 30493794, 2024-05-14T17:16:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 4960, 2024-05-14T17:16:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-14T17:16:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-14T17:16:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2557, 2024-05-14T17:16:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 30493794, 2024-05-14T17:16:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:00.222 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:00.230 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:00.239 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:00.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:00.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:00.274 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-14T17:16:00.289 INFO:tasks.check_counter:Daemon mds.b mds.exported=50 2024-05-14T17:16:00.301 INFO:tasks.check_counter:Daemon mds.b mds.imported=31 2024-05-14T17:16:00.314 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-05-14T17:16:00.351 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=26 2024-05-14T17:16:00.367 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=19 2024-05-14T17:16:00.374 DEBUG:tasks.check_counter:Getting stats from e 2024-05-14T17:16:00.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-14T17:16:00.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:53 smithi092 ceph-mon[35763]: pgmap v833: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:16:00.411 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:55 smithi092 ceph-mon[35763]: pgmap v834: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:16:00.419 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:57 smithi092 ceph-mon[35763]: pgmap v835: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:16:00.438 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:15:59 smithi092 ceph-mon[35763]: pgmap v836: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:16:00.449 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:55 smithi203 ceph-mon[35787]: pgmap v834: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:16:00.458 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:57 smithi203 ceph-mon[35787]: pgmap v835: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:16:00.468 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:15:59 smithi203 ceph-mon[35787]: pgmap v836: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:16:00.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:55 smithi069 ceph-mon[31507]: pgmap v834: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:16:00.484 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:57 smithi069 ceph-mon[31507]: pgmap v835: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:16:00.491 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:15:59 smithi069 ceph-mon[31507]: pgmap v836: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:16:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:16:01.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:01 smithi069 ceph-mon[31507]: pgmap v837: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:01.465 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:01 smithi203 ceph-mon[35787]: pgmap v837: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:01.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:01 smithi092 ceph-mon[35763]: pgmap v837: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:02.416 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-14T17:16:02.428 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:16:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1461, 2024-05-14T17:16:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1463, 2024-05-14T17:16:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 4568203, 2024-05-14T17:16:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 464662, 2024-05-14T17:16:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 8, 2024-05-14T17:16:02.530 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-05-14T17:16:02.539 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.126956386, 2024-05-14T17:16:02.548 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.043851002, 2024-05-14T17:16:02.580 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.031810912, 2024-05-14T17:16:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.039055587, 2024-05-14T17:16:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1461, 2024-05-14T17:16:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.058282791, 2024-05-14T17:16:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000039892 2024-05-14T17:16:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:02.707 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:02.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:02.728 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:16:02.747 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:16:02.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:16:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1111, 2024-05-14T17:16:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1470, 2024-05-14T17:16:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 4476044, 2024-05-14T17:16:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 1152339, 2024-05-14T17:16:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 7, 2024-05-14T17:16:02.858 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-05-14T17:16:02.866 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.106768990, 2024-05-14T17:16:02.902 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.037634201, 2024-05-14T17:16:02.908 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.026434275, 2024-05-14T17:16:02.920 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.030683395, 2024-05-14T17:16:02.932 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:02.941 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1468, 2024-05-14T17:16:02.950 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.075585123, 2024-05-14T17:16:02.959 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000051488 2024-05-14T17:16:02.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:02.977 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:03.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-14T17:16:03.035 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 800944 2024-05-14T17:16:03.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:03.064 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:16:03.069 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 3296, 2024-05-14T17:16:03.075 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 3247, 2024-05-14T17:16:03.082 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 3931688, 2024-05-14T17:16:03.099 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 1168398, 2024-05-14T17:16:03.111 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-05-14T17:16:03.124 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-05-14T17:16:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.361200441, 2024-05-14T17:16:03.148 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.119333943, 2024-05-14T17:16:03.160 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.124476672, 2024-05-14T17:16:03.171 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.101552151, 2024-05-14T17:16:03.182 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:03.192 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3246, 2024-05-14T17:16:03.228 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.157296568, 2024-05-14T17:16:03.236 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000048458 2024-05-14T17:16:03.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:03.305 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1991360, 2024-05-14T17:16:03.432 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 303120 2024-05-14T17:16:03.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:03.477 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-14T17:16:03.485 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-05-14T17:16:03.493 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-14T17:16:03.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-05-14T17:16:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-14T17:16:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-14T17:16:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 8937, 2024-05-14T17:16:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.246251677, 2024-05-14T17:16:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000027554 2024-05-14T17:16:03.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:03.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:03.651 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-05-14T17:16:03.684 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-14T17:16:03.692 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-14T17:16:03.703 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-14T17:16:03.805 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-05-14T17:16:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-05-14T17:16:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-05-14T17:16:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:03.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:03.926 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-05-14T17:16:03.961 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-05-14T17:16:03.970 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-05-14T17:16:03.979 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-05-14T17:16:03.988 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-05-14T17:16:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-05-14T17:16:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-05-14T17:16:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 1372, 2024-05-14T17:16:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 523, 2024-05-14T17:16:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 849, 2024-05-14T17:16:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-05-14T17:16:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 625, 2024-05-14T17:16:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-05-14T17:16:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-05-14T17:16:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-05-14T17:16:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 12, 2024-05-14T17:16:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-05-14T17:16:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-05-14T17:16:04.197 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-05-14T17:16:04.206 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-05-14T17:16:04.214 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:16:04.221 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-05-14T17:16:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-05-14T17:16:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-05-14T17:16:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-05-14T17:16:04.358 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-05-14T17:16:04.367 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-05-14T17:16:04.376 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-05-14T17:16:04.388 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-05-14T17:16:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-05-14T17:16:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:16:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-05-14T17:16:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 0, 2024-05-14T17:16:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 0, 2024-05-14T17:16:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-05-14T17:16:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:16:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-05-14T17:16:04.528 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:16:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:16:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:16:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:16:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:16:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-05-14T17:16:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-05-14T17:16:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T17:16:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-05-14T17:16:04.665 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:16:04.677 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:16:04.691 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:16:04.703 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-05-14T17:16:04.735 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-05-14T17:16:04.746 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-05-14T17:16:04.757 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-05-14T17:16:04.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:04.779 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-05-14T17:16:04.794 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-05-14T17:16:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-05-14T17:16:04.812 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-05-14T17:16:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-05-14T17:16:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-05-14T17:16:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-05-14T17:16:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-05-14T17:16:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:16:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-05-14T17:16:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-05-14T17:16:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-05-14T17:16:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-05-14T17:16:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-05-14T17:16:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:16:04.937 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:16:04.944 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-05-14T17:16:04.954 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-05-14T17:16:04.969 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-05-14T17:16:04.981 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-05-14T17:16:05.020 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:16:05.029 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-05-14T17:16:05.038 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-05-14T17:16:05.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-05-14T17:16:05.056 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-05-14T17:16:05.065 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-05-14T17:16:05.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:05.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-05-14T17:16:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-05-14T17:16:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-05-14T17:16:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-05-14T17:16:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-05-14T17:16:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 1901, 2024-05-14T17:16:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-05-14T17:16:05.207 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-05-14T17:16:05.218 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-05-14T17:16:05.227 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-05-14T17:16:05.235 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-05-14T17:16:05.244 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 5, 2024-05-14T17:16:05.254 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-05-14T17:16:05.262 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-05-14T17:16:05.274 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-05-14T17:16:05.285 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-05-14T17:16:05.294 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 13688666, 2024-05-14T17:16:05.310 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 13687573, 2024-05-14T17:16:05.321 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-05-14T17:16:05.333 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:05.347 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:05.367 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:05.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 1901 2024-05-14T17:16:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-05-14T17:16:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1302, 2024-05-14T17:16:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 1597, 2024-05-14T17:16:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 295, 2024-05-14T17:16:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 74, 2024-05-14T17:16:05.466 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 79, 2024-05-14T17:16:05.473 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 5, 2024-05-14T17:16:05.482 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 1372, 2024-05-14T17:16:05.494 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 1666, 2024-05-14T17:16:05.534 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 294, 2024-05-14T17:16:05.543 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-05-14T17:16:05.552 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-05-14T17:16:05.573 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-05-14T17:16:05.595 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 58624, 2024-05-14T17:16:05.605 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 215416 2024-05-14T17:16:05.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:05.658 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-05-14T17:16:05.666 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-05-14T17:16:05.673 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-05-14T17:16:05.683 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-05-14T17:16:05.690 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-05-14T17:16:05.699 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-05-14T17:16:05.708 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-05-14T17:16:05.719 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:05.729 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:05.737 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-05-14T17:16:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-05-14T17:16:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:05.880 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:05.888 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:05.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:05.906 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-05-14T17:16:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-05-14T17:16:05.957 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:05.978 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:05.988 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.067 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-05-14T17:16:06.127 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.167 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.186 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.209 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-05-14T17:16:06.221 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.229 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.241 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.263 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-05-14T17:16:06.275 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.286 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.312 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-05-14T17:16:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-05-14T17:16:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.455 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-05-14T17:16:06.484 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.491 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.498 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-05-14T17:16:06.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:03 smithi203 ceph-mon[35787]: pgmap v838: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-14T17:16:06.544 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:05 smithi203 ceph-mon[35787]: pgmap v839: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-14T17:16:06.573 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:03 smithi069 ceph-mon[31507]: pgmap v838: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-14T17:16:06.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:05 smithi069 ceph-mon[31507]: pgmap v839: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-14T17:16:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.655 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.676 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-05-14T17:16:06.685 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.724 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-05-14T17:16:06.736 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.749 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.779 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.802 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-05-14T17:16:06.824 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.845 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.873 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.896 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-05-14T17:16:06.908 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-05-14T17:16:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:06.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-05-14T17:16:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.041 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.097 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-05-14T17:16:07.104 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.113 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.120 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.134 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-05-14T17:16:07.143 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.149 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.156 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.171 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-05-14T17:16:07.177 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.193 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.201 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.216 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-05-14T17:16:07.247 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.255 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.265 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.289 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-05-14T17:16:07.299 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.309 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.320 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.342 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-05-14T17:16:07.354 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.366 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.379 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.413 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-05-14T17:16:07.424 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.449 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.491 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-05-14T17:16:07.531 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.542 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.552 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.569 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-05-14T17:16:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.603 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-05-14T17:16:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-05-14T17:16:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.690 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.697 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.715 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:16:07.722 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:16:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-05-14T17:16:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-05-14T17:16:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-05-14T17:16:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-05-14T17:16:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-05-14T17:16:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-05-14T17:16:07.922 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-05-14T17:16:07.934 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-05-14T17:16:07.947 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-05-14T17:16:07.960 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 4449, 2024-05-14T17:16:07.974 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-05-14T17:16:07.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:07.998 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-14T17:16:08.011 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:16:08.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-14T17:16:08.037 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:16:08.050 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:16:08.064 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:16:08.095 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:16:08.110 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:16:08.126 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:16:08.191 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:16:08.230 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:16:08.241 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:16:08.252 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:16:08.266 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:16:08.277 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:16:08.288 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:16:08.302 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-14T17:16:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:16:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-14T17:16:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:16:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:16:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:16:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:16:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:16:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:16:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:16:08.484 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-14T17:16:08.525 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:16:08.545 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:16:08.559 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:16:08.570 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-14T17:16:08.582 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-14T17:16:08.591 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-14T17:16:08.600 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:16:08.614 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:16:08.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:16:08.663 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:16:08.674 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 259039, 2024-05-14T17:16:08.688 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 1427, 2024-05-14T17:16:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:16:08.732 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-05-14T17:16:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-14T17:16:08.752 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-14T17:16:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:16:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-14T17:16:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:16:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-14T17:16:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8128, 2024-05-14T17:16:08.865 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 169, 2024-05-14T17:16:08.875 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:16:08.883 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:16:08.889 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-14T17:16:08.898 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-14T17:16:08.906 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 4098802, 2024-05-14T17:16:08.915 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 41382, 2024-05-14T17:16:08.922 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-14T17:16:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-14T17:16:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-14T17:16:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-14T17:16:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-05-14T17:16:09.049 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-05-14T17:16:09.065 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-05-14T17:16:09.078 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 5287, 2024-05-14T17:16:09.094 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 21, 2024-05-14T17:16:09.106 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 3, 2024-05-14T17:16:09.119 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 5287, 2024-05-14T17:16:09.135 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-05-14T17:16:09.152 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5287, 2024-05-14T17:16:09.166 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 9.142747100, 2024-05-14T17:16:09.181 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001729288 2024-05-14T17:16:09.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:09.208 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-05-14T17:16:09.231 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-05-14T17:16:09.243 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5287, 2024-05-14T17:16:09.253 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 5288 2024-05-14T17:16:09.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:09.291 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 5287, 2024-05-14T17:16:09.335 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 5287, 2024-05-14T17:16:09.375 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-05-14T17:16:09.387 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-05-14T17:16:09.396 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-05-14T17:16:09.409 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 3294, 2024-05-14T17:16:09.425 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-05-14T17:16:09.436 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 1991, 2024-05-14T17:16:09.448 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-05-14T17:16:09.460 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-05-14T17:16:09.471 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-05-14T17:16:09.485 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-05-14T17:16:09.497 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-05-14T17:16:09.524 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-05-14T17:16:09.535 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-05-14T17:16:09.549 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-05-14T17:16:09.557 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-05-14T17:16:09.585 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-05-14T17:16:09.627 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 1, 2024-05-14T17:16:09.641 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-05-14T17:16:09.651 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:16:09.661 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-05-14T17:16:09.671 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:16:09.684 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-05-14T17:16:09.695 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-05-14T17:16:09.706 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-05-14T17:16:09.714 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:16:09.726 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-05-14T17:16:09.740 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:16:09.752 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-05-14T17:16:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-05-14T17:16:09.788 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-05-14T17:16:09.802 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-05-14T17:16:09.818 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-05-14T17:16:09.861 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-05-14T17:16:09.911 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-05-14T17:16:09.926 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-05-14T17:16:09.938 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-05-14T17:16:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-05-14T17:16:09.965 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-05-14T17:16:09.980 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-05-14T17:16:09.993 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-05-14T17:16:10.003 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-05-14T17:16:10.018 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-05-14T17:16:10.028 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-05-14T17:16:10.040 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-05-14T17:16:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 124, 2024-05-14T17:16:10.096 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-05-14T17:16:10.106 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 46, 2024-05-14T17:16:10.114 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 7, 2024-05-14T17:16:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 8, 2024-05-14T17:16:10.166 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 1, 2024-05-14T17:16:10.203 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-05-14T17:16:10.219 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-05-14T17:16:10.225 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-05-14T17:16:10.231 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-05-14T17:16:10.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:10.250 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-05-14T17:16:10.262 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-05-14T17:16:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:16:10.288 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-05-14T17:16:10.299 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-05-14T17:16:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:10.321 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-05-14T17:16:10.340 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-05-14T17:16:10.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:10.364 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-05-14T17:16:10.377 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-05-14T17:16:10.417 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T17:16:10.455 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-05-14T17:16:10.467 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-05-14T17:16:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-05-14T17:16:10.494 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-05-14T17:16:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-05-14T17:16:10.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:16:10.541 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-14T17:16:10.561 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:10.570 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 5864, 2024-05-14T17:16:10.590 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 12416028, 2024-05-14T17:16:10.598 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:10.607 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 5864, 2024-05-14T17:16:10.615 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-14T17:16:10.623 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-14T17:16:10.658 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 5864, 2024-05-14T17:16:10.698 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 12416028, 2024-05-14T17:16:10.713 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:10.725 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:10.737 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:10.748 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:10.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:10.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:10.784 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-05-14T17:16:10.796 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:10.809 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-14T17:16:10.826 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:10.858 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-14T17:16:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-14T17:16:10.881 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-14T17:16:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 5287, 2024-05-14T17:16:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 9495933, 2024-05-14T17:16:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 5287, 2024-05-14T17:16:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 9495933, 2024-05-14T17:16:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:11.000 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:11.010 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:11.021 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:11.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:11.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:11.049 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-05-14T17:16:11.057 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:11.066 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-05-14T17:16:11.102 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:11.112 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-14T17:16:11.122 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-14T17:16:11.133 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:11.146 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-14T17:16:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 5287, 2024-05-14T17:16:11.201 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 5287, 2024-05-14T17:16:11.216 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 5287, 2024-05-14T17:16:11.227 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 5287, 2024-05-14T17:16:11.240 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:11.254 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:11.268 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:11.282 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:11.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:11.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:11.319 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:16:11.332 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:11.377 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-14T17:16:11.388 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:11.400 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-14T17:16:11.412 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-14T17:16:11.421 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:11.439 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-14T17:16:11.470 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-14T17:16:11.481 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-14T17:16:11.490 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-05-14T17:16:11.499 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-05-14T17:16:11.509 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:11.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:11.525 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:11.535 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:11.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:11.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:11.568 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x56551ab0c6e0": { 2024-05-14T17:16:11.600 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:11.622 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-14T17:16:11.634 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:11.650 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-14T17:16:11.663 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-14T17:16:11.680 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:11.728 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-14T17:16:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-14T17:16:11.759 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-14T17:16:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-05-14T17:16:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-05-14T17:16:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:11.867 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-14T17:16:11.886 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-14T17:16:11.894 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-14T17:16:11.902 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-05-14T17:16:11.910 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:16:11.942 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:16:11.957 DEBUG:tasks.check_counter:Getting stats from h 2024-05-14T17:16:11.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-14T17:16:11.972 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:03 smithi092 ceph-mon[35763]: pgmap v838: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-14T17:16:11.977 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:05 smithi092 ceph-mon[35763]: pgmap v839: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-14T17:16:11.983 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:07 smithi092 ceph-mon[35763]: pgmap v840: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-14T17:16:11.989 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:09 smithi092 ceph-mon[35763]: pgmap v841: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:16:11.996 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:10 smithi092 ceph-mon[35763]: osdmap e125: 12 total, 12 up, 12 in 2024-05-14T17:16:12.004 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:11 smithi092 ceph-mon[35763]: pgmap v843: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-14T17:16:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:07 smithi203 ceph-mon[35787]: pgmap v840: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-14T17:16:12.022 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:09 smithi203 ceph-mon[35787]: pgmap v841: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:16:12.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:10 smithi203 ceph-mon[35787]: osdmap e125: 12 total, 12 up, 12 in 2024-05-14T17:16:12.052 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:11 smithi203 ceph-mon[35787]: pgmap v843: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-14T17:16:12.071 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:07 smithi069 ceph-mon[31507]: pgmap v840: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-14T17:16:12.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:09 smithi069 ceph-mon[31507]: pgmap v841: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:16:12.088 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:10 smithi069 ceph-mon[31507]: osdmap e125: 12 total, 12 up, 12 in 2024-05-14T17:16:12.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:11 smithi069 ceph-mon[31507]: pgmap v843: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-14T17:16:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:16:13.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:13 smithi092 ceph-mon[35763]: pgmap v844: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:13.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:13 smithi069 ceph-mon[31507]: pgmap v844: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:13.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:13 smithi203 ceph-mon[35787]: pgmap v844: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:13.983 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-14T17:16:13.996 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:16:14.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 4063, 2024-05-14T17:16:14.024 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 3543, 2024-05-14T17:16:14.036 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2254210, 2024-05-14T17:16:14.047 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 5770624, 2024-05-14T17:16:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 11, 2024-05-14T17:16:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-05-14T17:16:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.316071913, 2024-05-14T17:16:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.145590147, 2024-05-14T17:16:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.154925657, 2024-05-14T17:16:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.032326262, 2024-05-14T17:16:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:14.168 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3543, 2024-05-14T17:16:14.176 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.168132906, 2024-05-14T17:16:14.199 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000047454 2024-05-14T17:16:14.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.214 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:14.222 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5695, 2024-05-14T17:16:14.230 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001191956, 2024-05-14T17:16:14.238 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000209 2024-05-14T17:16:14.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.251 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:16:14.257 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:16:14.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.271 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:16:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1074, 2024-05-14T17:16:14.299 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1032, 2024-05-14T17:16:14.307 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2155641, 2024-05-14T17:16:14.346 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 4806786, 2024-05-14T17:16:14.358 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 9, 2024-05-14T17:16:14.368 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 8, 2024-05-14T17:16:14.379 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.131650164, 2024-05-14T17:16:14.391 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.049134583, 2024-05-14T17:16:14.402 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.062117666, 2024-05-14T17:16:14.413 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.025569720, 2024-05-14T17:16:14.424 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:14.436 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1029, 2024-05-14T17:16:14.448 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.068276178, 2024-05-14T17:16:14.460 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000066351 2024-05-14T17:16:14.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.483 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:14.496 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 20, 2024-05-14T17:16:14.513 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000003357, 2024-05-14T17:16:14.525 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000167 2024-05-14T17:16:14.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2007744, 2024-05-14T17:16:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 313568 2024-05-14T17:16:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:16:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 5058, 2024-05-14T17:16:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 7054, 2024-05-14T17:16:14.646 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2088281, 2024-05-14T17:16:14.651 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 5952412, 2024-05-14T17:16:14.659 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 15, 2024-05-14T17:16:14.666 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-05-14T17:16:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.328411730, 2024-05-14T17:16:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.151411543, 2024-05-14T17:16:14.690 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.205143108, 2024-05-14T17:16:14.697 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.003394214, 2024-05-14T17:16:14.704 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:14.731 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 7052, 2024-05-14T17:16:14.739 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.359478258, 2024-05-14T17:16:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000050975 2024-05-14T17:16:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.775 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:14.782 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 9452, 2024-05-14T17:16:14.789 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001503482, 2024-05-14T17:16:14.795 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000159 2024-05-14T17:16:14.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.810 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1307360, 2024-05-14T17:16:14.817 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 3865280 2024-05-14T17:16:14.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.833 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-14T17:16:14.839 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-05-14T17:16:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-14T17:16:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.872 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-05-14T17:16:14.881 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-14T17:16:14.888 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-14T17:16:14.926 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1954, 2024-05-14T17:16:14.937 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.372412678, 2024-05-14T17:16:14.949 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000702360 2024-05-14T17:16:14.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-05-14T17:16:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-14T17:16:15.027 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-14T17:16:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 19, 2024-05-14T17:16:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000715482, 2024-05-14T17:16:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000037656 2024-05-14T17:16:15.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:15.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:15.084 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-14T17:16:15.106 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 1148, 2024-05-14T17:16:15.114 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 1083, 2024-05-14T17:16:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-05-14T17:16:15.131 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1083, 2024-05-14T17:16:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 48.007644225, 2024-05-14T17:16:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.044328388 2024-05-14T17:16:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-05-14T17:16:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 65, 2024-05-14T17:16:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 2, 2024-05-14T17:16:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-05-14T17:16:15.223 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 43, 2024-05-14T17:16:15.230 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 4, 2024-05-14T17:16:15.237 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-05-14T17:16:15.244 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 1389, 2024-05-14T17:16:15.251 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 156, 2024-05-14T17:16:15.258 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 1221, 2024-05-14T17:16:15.307 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 12, 2024-05-14T17:16:15.318 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 406, 2024-05-14T17:16:15.325 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 315, 2024-05-14T17:16:15.363 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 6, 2024-05-14T17:16:15.406 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 6, 2024-05-14T17:16:15.414 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 19, 2024-05-14T17:16:15.440 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 2133, 2024-05-14T17:16:15.451 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 1982, 2024-05-14T17:16:15.463 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 59, 2024-05-14T17:16:15.475 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 36, 2024-05-14T17:16:15.487 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:16:15.500 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-05-14T17:16:15.511 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 13, 2024-05-14T17:16:15.522 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-05-14T17:16:15.532 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-05-14T17:16:15.543 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 4, 2024-05-14T17:16:15.569 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 288, 2024-05-14T17:16:15.581 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 21, 2024-05-14T17:16:15.590 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 1226, 2024-05-14T17:16:15.601 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-05-14T17:16:15.650 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:16:15.662 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-05-14T17:16:15.673 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 1013, 2024-05-14T17:16:15.697 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 1462928311, 2024-05-14T17:16:15.704 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 7, 2024-05-14T17:16:15.714 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:16:15.723 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-05-14T17:16:15.731 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:16:15.738 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:16:15.744 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:16:15.751 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:16:15.757 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:16:15.762 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-05-14T17:16:15.769 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 17, 2024-05-14T17:16:15.803 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 1580, 2024-05-14T17:16:15.814 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 1352, 2024-05-14T17:16:15.820 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:16:15.867 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:16:15.874 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:16:15.881 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 2222, 2024-05-14T17:16:15.909 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 370, 2024-05-14T17:16:15.917 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 46, 2024-05-14T17:16:15.925 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 87 2024-05-14T17:16:15.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:15.946 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-05-14T17:16:15.958 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-05-14T17:16:15.970 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-05-14T17:16:15.982 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-05-14T17:16:15.994 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 43, 2024-05-14T17:16:16.012 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 20, 2024-05-14T17:16:16.022 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-05-14T17:16:16.035 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-05-14T17:16:16.070 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:16:16.087 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 18, 2024-05-14T17:16:16.100 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 5, 2024-05-14T17:16:16.115 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 11, 2024-05-14T17:16:16.124 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 11, 2024-05-14T17:16:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-05-14T17:16:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:16:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:16:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-05-14T17:16:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-05-14T17:16:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 12, 2024-05-14T17:16:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 9004, 2024-05-14T17:16:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:16:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 7, 2024-05-14T17:16:16.250 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-05-14T17:16:16.256 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 4, 2024-05-14T17:16:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-05-14T17:16:16.301 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-05-14T17:16:16.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:16.313 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-05-14T17:16:16.332 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-05-14T17:16:16.338 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 1901, 2024-05-14T17:16:16.351 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-05-14T17:16:16.358 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-05-14T17:16:16.364 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 1901, 2024-05-14T17:16:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-05-14T17:16:16.378 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-05-14T17:16:16.386 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 5, 2024-05-14T17:16:16.392 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-05-14T17:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-05-14T17:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 5, 2024-05-14T17:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-05-14T17:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-05-14T17:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-05-14T17:16:16.502 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-05-14T17:16:16.512 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 13688666, 2024-05-14T17:16:16.523 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-05-14T17:16:16.541 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-05-14T17:16:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1118, 2024-05-14T17:16:16.568 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 8.063793761, 2024-05-14T17:16:16.579 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.007212695 2024-05-14T17:16:16.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:16.603 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-05-14T17:16:16.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:16.625 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-05-14T17:16:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1323, 2024-05-14T17:16:16.655 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 1608, 2024-05-14T17:16:16.667 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 285, 2024-05-14T17:16:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 90, 2024-05-14T17:16:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 96, 2024-05-14T17:16:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 6, 2024-05-14T17:16:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 1389, 2024-05-14T17:16:16.763 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 1714, 2024-05-14T17:16:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 325, 2024-05-14T17:16:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 6, 2024-05-14T17:16:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 1420, 2024-05-14T17:16:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 1414, 2024-05-14T17:16:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 71636, 2024-05-14T17:16:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 223608 2024-05-14T17:16:16.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:16.843 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-05-14T17:16:16.849 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 1424, 2024-05-14T17:16:16.856 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 253, 2024-05-14T17:16:16.862 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 1148, 2024-05-14T17:16:16.887 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 43, 2024-05-14T17:16:16.893 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 367, 2024-05-14T17:16:16.926 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-05-14T17:16:16.938 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-05-14T17:16:16.961 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.005288736, 2024-05-14T17:16:16.971 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005288736 2024-05-14T17:16:16.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:16.991 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-05-14T17:16:17.000 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 346, 2024-05-14T17:16:17.018 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 26.045131626, 2024-05-14T17:16:17.026 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.075274946 2024-05-14T17:16:17.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.042 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-05-14T17:16:17.052 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:17.059 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:17.067 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-05-14T17:16:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 11, 2024-05-14T17:16:17.180 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.095617232, 2024-05-14T17:16:17.192 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.008692475 2024-05-14T17:16:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-05-14T17:16:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 485, 2024-05-14T17:16:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 21.110392975, 2024-05-14T17:16:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.043526583 2024-05-14T17:16:17.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-05-14T17:16:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.391 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-05-14T17:16:17.398 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:17.407 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:17.442 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:17.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.470 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-05-14T17:16:17.476 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:17.485 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:17.492 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:17.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.506 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-05-14T17:16:17.516 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:17.523 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:17.528 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:17.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.539 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-05-14T17:16:17.545 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:17.557 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:17.565 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-05-14T17:16:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:17.681 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:15 smithi203 ceph-mon[35787]: pgmap v845: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:17.689 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:17 smithi203 ceph-mon[35787]: pgmap v846: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:17.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:15 smithi069 ceph-mon[31507]: pgmap v845: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:17.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:17 smithi069 ceph-mon[31507]: pgmap v846: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-05-14T17:16:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-05-14T17:16:17.858 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:17.892 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:17.913 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:17.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:17.929 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-05-14T17:16:17.986 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:17.993 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-05-14T17:16:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 157, 2024-05-14T17:16:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.175047768, 2024-05-14T17:16:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001114953 2024-05-14T17:16:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-05-14T17:16:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 45, 2024-05-14T17:16:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.034564307, 2024-05-14T17:16:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000768095 2024-05-14T17:16:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-05-14T17:16:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 20, 2024-05-14T17:16:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.490487865, 2024-05-14T17:16:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.024524393 2024-05-14T17:16:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-05-14T17:16:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.233 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.247 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-05-14T17:16:18.254 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.261 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.291 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.304 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-05-14T17:16:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.361 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.372 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.409 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-05-14T17:16:18.422 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.432 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.443 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.471 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-05-14T17:16:18.480 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.494 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.505 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-05-14T17:16:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.655 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-05-14T17:16:18.672 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.696 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.746 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-05-14T17:16:18.754 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.794 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.818 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.830 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-05-14T17:16:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-05-14T17:16:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-05-14T17:16:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:19.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:19.031 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-05-14T17:16:19.083 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 18, 2024-05-14T17:16:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.051113716, 2024-05-14T17:16:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002839650 2024-05-14T17:16:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:16:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:16:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-05-14T17:16:19.190 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:19.199 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:19.210 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:19.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:19.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:19.240 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-05-14T17:16:19.252 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-05-14T17:16:19.258 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-05-14T17:16:19.277 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-05-14T17:16:19.309 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-05-14T17:16:19.317 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-05-14T17:16:19.325 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 89, 2024-05-14T17:16:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 89, 2024-05-14T17:16:19.373 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1579, 2024-05-14T17:16:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-05-14T17:16:19.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-14T17:16:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:16:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-14T17:16:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:16:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:16:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:16:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:16:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:16:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:16:19.503 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:16:19.510 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:16:19.535 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:16:19.540 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:16:19.545 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:16:19.593 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:16:19.608 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:16:19.614 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-14T17:16:19.621 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:16:19.633 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-14T17:16:19.641 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:16:19.649 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:16:19.660 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:16:19.670 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:16:19.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:16:19.719 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:16:19.728 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:16:19.744 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-14T17:16:19.779 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:16:19.790 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:16:19.801 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:16:19.836 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-14T17:16:19.848 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-14T17:16:19.858 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-14T17:16:19.883 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:16:19.894 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:16:19.904 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:16:19.913 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:16:19.919 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 504896, 2024-05-14T17:16:19.925 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 1234, 2024-05-14T17:16:19.931 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:16:19.937 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-05-14T17:16:19.944 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-14T17:16:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-14T17:16:19.962 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:16:19.996 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-14T17:16:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:16:20.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-14T17:16:20.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8248, 2024-05-14T17:16:20.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 172, 2024-05-14T17:16:20.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:16:20.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:16:20.092 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-14T17:16:20.098 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-14T17:16:20.109 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 4261805, 2024-05-14T17:16:20.116 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 52896, 2024-05-14T17:16:20.123 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-14T17:16:20.129 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-14T17:16:20.136 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-14T17:16:20.157 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-14T17:16:20.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:20.178 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-05-14T17:16:20.210 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-05-14T17:16:20.221 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-05-14T17:16:20.232 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 1390, 2024-05-14T17:16:20.263 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 9819155, 2024-05-14T17:16:20.275 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 33, 2024-05-14T17:16:20.287 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 1390, 2024-05-14T17:16:20.310 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-05-14T17:16:20.322 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1390, 2024-05-14T17:16:20.333 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 8.808757027, 2024-05-14T17:16:20.343 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.006337235 2024-05-14T17:16:20.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-05-14T17:16:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-05-14T17:16:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1390, 2024-05-14T17:16:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1583 2024-05-14T17:16:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 1390, 2024-05-14T17:16:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 3, 2024-05-14T17:16:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 1387, 2024-05-14T17:16:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-05-14T17:16:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-05-14T17:16:20.542 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-05-14T17:16:20.560 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-05-14T17:16:20.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 1, 2024-05-14T17:16:20.572 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 1119, 2024-05-14T17:16:20.578 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 71, 2024-05-14T17:16:20.583 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-05-14T17:16:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-05-14T17:16:20.596 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-05-14T17:16:20.602 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-05-14T17:16:20.610 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 23, 2024-05-14T17:16:20.620 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-05-14T17:16:20.626 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-05-14T17:16:20.646 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-05-14T17:16:20.653 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-05-14T17:16:20.660 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-05-14T17:16:20.692 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:16:20.698 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-05-14T17:16:20.706 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:16:20.723 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-05-14T17:16:20.731 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-05-14T17:16:20.740 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-05-14T17:16:20.749 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:16:20.758 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-05-14T17:16:20.765 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:16:20.772 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-05-14T17:16:20.782 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-05-14T17:16:20.790 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-05-14T17:16:20.820 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-05-14T17:16:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-05-14T17:16:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-05-14T17:16:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-05-14T17:16:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-05-14T17:16:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-05-14T17:16:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-05-14T17:16:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-05-14T17:16:20.932 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-05-14T17:16:20.945 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-05-14T17:16:20.953 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-05-14T17:16:20.965 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-05-14T17:16:20.975 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-05-14T17:16:20.985 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-05-14T17:16:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 125, 2024-05-14T17:16:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-05-14T17:16:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 47, 2024-05-14T17:16:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 11, 2024-05-14T17:16:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 11, 2024-05-14T17:16:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-05-14T17:16:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-05-14T17:16:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 274, 2024-05-14T17:16:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 4, 2024-05-14T17:16:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 61 2024-05-14T17:16:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:21.120 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-05-14T17:16:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-05-14T17:16:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:16:21.167 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-05-14T17:16:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-05-14T17:16:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-05-14T17:16:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 8139 2024-05-14T17:16:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-05-14T17:16:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-05-14T17:16:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 5, 2024-05-14T17:16:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-05-14T17:16:21.260 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 5, 2024-05-14T17:16:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 5, 2024-05-14T17:16:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 5, 2024-05-14T17:16:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-05-14T17:16:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:16:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-14T17:16:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 10191, 2024-05-14T17:16:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 5378592, 2024-05-14T17:16:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 10191, 2024-05-14T17:16:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-14T17:16:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-14T17:16:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 10191, 2024-05-14T17:16:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 5378592, 2024-05-14T17:16:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:21.488 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:21.498 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:21.507 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:21.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:21.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:21.543 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-05-14T17:16:21.551 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:21.560 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-14T17:16:21.570 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:21.579 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-14T17:16:21.587 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-14T17:16:21.600 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:21.624 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-14T17:16:21.666 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1390, 2024-05-14T17:16:21.684 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 9819828, 2024-05-14T17:16:21.695 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1390, 2024-05-14T17:16:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 9819828, 2024-05-14T17:16:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-05-14T17:16:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-05-14T17:16:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-14T17:16:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-14T17:16:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-14T17:16:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1390, 2024-05-14T17:16:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 1390, 2024-05-14T17:16:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1390, 2024-05-14T17:16:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1390, 2024-05-14T17:16:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:22.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:22.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:22.103 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:16:22.115 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:22.134 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-14T17:16:22.152 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:22.168 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 5, 2024-05-14T17:16:22.186 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 705, 2024-05-14T17:16:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 5, 2024-05-14T17:16:22.242 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-14T17:16:22.290 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-14T17:16:22.307 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1, 2024-05-14T17:16:22.320 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 705, 2024-05-14T17:16:22.372 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:22.411 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:22.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:22.453 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x555dcad68500": { 2024-05-14T17:16:22.464 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-14T17:16:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 1901, 2024-05-14T17:16:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 9494362, 2024-05-14T17:16:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 1901, 2024-05-14T17:16:22.602 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-14T17:16:22.613 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-14T17:16:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1118, 2024-05-14T17:16:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 9494362, 2024-05-14T17:16:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:22.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:22.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:22.742 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-14T17:16:22.759 INFO:tasks.check_counter:Daemon mds.h mds.exported=4 2024-05-14T17:16:22.773 INFO:tasks.check_counter:Daemon mds.h mds.imported=21 2024-05-14T17:16:22.800 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-05-14T17:16:22.814 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:16:22.829 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:16:22.873 DEBUG:tasks.check_counter:Getting stats from k 2024-05-14T17:16:22.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-14T17:16:22.907 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:15 smithi092 ceph-mon[35763]: pgmap v845: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:22.923 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:17 smithi092 ceph-mon[35763]: pgmap v846: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:22.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:19 smithi092 ceph-mon[35763]: pgmap v847: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:22.959 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:19 smithi092 ceph-mon[35763]: osdmap e126: 12 total, 12 up, 12 in 2024-05-14T17:16:22.969 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:21 smithi092 ceph-mon[35763]: pgmap v849: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:22.982 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:19 smithi203 ceph-mon[35787]: pgmap v847: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:22.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:19 smithi203 ceph-mon[35787]: osdmap e126: 12 total, 12 up, 12 in 2024-05-14T17:16:23.008 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:21 smithi203 ceph-mon[35787]: pgmap v849: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:23.020 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:19 smithi069 ceph-mon[31507]: pgmap v847: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:23.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:19 smithi069 ceph-mon[31507]: osdmap e126: 12 total, 12 up, 12 in 2024-05-14T17:16:23.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:21 smithi069 ceph-mon[31507]: pgmap v849: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:16:23.351 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:23 smithi092 ceph-mon[35763]: pgmap v850: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:23.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:23 smithi069 ceph-mon[31507]: pgmap v850: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:23.467 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:23 smithi203 ceph-mon[35787]: pgmap v850: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:24.972 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-14T17:16:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:16:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 4, 2024-05-14T17:16:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 4, 2024-05-14T17:16:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2944, 2024-05-14T17:16:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 448, 2024-05-14T17:16:25.040 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-05-14T17:16:25.051 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 0, 2024-05-14T17:16:25.064 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.001358538, 2024-05-14T17:16:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000036447, 2024-05-14T17:16:25.093 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.000573182, 2024-05-14T17:16:25.100 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:16:25.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:25.114 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3, 2024-05-14T17:16:25.147 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000784746, 2024-05-14T17:16:25.162 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000261582 2024-05-14T17:16:25.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:25.190 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:25.220 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:25.229 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-14T17:16:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T17:16:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:16:25.344 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 3, 2024-05-14T17:16:25.355 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 371, 2024-05-14T17:16:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:16:25.384 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 301504, 2024-05-14T17:16:25.398 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 6, 2024-05-14T17:16:25.411 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-05-14T17:16:25.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.021272548, 2024-05-14T17:16:25.501 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.015105639, 2024-05-14T17:16:25.523 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.001819772, 2024-05-14T17:16:25.564 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:16:25.583 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:25.594 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 368, 2024-05-14T17:16:25.610 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.032220759, 2024-05-14T17:16:25.621 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000087556 2024-05-14T17:16:25.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:25.661 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:25.679 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:25.688 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:25.705 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:25.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:16:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 301504 2024-05-14T17:16:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:16:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 496, 2024-05-14T17:16:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 485, 2024-05-14T17:16:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 1789520, 2024-05-14T17:16:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 300288, 2024-05-14T17:16:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 2, 2024-05-14T17:16:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-05-14T17:16:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.081273178, 2024-05-14T17:16:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.028581956, 2024-05-14T17:16:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.046808937, 2024-05-14T17:16:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.012115832, 2024-05-14T17:16:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:26.093 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 484, 2024-05-14T17:16:26.109 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.026755434, 2024-05-14T17:16:26.122 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000055279 2024-05-14T17:16:26.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:26.165 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:26.184 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:26.195 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:26.210 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1789520, 2024-05-14T17:16:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 300288 2024-05-14T17:16:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-14T17:16:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 0, 2024-05-14T17:16:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-14T17:16:26.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-14T17:16:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:16:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-14T17:16:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:16:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:16:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:16:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:16:26.506 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:16:26.517 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:16:26.533 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:16:26.544 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:16:26.561 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:16:26.572 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:16:26.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:16:26.649 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:16:26.662 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:16:26.672 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-14T17:16:26.688 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:16:26.705 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-14T17:16:26.722 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:16:26.733 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:16:26.749 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:16:26.760 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:16:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:16:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:16:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:16:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-14T17:16:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:16:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:16:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:16:26.874 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-14T17:16:26.890 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-14T17:16:26.896 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-14T17:16:26.910 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:16:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:16:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:16:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:16:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 148704, 2024-05-14T17:16:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 29, 2024-05-14T17:16:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:16:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-05-14T17:16:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-14T17:16:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-14T17:16:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:16:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-14T17:16:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:16:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-14T17:16:27.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 0, 2024-05-14T17:16:27.123 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 0, 2024-05-14T17:16:27.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:16:27.150 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:16:27.169 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-14T17:16:27.174 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-14T17:16:27.184 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 928, 2024-05-14T17:16:27.193 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 2, 2024-05-14T17:16:27.202 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-14T17:16:27.211 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-14T17:16:27.220 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-14T17:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-14T17:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-14T17:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-14T17:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-14T17:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-14T17:16:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 499, 2024-05-14T17:16:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1719421, 2024-05-14T17:16:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 499, 2024-05-14T17:16:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-14T17:16:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-14T17:16:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 499, 2024-05-14T17:16:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1719421, 2024-05-14T17:16:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-14T17:16:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-14T17:16:27.499 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-14T17:16:27.508 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-14T17:16:27.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:27.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-14T17:16:27.534 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-14T17:16:27.554 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:25 smithi092 ceph-mon[35763]: pgmap v851: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:27.562 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:25 smithi203 ceph-mon[35787]: pgmap v851: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:27.615 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:27 smithi203 ceph-mon[35787]: pgmap v852: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:27.649 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:25 smithi069 ceph-mon[31507]: pgmap v851: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:27.662 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:27 smithi069 ceph-mon[31507]: pgmap v852: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:27.673 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-14T17:16:27.685 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-14T17:16:27.692 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-14T17:16:27.712 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-14T17:16:27.725 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-14T17:16:27.750 DEBUG:tasks.check_counter:Getting stats from c 2024-05-14T17:16:27.761 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-14T17:16:27.776 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:27 smithi092 ceph-mon[35763]: pgmap v852: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:28.035 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T17:16:29.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:29 smithi069 ceph-mon[31507]: pgmap v853: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:16:29.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:29 smithi203 ceph-mon[35787]: pgmap v853: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:16:29.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:29 smithi092 ceph-mon[35763]: pgmap v853: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:16:30.095 INFO:teuthology.orchestra.run.smithi203.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-05-14T17:16:30.730 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-14T17:16:30.731 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi203 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump' 2024-05-14T17:16:30.732 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-14T17:16:30.744 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-14T17:16:30.744 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T17:16:30.744 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T17:16:30.766 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T17:16:30.767 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T17:16:30.827 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-14T17:16:31.121 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:31.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:31 smithi069 ceph-mon[31507]: pgmap v854: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T17:16:31.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:31 smithi203 ceph-mon[35787]: pgmap v854: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T17:16:31.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:31 smithi092 ceph-mon[35763]: pgmap v854: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T17:16:32.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.888+0000 7faa0b979700 1 -- 172.21.15.69:0/3990308060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04102440 msgr2=0x7faa041028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:32.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.888+0000 7faa0b979700 1 --2- 172.21.15.69:0/3990308060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04102440 0x7faa041028c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009990 tx=0x7fa9f4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.888+0000 7faa0b979700 1 -- 172.21.15.69:0/3990308060 shutdown_connections 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.888+0000 7faa0b979700 1 --2- 172.21.15.69:0/3990308060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa04102e00 0x7faa0410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.888+0000 7faa0b979700 1 --2- 172.21.15.69:0/3990308060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa04108520 0x7faa04108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.888+0000 7faa0b979700 1 --2- 172.21.15.69:0/3990308060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04102440 0x7faa041028c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.888+0000 7faa0b979700 1 -- 172.21.15.69:0/3990308060 >> 172.21.15.69:0/3990308060 conn(0x7faa040793f0 msgr2=0x7faa04079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.888+0000 7faa0b979700 1 -- 172.21.15.69:0/3990308060 shutdown_connections 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.888+0000 7faa0b979700 1 -- 172.21.15.69:0/3990308060 wait complete. 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa0b979700 1 Processor -- start 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa0b979700 1 -- start start 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa0b979700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa04102440 0x7faa0419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa0b979700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa04102e00 0x7faa0419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa0b979700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04108520 0x7faa04199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:32.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa0b979700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faa04112650 con 0x7faa04108520 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa0b979700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa041124d0 con 0x7faa04102440 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa0b979700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa041127d0 con 0x7faa04102e00 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa09f16700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04108520 0x7faa04199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa09f16700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04108520 0x7faa04199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56984/0 (socket says 172.21.15.69:56984) 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa09f16700 1 -- 172.21.15.69:0/1198733586 learned_addr learned my addr 172.21.15.69:0/1198733586 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.889+0000 7faa09715700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa04102440 0x7faa0419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7faa08f14700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa04102e00 0x7faa0419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7faa09f16700 1 -- 172.21.15.69:0/1198733586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa04102e00 msgr2=0x7faa0419fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7faa09f16700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa04102e00 0x7faa0419fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7faa09f16700 1 -- 172.21.15.69:0/1198733586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa04102440 msgr2=0x7faa0419f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7faa09f16700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa04102440 0x7faa0419f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7faa09f16700 1 -- 172.21.15.69:0/1198733586 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0419a2a0 con 0x7faa04108520 2024-05-14T17:16:32.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7faa09f16700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04108520 0x7faa04199b40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc007d70 tx=0x7fa9fc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:32.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7fa9fa7fc700 1 -- 172.21.15.69:0/1198733586 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc016020 con 0x7faa04108520 2024-05-14T17:16:32.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7fa9fa7fc700 1 -- 172.21.15.69:0/1198733586 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9fc00ad40 con 0x7faa04108520 2024-05-14T17:16:32.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7faa0b979700 1 -- 172.21.15.69:0/1198733586 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0419a590 con 0x7faa04108520 2024-05-14T17:16:32.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7fa9fa7fc700 1 -- 172.21.15.69:0/1198733586 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0115e0 con 0x7faa04108520 2024-05-14T17:16:32.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.890+0000 7faa0b979700 1 -- 172.21.15.69:0/1198733586 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa041a6800 con 0x7faa04108520 2024-05-14T17:16:32.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.891+0000 7faa0b979700 1 -- 172.21.15.69:0/1198733586 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0406a230 con 0x7faa04108520 2024-05-14T17:16:32.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.893+0000 7fa9fa7fc700 1 -- 172.21.15.69:0/1198733586 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa9fc019020 con 0x7faa04108520 2024-05-14T17:16:32.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.894+0000 7fa9fa7fc700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9f00b29f0 0x7fa9f00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:32.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.894+0000 7faa09715700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9f00b29f0 0x7fa9f00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:32.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.894+0000 7fa9fa7fc700 1 -- 172.21.15.69:0/1198733586 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7fa9fc0e28a0 con 0x7faa04108520 2024-05-14T17:16:32.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.894+0000 7faa09715700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9f00b29f0 0x7fa9f00b4eb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa00009a10 tx=0x7faa00009190 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:32.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:32.897+0000 7fa9fa7fc700 1 -- 172.21.15.69:0/1198733586 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9fc020020 con 0x7faa04108520 2024-05-14T17:16:33.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.121+0000 7faa0b979700 1 -- 172.21.15.69:0/1198733586 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faa0419afe0 con 0x7faa04108520 2024-05-14T17:16:33.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.121+0000 7fa9fa7fc700 1 -- 172.21.15.69:0/1198733586 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v126) ==== 81+0+942 (secure 0 0 0) 0x7fa9fc0aa380 con 0x7faa04108520 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/4012740750 2024-05-15T16:49:54.600513+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/4012740750 2024-05-15T16:49:54.600513+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/70559253 2024-05-15T16:49:54.600513+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3194136272 2024-05-15T16:39:31.768055+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3983720568 2024-05-15T16:39:31.768055+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2944165482 2024-05-15T16:49:54.600513+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3714306679 2024-05-15T16:39:31.768055+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1931804604 2024-05-15T16:49:54.600513+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/421328414 2024-05-15T16:39:31.768055+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3752620885 2024-05-15T16:39:05.658063+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3176577502 2024-05-15T16:49:54.600513+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/4209517756 2024-05-15T16:39:05.658063+0000 2024-05-14T17:16:33.121 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/421328414 2024-05-15T16:39:31.768055+0000 2024-05-14T17:16:33.122 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2746326029 2024-05-15T16:39:05.658063+0000 2024-05-14T17:16:33.122 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4187352755 2024-05-15T16:39:05.658063+0000 2024-05-14T17:16:33.122 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/4209517756 2024-05-15T16:39:05.658063+0000 2024-05-14T17:16:33.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.123+0000 7faa0b979700 1 -- 172.21.15.69:0/1198733586 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9f00b29f0 msgr2=0x7fa9f00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.123+0000 7faa0b979700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9f00b29f0 0x7fa9f00b4eb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa00009a10 tx=0x7faa00009190 comp rx=0 tx=0).stop 2024-05-14T17:16:33.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.123+0000 7faa0b979700 1 -- 172.21.15.69:0/1198733586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04108520 msgr2=0x7faa04199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.123+0000 7faa0b979700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04108520 0x7faa04199b40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc007d70 tx=0x7fa9fc00a340 comp rx=0 tx=0).stop 2024-05-14T17:16:33.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.124+0000 7faa0b979700 1 -- 172.21.15.69:0/1198733586 shutdown_connections 2024-05-14T17:16:33.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.124+0000 7faa0b979700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fa9f00b29f0 0x7fa9f00b4eb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.124+0000 7faa0b979700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04108520 0x7faa04199b40 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.124+0000 7faa0b979700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa04102440 0x7faa0419f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.124+0000 7faa0b979700 1 --2- 172.21.15.69:0/1198733586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa04102e00 0x7faa0419fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.124+0000 7faa0b979700 1 -- 172.21.15.69:0/1198733586 >> 172.21.15.69:0/1198733586 conn(0x7faa040793f0 msgr2=0x7faa0406b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:33.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.124+0000 7faa0b979700 1 -- 172.21.15.69:0/1198733586 shutdown_connections 2024-05-14T17:16:33.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:33.124+0000 7faa0b979700 1 -- 172.21.15.69:0/1198733586 wait complete. 2024-05-14T17:16:33.124 INFO:teuthology.orchestra.run.smithi069.stderr:listed 16 entries 2024-05-14T17:16:33.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:33 smithi069 ceph-mon[31507]: pgmap v855: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:16:33.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:33 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1198733586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T17:16:33.458 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:33 smithi203 ceph-mon[35787]: pgmap v855: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:16:33.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:33 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1198733586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T17:16:33.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:33 smithi092 ceph-mon[35763]: pgmap v855: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:16:33.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:33 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1198733586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T17:16:33.813 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-14T17:16:33.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:16:33.813 DEBUG:teuthology.orchestra.run.smithi069:> sudo umount /tmp/cephfs 2024-05-14T17:16:33.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T17:16:33.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:16:33.862 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-14T17:16:33.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:16:33.920 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-05-14T17:16:33.974 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-14T17:16:33.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:16:33.975 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-14T17:16:34.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:16:34.056 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-05-14T17:16:34.161 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-14T17:16:34.161 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-14T17:16:34.162 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T17:16:34.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056292,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=608 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/a2e95a0c242fd385e91c20a70e1eb5a870701ddd0cbd703215acf0c04a15325b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/EZRXFJTFYXFGGBICH7EPEJN2ZI:/var/lib/containers/storage/overlay/l/ZHSSIN4HPD43EADSS4UNWZBN6N,upperdir=/var/lib/containers/storage/overlay/a2e95a0c242fd385e91c20a70e1eb5a870701ddd0cbd703215acf0c04a15325b/diff,workdir=/var/lib/containers/storage/overlay/a2e95a0c242fd385e91c20a70e1eb5a870701ddd0cbd703215acf0c04a15325b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ce871c42dd8c83056a356e65cc94b158f7c0520a9b1509698c7f51d149737287/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/EZRXFJTFYXFGGBICH7EPEJN2ZI:/var/lib/containers/storage/overlay/l/ZHSSIN4HPD43EADSS4UNWZBN6N,upperdir=/var/lib/containers/storage/overlay/ce871c42dd8c83056a356e65cc94b158f7c0520a9b1509698c7f51d149737287/diff,workdir=/var/lib/containers/storage/overlay/ce871c42dd8c83056a356e65cc94b158f7c0520a9b1509698c7f51d149737287/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/3077466b27beed2db13f83791b174c259f60b558ebca38a5c45135d734807879/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/EZRXFJTFYXFGGBICH7EPEJN2ZI:/var/lib/containers/storage/overlay/l/ZHSSIN4HPD43EADSS4UNWZBN6N,upperdir=/var/lib/containers/storage/overlay/3077466b27beed2db13f83791b174c259f60b558ebca38a5c45135d734807879/diff,workdir=/var/lib/containers/storage/overlay/3077466b27beed2db13f83791b174c259f60b558ebca38a5c45135d734807879/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/dad8659dce793144570166e51062f68372395e44626dee64a38a91075d3bc4b5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/EZRXFJTFYXFGGBICH7EPEJN2ZI:/var/lib/containers/storage/overlay/l/ZHSSIN4HPD43EADSS4UNWZBN6N,upperdir=/var/lib/containers/storage/overlay/dad8659dce793144570166e51062f68372395e44626dee64a38a91075d3bc4b5/diff,workdir=/var/lib/containers/storage/overlay/dad8659dce793144570166e51062f68372395e44626dee64a38a91075d3bc4b5/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8df521b526097a561fc676fc093d94d5b889c5ffeb33f483fa9182a9326dfc53/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/EZRXFJTFYXFGGBICH7EPEJN2ZI:/var/lib/containers/storage/overlay/l/ZHSSIN4HPD43EADSS4UNWZBN6N,upperdir=/var/lib/containers/storage/overlay/8df521b526097a561fc676fc093d94d5b889c5ffeb33f483fa9182a9326dfc53/diff,workdir=/var/lib/containers/storage/overlay/8df521b526097a561fc676fc093d94d5b889c5ffeb33f483fa9182a9326dfc53/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a7e78c8f749ed9200d6f8ee9f22de35fd1b6a60a1d9af9ba988090cf6fada6df/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/EZRXFJTFYXFGGBICH7EPEJN2ZI:/var/lib/containers/storage/overlay/l/ZHSSIN4HPD43EADSS4UNWZBN6N,upperdir=/var/lib/containers/storage/overlay/a7e78c8f749ed9200d6f8ee9f22de35fd1b6a60a1d9af9ba988090cf6fada6df/diff,workdir=/var/lib/containers/storage/overlay/a7e78c8f749ed9200d6f8ee9f22de35fd1b6a60a1d9af9ba988090cf6fada6df/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e79579b0b5d1f664238fc2d99c4018d244d575c662fa39844fc6e7488ba0f383/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/EZRXFJTFYXFGGBICH7EPEJN2ZI:/var/lib/containers/storage/overlay/l/ZHSSIN4HPD43EADSS4UNWZBN6N,upperdir=/var/lib/containers/storage/overlay/e79579b0b5d1f664238fc2d99c4018d244d575c662fa39844fc6e7488ba0f383/diff,workdir=/var/lib/containers/storage/overlay/e79579b0b5d1f664238fc2d99c4018d244d575c662fa39844fc6e7488ba0f383/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9377b26809c11466489deed0cc2e3dabfe3bcb257b9d44d965ecc94e377a08b6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/EZRXFJTFYXFGGBICH7EPEJN2ZI:/var/lib/containers/storage/overlay/l/ZHSSIN4HPD43EADSS4UNWZBN6N,upperdir=/var/lib/containers/storage/overlay/9377b26809c11466489deed0cc2e3dabfe3bcb257b9d44d965ecc94e377a08b6/diff,workdir=/var/lib/containers/storage/overlay/9377b26809c11466489deed0cc2e3dabfe3bcb257b9d44d965ecc94e377a08b6/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6756fd1c12c7722a42b14319bb00c68e74782d2cf83c029dbb8fd558a8be5790/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/EZRXFJTFYXFGGBICH7EPEJN2ZI:/var/lib/containers/storage/overlay/l/ZHSSIN4HPD43EADSS4UNWZBN6N,upperdir=/var/lib/containers/storage/overlay/6756fd1c12c7722a42b14319bb00c68e74782d2cf83c029dbb8fd558a8be5790/diff,workdir=/var/lib/containers/storage/overlay/6756fd1c12c7722a42b14319bb00c68e74782d2cf83c029dbb8fd558a8be5790/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/199c31a80a3cd5556f9bc06c625c784050063c55cc42e6ec6bdadb456e2cc7ff/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/EZRXFJTFYXFGGBICH7EPEJN2ZI:/var/lib/containers/storage/overlay/l/ZHSSIN4HPD43EADSS4UNWZBN6N,upperdir=/var/lib/containers/storage/overlay/199c31a80a3cd5556f9bc06c625c784050063c55cc42e6ec6bdadb456e2cc7ff/diff,workdir=/var/lib/containers/storage/overlay/199c31a80a3cd5556f9bc06c625c784050063c55cc42e6ec6bdadb456e2cc7ff/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755,inode64 0 0 2024-05-14T17:16:34.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:16:34.209 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-05-14T17:16:34.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:16:34.247 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-05-14T17:16:34.335 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2024-05-14T17:16:34.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:16:34.338 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-14T17:16:34.348 INFO:tasks.cephadm:Teardown begin 2024-05-14T17:16:34.348 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T17:16:34.348 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:16:34.594 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:35.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:35 smithi069 ceph-mon[31507]: pgmap v856: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:16:35.458 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:35 smithi203 ceph-mon[35787]: pgmap v856: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:16:35.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:35 smithi092 ceph-mon[35763]: pgmap v856: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:16:36.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 -- 172.21.15.69:0/1132335596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f633006c630 msgr2=0x7f63301103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:36.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 --2- 172.21.15.69:0/1132335596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f633006c630 0x7f63301103d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f632800b0a0 tx=0x7f632800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 -- 172.21.15.69:0/1132335596 shutdown_connections 2024-05-14T17:16:36.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 --2- 172.21.15.69:0/1132335596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f633006c630 0x7f63301103d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 --2- 172.21.15.69:0/1132335596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63301095b0 0x7f6330109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 --2- 172.21.15.69:0/1132335596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633006bc70 0x7f633006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 -- 172.21.15.69:0/1132335596 >> 172.21.15.69:0/1132335596 conn(0x7f63300793f0 msgr2=0x7f6330079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:36.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 -- 172.21.15.69:0/1132335596 shutdown_connections 2024-05-14T17:16:36.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 -- 172.21.15.69:0/1132335596 wait complete. 2024-05-14T17:16:36.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 Processor -- start 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.300+0000 7f6337ab9700 1 -- start start 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6337ab9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f633006bc70 0x7f633019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6337ab9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633006c630 0x7f633019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6337ab9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63301095b0 0x7f6330199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6337ab9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63301136e0 con 0x7f633006bc70 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6337ab9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6330113560 con 0x7f633006c630 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6337ab9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6330113860 con 0x7f63301095b0 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6335855700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f633006bc70 0x7f633019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6335855700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f633006bc70 0x7f633019f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33196/0 (socket says 172.21.15.69:33196) 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6335855700 1 -- 172.21.15.69:0/2823625418 learned_addr learned my addr 172.21.15.69:0/2823625418 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6335054700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633006c630 0x7f633019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6336056700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63301095b0 0x7f6330199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6335855700 1 -- 172.21.15.69:0/2823625418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63301095b0 msgr2=0x7f6330199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6335855700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63301095b0 0x7f6330199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6335855700 1 -- 172.21.15.69:0/2823625418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633006c630 msgr2=0x7f633019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:36.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6335855700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633006c630 0x7f633019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6335855700 1 -- 172.21.15.69:0/2823625418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633019a1a0 con 0x7f633006bc70 2024-05-14T17:16:36.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6335855700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f633006bc70 0x7f633019f880 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f632000e9a0 tx=0x7f632000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:36.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6326ffd700 1 -- 172.21.15.69:0/2823625418 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632000ca10 con 0x7f633006bc70 2024-05-14T17:16:36.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.301+0000 7f6326ffd700 1 -- 172.21.15.69:0/2823625418 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f632000cbb0 con 0x7f633006bc70 2024-05-14T17:16:36.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.302+0000 7f6337ab9700 1 -- 172.21.15.69:0/2823625418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633019a490 con 0x7f633006bc70 2024-05-14T17:16:36.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.302+0000 7f6326ffd700 1 -- 172.21.15.69:0/2823625418 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320018680 con 0x7f633006bc70 2024-05-14T17:16:36.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.302+0000 7f6337ab9700 1 -- 172.21.15.69:0/2823625418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63301a6750 con 0x7f633006bc70 2024-05-14T17:16:36.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.302+0000 7f6337ab9700 1 -- 172.21.15.69:0/2823625418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633006a230 con 0x7f633006bc70 2024-05-14T17:16:36.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.304+0000 7f6326ffd700 1 -- 172.21.15.69:0/2823625418 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6320010430 con 0x7f633006bc70 2024-05-14T17:16:36.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.305+0000 7f6326ffd700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f631c0b29f0 0x7f631c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:36.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.305+0000 7f6335054700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f631c0b29f0 0x7f631c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:36.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.305+0000 7f6326ffd700 1 -- 172.21.15.69:0/2823625418 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7f63200e2b00 con 0x7f633006bc70 2024-05-14T17:16:36.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.305+0000 7f6335054700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f631c0b29f0 0x7f631c0b4eb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6330079aa0 tx=0x7f632c00b3c0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:36.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.309+0000 7f6326ffd700 1 -- 172.21.15.69:0/2823625418 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f63200aa470 con 0x7f633006bc70 2024-05-14T17:16:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.561+0000 7f6337ab9700 1 -- 172.21.15.69:0/2823625418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f633019af70 con 0x7f633006bc70 2024-05-14T17:16:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.562+0000 7f6326ffd700 1 -- 172.21.15.69:0/2823625418 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11031 (secure 0 0 0) 0x7f63200a9d70 con 0x7f633006bc70 2024-05-14T17:16:36.562 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:16:36.562 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:08:18.843541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24385":{"gid":24385,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T17:16:36.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.564+0000 7f6337ab9700 1 -- 172.21.15.69:0/2823625418 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f631c0b29f0 msgr2=0x7f631c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:36.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.564+0000 7f6337ab9700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f631c0b29f0 0x7f631c0b4eb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6330079aa0 tx=0x7f632c00b3c0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.564+0000 7f6337ab9700 1 -- 172.21.15.69:0/2823625418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f633006bc70 msgr2=0x7f633019f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:36.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.564+0000 7f6337ab9700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f633006bc70 0x7f633019f880 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f632000e9a0 tx=0x7f632000ecb0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.565+0000 7f6337ab9700 1 -- 172.21.15.69:0/2823625418 shutdown_connections 2024-05-14T17:16:36.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.565+0000 7f6337ab9700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f631c0b29f0 0x7f631c0b4eb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.565+0000 7f6337ab9700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f633006bc70 0x7f633019f880 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.565+0000 7f6337ab9700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63301095b0 0x7f6330199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.565+0000 7f6337ab9700 1 --2- 172.21.15.69:0/2823625418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633006c630 0x7f633019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.565+0000 7f6337ab9700 1 -- 172.21.15.69:0/2823625418 >> 172.21.15.69:0/2823625418 conn(0x7f63300793f0 msgr2=0x7f6330101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.565+0000 7f6337ab9700 1 -- 172.21.15.69:0/2823625418 shutdown_connections 2024-05-14T17:16:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:36.565+0000 7f6337ab9700 1 -- 172.21.15.69:0/2823625418 wait complete. 2024-05-14T17:16:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 61 2024-05-14T17:16:37.212 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:16:37.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:37 smithi069 ceph-mon[31507]: pgmap v857: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T17:16:37.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:37 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2823625418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:16:37.453 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:37.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:37 smithi203 ceph-mon[35787]: pgmap v857: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T17:16:37.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:37 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2823625418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:16:37.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:37 smithi092 ceph-mon[35763]: pgmap v857: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T17:16:37.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:37 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2823625418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:16:38.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:38 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:16:38.458 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:38 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:16:38.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:38 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:16:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.184+0000 7fbeb8c52700 1 -- 172.21.15.69:0/97165199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb4102440 msgr2=0x7fbeb41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.184+0000 7fbeb8c52700 1 --2- 172.21.15.69:0/97165199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb4102440 0x7fbeb41028c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009990 tx=0x7fbea8009ca0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.184+0000 7fbeb8c52700 1 -- 172.21.15.69:0/97165199 shutdown_connections 2024-05-14T17:16:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.184+0000 7fbeb8c52700 1 --2- 172.21.15.69:0/97165199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb4102e00 0x7fbeb410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.184+0000 7fbeb8c52700 1 --2- 172.21.15.69:0/97165199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeb4108520 0x7fbeb4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.184+0000 7fbeb8c52700 1 --2- 172.21.15.69:0/97165199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb4102440 0x7fbeb41028c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.184+0000 7fbeb8c52700 1 -- 172.21.15.69:0/97165199 >> 172.21.15.69:0/97165199 conn(0x7fbeb40793f0 msgr2=0x7fbeb4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.184+0000 7fbeb8c52700 1 -- 172.21.15.69:0/97165199 shutdown_connections 2024-05-14T17:16:39.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.184+0000 7fbeb8c52700 1 -- 172.21.15.69:0/97165199 wait complete. 2024-05-14T17:16:39.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.185+0000 7fbeb8c52700 1 Processor -- start 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.185+0000 7fbeb8c52700 1 -- start start 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb8c52700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb4102440 0x7fbeb419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb8c52700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb4102e00 0x7fbeb419afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb8c52700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeb4108520 0x7fbeb41a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb8c52700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb41125f0 con 0x7fbeb4102e00 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb8c52700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4112470 con 0x7fbeb4108520 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb8c52700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4112770 con 0x7fbeb4102440 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb4102e00 0x7fbeb419afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb4102e00 0x7fbeb419afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33218/0 (socket says 172.21.15.69:33218) 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb1d9b700 1 -- 172.21.15.69:0/2019678957 learned_addr learned my addr 172.21.15.69:0/2019678957 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb259c700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb4102440 0x7fbeb419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb2d9d700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeb4108520 0x7fbeb41a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb1d9b700 1 -- 172.21.15.69:0/2019678957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb4102440 msgr2=0x7fbeb419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:39.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb1d9b700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb4102440 0x7fbeb419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb1d9b700 1 -- 172.21.15.69:0/2019678957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeb4108520 msgr2=0x7fbeb41a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb1d9b700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeb4108520 0x7fbeb41a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb1d9b700 1 -- 172.21.15.69:0/2019678957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb41a6030 con 0x7fbeb4102e00 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb2d9d700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeb4108520 0x7fbeb41a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb1d9b700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb4102e00 0x7fbeb419afb0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbea8000c00 tx=0x7fbea80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbea37fe700 1 -- 172.21.15.69:0/2019678957 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8027070 con 0x7fbeb4102e00 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbea37fe700 1 -- 172.21.15.69:0/2019678957 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea8003f20 con 0x7fbeb4102e00 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbea37fe700 1 -- 172.21.15.69:0/2019678957 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8022cf0 con 0x7fbeb4102e00 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.186+0000 7fbeb8c52700 1 -- 172.21.15.69:0/2019678957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb41a6200 con 0x7fbeb4102e00 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.187+0000 7fbeb8c52700 1 -- 172.21.15.69:0/2019678957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb41a6650 con 0x7fbeb4102e00 2024-05-14T17:16:39.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.187+0000 7fbeb8c52700 1 -- 172.21.15.69:0/2019678957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb406a230 con 0x7fbeb4102e00 2024-05-14T17:16:39.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.189+0000 7fbea37fe700 1 -- 172.21.15.69:0/2019678957 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbea802a070 con 0x7fbeb4102e00 2024-05-14T17:16:39.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.190+0000 7fbea37fe700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe9c0b29f0 0x7fbe9c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:39.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.190+0000 7fbeb259c700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe9c0b29f0 0x7fbe9c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:39.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.190+0000 7fbea37fe700 1 -- 172.21.15.69:0/2019678957 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7fbea80e3720 con 0x7fbeb4102e00 2024-05-14T17:16:39.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.190+0000 7fbeb259c700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe9c0b29f0 0x7fbe9c0b4eb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbea4007b60 tx=0x7fbea40073d0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:39.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.193+0000 7fbea37fe700 1 -- 172.21.15.69:0/2019678957 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbea8013070 con 0x7fbeb4102e00 2024-05-14T17:16:39.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:39 smithi069 ceph-mon[31507]: pgmap v858: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T17:16:39.414 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:39 smithi203 ceph-mon[35787]: pgmap v858: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T17:16:39.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.446+0000 7fbeb8c52700 1 -- 172.21.15.69:0/2019678957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeb402e720 con 0x7fbeb4102e00 2024-05-14T17:16:39.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.447+0000 7fbea37fe700 1 -- 172.21.15.69:0/2019678957 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11031 (secure 0 0 0) 0x7fbea80ab180 con 0x7fbeb4102e00 2024-05-14T17:16:39.446 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:16:39.446 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/894337492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":894337492},{"type":"v1","addr":"172.21.15.69:6835","nonce":894337492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1744189392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1744189392},{"type":"v1","addr":"172.21.15.69:6837","nonce":1744189392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3026437817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3026437817},{"type":"v1","addr":"172.21.15.69:6839","nonce":3026437817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/3552362754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":3552362754},{"type":"v1","addr":"172.21.15.69:6841","nonce":3552362754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/786223395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":786223395},{"type":"v1","addr":"172.21.15.92:6833","nonce":786223395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:46:45.333110+0000","modified":"2024-05-14T17:08:18.843541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":24433,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24385":{"gid":24385,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/85237132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":85237132},{"type":"v1","addr":"172.21.15.203:6833","nonce":85237132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/2453543352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2453543352},{"type":"v1","addr":"172.21.15.92:6835","nonce":2453543352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3334129544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3334129544},{"type":"v1","addr":"172.21.15.203:6835","nonce":3334129544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/1194779586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1194779586},{"type":"v1","addr":"172.21.15.92:6837","nonce":1194779586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.203:6839/398835794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":398835794},{"type":"v1","addr":"172.21.15.203:6839","nonce":398835794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/2169416341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":2169416341},{"type":"v1","addr":"172.21.15.92:6839","nonce":2169416341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[24422,24433,24446]},"id":1}]} 2024-05-14T17:16:39.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 -- 172.21.15.69:0/2019678957 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe9c0b29f0 msgr2=0x7fbe9c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:39.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe9c0b29f0 0x7fbe9c0b4eb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbea4007b60 tx=0x7fbea40073d0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 -- 172.21.15.69:0/2019678957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb4102e00 msgr2=0x7fbeb419afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb4102e00 0x7fbeb419afb0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbea8000c00 tx=0x7fbea80047b0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 -- 172.21.15.69:0/2019678957 shutdown_connections 2024-05-14T17:16:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeb4108520 0x7fbeb41a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbeb4102440 0x7fbeb419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7fbe9c0b29f0 0x7fbe9c0b4eb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 --2- 172.21.15.69:0/2019678957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb4102e00 0x7fbeb419afb0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 -- 172.21.15.69:0/2019678957 >> 172.21.15.69:0/2019678957 conn(0x7fbeb40793f0 msgr2=0x7fbeb4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 -- 172.21.15.69:0/2019678957 shutdown_connections 2024-05-14T17:16:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:39.449+0000 7fbeb8c52700 1 -- 172.21.15.69:0/2019678957 wait complete. 2024-05-14T17:16:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 61 2024-05-14T17:16:39.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:39 smithi092 ceph-mon[35763]: pgmap v858: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T17:16:40.088 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:16:40.330 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:40.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:40 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/2019678957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:16:40.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:40 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/2019678957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:16:40.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:40 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/2019678957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:16:41.371 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:41 smithi203 ceph-mon[35787]: pgmap v859: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T17:16:41.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:41 smithi069 ceph-mon[31507]: pgmap v859: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T17:16:41.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:41 smithi092 ceph-mon[35763]: pgmap v859: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T17:16:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.984+0000 7f70d06ad700 1 -- 172.21.15.69:0/768624452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70c8108460 msgr2=0x7f70c806bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.984+0000 7f70d06ad700 1 --2- 172.21.15.69:0/768624452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70c8108460 0x7f70c806bc70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009990 tx=0x7f70c4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:16:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.984+0000 7f70d06ad700 1 -- 172.21.15.69:0/768624452 shutdown_connections 2024-05-14T17:16:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.984+0000 7f70d06ad700 1 --2- 172.21.15.69:0/768624452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70c8108460 0x7f70c806bc70 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.984+0000 7f70d06ad700 1 --2- 172.21.15.69:0/768624452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c8105f70 0x7f70c8106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.984+0000 7f70d06ad700 1 --2- 172.21.15.69:0/768624452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70c806c1b0 0x7f70c806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.984+0000 7f70d06ad700 1 -- 172.21.15.69:0/768624452 >> 172.21.15.69:0/768624452 conn(0x7f70c80793f0 msgr2=0x7f70c8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.984+0000 7f70d06ad700 1 -- 172.21.15.69:0/768624452 shutdown_connections 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.984+0000 7f70d06ad700 1 -- 172.21.15.69:0/768624452 wait complete. 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.984+0000 7f70d06ad700 1 Processor -- start 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70d06ad700 1 -- start start 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70d06ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70c806c1b0 0x7f70c81a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70d06ad700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70c8105f70 0x7f70c81a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70d06ad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c8108460 0x7f70c81a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70d06ad700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8119cd0 con 0x7f70c8105f70 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70d06ad700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8119b50 con 0x7f70c806c1b0 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70d06ad700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8119e50 con 0x7f70c8108460 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70cdc48700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70c8105f70 0x7f70c81a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70cdc48700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70c8105f70 0x7f70c81a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33242/0 (socket says 172.21.15.69:33242) 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70cdc48700 1 -- 172.21.15.69:0/1932720161 learned_addr learned my addr 172.21.15.69:0/1932720161 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70cec4a700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c8108460 0x7f70c81a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70ce449700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70c806c1b0 0x7f70c81a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70cdc48700 1 -- 172.21.15.69:0/1932720161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c8108460 msgr2=0x7f70c81a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70cdc48700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c8108460 0x7f70c81a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70cdc48700 1 -- 172.21.15.69:0/1932720161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70c806c1b0 msgr2=0x7f70c81a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70cdc48700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70c806c1b0 0x7f70c81a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70cdc48700 1 -- 172.21.15.69:0/1932720161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c81a52b0 con 0x7f70c8105f70 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70ce449700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70c806c1b0 0x7f70c81a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70cdc48700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70c8105f70 0x7f70c81a07e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c920 tx=0x7f70b800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.985+0000 7f70bf7fe700 1 -- 172.21.15.69:0/1932720161 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8007b00 con 0x7f70c8105f70 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.986+0000 7f70bf7fe700 1 -- 172.21.15.69:0/1932720161 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b8004ce0 con 0x7f70c8105f70 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.986+0000 7f70d06ad700 1 -- 172.21.15.69:0/1932720161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c81a55a0 con 0x7f70c8105f70 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.986+0000 7f70bf7fe700 1 -- 172.21.15.69:0/1932720161 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8005550 con 0x7f70c8105f70 2024-05-14T17:16:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.986+0000 7f70d06ad700 1 -- 172.21.15.69:0/1932720161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c81ace20 con 0x7f70c8105f70 2024-05-14T17:16:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.986+0000 7f70d06ad700 1 -- 172.21.15.69:0/1932720161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c806a230 con 0x7f70c8105f70 2024-05-14T17:16:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.988+0000 7f70bf7fe700 1 -- 172.21.15.69:0/1932720161 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f70b8007ca0 con 0x7f70c8105f70 2024-05-14T17:16:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.989+0000 7f70bf7fe700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70b40b29f0 0x7f70b40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.989+0000 7f70ce449700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70b40b29f0 0x7f70b40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.989+0000 7f70bf7fe700 1 -- 172.21.15.69:0/1932720161 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7f70b80e25a0 con 0x7f70c8105f70 2024-05-14T17:16:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.989+0000 7f70ce449700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70b40b29f0 0x7f70b40b4eb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f70c400b5c0 tx=0x7f70c4005fb0 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:41.992+0000 7f70bf7fe700 1 -- 172.21.15.69:0/1932720161 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70b8014030 con 0x7f70c8105f70 2024-05-14T17:16:42.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.196+0000 7f70d06ad700 1 -- 172.21.15.69:0/1932720161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70c81a1600 con 0x7f70c8105f70 2024-05-14T17:16:42.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.198+0000 7f70bf7fe700 1 -- 172.21.15.69:0/1932720161 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+23134 (secure 0 0 0) 0x7f70c81a1600 con 0x7f70c8105f70 2024-05-14T17:16:42.198 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-14T17:16:42.199 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":126,"fsid":"356e185e-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:38:33.684567+0000","modified":"2024-05-14T17:16:18.956688+0000","last_up_change":"2024-05-14T16:45:39.686515+0000","last_in_change":"2024-05-14T16:45:19.441818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:41:53.859789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:46:38.843325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:46:42.395401+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":125,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8e3c095-398a-4772-b0ef-655789199a67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6803","nonce":3097069954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6805","nonce":3097069954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6809","nonce":3097069954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3097069954},{"type":"v1","addr":"172.21.15.69:6807","nonce":3097069954}]},"public_addr":"172.21.15.69:6803/3097069954","cluster_addr":"172.21.15.69:6805/3097069954","heartbeat_back_addr":"172.21.15.69:6809/3097069954","heartbeat_front_addr":"172.21.15.69:6807/3097069954","state":["exists","up"]},{"osd":1,"uuid":"0678a843-a395-41d6-892f-7d0d24b30e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6801","nonce":1169355341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6803","nonce":1169355341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6807","nonce":1169355341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1169355341},{"type":"v1","addr":"172.21.15.92:6805","nonce":1169355341}]},"public_addr":"172.21.15.92:6801/1169355341","cluster_addr":"172.21.15.92:6803/1169355341","heartbeat_back_addr":"172.21.15.92:6807/1169355341","heartbeat_front_addr":"172.21.15.92:6805/1169355341","state":["exists","up"]},{"osd":2,"uuid":"9f4e05d2-6132-476c-9c8e-64526c9bfa08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6801","nonce":2458206770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6803","nonce":2458206770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6807","nonce":2458206770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2458206770},{"type":"v1","addr":"172.21.15.203:6805","nonce":2458206770}]},"public_addr":"172.21.15.203:6801/2458206770","cluster_addr":"172.21.15.203:6803/2458206770","heartbeat_back_addr":"172.21.15.203:6807/2458206770","heartbeat_front_addr":"172.21.15.203:6805/2458206770","state":["exists","up"]},{"osd":3,"uuid":"097ec507-bd81-441b-9b33-61e9c55c9f7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6811","nonce":1656710465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6813","nonce":1656710465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6817","nonce":1656710465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1656710465},{"type":"v1","addr":"172.21.15.69:6815","nonce":1656710465}]},"public_addr":"172.21.15.69:6811/1656710465","cluster_addr":"172.21.15.69:6813/1656710465","heartbeat_back_addr":"172.21.15.69:6817/1656710465","heartbeat_front_addr":"172.21.15.69:6815/1656710465","state":["exists","up"]},{"osd":4,"uuid":"76556e0f-a08a-4c01-a662-f84af601cf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6809","nonce":301070380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6811","nonce":301070380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6815","nonce":301070380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":301070380},{"type":"v1","addr":"172.21.15.92:6813","nonce":301070380}]},"public_addr":"172.21.15.92:6809/301070380","cluster_addr":"172.21.15.92:6811/301070380","heartbeat_back_addr":"172.21.15.92:6815/301070380","heartbeat_front_addr":"172.21.15.92:6813/301070380","state":["exists","up"]},{"osd":5,"uuid":"23ebc58a-f720-4000-a6aa-a2cfa0223481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6809","nonce":3198746082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6811","nonce":3198746082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6815","nonce":3198746082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3198746082},{"type":"v1","addr":"172.21.15.203:6813","nonce":3198746082}]},"public_addr":"172.21.15.203:6809/3198746082","cluster_addr":"172.21.15.203:6811/3198746082","heartbeat_back_addr":"172.21.15.203:6815/3198746082","heartbeat_front_addr":"172.21.15.203:6813/3198746082","state":["exists","up"]},{"osd":6,"uuid":"0bfda64f-86f0-4ec6-9f4a-9d4fe3c4cca2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6819","nonce":2234812220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6821","nonce":2234812220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6825","nonce":2234812220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2234812220},{"type":"v1","addr":"172.21.15.69:6823","nonce":2234812220}]},"public_addr":"172.21.15.69:6819/2234812220","cluster_addr":"172.21.15.69:6821/2234812220","heartbeat_back_addr":"172.21.15.69:6825/2234812220","heartbeat_front_addr":"172.21.15.69:6823/2234812220","state":["exists","up"]},{"osd":7,"uuid":"b87d086b-bce8-441b-83bb-9d21051d300c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6817","nonce":144400151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6819","nonce":144400151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6823","nonce":144400151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":144400151},{"type":"v1","addr":"172.21.15.92:6821","nonce":144400151}]},"public_addr":"172.21.15.92:6817/144400151","cluster_addr":"172.21.15.92:6819/144400151","heartbeat_back_addr":"172.21.15.92:6823/144400151","heartbeat_front_addr":"172.21.15.92:6821/144400151","state":["exists","up"]},{"osd":8,"uuid":"59490349-0db6-4e24-bc5b-b1cf52663838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6817","nonce":1294563875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6819","nonce":1294563875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6823","nonce":1294563875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1294563875},{"type":"v1","addr":"172.21.15.203:6821","nonce":1294563875}]},"public_addr":"172.21.15.203:6817/1294563875","cluster_addr":"172.21.15.203:6819/1294563875","heartbeat_back_addr":"172.21.15.203:6823/1294563875","heartbeat_front_addr":"172.21.15.203:6821/1294563875","state":["exists","up"]},{"osd":9,"uuid":"0b029d15-d987-448e-8ea7-513aff17fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6827","nonce":1618024425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6829","nonce":1618024425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6833","nonce":1618024425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1618024425},{"type":"v1","addr":"172.21.15.69:6831","nonce":1618024425}]},"public_addr":"172.21.15.69:6827/1618024425","cluster_addr":"172.21.15.69:6829/1618024425","heartbeat_back_addr":"172.21.15.69:6833/1618024425","heartbeat_front_addr":"172.21.15.69:6831/1618024425","state":["exists","up"]},{"osd":10,"uuid":"bce9e619-6203-4ae1-b02c-322eee6f809a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6825","nonce":1052364226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6827","nonce":1052364226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6831","nonce":1052364226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1052364226},{"type":"v1","addr":"172.21.15.92:6829","nonce":1052364226}]},"public_addr":"172.21.15.92:6825/1052364226","cluster_addr":"172.21.15.92:6827/1052364226","heartbeat_back_addr":"172.21.15.92:6831/1052364226","heartbeat_front_addr":"172.21.15.92:6829/1052364226","state":["exists","up"]},{"osd":11,"uuid":"9acab8e8-82ae-44b8-aba6-87b3a99ebe62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6825","nonce":3400874682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6827","nonce":3400874682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6831","nonce":3400874682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":3400874682},{"type":"v1","addr":"172.21.15.203:6829","nonce":3400874682}]},"public_addr":"172.21.15.203:6825/3400874682","cluster_addr":"172.21.15.203:6827/3400874682","heartbeat_back_addr":"172.21.15.203:6831/3400874682","heartbeat_front_addr":"172.21.15.203:6829/3400874682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:05.292703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:28.878800+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:41:52.196983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:16.299657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:42:40.859475+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:05.054030+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:29.531582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:43:55.402564+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:20.827621+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:44:46.390200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:11.978068+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:37.808403+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4012740750":"2024-05-15T16:49:54.600513+0000","172.21.15.69:6800/4012740750":"2024-05-15T16:49:54.600513+0000","172.21.15.69:0/70559253":"2024-05-15T16:49:54.600513+0000","172.21.15.69:0/3194136272":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3983720568":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2944165482":"2024-05-15T16:49:54.600513+0000","172.21.15.69:0/3714306679":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/1931804604":"2024-05-15T16:49:54.600513+0000","172.21.15.69:6801/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/3752620885":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/3176577502":"2024-05-15T16:49:54.600513+0000","172.21.15.69:6801/4209517756":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/421328414":"2024-05-15T16:39:31.768055+0000","172.21.15.69:0/2746326029":"2024-05-15T16:39:05.658063+0000","172.21.15.69:0/4187352755":"2024-05-15T16:39:05.658063+0000","172.21.15.69:6800/4209517756":"2024-05-15T16:39:05.658063+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":35,"length":1},{"begin":48,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.200+0000 7f70d06ad700 1 -- 172.21.15.69:0/1932720161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70b40b29f0 msgr2=0x7f70b40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.200+0000 7f70d06ad700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70b40b29f0 0x7f70b40b4eb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f70c400b5c0 tx=0x7f70c4005fb0 comp rx=0 tx=0).stop 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.200+0000 7f70d06ad700 1 -- 172.21.15.69:0/1932720161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70c8105f70 msgr2=0x7f70c81a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.200+0000 7f70d06ad700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70c8105f70 0x7f70c81a07e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c920 tx=0x7f70b800cc30 comp rx=0 tx=0).stop 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.200+0000 7f70d06ad700 1 -- 172.21.15.69:0/1932720161 shutdown_connections 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.200+0000 7f70d06ad700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c8108460 0x7f70c81a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.200+0000 7f70d06ad700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70c806c1b0 0x7f70c81a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.200+0000 7f70d06ad700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f70b40b29f0 0x7f70b40b4eb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.200+0000 7f70d06ad700 1 --2- 172.21.15.69:0/1932720161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70c8105f70 0x7f70c81a07e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.200+0000 7f70d06ad700 1 -- 172.21.15.69:0/1932720161 >> 172.21.15.69:0/1932720161 conn(0x7f70c80793f0 msgr2=0x7f70c8101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:42.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.201+0000 7f70d06ad700 1 -- 172.21.15.69:0/1932720161 shutdown_connections 2024-05-14T17:16:42.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:42.201+0000 7f70d06ad700 1 -- 172.21.15.69:0/1932720161 wait complete. 2024-05-14T17:16:42.839 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-14T17:16:43.083 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:43.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:43 smithi069 ceph-mon[31507]: pgmap v860: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T17:16:43.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:43 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/1932720161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:16:43.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:43 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:16:43.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:43 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:16:43.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:43 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:16:43.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:43 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:16:43.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:43 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:16:43.458 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:43 smithi203 ceph-mon[35787]: pgmap v860: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T17:16:43.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:43 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/1932720161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:16:43.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:43 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:16:43.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:43 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:16:43.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:43 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:16:43.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:43 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:16:43.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:43 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:16:43.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:43 smithi092 ceph-mon[35763]: pgmap v860: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T17:16:43.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:43 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/1932720161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:16:43.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:43 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:16:43.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:43 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:16:43.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:43 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:16:43.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:43 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:16:43.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:43 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' 2024-05-14T17:16:44.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:44 smithi069 ceph-mon[31507]: pgmap v861: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 766 B/s wr, 7 op/s 2024-05-14T17:16:44.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:44 smithi069 ceph-mon[31507]: pgmap v862: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 943 B/s wr, 8 op/s 2024-05-14T17:16:44.458 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:44 smithi203 ceph-mon[35787]: pgmap v861: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 766 B/s wr, 7 op/s 2024-05-14T17:16:44.458 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:44 smithi203 ceph-mon[35787]: pgmap v862: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 943 B/s wr, 8 op/s 2024-05-14T17:16:44.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:44 smithi092 ceph-mon[35763]: pgmap v861: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 766 B/s wr, 7 op/s 2024-05-14T17:16:44.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:44 smithi092 ceph-mon[35763]: pgmap v862: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 943 B/s wr, 8 op/s 2024-05-14T17:16:44.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.676+0000 7f473164e700 1 -- 172.21.15.69:0/1069036208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c103fc0 msgr2=0x7f472c1103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:44.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.676+0000 7f473164e700 1 --2- 172.21.15.69:0/1069036208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c103fc0 0x7f472c1103b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f47240099c0 tx=0x7f4724009cd0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.676+0000 7f473164e700 1 -- 172.21.15.69:0/1069036208 shutdown_connections 2024-05-14T17:16:44.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.676+0000 7f473164e700 1 --2- 172.21.15.69:0/1069036208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c103fc0 0x7f472c1103b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.676+0000 7f473164e700 1 --2- 172.21.15.69:0/1069036208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f472c1095e0 0x7f472c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.676+0000 7f473164e700 1 --2- 172.21.15.69:0/1069036208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c103600 0x7f472c103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.676+0000 7f473164e700 1 -- 172.21.15.69:0/1069036208 >> 172.21.15.69:0/1069036208 conn(0x7f472c07a700 msgr2=0x7f472c07ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:44.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.676+0000 7f473164e700 1 -- 172.21.15.69:0/1069036208 shutdown_connections 2024-05-14T17:16:44.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.676+0000 7f473164e700 1 -- 172.21.15.69:0/1069036208 wait complete. 2024-05-14T17:16:44.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f473164e700 1 Processor -- start 2024-05-14T17:16:44.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f473164e700 1 -- start start 2024-05-14T17:16:44.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f473164e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f472c103600 0x7f472c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:44.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f473164e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c103fc0 0x7f472c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:44.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f473164e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c1095e0 0x7f472c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:44.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f473164e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f472c113740 con 0x7f472c103fc0 2024-05-14T17:16:44.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f473164e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1135c0 con 0x7f472c1095e0 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f473164e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1138c0 con 0x7f472c103600 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c103fc0 0x7f472c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c103fc0 0x7f472c075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33750/0 (socket says 172.21.15.69:33750) 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472a7fc700 1 -- 172.21.15.69:0/933803162 learned_addr learned my addr 172.21.15.69:0/933803162 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472affd700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f472c103600 0x7f472c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472b7fe700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c1095e0 0x7f472c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472a7fc700 1 -- 172.21.15.69:0/933803162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f472c103600 msgr2=0x7f472c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472a7fc700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f472c103600 0x7f472c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472a7fc700 1 -- 172.21.15.69:0/933803162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c1095e0 msgr2=0x7f472c0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472a7fc700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c1095e0 0x7f472c0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472a7fc700 1 -- 172.21.15.69:0/933803162 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c1a8060 con 0x7f472c103fc0 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.677+0000 7f472b7fe700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c1095e0 0x7f472c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.678+0000 7f472a7fc700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c103fc0 0x7f472c075d80 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f471c00ef10 tx=0x7f471c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.678+0000 7f4713fff700 1 -- 172.21.15.69:0/933803162 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c00cad0 con 0x7f472c103fc0 2024-05-14T17:16:44.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.678+0000 7f4713fff700 1 -- 172.21.15.69:0/933803162 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f471c019070 con 0x7f472c103fc0 2024-05-14T17:16:44.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.678+0000 7f4713fff700 1 -- 172.21.15.69:0/933803162 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c0146e0 con 0x7f472c103fc0 2024-05-14T17:16:44.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.678+0000 7f473164e700 1 -- 172.21.15.69:0/933803162 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c1a82f0 con 0x7f472c103fc0 2024-05-14T17:16:44.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.678+0000 7f473164e700 1 -- 172.21.15.69:0/933803162 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c1a87e0 con 0x7f472c103fc0 2024-05-14T17:16:44.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.679+0000 7f473164e700 1 -- 172.21.15.69:0/933803162 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f472c10ba70 con 0x7f472c103fc0 2024-05-14T17:16:44.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.680+0000 7f4713fff700 1 -- 172.21.15.69:0/933803162 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f471c012050 con 0x7f472c103fc0 2024-05-14T17:16:44.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.681+0000 7f4713fff700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47140b29f0 0x7f47140b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:44.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.681+0000 7f472affd700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47140b29f0 0x7f47140b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:44.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.681+0000 7f4713fff700 1 -- 172.21.15.69:0/933803162 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7f471c0e26e0 con 0x7f472c103fc0 2024-05-14T17:16:44.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.681+0000 7f472affd700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47140b29f0 0x7f47140b4eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f472000bbb0 tx=0x7f472000b590 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:44.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.685+0000 7f4713fff700 1 -- 172.21.15.69:0/933803162 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f471c010040 con 0x7f472c103fc0 2024-05-14T17:16:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.933+0000 7f473164e700 1 -- 172.21.15.69:0/933803162 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f472c10b450 con 0x7f472c103fc0 2024-05-14T17:16:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.935+0000 7f4713fff700 1 -- 172.21.15.69:0/933803162 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7f471c0aa0b0 con 0x7f472c103fc0 2024-05-14T17:16:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.938+0000 7f473164e700 1 -- 172.21.15.69:0/933803162 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47140b29f0 msgr2=0x7f47140b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.938+0000 7f473164e700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47140b29f0 0x7f47140b4eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f472000bbb0 tx=0x7f472000b590 comp rx=0 tx=0).stop 2024-05-14T17:16:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.939+0000 7f473164e700 1 -- 172.21.15.69:0/933803162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c103fc0 msgr2=0x7f472c075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.939+0000 7f473164e700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c103fc0 0x7f472c075d80 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f471c00ef10 tx=0x7f471c00c3a0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.939+0000 7f473164e700 1 -- 172.21.15.69:0/933803162 shutdown_connections 2024-05-14T17:16:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.939+0000 7f473164e700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c1095e0 0x7f472c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.939+0000 7f473164e700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f472c103600 0x7f472c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.939+0000 7f473164e700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f47140b29f0 0x7f47140b4eb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.939+0000 7f473164e700 1 --2- 172.21.15.69:0/933803162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c103fc0 0x7f472c075d80 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.939+0000 7f473164e700 1 -- 172.21.15.69:0/933803162 >> 172.21.15.69:0/933803162 conn(0x7f472c07a700 msgr2=0x7f472c102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.939+0000 7f473164e700 1 -- 172.21.15.69:0/933803162 shutdown_connections 2024-05-14T17:16:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:44.939+0000 7f473164e700 1 -- 172.21.15.69:0/933803162 wait complete. 2024-05-14T17:16:44.940 INFO:teuthology.orchestra.run.smithi069.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T17:16:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:44 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a[31503]: 2024-05-14T17:16:44.934+0000 7f937ece6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:16:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:45 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/933803162' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T17:16:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:45 smithi069 ceph-mon[31507]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T17:16:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:45 smithi069 ceph-mon[31507]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:16:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:45 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/933803162' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T17:16:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:45 smithi069 ceph-mon[31507]: osdmap e127: 12 total, 12 up, 12 in 2024-05-14T17:16:45.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:45 smithi069 ceph-mon[31507]: fsmap cephfs:0/3 5 up:standby, 3 failed 2024-05-14T17:16:45.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:45 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/933803162' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T17:16:45.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:45 smithi092 ceph-mon[35763]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T17:16:45.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:45 smithi092 ceph-mon[35763]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:16:45.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:45 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/933803162' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T17:16:45.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:45 smithi092 ceph-mon[35763]: osdmap e127: 12 total, 12 up, 12 in 2024-05-14T17:16:45.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:45 smithi092 ceph-mon[35763]: fsmap cephfs:0/3 5 up:standby, 3 failed 2024-05-14T17:16:45.250 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 17:16:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b[66051]: ignoring --setuser ceph since I am not root 2024-05-14T17:16:45.250 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 17:16:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b[66051]: ignoring --setgroup ceph since I am not root 2024-05-14T17:16:45.251 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 17:16:44 smithi092 ceph-mds[66055]: -- 172.21.15.92:0/111719128 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614a7a45a40 con 0x5614a7a18c00 2024-05-14T17:16:45.251 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 17:16:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b[66051]: starting mds.b at 2024-05-14T17:16:45.251 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 17:16:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e[65431]: ignoring --setuser ceph since I am not root 2024-05-14T17:16:45.251 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 17:16:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e[65431]: ignoring --setgroup ceph since I am not root 2024-05-14T17:16:45.251 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 17:16:44 smithi092 ceph-mds[65435]: -- 172.21.15.92:0/3032258500 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d8eeffa40 con 0x564d8eed2800 2024-05-14T17:16:45.251 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 17:16:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e[65431]: starting mds.e at 2024-05-14T17:16:45.251 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 17:16:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h[65746]: ignoring --setuser ceph since I am not root 2024-05-14T17:16:45.251 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 17:16:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h[65746]: ignoring --setgroup ceph since I am not root 2024-05-14T17:16:45.251 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 17:16:44 smithi092 ceph-mds[65750]: -- 172.21.15.92:0/3459362222 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f39013fa40 con 0x55f390112c00 2024-05-14T17:16:45.252 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 17:16:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h[65746]: starting mds.h at 2024-05-14T17:16:45.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:45 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/933803162' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T17:16:45.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:45 smithi203 ceph-mon[35787]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T17:16:45.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:45 smithi203 ceph-mon[35787]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:16:45.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:45 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/933803162' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T17:16:45.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:45 smithi203 ceph-mon[35787]: osdmap e127: 12 total, 12 up, 12 in 2024-05-14T17:16:45.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:45 smithi203 ceph-mon[35787]: fsmap cephfs:0/3 5 up:standby, 3 failed 2024-05-14T17:16:45.463 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 17:16:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f[65972]: ignoring --setuser ceph since I am not root 2024-05-14T17:16:45.463 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 17:16:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f[65972]: ignoring --setgroup ceph since I am not root 2024-05-14T17:16:45.463 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 17:16:44 smithi203 ceph-mds[65976]: -- 172.21.15.203:0/793245539 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560afe637a40 con 0x560afe60b000 2024-05-14T17:16:45.463 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 17:16:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f[65972]: starting mds.f at 2024-05-14T17:16:45.463 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 17:16:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i[66586]: ignoring --setuser ceph since I am not root 2024-05-14T17:16:45.463 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 17:16:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i[66586]: ignoring --setgroup ceph since I am not root 2024-05-14T17:16:45.464 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 17:16:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i[66586]: starting mds.i at 2024-05-14T17:16:45.464 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:16:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-l[65668]: ignoring --setuser ceph since I am not root 2024-05-14T17:16:45.464 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:16:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-l[65668]: ignoring --setgroup ceph since I am not root 2024-05-14T17:16:45.464 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:16:44 smithi203 ceph-mds[65672]: -- 172.21.15.203:0/2142830852 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc5cdb7a40 con 0x55bc5cd8ac00 2024-05-14T17:16:45.464 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:16:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-l[65668]: starting mds.l at 2024-05-14T17:16:45.643 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T17:16:45.885 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:46.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: pgmap v863: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 589 B/s wr, 7 op/s 2024-05-14T17:16:46.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.92:6834/724767637,v1:172.21.15.92:6835/724767637] up:boot 2024-05-14T17:16:46.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6832/2446943797,v1:172.21.15.203:6833/2446943797] up:boot 2024-05-14T17:16:46.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6834/3290447856,v1:172.21.15.203:6835/3290447856] up:boot 2024-05-14T17:16:46.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.92:6836/3886954279,v1:172.21.15.92:6838/3886954279] up:boot 2024-05-14T17:16:46.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.92:6837/2425182593,v1:172.21.15.92:6839/2425182593] up:boot 2024-05-14T17:16:46.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: mds.? [v2:172.21.15.203:6836/2641476531,v1:172.21.15.203:6837/2641476531] up:boot 2024-05-14T17:16:46.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: fsmap cephfs:0/3 11 up:standby, 3 failed 2024-05-14T17:16:46.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T17:16:46.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:16:46.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T17:16:46.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:16:46.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:16:46.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:46 smithi069 ceph-mon[31507]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:16:46.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: pgmap v863: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 589 B/s wr, 7 op/s 2024-05-14T17:16:46.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.92:6834/724767637,v1:172.21.15.92:6835/724767637] up:boot 2024-05-14T17:16:46.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6832/2446943797,v1:172.21.15.203:6833/2446943797] up:boot 2024-05-14T17:16:46.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6834/3290447856,v1:172.21.15.203:6835/3290447856] up:boot 2024-05-14T17:16:46.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.92:6836/3886954279,v1:172.21.15.92:6838/3886954279] up:boot 2024-05-14T17:16:46.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.92:6837/2425182593,v1:172.21.15.92:6839/2425182593] up:boot 2024-05-14T17:16:46.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: mds.? [v2:172.21.15.203:6836/2641476531,v1:172.21.15.203:6837/2641476531] up:boot 2024-05-14T17:16:46.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: fsmap cephfs:0/3 11 up:standby, 3 failed 2024-05-14T17:16:46.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T17:16:46.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:16:46.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T17:16:46.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:16:46.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:16:46.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:46 smithi203 ceph-mon[35787]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:16:46.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: pgmap v863: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 589 B/s wr, 7 op/s 2024-05-14T17:16:46.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.92:6834/724767637,v1:172.21.15.92:6835/724767637] up:boot 2024-05-14T17:16:46.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6832/2446943797,v1:172.21.15.203:6833/2446943797] up:boot 2024-05-14T17:16:46.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6834/3290447856,v1:172.21.15.203:6835/3290447856] up:boot 2024-05-14T17:16:46.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.92:6836/3886954279,v1:172.21.15.92:6838/3886954279] up:boot 2024-05-14T17:16:46.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.92:6837/2425182593,v1:172.21.15.92:6839/2425182593] up:boot 2024-05-14T17:16:46.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: mds.? [v2:172.21.15.203:6836/2641476531,v1:172.21.15.203:6837/2641476531] up:boot 2024-05-14T17:16:46.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: fsmap cephfs:0/3 11 up:standby, 3 failed 2024-05-14T17:16:46.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T17:16:46.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:16:46.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T17:16:46.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:16:46.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:16:46.501 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:46 smithi092 ceph-mon[35763]: from='mgr.24764 172.21.15.69:0/286270307' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:16:47.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 -- 172.21.15.69:0/2089040959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b981095b0 msgr2=0x7f1b98109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 --2- 172.21.15.69:0/2089040959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b981095b0 0x7f1b98109990 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009990 tx=0x7f1b94009ca0 comp rx=0 tx=0).stop 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 -- 172.21.15.69:0/2089040959 shutdown_connections 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 --2- 172.21.15.69:0/2089040959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b981095b0 0x7f1b98109990 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 --2- 172.21.15.69:0/2089040959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9806bbe0 0x7f1b981103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 --2- 172.21.15.69:0/2089040959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9806b220 0x7f1b9806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 -- 172.21.15.69:0/2089040959 >> 172.21.15.69:0/2089040959 conn(0x7f1b980793f0 msgr2=0x7f1b98079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 -- 172.21.15.69:0/2089040959 shutdown_connections 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 -- 172.21.15.69:0/2089040959 wait complete. 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 Processor -- start 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.717+0000 7f1ba0129700 1 -- start start 2024-05-14T17:16:47.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1ba0129700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9806b220 0x7f1b9819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1ba0129700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9806bbe0 0x7f1b9819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1ba0129700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b981095b0 0x7f1b981999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1ba0129700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b981136c0 con 0x7f1b981095b0 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1ba0129700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98113540 con 0x7f1b9806bbe0 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1ba0129700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98113840 con 0x7f1b9806b220 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9e6c6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b981095b0 0x7f1b981999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9e6c6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b981095b0 0x7f1b981999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33760/0 (socket says 172.21.15.69:33760) 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9e6c6700 1 -- 172.21.15.69:0/698665959 learned_addr learned my addr 172.21.15.69:0/698665959 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9d6c4700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9806bbe0 0x7f1b9819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9dec5700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9806b220 0x7f1b9819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9e6c6700 1 -- 172.21.15.69:0/698665959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9806b220 msgr2=0x7f1b9819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9e6c6700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9806b220 0x7f1b9819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:47.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9e6c6700 1 -- 172.21.15.69:0/698665959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9806bbe0 msgr2=0x7f1b9819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:47.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9e6c6700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9806bbe0 0x7f1b9819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:47.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9e6c6700 1 -- 172.21.15.69:0/698665959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9819a100 con 0x7f1b981095b0 2024-05-14T17:16:47.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b9e6c6700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b981095b0 0x7f1b981999a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1b90007d70 tx=0x7f1b9000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:47.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b8effd700 1 -- 172.21.15.69:0/698665959 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90016020 con 0x7f1b981095b0 2024-05-14T17:16:47.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1ba0129700 1 -- 172.21.15.69:0/698665959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9819a3f0 con 0x7f1b981095b0 2024-05-14T17:16:47.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b8effd700 1 -- 172.21.15.69:0/698665959 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b9000ac90 con 0x7f1b981095b0 2024-05-14T17:16:47.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.718+0000 7f1b8effd700 1 -- 172.21.15.69:0/698665959 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900115e0 con 0x7f1b981095b0 2024-05-14T17:16:47.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.719+0000 7f1ba0129700 1 -- 172.21.15.69:0/698665959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b981a6660 con 0x7f1b981095b0 2024-05-14T17:16:47.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.719+0000 7f1ba0129700 1 -- 172.21.15.69:0/698665959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9810ba90 con 0x7f1b981095b0 2024-05-14T17:16:47.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.722+0000 7f1b8effd700 1 -- 172.21.15.69:0/698665959 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1b90019020 con 0x7f1b981095b0 2024-05-14T17:16:47.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.722+0000 7f1b8effd700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b840b29f0 0x7f1b840b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:47.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.722+0000 7f1b9dec5700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b840b29f0 0x7f1b840b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:47.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.723+0000 7f1b8effd700 1 -- 172.21.15.69:0/698665959 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8756+0+0 (secure 0 0 0) 0x7f1b900e29e0 con 0x7f1b981095b0 2024-05-14T17:16:47.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.723+0000 7f1b9dec5700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b840b29f0 0x7f1b840b4eb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b98100c00 tx=0x7f1b94005960 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:47.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.726+0000 7f1b8effd700 1 -- 172.21.15.69:0/698665959 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1b90020020 con 0x7f1b981095b0 2024-05-14T17:16:47.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:47.972+0000 7f1ba0129700 1 -- 172.21.15.69:0/698665959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1b9810b470 con 0x7f1b981095b0 2024-05-14T17:16:48.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.183+0000 7f1b8effd700 1 -- 172.21.15.69:0/698665959 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) ==== 104+0+0 (secure 0 0 0) 0x7f1b900aa3e0 con 0x7f1b981095b0 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 -- 172.21.15.69:0/698665959 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b840b29f0 msgr2=0x7f1b840b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b840b29f0 0x7f1b840b4eb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b98100c00 tx=0x7f1b94005960 comp rx=0 tx=0).stop 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 -- 172.21.15.69:0/698665959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b981095b0 msgr2=0x7f1b981999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b981095b0 0x7f1b981999a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1b90007d70 tx=0x7f1b9000a340 comp rx=0 tx=0).stop 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 -- 172.21.15.69:0/698665959 shutdown_connections 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f1b840b29f0 0x7f1b840b4eb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b981095b0 0x7f1b981999a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9806b220 0x7f1b9819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 --2- 172.21.15.69:0/698665959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9806bbe0 0x7f1b9819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 -- 172.21.15.69:0/698665959 >> 172.21.15.69:0/698665959 conn(0x7f1b980793f0 msgr2=0x7f1b98100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.184+0000 7f1ba0129700 1 -- 172.21.15.69:0/698665959 shutdown_connections 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:48.185+0000 7f1ba0129700 1 -- 172.21.15.69:0/698665959 wait complete. 2024-05-14T17:16:48.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:48 smithi069 ceph-mon[31507]: pgmap v865: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-14T17:16:48.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:48 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/698665959' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T17:16:48.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:48 smithi203 ceph-mon[35787]: pgmap v865: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-14T17:16:48.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:48 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/698665959' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T17:16:48.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:48 smithi092 ceph-mon[35763]: pgmap v865: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-14T17:16:48.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:48 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/698665959' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T17:16:48.837 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T17:16:49.079 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:49.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:49 smithi069 ceph-mon[31507]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T17:16:49.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:49 smithi069 ceph-mon[31507]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T17:16:49.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:49 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/698665959' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T17:16:49.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:49 smithi069 ceph-mon[31507]: fsmap 11 up:standby 2024-05-14T17:16:49.458 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:49 smithi203 ceph-mon[35787]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T17:16:49.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:49 smithi203 ceph-mon[35787]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T17:16:49.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:49 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/698665959' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T17:16:49.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:49 smithi203 ceph-mon[35787]: fsmap 11 up:standby 2024-05-14T17:16:49.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:49 smithi092 ceph-mon[35763]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T17:16:49.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:49 smithi092 ceph-mon[35763]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T17:16:49.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:49 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/698665959' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T17:16:49.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:49 smithi092 ceph-mon[35763]: fsmap 11 up:standby 2024-05-14T17:16:50.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:50 smithi069 ceph-mon[31507]: pgmap v866: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-14T17:16:50.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:50 smithi203 ceph-mon[35787]: pgmap v866: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-14T17:16:50.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:50 smithi092 ceph-mon[35763]: pgmap v866: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-14T17:16:50.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.733+0000 7f03b65e3700 1 -- 172.21.15.69:0/871335099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b0102440 msgr2=0x7f03b01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:50.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.733+0000 7f03b65e3700 1 --2- 172.21.15.69:0/871335099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b0102440 0x7f03b01028c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009990 tx=0x7f03a4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:16:50.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.733+0000 7f03b65e3700 1 -- 172.21.15.69:0/871335099 shutdown_connections 2024-05-14T17:16:50.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.733+0000 7f03b65e3700 1 --2- 172.21.15.69:0/871335099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b0102e00 0x7f03b010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:50.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.733+0000 7f03b65e3700 1 --2- 172.21.15.69:0/871335099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b0108520 0x7f03b0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:50.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.733+0000 7f03b65e3700 1 --2- 172.21.15.69:0/871335099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b0102440 0x7f03b01028c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:50.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.733+0000 7f03b65e3700 1 -- 172.21.15.69:0/871335099 >> 172.21.15.69:0/871335099 conn(0x7f03b00793f0 msgr2=0x7f03b0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:50.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.733+0000 7f03b65e3700 1 -- 172.21.15.69:0/871335099 shutdown_connections 2024-05-14T17:16:50.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.733+0000 7f03b65e3700 1 -- 172.21.15.69:0/871335099 wait complete. 2024-05-14T17:16:50.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03b65e3700 1 Processor -- start 2024-05-14T17:16:50.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03b65e3700 1 -- start start 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03b65e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b0102440 0x7f03b0104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03b65e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b0102e00 0x7f03b01045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03b65e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b0108520 0x7f03b0104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03b65e3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03b01125d0 con 0x7f03b0102440 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03b65e3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03b0112450 con 0x7f03b0102e00 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03b65e3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03b0112750 con 0x7f03b0108520 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03affff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b0102440 0x7f03b0104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03affff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b0102440 0x7f03b0104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33778/0 (socket says 172.21.15.69:33778) 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03affff700 1 -- 172.21.15.69:0/3669030647 learned_addr learned my addr 172.21.15.69:0/3669030647 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.734+0000 7f03af7fe700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b0102e00 0x7f03b01045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03b4b80700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b0108520 0x7f03b0104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03affff700 1 -- 172.21.15.69:0/3669030647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b0108520 msgr2=0x7f03b0104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03affff700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b0108520 0x7f03b0104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03affff700 1 -- 172.21.15.69:0/3669030647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b0102e00 msgr2=0x7f03b01045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03affff700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b0102e00 0x7f03b01045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03affff700 1 -- 172.21.15.69:0/3669030647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b01a8170 con 0x7f03b0102440 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03affff700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b0102440 0x7f03b0104080 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f03a000d950 tx=0x7f03a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03ad7fa700 1 -- 172.21.15.69:0/3669030647 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0016070 con 0x7f03b0102440 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03b65e3700 1 -- 172.21.15.69:0/3669030647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b01a8460 con 0x7f03b0102440 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03ad7fa700 1 -- 172.21.15.69:0/3669030647 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a0009df0 con 0x7f03b0102440 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03ad7fa700 1 -- 172.21.15.69:0/3669030647 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a000f3c0 con 0x7f03b0102440 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.735+0000 7f03b65e3700 1 -- 172.21.15.69:0/3669030647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b01a89c0 con 0x7f03b0102440 2024-05-14T17:16:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.736+0000 7f03b65e3700 1 -- 172.21.15.69:0/3669030647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b006a230 con 0x7f03b0102440 2024-05-14T17:16:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.738+0000 7f03ad7fa700 1 -- 172.21.15.69:0/3669030647 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f03a0019070 con 0x7f03b0102440 2024-05-14T17:16:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.739+0000 7f03ad7fa700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03980b29f0 0x7f03980b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.739+0000 7f03af7fe700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03980b29f0 0x7f03980b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.739+0000 7f03ad7fa700 1 -- 172.21.15.69:0/3669030647 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8756+0+0 (secure 0 0 0) 0x7f03a00e21c0 con 0x7f03b0102440 2024-05-14T17:16:50.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.739+0000 7f03af7fe700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03980b29f0 0x7f03980b4eb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03a400b5c0 tx=0x7f03a4005960 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.742+0000 7f03ad7fa700 1 -- 172.21.15.69:0/3669030647 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f03a00a9b50 con 0x7f03b0102440 2024-05-14T17:16:50.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:50.944+0000 7f03b65e3700 1 -- 172.21.15.69:0/3669030647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f03b01a8bf0 con 0x7f03b0102440 2024-05-14T17:16:51.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.202+0000 7f03ad7fa700 1 -- 172.21.15.69:0/3669030647 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v128) ==== 181+0+0 (secure 0 0 0) 0x7f03a00a9450 con 0x7f03b0102440 2024-05-14T17:16:51.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.205+0000 7f03b65e3700 1 -- 172.21.15.69:0/3669030647 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03980b29f0 msgr2=0x7f03980b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:51.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.205+0000 7f03b65e3700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03980b29f0 0x7f03980b4eb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03a400b5c0 tx=0x7f03a4005960 comp rx=0 tx=0).stop 2024-05-14T17:16:51.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.205+0000 7f03b65e3700 1 -- 172.21.15.69:0/3669030647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b0102440 msgr2=0x7f03b0104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:51.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.205+0000 7f03b65e3700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b0102440 0x7f03b0104080 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f03a000d950 tx=0x7f03a000dc60 comp rx=0 tx=0).stop 2024-05-14T17:16:51.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.205+0000 7f03b65e3700 1 -- 172.21.15.69:0/3669030647 shutdown_connections 2024-05-14T17:16:51.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.205+0000 7f03b65e3700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f03980b29f0 0x7f03980b4eb0 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03a400b5c0 tx=0x7f03a4005960 comp rx=0 tx=0).stop 2024-05-14T17:16:51.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.205+0000 7f03b65e3700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b0102440 0x7f03b0104080 secure :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f03a000d950 tx=0x7f03a000dc60 comp rx=0 tx=0).stop 2024-05-14T17:16:51.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.205+0000 7f03b65e3700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b0108520 0x7f03b0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:51.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.205+0000 7f03b65e3700 1 --2- 172.21.15.69:0/3669030647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b0102e00 0x7f03b01045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:51.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.205+0000 7f03b65e3700 1 -- 172.21.15.69:0/3669030647 >> 172.21.15.69:0/3669030647 conn(0x7f03b00793f0 msgr2=0x7f03b01063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:51.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.206+0000 7f03b65e3700 1 -- 172.21.15.69:0/3669030647 shutdown_connections 2024-05-14T17:16:51.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:51.206+0000 7f03b65e3700 1 -- 172.21.15.69:0/3669030647 wait complete. 2024-05-14T17:16:51.206 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' removed 2024-05-14T17:16:51.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:51 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3669030647' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:16:51.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:51 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3669030647' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:16:51.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:51 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3669030647' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:16:51.884 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T17:16:52.133 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:52.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:52 smithi069 ceph-mon[31507]: pgmap v867: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-14T17:16:52.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:52 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3669030647' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:16:52.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:52 smithi069 ceph-mon[31507]: osdmap e128: 12 total, 12 up, 12 in 2024-05-14T17:16:52.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:52 smithi203 ceph-mon[35787]: pgmap v867: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-14T17:16:52.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:52 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3669030647' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:16:52.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:52 smithi203 ceph-mon[35787]: osdmap e128: 12 total, 12 up, 12 in 2024-05-14T17:16:52.499 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:52 smithi092 ceph-mon[35763]: pgmap v867: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-14T17:16:52.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:52 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3669030647' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:16:52.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:52 smithi092 ceph-mon[35763]: osdmap e128: 12 total, 12 up, 12 in 2024-05-14T17:16:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.861+0000 7f3b7536f700 1 -- 172.21.15.69:0/2606496736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b7006b220 msgr2=0x7f3b7006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.861+0000 7f3b7536f700 1 --2- 172.21.15.69:0/2606496736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b7006b220 0x7f3b7006b6a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009990 tx=0x7f3b58009ca0 comp rx=0 tx=0).stop 2024-05-14T17:16:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.861+0000 7f3b7536f700 1 -- 172.21.15.69:0/2606496736 shutdown_connections 2024-05-14T17:16:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.861+0000 7f3b7536f700 1 --2- 172.21.15.69:0/2606496736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b7006bbe0 0x7f3b701103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.861+0000 7f3b7536f700 1 --2- 172.21.15.69:0/2606496736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b701095b0 0x7f3b70109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.861+0000 7f3b7536f700 1 --2- 172.21.15.69:0/2606496736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b7006b220 0x7f3b7006b6a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.861+0000 7f3b7536f700 1 -- 172.21.15.69:0/2606496736 >> 172.21.15.69:0/2606496736 conn(0x7f3b700793f0 msgr2=0x7f3b70079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.861+0000 7f3b7536f700 1 -- 172.21.15.69:0/2606496736 shutdown_connections 2024-05-14T17:16:53.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.861+0000 7f3b7536f700 1 -- 172.21.15.69:0/2606496736 wait complete. 2024-05-14T17:16:53.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b7536f700 1 Processor -- start 2024-05-14T17:16:53.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b7536f700 1 -- start start 2024-05-14T17:16:53.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b7536f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b7006b220 0x7f3b7019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b7536f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b7006bbe0 0x7f3b7019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b7536f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b701095b0 0x7f3b70199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b7536f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701136e0 con 0x7f3b7006bbe0 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b7536f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70113560 con 0x7f3b701095b0 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b7536f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70113860 con 0x7f3b7006b220 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b667fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b7006bbe0 0x7f3b7019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b667fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b7006bbe0 0x7f3b7019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33808/0 (socket says 172.21.15.69:33808) 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b667fc700 1 -- 172.21.15.69:0/3112840341 learned_addr learned my addr 172.21.15.69:0/3112840341 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b6f7fe700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b701095b0 0x7f3b70199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.862+0000 7f3b6effd700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b7006b220 0x7f3b7019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b667fc700 1 -- 172.21.15.69:0/3112840341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b7006b220 msgr2=0x7f3b7019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b667fc700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b7006b220 0x7f3b7019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b667fc700 1 -- 172.21.15.69:0/3112840341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b701095b0 msgr2=0x7f3b70199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b667fc700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b701095b0 0x7f3b70199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:53.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b667fc700 1 -- 172.21.15.69:0/3112840341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7019a180 con 0x7f3b7006bbe0 2024-05-14T17:16:53.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b667fc700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b7006bbe0 0x7f3b7019fda0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3b58000c00 tx=0x7f3b58009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:53.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b6cff9700 1 -- 172.21.15.69:0/3112840341 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58027070 con 0x7f3b7006bbe0 2024-05-14T17:16:53.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b6cff9700 1 -- 172.21.15.69:0/3112840341 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b58022470 con 0x7f3b7006bbe0 2024-05-14T17:16:53.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b6cff9700 1 -- 172.21.15.69:0/3112840341 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58017650 con 0x7f3b7006bbe0 2024-05-14T17:16:53.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b7536f700 1 -- 172.21.15.69:0/3112840341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7019a410 con 0x7f3b7006bbe0 2024-05-14T17:16:53.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.863+0000 7f3b7536f700 1 -- 172.21.15.69:0/3112840341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b701a6730 con 0x7f3b7006bbe0 2024-05-14T17:16:53.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.864+0000 7f3b7536f700 1 -- 172.21.15.69:0/3112840341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b7010ba90 con 0x7f3b7006bbe0 2024-05-14T17:16:53.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.866+0000 7f3b6cff9700 1 -- 172.21.15.69:0/3112840341 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3b5802a070 con 0x7f3b7006bbe0 2024-05-14T17:16:53.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.867+0000 7f3b6cff9700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3b5c0b29f0 0x7f3b5c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:53.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.867+0000 7f3b6effd700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3b5c0b29f0 0x7f3b5c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:53.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.868+0000 7f3b6cff9700 1 -- 172.21.15.69:0/3112840341 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8231+0+0 (secure 0 0 0) 0x7f3b580e37c0 con 0x7f3b7006bbe0 2024-05-14T17:16:53.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.868+0000 7f3b6effd700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3b5c0b29f0 0x7f3b5c0b4eb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3b60005e10 tx=0x7f3b60005d40 comp rx=0 tx=0).ready entity=mgr.24764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:53.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:53.870+0000 7f3b6cff9700 1 -- 172.21.15.69:0/3112840341 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b58013070 con 0x7f3b7006bbe0 2024-05-14T17:16:54.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.074+0000 7f3b7536f700 1 -- 172.21.15.69:0/3112840341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3b70104c70 con 0x7f3b7006bbe0 2024-05-14T17:16:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.205+0000 7f3b6cff9700 1 -- 172.21.15.69:0/3112840341 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v129) ==== 169+0+0 (secure 0 0 0) 0x7f3b70104c70 con 0x7f3b7006bbe0 2024-05-14T17:16:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.208+0000 7f3b7536f700 1 -- 172.21.15.69:0/3112840341 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3b5c0b29f0 msgr2=0x7f3b5c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.208+0000 7f3b7536f700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3b5c0b29f0 0x7f3b5c0b4eb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3b60005e10 tx=0x7f3b60005d40 comp rx=0 tx=0).stop 2024-05-14T17:16:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.208+0000 7f3b7536f700 1 -- 172.21.15.69:0/3112840341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b7006bbe0 msgr2=0x7f3b7019fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.208+0000 7f3b7536f700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b7006bbe0 0x7f3b7019fda0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3b58000c00 tx=0x7f3b58009580 comp rx=0 tx=0).stop 2024-05-14T17:16:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.210+0000 7f3b7536f700 1 -- 172.21.15.69:0/3112840341 shutdown_connections 2024-05-14T17:16:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.210+0000 7f3b7536f700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b701095b0 0x7f3b70199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.210+0000 7f3b7536f700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b7006b220 0x7f3b7019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.210+0000 7f3b7536f700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.69:6800/4217409403,v1:172.21.15.69:6801/4217409403] conn(0x7f3b5c0b29f0 0x7f3b5c0b4eb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.210+0000 7f3b7536f700 1 --2- 172.21.15.69:0/3112840341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b7006bbe0 0x7f3b7019fda0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.210+0000 7f3b7536f700 1 -- 172.21.15.69:0/3112840341 >> 172.21.15.69:0/3112840341 conn(0x7f3b700793f0 msgr2=0x7f3b701010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.211+0000 7f3b7536f700 1 -- 172.21.15.69:0/3112840341 shutdown_connections 2024-05-14T17:16:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-14T17:16:54.211+0000 7f3b7536f700 1 -- 172.21.15.69:0/3112840341 wait complete. 2024-05-14T17:16:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' removed 2024-05-14T17:16:54.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:54 smithi092 ceph-mon[35763]: pgmap v869: 65 pgs: 65 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:16:54.500 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:54 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3112840341' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:16:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:54 smithi069 ceph-mon[31507]: pgmap v869: 65 pgs: 65 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:16:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:54 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3112840341' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:16:54.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:54 smithi203 ceph-mon[35787]: pgmap v869: 65 pgs: 65 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:16:54.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:54 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3112840341' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:16:54.897 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:16:54.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:16:54.991 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:16:55.033 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-14T17:16:55.033 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T17:16:55.055 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T17:16:55.096 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T17:16:55.114 INFO:tasks.cephadm:Stopping all daemons... 2024-05-14T17:16:55.114 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-14T17:16:55.114 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.a 2024-05-14T17:16:55.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:55 smithi092 ceph-mon[35763]: from='client.? 172.21.15.69:0/3112840341' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:16:55.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:55 smithi092 ceph-mon[35763]: osdmap e129: 12 total, 12 up, 12 in 2024-05-14T17:16:55.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:55 smithi069 systemd[1]: Stopping Ceph mon.a for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:16:55.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:55 smithi069 ceph-mon[31507]: from='client.? 172.21.15.69:0/3112840341' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:16:55.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:55 smithi069 ceph-mon[31507]: osdmap e129: 12 total, 12 up, 12 in 2024-05-14T17:16:55.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:55 smithi203 ceph-mon[35787]: from='client.? 172.21.15.69:0/3112840341' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:16:55.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:55 smithi203 ceph-mon[35787]: osdmap e129: 12 total, 12 up, 12 in 2024-05-14T17:16:55.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:55 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a[31503]: 2024-05-14T17:16:55.439+0000 7f93844f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:16:55.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:55 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a[31503]: 2024-05-14T17:16:55.439+0000 7f93844f1700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-14T17:16:55.703 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:55 smithi069 podman[111386]: 2024-05-14 17:16:55.602988652 +0000 UTC m=+0.284062699 container died 3fcc6b76fa602cf3d2d4361bb39e6d84ea3c6d1620c3d241d095e46da6ef2989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-14T17:16:56.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:55 smithi069 podman[111386]: 2024-05-14 17:16:55.999152527 +0000 UTC m=+0.680226573 container cleanup 3fcc6b76fa602cf3d2d4361bb39e6d84ea3c6d1620c3d241d095e46da6ef2989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-14T17:16:56.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:May 14 17:16:55 smithi069 bash[111386]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-a 2024-05-14T17:16:56.399 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.a.service' 2024-05-14T17:16:56.453 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:16:56.453 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-14T17:16:56.453 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-14T17:16:56.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.b 2024-05-14T17:16:56.750 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:56 smithi092 systemd[1]: Stopping Ceph mon.b for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:16:57.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:56 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-b[35758]: 2024-05-14T17:16:56.769+0000 7f9725aaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:16:57.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:56 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-b[35758]: 2024-05-14T17:16:56.769+0000 7f9725aaf700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-14T17:16:57.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:56 smithi092 podman[77300]: 2024-05-14 17:16:56.840967737 +0000 UTC m=+0.205892092 container died 9105d8cf405ae50d0bdc04892a1c776644eb6947869072510eea897e480c264f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514) 2024-05-14T17:16:57.606 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:57 smithi092 podman[77300]: 2024-05-14 17:16:57.381224522 +0000 UTC m=+0.746148871 container cleanup 9105d8cf405ae50d0bdc04892a1c776644eb6947869072510eea897e480c264f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-b, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-14T17:16:57.606 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:57 smithi092 bash[77300]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-b 2024-05-14T17:16:57.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.b.service' 2024-05-14T17:16:57.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:57 smithi092 podman[77312]: 2024-05-14 17:16:57.607675588 +0000 UTC m=+0.763034867 container remove 9105d8cf405ae50d0bdc04892a1c776644eb6947869072510eea897e480c264f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-14T17:16:57.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:57 smithi092 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.b.service: Deactivated successfully. 2024-05-14T17:16:57.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:57 smithi092 systemd[1]: Stopped Ceph mon.b for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:16:57.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 14 17:16:57 smithi092 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.b.service: Consumed 18.971s CPU time. 2024-05-14T17:16:57.904 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:16:57.904 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-14T17:16:57.905 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-14T17:16:57.905 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.c 2024-05-14T17:16:58.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:57 smithi203 systemd[1]: Stopping Ceph mon.c for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:16:58.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:58 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-c[35761]: 2024-05-14T17:16:58.359+0000 7fbda4640700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:16:58.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:58 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-c[35761]: 2024-05-14T17:16:58.359+0000 7fbda4640700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-14T17:16:58.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:58 smithi203 podman[79084]: 2024-05-14 17:16:58.523903531 +0000 UTC m=+0.427772860 container died 92ff3665e8b07f36fa7f0773e265175a56c978e384c0b418373314b926cb55cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-14T17:16:59.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:58 smithi203 podman[79084]: 2024-05-14 17:16:58.919641915 +0000 UTC m=+0.823511243 container cleanup 92ff3665e8b07f36fa7f0773e265175a56c978e384c0b418373314b926cb55cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-c, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-14T17:16:59.209 INFO:journalctl@ceph.mon.c.smithi203.stdout:May 14 17:16:58 smithi203 bash[79084]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mon-c 2024-05-14T17:16:59.509 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mon.c.service' 2024-05-14T17:16:59.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:16:59.556 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-14T17:16:59.557 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-14T17:16:59.557 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.x 2024-05-14T17:16:59.953 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 17:16:59 smithi069 systemd[1]: Stopping Ceph mgr.x for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:17:00.238 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 17:17:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T17:17:00.012+0000 7f134c4a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:17:00.239 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 17:17:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x[31732]: 2024-05-14T17:17:00.012+0000 7f134c4a5700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-14T17:17:00.239 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 17:17:00 smithi069 podman[111506]: 2024-05-14 17:17:00.0435401 +0000 UTC m=+0.282367015 container died 24c9b3caf1d97601bd3bf9a7d2151b40d4770404d120c7b8eee3d868f5e31367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:17:00.664 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 17:17:00 smithi069 podman[111506]: 2024-05-14 17:17:00.455053883 +0000 UTC m=+0.693880790 container cleanup 24c9b3caf1d97601bd3bf9a7d2151b40d4770404d120c7b8eee3d868f5e31367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:17:00.664 INFO:journalctl@ceph.mgr.x.smithi069.stdout:May 14 17:17:00 smithi069 bash[111506]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-x 2024-05-14T17:17:00.887 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-14T17:17:00.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:17:00.925 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-14T17:17:00.925 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-14T17:17:00.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.y 2024-05-14T17:17:01.249 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 17:17:00 smithi092 systemd[1]: Stopping Ceph mgr.y for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:17:01.749 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 17:17:01 smithi092 podman[77423]: 2024-05-14 17:17:01.458251925 +0000 UTC m=+0.335100144 container died d1f975a7b58b84ce254fd7122fb4c3366b58e162f73b6110c7265d0a874e4425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:17:02.119 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 17:17:01 smithi092 podman[77423]: 2024-05-14 17:17:01.860941724 +0000 UTC m=+0.737789948 container cleanup d1f975a7b58b84ce254fd7122fb4c3366b58e162f73b6110c7265d0a874e4425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-14T17:17:02.120 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 14 17:17:01 smithi092 bash[77423]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-y 2024-05-14T17:17:02.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-14T17:17:02.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:17:02.332 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-14T17:17:02.332 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-14T17:17:02.332 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.z 2024-05-14T17:17:02.680 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 17:17:02 smithi203 systemd[1]: Stopping Ceph mgr.z for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:17:02.958 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 17:17:02 smithi203 podman[79209]: 2024-05-14 17:17:02.681210777 +0000 UTC m=+0.151310750 container died de0a6ecb180747f755d6d6abc7d6fc3f159e53d22a993d78008eb2739f67ea9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-14T17:17:03.461 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 17:17:03 smithi203 podman[79209]: 2024-05-14 17:17:03.126711345 +0000 UTC m=+0.596811318 container cleanup de0a6ecb180747f755d6d6abc7d6fc3f159e53d22a993d78008eb2739f67ea9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T17:17:03.461 INFO:journalctl@ceph.mgr.z.smithi203.stdout:May 14 17:17:03 smithi203 bash[79209]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mgr-z 2024-05-14T17:17:04.020 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-14T17:17:04.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:17:04.054 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-14T17:17:04.054 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-14T17:17:04.055 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.0 2024-05-14T17:17:04.376 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:04 smithi069 systemd[1]: Stopping Ceph osd.0 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:17:04.703 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0[43503]: 2024-05-14T17:17:04.376+0000 7f810b24d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:17:04.703 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0[43503]: 2024-05-14T17:17:04.376+0000 7f810b24d700 -1 osd.0 129 *** Got signal Terminated *** 2024-05-14T17:17:04.703 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0[43503]: 2024-05-14T17:17:04.376+0000 7f810b24d700 -1 osd.0 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:17:10.265 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:10 smithi069 podman[111636]: 2024-05-14 17:17:10.016083759 +0000 UTC m=+5.764619171 container died 24665bcfc0b55f6eb7f94c5cf8ca8131c6d3d21f5250507538c28cc42977f43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-14T17:17:10.644 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:10 smithi069 podman[111636]: 2024-05-14 17:17:10.43612695 +0000 UTC m=+6.184662360 container cleanup 24665bcfc0b55f6eb7f94c5cf8ca8131c6d3d21f5250507538c28cc42977f43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:17:10.645 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:10 smithi069 bash[111636]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0 2024-05-14T17:17:10.953 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:10 smithi069 podman[111652]: 2024-05-14 17:17:10.64636778 +0000 UTC m=+0.631265477 container remove 24665bcfc0b55f6eb7f94c5cf8ca8131c6d3d21f5250507538c28cc42977f43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:17:11.382 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:11 smithi069 podman[111744]: 2024-05-14 17:17:10.95952518 +0000 UTC m=+0.017006012 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:17:11.383 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:11 smithi069 podman[111744]: 2024-05-14 17:17:11.064156136 +0000 UTC m=+0.121636955 container create 6d79d95aa1e37df3e043b38211c29aad688282e43fa8d1019f809595107d4539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-14T17:17:11.383 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:11 smithi069 podman[111744]: 2024-05-14 17:17:11.291818704 +0000 UTC m=+0.349299525 container init 6d79d95aa1e37df3e043b38211c29aad688282e43fa8d1019f809595107d4539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-14T17:17:11.383 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:11 smithi069 podman[111744]: 2024-05-14 17:17:11.295180198 +0000 UTC m=+0.352661022 container start 6d79d95aa1e37df3e043b38211c29aad688282e43fa8d1019f809595107d4539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:17:11.383 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:11 smithi069 podman[111744]: 2024-05-14 17:17:11.383605668 +0000 UTC m=+0.441086494 container attach 6d79d95aa1e37df3e043b38211c29aad688282e43fa8d1019f809595107d4539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , ceph=True) 2024-05-14T17:17:11.703 INFO:journalctl@ceph.osd.0.smithi069.stdout:May 14 17:17:11 smithi069 podman[111744]: 2024-05-14 17:17:11.598256983 +0000 UTC m=+0.655737804 container died 6d79d95aa1e37df3e043b38211c29aad688282e43fa8d1019f809595107d4539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-05-14T17:17:12.302 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.0.service' 2024-05-14T17:17:12.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:17:12.345 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-14T17:17:12.345 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-14T17:17:12.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.1 2024-05-14T17:17:12.750 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:12 smithi092 systemd[1]: Stopping Ceph osd.1 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:17:13.249 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:12 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1[40326]: 2024-05-14T17:17:12.755+0000 7f3ab4a40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:17:13.250 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:12 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1[40326]: 2024-05-14T17:17:12.755+0000 7f3ab4a40700 -1 osd.1 129 *** Got signal Terminated *** 2024-05-14T17:17:13.250 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:12 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1[40326]: 2024-05-14T17:17:12.755+0000 7f3ab4a40700 -1 osd.1 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:17:18.689 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:18 smithi092 podman[77548]: 2024-05-14 17:17:18.385689308 +0000 UTC m=+5.843539651 container died 3c35fbb5141f340bb5b09d79ac54e31856ec7168c0c9218ac8878be46ec648d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-14T17:17:19.000 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:18 smithi092 podman[77548]: 2024-05-14 17:17:18.789943549 +0000 UTC m=+6.247793886 container cleanup 3c35fbb5141f340bb5b09d79ac54e31856ec7168c0c9218ac8878be46ec648d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-14T17:17:19.000 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:18 smithi092 bash[77548]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1 2024-05-14T17:17:19.351 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:19 smithi092 podman[77564]: 2024-05-14 17:17:19.02461962 +0000 UTC m=+0.641043515 container remove 3c35fbb5141f340bb5b09d79ac54e31856ec7168c0c9218ac8878be46ec648d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:17:19.626 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:19 smithi092 podman[77652]: 2024-05-14 17:17:19.255231713 +0000 UTC m=+0.017540784 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:17:19.627 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:19 smithi092 podman[77652]: 2024-05-14 17:17:19.358725862 +0000 UTC m=+0.121034938 container create ddae0f35b23d50ba22bea563b87a182731b55d66a6930ec1364b062a65afc56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5) 2024-05-14T17:17:19.627 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:19 smithi092 podman[77652]: 2024-05-14 17:17:19.628098419 +0000 UTC m=+0.390407490 container init ddae0f35b23d50ba22bea563b87a182731b55d66a6930ec1364b062a65afc56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-14T17:17:19.925 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:19 smithi092 podman[77652]: 2024-05-14 17:17:19.631537552 +0000 UTC m=+0.393846624 container start ddae0f35b23d50ba22bea563b87a182731b55d66a6930ec1364b062a65afc56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-14T17:17:19.925 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:19 smithi092 podman[77652]: 2024-05-14 17:17:19.728316225 +0000 UTC m=+0.490625313 container attach ddae0f35b23d50ba22bea563b87a182731b55d66a6930ec1364b062a65afc56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:17:20.249 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 14 17:17:19 smithi092 podman[77652]: 2024-05-14 17:17:19.926747284 +0000 UTC m=+0.689056361 container died ddae0f35b23d50ba22bea563b87a182731b55d66a6930ec1364b062a65afc56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-14T17:17:20.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.1.service' 2024-05-14T17:17:20.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:17:20.647 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-14T17:17:20.647 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-14T17:17:20.647 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.2 2024-05-14T17:17:20.959 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:20 smithi203 systemd[1]: Stopping Ceph osd.2 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:17:21.218 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:20 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2[40916]: 2024-05-14T17:17:20.971+0000 7fc9d8ed7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:17:21.218 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:20 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2[40916]: 2024-05-14T17:17:20.971+0000 7fc9d8ed7700 -1 osd.2 129 *** Got signal Terminated *** 2024-05-14T17:17:21.219 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:20 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2[40916]: 2024-05-14T17:17:20.971+0000 7fc9d8ed7700 -1 osd.2 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:17:26.890 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:26 smithi203 podman[79336]: 2024-05-14 17:17:26.597735271 +0000 UTC m=+5.751114859 container died f373bf8f11702aed7708618f81058707047ba3604f77821ca95119e2e5b3f3d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:17:27.209 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:26 smithi203 podman[79336]: 2024-05-14 17:17:26.982366484 +0000 UTC m=+6.135746077 container cleanup f373bf8f11702aed7708618f81058707047ba3604f77821ca95119e2e5b3f3d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-14T17:17:27.209 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:26 smithi203 bash[79336]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2 2024-05-14T17:17:27.540 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:27 smithi203 podman[79365]: 2024-05-14 17:17:27.222831716 +0000 UTC m=+0.616225465 container remove f373bf8f11702aed7708618f81058707047ba3604f77821ca95119e2e5b3f3d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-14T17:17:27.792 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:27 smithi203 podman[79453]: 2024-05-14 17:17:27.444388037 +0000 UTC m=+0.017439150 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:17:27.792 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:27 smithi203 podman[79453]: 2024-05-14 17:17:27.548980051 +0000 UTC m=+0.122031153 container create 88c39cf28382b1bc3671c62864a749f6030928f7f74bf84588a2e0fbeef09560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-05-14T17:17:28.091 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:27 smithi203 podman[79453]: 2024-05-14 17:17:27.793722156 +0000 UTC m=+0.366773254 container init 88c39cf28382b1bc3671c62864a749f6030928f7f74bf84588a2e0fbeef09560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-14T17:17:28.092 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:27 smithi203 podman[79453]: 2024-05-14 17:17:27.797149519 +0000 UTC m=+0.370200619 container start 88c39cf28382b1bc3671c62864a749f6030928f7f74bf84588a2e0fbeef09560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:17:28.092 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:27 smithi203 podman[79453]: 2024-05-14 17:17:27.885338354 +0000 UTC m=+0.458389465 container attach 88c39cf28382b1bc3671c62864a749f6030928f7f74bf84588a2e0fbeef09560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:17:28.455 INFO:journalctl@ceph.osd.2.smithi203.stdout:May 14 17:17:28 smithi203 podman[79453]: 2024-05-14 17:17:28.0934037 +0000 UTC m=+0.666454799 container died 88c39cf28382b1bc3671c62864a749f6030928f7f74bf84588a2e0fbeef09560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-14T17:17:28.765 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.2.service' 2024-05-14T17:17:28.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:17:28.806 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-14T17:17:28.806 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-14T17:17:28.806 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.3 2024-05-14T17:17:29.148 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:28 smithi069 systemd[1]: Stopping Ceph osd.3 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:17:29.453 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:29 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3[50250]: 2024-05-14T17:17:29.147+0000 7f8c65003700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:17:29.453 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:29 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3[50250]: 2024-05-14T17:17:29.147+0000 7f8c65003700 -1 osd.3 129 *** Got signal Terminated *** 2024-05-14T17:17:29.453 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:29 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3[50250]: 2024-05-14T17:17:29.147+0000 7f8c65003700 -1 osd.3 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:17:31.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:30 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:30.797+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:31.355 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:31 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:31.064+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:32.129 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:31 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:31.762+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:32.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:32 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:32.083+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:32.458 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:32 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:32.043+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:32.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:32 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:32.130+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:33.144 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:32 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:32.798+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:33.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:33 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:33.077+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:33.458 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:33 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:33.056+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:33.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:33 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:33.145+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:33.953 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:33 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3[50250]: 2024-05-14T17:17:33.577+0000 7f8c615f7700 -1 osd.3 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.998122+0000 front 2024-05-14T17:17:07.998145+0000 (oldest deadline 2024-05-14T17:17:33.298354+0000) 2024-05-14T17:17:34.168 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:33 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:33.753+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:34.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:34 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:34.111+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:34.458 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:34.057+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:34.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:34 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:34.169+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:34.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:34.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:34.999 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:34 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:34.723+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:35.158 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:35 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:35.057+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:35.158 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:34 smithi069 podman[111860]: 2024-05-14 17:17:34.854504062 +0000 UTC m=+5.843168731 container died e04c88b27f1ee1480c65d74aba4f1731fd1a5581e4a6832dd740764e61cec16b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-05-14T17:17:35.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:35 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:35.160+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:35.453 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:35 smithi069 podman[111860]: 2024-05-14 17:17:35.257094797 +0000 UTC m=+6.245759467 container cleanup e04c88b27f1ee1480c65d74aba4f1731fd1a5581e4a6832dd740764e61cec16b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T17:17:35.453 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:35 smithi069 bash[111860]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3 2024-05-14T17:17:35.458 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:35 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:35.103+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:35.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:35 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:35.176+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:35.814 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:35 smithi069 podman[111886]: 2024-05-14 17:17:35.500658385 +0000 UTC m=+0.644990534 container remove e04c88b27f1ee1480c65d74aba4f1731fd1a5581e4a6832dd740764e61cec16b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS) 2024-05-14T17:17:35.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:35 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:35.526+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:35.999 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:35 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:35.746+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:36.069 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:35 smithi069 podman[111974]: 2024-05-14 17:17:35.718208645 +0000 UTC m=+0.017653673 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:17:36.069 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:35 smithi069 podman[111974]: 2024-05-14 17:17:35.834981959 +0000 UTC m=+0.134426965 container create 2e653838b3f7583c52264126209fb9a763b6c2fbaf0ec94bbc7c77f42fa1b092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-14T17:17:36.069 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:36 smithi069 podman[111974]: 2024-05-14 17:17:36.070763055 +0000 UTC m=+0.370208074 container init 2e653838b3f7583c52264126209fb9a763b6c2fbaf0ec94bbc7c77f42fa1b092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-14T17:17:36.366 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:36 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:36.133+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:36.367 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:36 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:36.102+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:36.367 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:36 smithi069 podman[111974]: 2024-05-14 17:17:36.074330777 +0000 UTC m=+0.373775808 container start 2e653838b3f7583c52264126209fb9a763b6c2fbaf0ec94bbc7c77f42fa1b092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-14T17:17:36.367 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:36 smithi069 podman[111974]: 2024-05-14 17:17:36.145716699 +0000 UTC m=+0.445161726 container attach 2e653838b3f7583c52264126209fb9a763b6c2fbaf0ec94bbc7c77f42fa1b092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-14T17:17:36.459 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:36 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:36.146+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:36.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:36 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:36.205+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:36.673 INFO:journalctl@ceph.osd.3.smithi069.stdout:May 14 17:17:36 smithi069 podman[111974]: 2024-05-14 17:17:36.368222832 +0000 UTC m=+0.667667860 container died 2e653838b3f7583c52264126209fb9a763b6c2fbaf0ec94bbc7c77f42fa1b092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-14T17:17:36.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:36 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:36.565+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:36.986 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.3.service' 2024-05-14T17:17:37.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:17:37.026 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-14T17:17:37.026 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-14T17:17:37.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.4 2024-05-14T17:17:37.051 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:36 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:36.772+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:37.351 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:37 smithi092 systemd[1]: Stopping Ceph osd.4 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:17:37.351 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:37 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:37.202+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:37.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:37 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:37.071+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:37.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:37 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:37.124+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:37.458 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:37 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:37.145+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:37.750 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:37 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:37.351+0000 7fa260a82700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:17:37.750 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:37 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:37.351+0000 7fa260a82700 -1 osd.4 129 *** Got signal Terminated *** 2024-05-14T17:17:37.750 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:37 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:37.351+0000 7fa260a82700 -1 osd.4 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:17:37.958 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:37 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:37.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:38.155 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:37 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:37.776+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:38.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:38 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:38.084+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:38.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:38 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:38.094+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:38.458 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:38.179+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:38.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:38 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:38.156+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:38.958 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:38.517+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:39.142 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:38 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:38.801+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:39.142 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:38 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:38.813+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:39.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:39 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:39.038+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:39.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:39 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:39.048+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:39.458 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:39 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:39.179+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:39.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:39 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:39.143+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:39.958 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:39 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:39.475+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:40.128 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:39 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:39.785+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:40.128 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:39 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:39.799+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:40.441 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:40 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:40.178+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:40.442 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:40 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:40.418+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:40.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:40 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:40.038+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:40.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:40 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:40.067+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:40.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:40 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:40.128+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:40.708 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:40 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:40.442+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:41.086 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:40 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:40.776+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:41.086 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:40 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:40.772+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:41.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:41 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:41.073+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:41.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:41 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:41.027+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:41.459 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:41 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:41.201+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:41.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:41 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:41.083+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:41.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:41 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:41.461+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:41.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:41 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:41.473+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:41.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:41 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:41.473+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:42.055 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:41 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:41.783+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:42.055 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:41 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:41.797+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:42.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:42 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:42.112+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:42.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:42 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:42.112+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:42.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:42 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:42.031+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:42.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:42 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:42.031+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:42.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:42 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4[47265]: 2024-05-14T17:17:42.055+0000 7fa25906e700 -1 osd.4 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:06.481281+0000 front 2024-05-14T17:17:06.481256+0000 (oldest deadline 2024-05-14T17:17:31.181397+0000) 2024-05-14T17:17:42.501 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:42 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:42.210+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:42.501 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:42 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:42.425+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:42.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:42 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:42.502+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:42.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:42 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:42.502+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:43.233 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:42 smithi092 podman[77769]: 2024-05-14 17:17:42.933966888 +0000 UTC m=+5.709516037 container died 6b3e9e8125c78120c0e3fda5ca58da92e6a03601895ccbd559b9a3e55fef1bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-14T17:17:43.234 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:42 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:42.786+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:43.234 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:42 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:42.785+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:43.234 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:42 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:42.785+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:43.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:43 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:43.124+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:43.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:43 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:43.124+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:43.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:43 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:43.015+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:43.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:43 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:43.015+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:43.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:43 smithi092 podman[77769]: 2024-05-14 17:17:43.326064642 +0000 UTC m=+6.101613780 container cleanup 6b3e9e8125c78120c0e3fda5ca58da92e6a03601895ccbd559b9a3e55fef1bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-14T17:17:43.499 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:43 smithi092 bash[77769]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4 2024-05-14T17:17:43.526 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:43 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:43.250+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:43.526 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:43 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:43.399+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:43.909 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:43 smithi092 podman[77784]: 2024-05-14 17:17:43.594145293 +0000 UTC m=+0.662603181 container remove 6b3e9e8125c78120c0e3fda5ca58da92e6a03601895ccbd559b9a3e55fef1bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-14T17:17:43.909 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:43 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:43.754+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:43.909 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:43 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:43.789+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:43.909 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:43 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:43.789+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:43.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:43 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:43.527+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:43.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:43 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:43.527+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:44.188 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:43 smithi092 podman[77870]: 2024-05-14 17:17:43.813164085 +0000 UTC m=+0.017002083 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:17:44.188 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:43 smithi092 podman[77870]: 2024-05-14 17:17:43.936568259 +0000 UTC m=+0.140406242 container create f2e617c3fd42f2480a7f02fed6e89ae9ef89ffac21ec2d69a9599ea776dbfb69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-14T17:17:44.188 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:44 smithi092 podman[77870]: 2024-05-14 17:17:44.189695367 +0000 UTC m=+0.393533337 container init f2e617c3fd42f2480a7f02fed6e89ae9ef89ffac21ec2d69a9599ea776dbfb69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-14T17:17:44.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:44 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:44.082+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:44.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:44 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:44.082+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:44.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:43 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:43.976+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:44.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:43 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:43.976+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:44.458 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:44.423+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:44.459 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:44.204+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:44.459 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:44.204+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.411421+0000 front 2024-05-14T17:17:17.411497+0000 (oldest deadline 2024-05-14T17:17:43.311558+0000) 2024-05-14T17:17:44.483 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:44 smithi092 podman[77870]: 2024-05-14 17:17:44.192887478 +0000 UTC m=+0.396725458 container start f2e617c3fd42f2480a7f02fed6e89ae9ef89ffac21ec2d69a9599ea776dbfb69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-05-14T17:17:44.483 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:44 smithi092 podman[77870]: 2024-05-14 17:17:44.289571891 +0000 UTC m=+0.493409879 container attach f2e617c3fd42f2480a7f02fed6e89ae9ef89ffac21ec2d69a9599ea776dbfb69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=) 2024-05-14T17:17:44.749 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 14 17:17:44 smithi092 podman[77870]: 2024-05-14 17:17:44.484844547 +0000 UTC m=+0.688682531 container died f2e617c3fd42f2480a7f02fed6e89ae9ef89ffac21ec2d69a9599ea776dbfb69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-14T17:17:44.958 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:44.553+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:44.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:44 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:44.553+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:45.108 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:44.798+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:45.109 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:44.773+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:45.109 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:44 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:44.773+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:45.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.4.service' 2024-05-14T17:17:45.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:44 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:44.950+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:45.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:44 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:44.950+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:45.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:45 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:45.041+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:45.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:45 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:45.041+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:45.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:17:45.217 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-14T17:17:45.218 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-14T17:17:45.218 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.5 2024-05-14T17:17:45.527 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:45.392+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:45.527 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:45.248+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:45.527 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:45.248+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.411421+0000 front 2024-05-14T17:17:17.411497+0000 (oldest deadline 2024-05-14T17:17:43.311558+0000) 2024-05-14T17:17:45.527 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:45 smithi203 systemd[1]: Stopping Ceph osd.5 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:17:45.959 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:45.560+0000 7f7a57a7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:17:45.959 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:45.560+0000 7f7a57a7e700 -1 osd.5 129 *** Got signal Terminated *** 2024-05-14T17:17:45.959 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:45.560+0000 7f7a57a7e700 -1 osd.5 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:17:45.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:45.528+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:45.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:45 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:45.528+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:46.249 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:45 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:45.759+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:46.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:45 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:45.792+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:46.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:45 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:45.792+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:46.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:46 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:46.044+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:46.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:46 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:46.044+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:46.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:45 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:45.964+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:46.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:45 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:45.964+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:46.568 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:46.217+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:46.568 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:46.217+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.411421+0000 front 2024-05-14T17:17:17.411497+0000 (oldest deadline 2024-05-14T17:17:43.311558+0000) 2024-05-14T17:17:46.568 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:46.422+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:46.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:46.569+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:46.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:46 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:46.569+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:46.999 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:46 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:46.719+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:46.999 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:46 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:46.753+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:46.999 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:46 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:46.753+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:47.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:47.002+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:47.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:47.002+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:47.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:46 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:46.943+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:47.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:46 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:46.943+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:47.459 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:47.205+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:47.459 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:47.205+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.411421+0000 front 2024-05-14T17:17:17.411497+0000 (oldest deadline 2024-05-14T17:17:43.311558+0000) 2024-05-14T17:17:47.459 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:47.426+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:47.459 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:47.426+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:47.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:47.598+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:47.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:47 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:47.598+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:47.999 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:47.715+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:48.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:47.802+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:48.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:47 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:47.802+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:48.454 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:47.992+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:48.454 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:47.992+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:48.454 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:47.995+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:48.454 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:47.995+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:48.454 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:47.995+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:48.605 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:48 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:48.446+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:48.606 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:48 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:48.446+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:48.606 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:48 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:48.222+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:48.606 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:48 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:48.222+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.411421+0000 front 2024-05-14T17:17:17.411497+0000 (oldest deadline 2024-05-14T17:17:43.311558+0000) 2024-05-14T17:17:48.606 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:48 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:48.222+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.312158+0000 front 2024-05-14T17:17:23.312089+0000 (oldest deadline 2024-05-14T17:17:47.412178+0000) 2024-05-14T17:17:48.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:48 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:48.606+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:48.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:48 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:48.606+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:49.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:48.711+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:49.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:48.711+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:49.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:48.772+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:49.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:48.772+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:49.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:48 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:48.772+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:49.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:48 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:48.958+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:49.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:48 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:48.958+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:49.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:49.001+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:49.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:49.001+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:49.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:49.001+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:49.580 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:49.495+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:49.580 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:49.495+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:49.580 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:49.258+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:49.580 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:49.258+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.411421+0000 front 2024-05-14T17:17:17.411497+0000 (oldest deadline 2024-05-14T17:17:43.311558+0000) 2024-05-14T17:17:49.580 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:49.258+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.312158+0000 front 2024-05-14T17:17:23.312089+0000 (oldest deadline 2024-05-14T17:17:47.412178+0000) 2024-05-14T17:17:49.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:49.581+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:49.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:49.581+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:49.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:49 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:49.581+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:50.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:49.672+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:50.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:49.672+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:50.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:49.748+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:50.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:49.748+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:50.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:49 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:49.748+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:50.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:49.993+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:50.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:49.993+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:50.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:49.979+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:50.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:49.979+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:50.454 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:49.979+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:50.543 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:50.212+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:07.207886+0000 front 2024-05-14T17:17:07.207910+0000 (oldest deadline 2024-05-14T17:17:31.308070+0000) 2024-05-14T17:17:50.543 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:50.212+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.411421+0000 front 2024-05-14T17:17:17.411497+0000 (oldest deadline 2024-05-14T17:17:43.311558+0000) 2024-05-14T17:17:50.543 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5[47853]: 2024-05-14T17:17:50.212+0000 7f7a5006a700 -1 osd.5 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.312158+0000 front 2024-05-14T17:17:23.312089+0000 (oldest deadline 2024-05-14T17:17:47.412178+0000) 2024-05-14T17:17:50.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:50.543+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:50.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:50.543+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:50.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:50.599+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:50.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:50.599+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:50.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:50 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:50.599+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:50.999 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:50 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:50.673+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:51.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:50 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:50.673+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:51.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:50 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:50.794+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:51.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:50 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:50.794+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:51.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:50 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:50.794+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:51.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:51.002+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:51.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:51.002+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:51.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:50 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:50.946+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:51.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:50 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:50.946+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:51.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:50 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:50.946+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:51.459 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:51 smithi203 podman[79557]: 2024-05-14 17:17:51.209165021 +0000 UTC m=+5.785568414 container died 3f0d1a8584f278e4413af54f5978f0b4b5029484e514d0cff2a5ace81d5766a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T17:17:51.868 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:51 smithi203 podman[79557]: 2024-05-14 17:17:51.601229633 +0000 UTC m=+6.177633072 container cleanup 3f0d1a8584f278e4413af54f5978f0b4b5029484e514d0cff2a5ace81d5766a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:17:51.868 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:51 smithi203 bash[79557]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5 2024-05-14T17:17:51.869 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:51 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:51.566+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:51.869 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:51 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:51.566+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:51.869 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:51 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:51.607+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:51.869 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:51 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:51.607+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:51.869 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:51 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:51.607+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:51.901 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:51 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:51.626+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:51.901 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:51 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:51.626+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:51.901 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:51 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:51.753+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:51.901 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:51 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:51.753+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:51.901 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:51 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:51.753+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:52.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:51.988+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:52.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:51.988+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:52.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:51.916+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:52.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:51.916+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:52.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:51 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:51.916+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:52.208 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:51 smithi203 podman[79573]: 2024-05-14 17:17:51.869752938 +0000 UTC m=+0.660188617 container remove 3f0d1a8584f278e4413af54f5978f0b4b5029484e514d0cff2a5ace81d5766a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-14T17:17:52.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:52 smithi203 podman[79660]: 2024-05-14 17:17:52.118963672 +0000 UTC m=+0.017109868 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:17:52.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:52 smithi203 podman[79660]: 2024-05-14 17:17:52.254353019 +0000 UTC m=+0.152499204 container create 3fd3313e6fb3fb765f4e7ab341ea41601361be4ac0ee5d93e7581566a7eb9574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-14T17:17:52.793 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:52.557+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:52.793 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:52.557+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:52.794 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:52 smithi203 podman[79660]: 2024-05-14 17:17:52.490408637 +0000 UTC m=+0.388554822 container init 3fd3313e6fb3fb765f4e7ab341ea41601361be4ac0ee5d93e7581566a7eb9574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:17:52.794 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:52 smithi203 podman[79660]: 2024-05-14 17:17:52.493453055 +0000 UTC m=+0.391599245 container start 3fd3313e6fb3fb765f4e7ab341ea41601361be4ac0ee5d93e7581566a7eb9574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-14T17:17:52.794 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:52 smithi203 podman[79660]: 2024-05-14 17:17:52.598949619 +0000 UTC m=+0.497095818 container attach 3fd3313e6fb3fb765f4e7ab341ea41601361be4ac0ee5d93e7581566a7eb9574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:17:52.794 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:52.616+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:52.794 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:52.616+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:52.794 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:52 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:52.616+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:52.999 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:52 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:52.615+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:53.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:52 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:52.615+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:53.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:52 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:52.789+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:53.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:52 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:52.789+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:53.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:52 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:52.789+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:53.162 INFO:journalctl@ceph.osd.5.smithi203.stdout:May 14 17:17:52 smithi203 podman[79660]: 2024-05-14 17:17:52.795112238 +0000 UTC m=+0.693258431 container died 3fd3313e6fb3fb765f4e7ab341ea41601361be4ac0ee5d93e7581566a7eb9574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD) 2024-05-14T17:17:53.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:52.919+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:53.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:52.919+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:53.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:52.919+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:53.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:52.962+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:53.204 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:52.962+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:53.204 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:52.962+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:17:53.478 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.5.service' 2024-05-14T17:17:53.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:17:53.517 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-14T17:17:53.517 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-14T17:17:53.517 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.6 2024-05-14T17:17:53.826 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:53 smithi069 systemd[1]: Stopping Ceph osd.6 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:17:53.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:53 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:53.509+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:53.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:53 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:53.509+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:53.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:53 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:53.591+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:53.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:53 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:53.591+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:53.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:53 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:53.591+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:54.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:53 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:53.609+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:54.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:53 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:53.609+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:54.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:53 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:53.805+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:54.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:53 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:53.805+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:54.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:53 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:53.805+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:54.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:53.950+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:54.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:53.950+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:54.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:53.950+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:17:54.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:53.826+0000 7fa402d15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:17:54.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:53.826+0000 7fa402d15700 -1 osd.6 129 *** Got signal Terminated *** 2024-05-14T17:17:54.204 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:53.826+0000 7fa402d15700 -1 osd.6 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:17:54.204 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:53.949+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:54.204 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:53.949+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:54.204 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:53 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:53.949+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:54.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:54 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:54.530+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:54.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:54 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:54.530+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:54.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:54 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:54.571+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:54.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:54 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:54.571+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:54.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:54 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:54.571+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:54.999 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:54 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:54.652+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:55.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:54 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:54.652+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:55.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:54 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:54.772+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:55.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:54 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:54.772+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:55.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:54 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:54.772+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:55.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:54.937+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:55.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:54.937+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:55.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:54.937+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:55.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:54.914+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:55.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:54.914+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:55.204 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:54 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:54.914+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:17:55.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:55 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:55.525+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:55.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:55 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:55.525+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:55.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:55 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:55.557+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:55.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:55 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:55.557+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:55.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:55 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:55.557+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:55.999 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:55 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:55.634+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:56.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:55 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:55.634+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:56.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:55 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:55.793+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:56.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:55 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:55.793+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:56.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:55 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:55.793+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:56.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:55 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:55.917+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:56.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:55 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:55.917+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:56.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:55 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:55.917+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:17:56.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:55 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:55.947+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:56.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:55 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:55.947+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:56.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:55 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:55.947+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:56.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:55 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:55.947+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.711022+0000 front 2024-05-14T17:17:32.711047+0000 (oldest deadline 2024-05-14T17:17:55.011186+0000) 2024-05-14T17:17:56.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:56 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:56.509+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:56.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:56 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:56.509+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:56.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:56 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:56.512+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:56.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:56 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:56.512+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:56.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:56 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:56.512+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:56.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:56 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:56.512+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:17:57.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:56 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:56.616+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:57.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:56 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:56.616+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:57.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:56 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:56.811+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:57.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:56 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:56.811+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:57.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:56 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:56.811+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:57.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:56 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:56.811+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:17:57.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:56 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:56.922+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:57.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:56 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:56.922+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:57.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:56 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:56.922+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:57.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:56 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:56.922+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.711022+0000 front 2024-05-14T17:17:32.711047+0000 (oldest deadline 2024-05-14T17:17:55.011186+0000) 2024-05-14T17:17:57.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:56 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:56.908+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:57.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:56 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:56.908+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:57.204 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:56 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:56.908+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:17:57.204 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:56 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:56.908+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:17:57.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:57 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:57.543+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:57.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:57 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:57.543+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:57.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:57 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:57.464+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:57.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:57 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:57.464+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:57.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:57 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:57.464+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:57.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:57 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:57.464+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:17:58.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:57 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:57.648+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:58.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:57 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:57.648+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:58.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:57 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:57.648+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.872999+0000 front 2024-05-14T17:17:31.872947+0000 (oldest deadline 2024-05-14T17:17:57.172797+0000) 2024-05-14T17:17:58.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:57 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:57.778+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:58.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:57 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:57.778+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:58.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:57 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:57.778+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:58.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:57 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:57.779+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:17:58.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:57 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:57.883+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.504661+0000 front 2024-05-14T17:17:08.504689+0000 (oldest deadline 2024-05-14T17:17:30.705601+0000) 2024-05-14T17:17:58.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:57 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:57.883+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.107567+0000 front 2024-05-14T17:17:17.107540+0000 (oldest deadline 2024-05-14T17:17:41.108268+0000) 2024-05-14T17:17:58.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:57 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:57.883+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:25.110018+0000 front 2024-05-14T17:17:25.109920+0000 (oldest deadline 2024-05-14T17:17:47.410022+0000) 2024-05-14T17:17:58.203 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:57 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6[56756]: 2024-05-14T17:17:57.883+0000 7fa3ff309700 -1 osd.6 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.711022+0000 front 2024-05-14T17:17:32.711047+0000 (oldest deadline 2024-05-14T17:17:55.011186+0000) 2024-05-14T17:17:58.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:57 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:57.918+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:58.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:57 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:57.918+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:58.204 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:57 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:57.918+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:17:58.204 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:57 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:57.918+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:17:58.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:58 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:58.545+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:58.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:58 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:58.545+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:58.959 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:58 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:58.545+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:17:58.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:58 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:58.475+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:58.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:58 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:58.475+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:58.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:58 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:58.475+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:58.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:58 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:58.475+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:17:59.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:58 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:58.681+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:17:59.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:58 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:58.681+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:17:59.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:58 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:58.681+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.872999+0000 front 2024-05-14T17:17:31.872947+0000 (oldest deadline 2024-05-14T17:17:57.172797+0000) 2024-05-14T17:17:59.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:58 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:58.807+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:17:59.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:58 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:58.807+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:17:59.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:58 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:58.807+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:17:59.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:58 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:58.807+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:17:59.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:58 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:58.875+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:17:59.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:58 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:58.876+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:17:59.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:58 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:58.876+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:17:59.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:58 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:58.876+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:17:59.703 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:59 smithi069 podman[112077]: 2024-05-14 17:17:59.443939948 +0000 UTC m=+5.729149707 container died 2b8e94f60e67b906dfd86acdec44e43c25a8f4675737d740bdd6b44780c0320e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-14T17:17:59.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:59 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:59.445+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:17:59.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:59 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:59.445+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:17:59.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:59 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:59.445+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:17:59.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:17:59 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:17:59.445+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:17:59.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:59 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:59.508+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:17:59.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:59 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:59.508+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:17:59.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:17:59 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:17:59.508+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:17:59.999 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:59 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:59.730+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:18:00.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:59 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:59.730+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:18:00.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:17:59 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:17:59.730+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.872999+0000 front 2024-05-14T17:17:31.872947+0000 (oldest deadline 2024-05-14T17:17:57.172797+0000) 2024-05-14T17:18:00.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:59 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:59.839+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:00.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:59 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:59.839+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:00.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:59 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:59.839+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:00.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:17:59 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:17:59.839+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:00.090 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:59 smithi069 podman[112077]: 2024-05-14 17:17:59.868454315 +0000 UTC m=+6.153664064 container cleanup 2b8e94f60e67b906dfd86acdec44e43c25a8f4675737d740bdd6b44780c0320e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:18:00.091 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:17:59 smithi069 bash[112077]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6 2024-05-14T17:18:00.091 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:59 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:59.910+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:00.091 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:59 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:59.910+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:00.091 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:59 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:59.910+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:00.091 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:17:59 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:17:59.910+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:00.453 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:18:00 smithi069 podman[112093]: 2024-05-14 17:18:00.128422237 +0000 UTC m=+0.681744779 container remove 2b8e94f60e67b906dfd86acdec44e43c25a8f4675737d740bdd6b44780c0320e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-14T17:18:00.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:00 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:00.459+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:00.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:00 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:00.459+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:00.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:00 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:00.459+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:00.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:00 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:00.432+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:00.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:00 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:00.432+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:00.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:00 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:00.432+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:00.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:00 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:00.432+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:00.848 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:18:00 smithi069 podman[112194]: 2024-05-14 17:18:00.472896336 +0000 UTC m=+0.016817481 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:18:00.848 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:18:00 smithi069 podman[112194]: 2024-05-14 17:18:00.579454565 +0000 UTC m=+0.123375684 container create 58daf20495fa8085984d0005d2e8c56eeef47817233c9af507002c068abe319b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-14T17:18:00.848 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:18:00 smithi069 podman[112194]: 2024-05-14 17:18:00.849826972 +0000 UTC m=+0.393748090 container init 58daf20495fa8085984d0005d2e8c56eeef47817233c9af507002c068abe319b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True) 2024-05-14T17:18:01.147 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:18:00 smithi069 podman[112194]: 2024-05-14 17:18:00.853358746 +0000 UTC m=+0.397279874 container start 58daf20495fa8085984d0005d2e8c56eeef47817233c9af507002c068abe319b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-14T17:18:01.147 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:18:00 smithi069 podman[112194]: 2024-05-14 17:18:00.93222421 +0000 UTC m=+0.476145340 container attach 58daf20495fa8085984d0005d2e8c56eeef47817233c9af507002c068abe319b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:18:01.147 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:00.931+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:01.147 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:00.931+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:01.147 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:00.931+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:01.147 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:00 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:00.931+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:01.249 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:00 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:00.758+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:18:01.249 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:00 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:00.758+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:18:01.250 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:00 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:00.758+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.872999+0000 front 2024-05-14T17:17:31.872947+0000 (oldest deadline 2024-05-14T17:17:57.172797+0000) 2024-05-14T17:18:01.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:00 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:00.829+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:01.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:00 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:00.829+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:01.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:00 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:00.829+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:01.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:00 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:00.829+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:01.434 INFO:journalctl@ceph.osd.6.smithi069.stdout:May 14 17:18:01 smithi069 podman[112194]: 2024-05-14 17:18:01.148463806 +0000 UTC m=+0.692384928 container died 58daf20495fa8085984d0005d2e8c56eeef47817233c9af507002c068abe319b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T17:18:01.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:01 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:01.413+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:01.710 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:01 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:01.413+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:01.710 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:01 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:01.413+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:01.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:01 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:01.466+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:01.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:01 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:01.466+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:01.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:01 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:01.466+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:01.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:01 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:01.466+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:01.736 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.6.service' 2024-05-14T17:18:01.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:01.777 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-14T17:18:01.777 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-14T17:18:01.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.7 2024-05-14T17:18:02.085 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:01 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:01.783+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:02.085 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:01 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:01.783+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:02.085 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:01 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:01.783+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:02.085 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:01 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:01.783+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:02.085 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:01 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:01.762+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:18:02.086 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:01 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:01.762+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:18:02.086 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:01 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:01.762+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.872999+0000 front 2024-05-14T17:17:31.872947+0000 (oldest deadline 2024-05-14T17:17:57.172797+0000) 2024-05-14T17:18:02.086 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:01 smithi092 systemd[1]: Stopping Ceph osd.7 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:02.086 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:02 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:02.086+0000 7fd8a1fad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:02.086 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:02 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:02.086+0000 7fd8a1fad700 -1 osd.7 129 *** Got signal Terminated *** 2024-05-14T17:18:02.086 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:02 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:02.086+0000 7fd8a1fad700 -1 osd.7 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:18:02.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:01 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:01.913+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:02.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:01 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:01.913+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:02.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:01 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:01.913+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:02.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:01 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:01.913+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:02.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:02 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:02.507+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:02.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:02 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:02.507+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:02.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:02 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:02.507+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:02.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:02 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:02.507+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:02.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:02 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:02.405+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:02.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:02 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:02.405+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:02.710 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:02 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:02.405+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:03.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:02 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:02.804+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:18:03.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:02 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:02.804+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:18:03.000 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:02 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:02.804+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.872999+0000 front 2024-05-14T17:17:31.872947+0000 (oldest deadline 2024-05-14T17:17:57.172797+0000) 2024-05-14T17:18:03.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:02 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:02.822+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:03.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:02 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:02.822+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:03.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:02 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:02.822+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:03.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:02 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:02.822+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:03.205 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:02 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:02.915+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:03.206 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:02 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:02.915+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:03.206 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:02 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:02.915+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:03.206 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:02 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:02.915+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:03.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:03.362+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:03.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:03.362+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:03.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:03.362+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:03.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:03.536+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:03.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:03.536+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:03.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:03.536+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:03.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:03 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:03.536+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:04.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:03 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:03.884+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:04.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:03 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:03.884+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:04.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:03 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:03.884+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:04.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:03 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:03.884+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:04.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:03 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:03.884+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:04.250 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:03 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:03.807+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:18:04.251 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:03 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:03.807+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:18:04.251 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:03 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:03.807+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.872999+0000 front 2024-05-14T17:17:31.872947+0000 (oldest deadline 2024-05-14T17:17:57.172797+0000) 2024-05-14T17:18:04.251 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:03 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:03.826+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:04.251 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:03 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:03.826+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:04.251 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:03 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:03.826+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:04.251 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:03 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:03.826+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:04.251 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:03 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:03.826+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:04.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:04.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:04.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:04.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:04.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:04.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:04.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:04.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:04.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:04.348+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:04.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:04.348+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:04.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:04.348+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:04.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:04 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:04.348+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:05.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:04.905+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:05.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:04.905+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:05.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:04.905+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:05.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:04.905+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:05.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:04 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:04.905+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:05.249 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:04 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:04.791+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:18:05.249 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:04 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:04.791+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:18:05.249 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:04 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:04.791+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.872999+0000 front 2024-05-14T17:17:31.872947+0000 (oldest deadline 2024-05-14T17:17:57.172797+0000) 2024-05-14T17:18:05.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:04 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:04.792+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:05.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:04 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:04.792+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:05.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:04 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:04.792+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:05.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:04 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:04.792+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:05.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:04 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:04.792+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:05.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:05 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:05.367+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:05.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:05 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:05.367+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:05.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:05 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:05.367+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:05.710 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:05 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:05.367+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:05.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:05 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:05.597+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:05.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:05 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:05.597+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:05.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:05 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:05.597+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:05.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:05 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:05.597+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:05.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:05 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:05.597+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:06.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:05 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:05.896+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:06.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:05 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:05.896+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:06.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:05 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:05.896+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:06.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:05 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:05.896+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:06.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:05 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:05.896+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:06.249 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:05 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:05.777+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:18:06.249 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:05 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:05.777+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:18:06.250 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:05 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:05.777+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.872999+0000 front 2024-05-14T17:17:31.872947+0000 (oldest deadline 2024-05-14T17:17:57.172797+0000) 2024-05-14T17:18:06.250 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:05 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:05.777+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.974169+0000 front 2024-05-14T17:17:39.974140+0000 (oldest deadline 2024-05-14T17:18:05.274380+0000) 2024-05-14T17:18:06.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:05 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:05.809+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:06.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:05 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:05.809+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:06.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:05 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:05.809+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:06.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:05 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:05.809+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:06.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:05 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:05.809+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:06.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:06.381+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:06.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:06.381+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:06.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:06.381+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:06.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:06.381+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:06.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:06.561+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:06.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:06.561+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:06.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:06.561+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:06.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:06.561+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:06.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:06 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:06.561+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:07.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:06 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:06.937+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:07.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:06 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:06.937+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:07.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:06 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:06.937+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:07.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:06 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:06.937+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:07.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:06 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:06.937+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:07.249 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:06.784+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.469837+0000 front 2024-05-14T17:17:17.469626+0000 (oldest deadline 2024-05-14T17:17:38.569770+0000) 2024-05-14T17:18:07.250 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:06.784+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.471079+0000 front 2024-05-14T17:17:24.470980+0000 (oldest deadline 2024-05-14T17:17:47.970861+0000) 2024-05-14T17:18:07.250 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:06.784+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.872999+0000 front 2024-05-14T17:17:31.872947+0000 (oldest deadline 2024-05-14T17:17:57.172797+0000) 2024-05-14T17:18:07.250 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7[54252]: 2024-05-14T17:18:06.784+0000 7fd89e5a1700 -1 osd.7 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.974169+0000 front 2024-05-14T17:17:39.974140+0000 (oldest deadline 2024-05-14T17:18:05.274380+0000) 2024-05-14T17:18:07.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:06.828+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:07.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:06.828+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:07.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:06.828+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:07.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:06.828+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:07.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:06.828+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:07.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:07 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:07.386+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:07.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:07 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:07.386+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:07.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:07 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:07.386+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:07.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:07 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:07.386+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:07.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:07 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:07.527+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:07.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:07 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:07.527+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:07.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:07 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:07.527+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:07.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:07 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:07.527+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:07.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:07 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:07.527+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:07.902 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:07 smithi092 podman[77988]: 2024-05-14 17:18:07.637365105 +0000 UTC m=+5.666810999 container died 4d095482f907cfd9a76229aa9ea3bdf4ec1e439234b62a218cc0aa9a6783cae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-14T17:18:07.902 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:07 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:07.798+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:07.902 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:07 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:07.798+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:07.903 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:07 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:07.798+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:07.903 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:07 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:07.798+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:07.903 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:07 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:07.798+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:08.249 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:07 smithi092 podman[77988]: 2024-05-14 17:18:07.985619976 +0000 UTC m=+6.015065853 container cleanup 4d095482f907cfd9a76229aa9ea3bdf4ec1e439234b62a218cc0aa9a6783cae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-14T17:18:08.250 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:07 smithi092 bash[77988]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7 2024-05-14T17:18:08.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:07 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:07.982+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:08.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:07 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:07.982+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:08.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:07 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:07.982+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:08.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:07 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:07.982+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:08.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:07 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:07.982+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:08.601 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:08 smithi092 podman[78003]: 2024-05-14 17:18:08.261685764 +0000 UTC m=+0.620327833 container remove 4d095482f907cfd9a76229aa9ea3bdf4ec1e439234b62a218cc0aa9a6783cae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-14T17:18:08.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:08 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:08.372+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:08.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:08 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:08.372+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:08.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:08 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:08.372+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:08.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:08 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:08.372+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:08.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:08 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:08.490+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:08.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:08 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:08.490+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:08.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:08 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:08.490+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:08.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:08 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:08.490+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:08.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:08 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:08.490+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:08.922 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:08 smithi092 podman[78089]: 2024-05-14 17:18:08.504975074 +0000 UTC m=+0.017287026 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:18:08.922 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:08 smithi092 podman[78089]: 2024-05-14 17:18:08.645881233 +0000 UTC m=+0.158193186 container create d913b2d9275dfeb02138b77ac1958e612b0883db62e6a2816fdd980e98464c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-14T17:18:08.922 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:08 smithi092 podman[78089]: 2024-05-14 17:18:08.923475988 +0000 UTC m=+0.435787941 container init d913b2d9275dfeb02138b77ac1958e612b0883db62e6a2816fdd980e98464c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:18:08.922 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:08 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:08.842+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:08.923 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:08 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:08.842+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:08.923 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:08 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:08.842+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:08.923 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:08 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:08.842+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:08.923 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:08 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:08.842+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:09.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:08 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:08.934+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:09.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:08 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:08.934+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:09.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:08 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:08.934+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:09.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:08 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:08.934+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:09.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:08 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:08.934+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:09.215 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:08 smithi092 podman[78089]: 2024-05-14 17:18:08.926615721 +0000 UTC m=+0.438927681 container start d913b2d9275dfeb02138b77ac1958e612b0883db62e6a2816fdd980e98464c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-14T17:18:09.215 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:09 smithi092 podman[78089]: 2024-05-14 17:18:09.015222986 +0000 UTC m=+0.527534953 container attach d913b2d9275dfeb02138b77ac1958e612b0883db62e6a2816fdd980e98464c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-05-14T17:18:09.488 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 14 17:18:09 smithi092 podman[78089]: 2024-05-14 17:18:09.216980223 +0000 UTC m=+0.729292177 container died d913b2d9275dfeb02138b77ac1958e612b0883db62e6a2816fdd980e98464c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T17:18:09.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:09 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:09.330+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:09.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:09 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:09.330+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:09.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:09 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:09.330+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:09.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:09 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:09.330+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:09.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:09 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:09.530+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:09.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:09 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:09.530+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:09.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:09 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:09.530+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:09.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:09 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:09.530+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:09.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:09 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:09.530+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:09.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.7.service' 2024-05-14T17:18:09.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:09.882 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-14T17:18:09.882 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-14T17:18:09.882 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.8 2024-05-14T17:18:10.182 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:09 smithi203 systemd[1]: Stopping Ceph osd.8 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:10.182 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:10.182+0000 7f481dd1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:10.182 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:10.182+0000 7f481dd1b700 -1 osd.8 129 *** Got signal Terminated *** 2024-05-14T17:18:10.182 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:10.182+0000 7f481dd1b700 -1 osd.8 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:18:10.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:09 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:09.920+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:10.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:09 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:09.920+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:10.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:09 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:09.920+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:10.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:09 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:09.920+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:10.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:09 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:09.920+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:10.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:09 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:09.879+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:10.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:09 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:09.879+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:10.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:09 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:09.879+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:10.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:09 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:09.879+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:10.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:09 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:09.879+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:10.459 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:10.288+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:10.459 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:10.288+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:10.459 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:10.288+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:10.459 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:10.288+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:10.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:10.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:10.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:10.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:10.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:10.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:10.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:10.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:10.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:10 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:10.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:11.211 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:10.955+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:11.211 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:10.955+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:11.211 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:10.955+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:11.211 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:10.955+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:11.211 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:10 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:10.955+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:11.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:10 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:10.865+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:11.275 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:10 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:10.865+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:11.276 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:10 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:10.865+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:11.276 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:10 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:10.865+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:11.276 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:10 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:10.865+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:11.547 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:11 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:11.262+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:11.547 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:11 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:11.262+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:11.547 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:11 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:11.262+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:11.548 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:11 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:11.262+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:11.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:11 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:11.548+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:11.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:11 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:11.548+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:11.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:11 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:11.548+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:11.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:11 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:11.548+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:11.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:11 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:11.548+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:12.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:11 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:11.920+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:12.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:11 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:11.920+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:12.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:11 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:11.920+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:12.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:11 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:11.920+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:12.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:11 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:11.920+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:12.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:11 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:11.829+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:12.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:11 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:11.829+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:12.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:11 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:11.829+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:12.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:11 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:11.829+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:12.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:11 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:11.829+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:12.518 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:12.268+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:12.518 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:12.268+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:12.518 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:12.268+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:12.518 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:12.268+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:12.518 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:12.268+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.505067+0000 front 2024-05-14T17:17:47.505115+0000 (oldest deadline 2024-05-14T17:18:12.205329+0000) 2024-05-14T17:18:12.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:12.518+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:12.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:12.518+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:12.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:12.518+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:12.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:12.518+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:12.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:12 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:12.518+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:13.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:12 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:12.940+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:13.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:12 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:12.940+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:13.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:12 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:12.940+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:13.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:12 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:12.940+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:13.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:12 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:12.940+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:13.249 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:12 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:12.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:13.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:12 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:12.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:13.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:12 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:12.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:13.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:12 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:12.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:13.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:12 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:12.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:13.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:13.294+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:13.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:13.294+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:13.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:13.294+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:13.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:13.294+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:13.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:13.294+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.505067+0000 front 2024-05-14T17:17:47.505115+0000 (oldest deadline 2024-05-14T17:18:12.205329+0000) 2024-05-14T17:18:13.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:13.520+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:13.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:13.520+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:13.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:13.520+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:13.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:13.520+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:13.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:13.520+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:13.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:13 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:13.520+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:14.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:13 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:13.822+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:14.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:13 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:13.822+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:14.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:13 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:13.822+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:14.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:13 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:13.822+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:14.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:13 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:13.822+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:14.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:13 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:13.966+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:14.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:13 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:13.966+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:14.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:13 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:13.966+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:14.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:13 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:13.966+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:14.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:13 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:13.966+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:14.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:13 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:13.966+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.407792+0000 front 2024-05-14T17:17:47.407879+0000 (oldest deadline 2024-05-14T17:18:13.307661+0000) 2024-05-14T17:18:14.560 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:14.289+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.498481+0000 front 2024-05-14T17:17:17.498530+0000 (oldest deadline 2024-05-14T17:17:40.398635+0000) 2024-05-14T17:18:14.560 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:14.289+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:24.900906+0000 front 2024-05-14T17:17:24.900856+0000 (oldest deadline 2024-05-14T17:17:46.601016+0000) 2024-05-14T17:18:14.560 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:14.289+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:31.802554+0000 front 2024-05-14T17:17:31.802517+0000 (oldest deadline 2024-05-14T17:17:57.603116+0000) 2024-05-14T17:18:14.560 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:14.289+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.904203+0000 front 2024-05-14T17:17:39.904219+0000 (oldest deadline 2024-05-14T17:18:04.004169+0000) 2024-05-14T17:18:14.560 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8[54417]: 2024-05-14T17:18:14.289+0000 7f481a30f700 -1 osd.8 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.505067+0000 front 2024-05-14T17:17:47.505115+0000 (oldest deadline 2024-05-14T17:18:12.205329+0000) 2024-05-14T17:18:14.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:14.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:14.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:14.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:14.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:14.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:14.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:14.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:14.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:14.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:14.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:14 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:14.560+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:15.249 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:14 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:14.849+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:15.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:14 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:14.849+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:15.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:14 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:14.849+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:15.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:14 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:14.849+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:15.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:14 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:14.849+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:15.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:14 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:14.849+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:15.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.009+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:15.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.009+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:15.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.009+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:15.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.009+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:15.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.009+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:15.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.009+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.407792+0000 front 2024-05-14T17:17:47.407879+0000 (oldest deadline 2024-05-14T17:18:13.307661+0000) 2024-05-14T17:18:15.851 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:15 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:15.563+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:15.851 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:15 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:15.563+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:15.851 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:15 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:15.563+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:15.851 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:15 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:15.563+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:15.852 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:15 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:15.563+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:15.852 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:15 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:15.563+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:16.140 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:15 smithi203 podman[79767]: 2024-05-14 17:18:15.852713899 +0000 UTC m=+5.772710759 container died c906dce75de92860da13ea9df5ae8f05bd00fb2b7b273f1c811e62739699a0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2) 2024-05-14T17:18:16.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:15 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:15.859+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:16.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:15 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:15.859+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:16.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:15 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:15.859+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:16.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:15 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:15.859+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:16.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:15 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:15.859+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:16.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:15 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:15.859+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:16.399 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:16 smithi203 podman[79767]: 2024-05-14 17:18:16.207652884 +0000 UTC m=+6.127649742 container cleanup c906dce75de92860da13ea9df5ae8f05bd00fb2b7b273f1c811e62739699a0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-14T17:18:16.399 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:16 smithi203 bash[79767]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8 2024-05-14T17:18:16.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.973+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:16.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.973+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:16.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.973+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:16.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.973+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:16.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.973+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:16.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:15 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:15.973+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.407792+0000 front 2024-05-14T17:17:47.407879+0000 (oldest deadline 2024-05-14T17:18:13.307661+0000) 2024-05-14T17:18:16.709 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:16 smithi203 podman[79784]: 2024-05-14 17:18:16.400831913 +0000 UTC m=+0.544937253 container remove c906dce75de92860da13ea9df5ae8f05bd00fb2b7b273f1c811e62739699a0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2) 2024-05-14T17:18:16.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:16 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:16.514+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:16.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:16 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:16.514+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:16.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:16 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:16.514+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:16.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:16 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:16.514+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:16.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:16 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:16.514+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:16.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:16 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:16.514+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:17.144 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:16 smithi203 podman[79869]: 2024-05-14 17:18:16.810065013 +0000 UTC m=+0.110783950 container create cf46f85fd64d721a29a5404232c6f389fbaa5a4e847272a0a14a4fc03054372b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-05-14T17:18:17.145 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:16 smithi203 podman[79869]: 2024-05-14 17:18:16.716964759 +0000 UTC m=+0.017683695 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:18:17.145 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:17 smithi203 podman[79869]: 2024-05-14 17:18:17.045832081 +0000 UTC m=+0.346551016 container init cf46f85fd64d721a29a5404232c6f389fbaa5a4e847272a0a14a4fc03054372b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:18:17.145 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:17 smithi203 podman[79869]: 2024-05-14 17:18:17.048852463 +0000 UTC m=+0.349571399 container start cf46f85fd64d721a29a5404232c6f389fbaa5a4e847272a0a14a4fc03054372b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:18:17.145 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:17 smithi203 podman[79869]: 2024-05-14 17:18:17.145845975 +0000 UTC m=+0.446564909 container attach cf46f85fd64d721a29a5404232c6f389fbaa5a4e847272a0a14a4fc03054372b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-14T17:18:17.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:16 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:16.810+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:17.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:16 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:16.810+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:17.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:16 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:16.810+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:17.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:16 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:16.810+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:17.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:16 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:16.810+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:17.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:16 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:16.810+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:17.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:17 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:17.020+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:17.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:17 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:17.020+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:17.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:17 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:17.020+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:17.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:17 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:17.020+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:17.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:17 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:17.020+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:17.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:17 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:17.020+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.407792+0000 front 2024-05-14T17:17:47.407879+0000 (oldest deadline 2024-05-14T17:18:13.307661+0000) 2024-05-14T17:18:17.459 INFO:journalctl@ceph.osd.8.smithi203.stdout:May 14 17:18:17 smithi203 podman[79869]: 2024-05-14 17:18:17.345357894 +0000 UTC m=+0.646076829 container died cf46f85fd64d721a29a5404232c6f389fbaa5a4e847272a0a14a4fc03054372b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-8-deactivate, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-14T17:18:17.957 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:17 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:17.555+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:17.957 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:17 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:17.555+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:17.957 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:17 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:17.555+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:17.957 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:17 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:17.555+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:17.957 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:17 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:17.555+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:17.957 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:17 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:17.555+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:17.979 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.8.service' 2024-05-14T17:18:18.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:18.018 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-14T17:18:18.019 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-14T17:18:18.019 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.9 2024-05-14T17:18:18.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:17 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:17.850+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:18.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:17 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:17.850+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:18.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:17 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:17.850+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:18.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:17 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:17.850+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:18.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:17 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:17.850+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:18.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:17 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:17.850+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:18.345 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:18 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:18.004+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:18.345 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:18 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:18.004+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:18.345 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:18 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:18.004+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:18.345 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:18 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:18.004+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:18.345 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:18 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:18.004+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:18.345 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:18 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:18.004+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.407792+0000 front 2024-05-14T17:17:47.407879+0000 (oldest deadline 2024-05-14T17:18:13.307661+0000) 2024-05-14T17:18:18.345 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:18 smithi069 systemd[1]: Stopping Ceph osd.9 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:18.703 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:18 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:18.345+0000 7f9b63ab6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:18.703 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:18 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:18.345+0000 7f9b63ab6700 -1 osd.9 129 *** Got signal Terminated *** 2024-05-14T17:18:18.703 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:18 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:18.345+0000 7f9b63ab6700 -1 osd.9 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:18:18.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:18 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:18.575+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:18.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:18 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:18.575+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:18.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:18 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:18.575+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:18.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:18 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:18.575+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:18.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:18 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:18.575+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:18.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:18 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:18.575+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:19.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:18 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:18.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:19.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:18 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:18.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:19.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:18 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:18.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:19.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:18 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:18.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:19.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:18 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:18.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:19.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:18 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:18.848+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:19.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.000+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:19.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.000+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:19.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.000+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:19.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.000+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:19.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.000+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:19.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.000+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.407792+0000 front 2024-05-14T17:17:47.407879+0000 (oldest deadline 2024-05-14T17:18:13.307661+0000) 2024-05-14T17:18:19.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:19 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:19.581+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:19.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:19 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:19.581+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:19.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:19 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:19.581+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:19.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:19 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:19.581+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:19.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:19 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:19.581+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:19.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:19 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:19.581+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:20.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.953+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:20.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.953+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:20.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.953+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:20.203 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.953+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:20.204 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.953+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:20.204 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:19 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:19.953+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.407792+0000 front 2024-05-14T17:17:47.407879+0000 (oldest deadline 2024-05-14T17:18:13.307661+0000) 2024-05-14T17:18:20.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:19 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:19.877+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:20.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:19 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:19.877+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:20.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:19 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:19.877+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:20.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:19 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:19.877+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:20.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:19 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:19.877+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:20.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:19 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:19.877+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:20.861 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:20 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:20.587+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:20.861 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:20 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:20.587+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:20.861 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:20 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:20.587+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:20.861 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:20 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:20.587+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:20.861 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:20 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:20.587+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:20.861 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:20 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:20.587+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:21.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:20 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:20.855+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:21.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:20 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:20.855+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:21.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:20 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:20.855+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:21.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:20 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:20.855+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:21.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:20 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:20.855+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:21.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:20 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:20.855+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:21.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:20 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:20.972+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:21.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:20 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:20.972+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:21.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:20 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:20.972+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:21.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:20 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:20.972+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:21.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:20 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:20.972+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:21.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:20 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:20.972+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.407792+0000 front 2024-05-14T17:17:47.407879+0000 (oldest deadline 2024-05-14T17:18:13.307661+0000) 2024-05-14T17:18:21.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:21 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:21.626+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:21.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:21 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:21.626+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:21.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:21 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:21.626+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:21.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:21 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:21.626+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:21.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:21 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:21.626+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:21.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:21 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:21.626+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:22.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:21 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:21.808+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:22.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:21 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:21.808+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:22.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:21 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:21.808+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:22.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:21 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:21.808+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:22.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:21 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:21.808+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:22.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:21 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:21.808+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:22.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:21 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:21.808+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-05-14T17:17:57.000234+0000 front 2024-05-14T17:17:57.000203+0000 (oldest deadline 2024-05-14T17:18:21.700141+0000) 2024-05-14T17:18:22.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.015+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:22.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.015+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:22.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.015+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:22.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.015+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:22.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.015+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:22.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.015+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.407792+0000 front 2024-05-14T17:17:47.407879+0000 (oldest deadline 2024-05-14T17:18:13.307661+0000) 2024-05-14T17:18:22.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:22 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:22.636+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:22.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:22 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:22.636+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:22.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:22 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:22.636+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:22.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:22 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:22.636+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:22.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:22 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:22.636+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:22.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:22 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:22.636+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:23.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:22 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:22.814+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:23.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:22 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:22.814+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:23.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:22 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:22.814+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:23.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:22 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:22.814+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:23.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:22 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:22.814+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:23.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:22 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:22.814+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:23.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:22 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:22.814+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-05-14T17:17:57.000234+0000 front 2024-05-14T17:17:57.000203+0000 (oldest deadline 2024-05-14T17:18:21.700141+0000) 2024-05-14T17:18:23.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.987+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.301366+0000 front 2024-05-14T17:17:08.301336+0000 (oldest deadline 2024-05-14T17:17:34.201519+0000) 2024-05-14T17:18:23.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.987+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.502633+0000 front 2024-05-14T17:17:16.502884+0000 (oldest deadline 2024-05-14T17:17:41.802850+0000) 2024-05-14T17:18:23.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.987+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.504035+0000 front 2024-05-14T17:17:23.503946+0000 (oldest deadline 2024-05-14T17:17:52.304343+0000) 2024-05-14T17:18:23.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.987+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:32.304691+0000 front 2024-05-14T17:17:32.304785+0000 (oldest deadline 2024-05-14T17:17:56.404882+0000) 2024-05-14T17:18:23.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.987+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:38.706153+0000 front 2024-05-14T17:17:38.706167+0000 (oldest deadline 2024-05-14T17:18:03.406145+0000) 2024-05-14T17:18:23.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:22 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9[63317]: 2024-05-14T17:18:22.987+0000 7f9b5c0a2700 -1 osd.9 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:47.407792+0000 front 2024-05-14T17:17:47.407879+0000 (oldest deadline 2024-05-14T17:18:13.307661+0000) 2024-05-14T17:18:23.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:23 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:23.594+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:23.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:23 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:23.594+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:23.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:23 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:23.594+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:23.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:23 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:23.594+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:23.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:23 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:23.594+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:23.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:23 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:23.594+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:24.153 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:23 smithi069 podman[112300]: 2024-05-14 17:18:23.866020746 +0000 UTC m=+5.651051863 container died 3dd1dc5b487bb4add21b994ea16b8f6292913a9a97489c5ceddde2a40ffc4ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-14T17:18:24.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:23 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:23.782+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:24.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:23 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:23.782+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:24.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:23 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:23.782+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:24.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:23 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:23.782+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:24.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:23 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:23.782+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:24.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:23 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:23.782+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:24.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:23 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:23.782+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-05-14T17:17:57.000234+0000 front 2024-05-14T17:17:57.000203+0000 (oldest deadline 2024-05-14T17:18:21.700141+0000) 2024-05-14T17:18:24.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:24 smithi069 podman[112300]: 2024-05-14 17:18:24.254028174 +0000 UTC m=+6.039059273 container cleanup 3dd1dc5b487bb4add21b994ea16b8f6292913a9a97489c5ceddde2a40ffc4ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-14T17:18:24.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:24 smithi069 bash[112300]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9 2024-05-14T17:18:24.830 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:24 smithi069 podman[112314]: 2024-05-14 17:18:24.547294105 +0000 UTC m=+0.677823439 container remove 3dd1dc5b487bb4add21b994ea16b8f6292913a9a97489c5ceddde2a40ffc4ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-14T17:18:24.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:24 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:24.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:24.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:24 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:24.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:24.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:24 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:24.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:24.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:24 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:24.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:24.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:24 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:24.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:24.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:24 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:24.570+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:25.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:24 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:24.739+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:25.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:24 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:24.739+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:25.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:24 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:24.739+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:25.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:24 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:24.739+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:25.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:24 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:24.739+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:25.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:24 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:24.739+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:25.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:24 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:24.739+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-05-14T17:17:57.000234+0000 front 2024-05-14T17:17:57.000203+0000 (oldest deadline 2024-05-14T17:18:21.700141+0000) 2024-05-14T17:18:25.165 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:24 smithi069 podman[112405]: 2024-05-14 17:18:24.831561812 +0000 UTC m=+0.084808048 container create 467e037f53e6c0561acb4be3df17a779db35b270e9aecfc7f970d2787dbc28af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-14T17:18:25.166 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:24 smithi069 podman[112405]: 2024-05-14 17:18:24.764379562 +0000 UTC m=+0.017625810 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:18:25.166 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:25 smithi069 podman[112405]: 2024-05-14 17:18:25.075401643 +0000 UTC m=+0.328647872 container init 467e037f53e6c0561acb4be3df17a779db35b270e9aecfc7f970d2787dbc28af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-14T17:18:25.166 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:25 smithi069 podman[112405]: 2024-05-14 17:18:25.078579734 +0000 UTC m=+0.331825982 container start 467e037f53e6c0561acb4be3df17a779db35b270e9aecfc7f970d2787dbc28af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-05-14T17:18:25.166 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:25 smithi069 podman[112405]: 2024-05-14 17:18:25.167103084 +0000 UTC m=+0.420349328 container attach 467e037f53e6c0561acb4be3df17a779db35b270e9aecfc7f970d2787dbc28af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:18:25.453 INFO:journalctl@ceph.osd.9.smithi069.stdout:May 14 17:18:25 smithi069 podman[112405]: 2024-05-14 17:18:25.389490181 +0000 UTC m=+0.642736420 container died 467e037f53e6c0561acb4be3df17a779db35b270e9aecfc7f970d2787dbc28af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-14T17:18:25.870 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.9.service' 2024-05-14T17:18:25.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:25.909 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-14T17:18:25.909 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-14T17:18:25.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.10 2024-05-14T17:18:25.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:25 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:25.553+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:25.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:25 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:25.553+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:25.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:25 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:25.553+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:25.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:25 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:25.553+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:25.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:25 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:25.553+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:25.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:25 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:25.553+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:26.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:25 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:25.748+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:26.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:25 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:25.748+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:26.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:25 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:25.748+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:26.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:25 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:25.748+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:26.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:25 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:25.748+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:26.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:25 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:25.748+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:26.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:25 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:25.748+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-05-14T17:17:57.000234+0000 front 2024-05-14T17:17:57.000203+0000 (oldest deadline 2024-05-14T17:18:21.700141+0000) 2024-05-14T17:18:26.001 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:25 smithi092 systemd[1]: Stopping Ceph osd.10 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:26.499 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:26 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:26.210+0000 7f86a228e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:26.500 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:26 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:26.210+0000 7f86a228e700 -1 osd.10 129 *** Got signal Terminated *** 2024-05-14T17:18:26.500 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:26 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:26.210+0000 7f86a228e700 -1 osd.10 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:18:26.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:26 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:26.568+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:26.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:26 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:26.568+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:26.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:26 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:26.568+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:26.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:26 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:26.568+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:26.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:26 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:26.568+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:26.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:26 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:26.568+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:27.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:26 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:26.713+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:27.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:26 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:26.713+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:27.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:26 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:26.713+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:27.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:26 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:26.713+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:27.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:26 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:26.713+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:27.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:26 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:26.713+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:27.000 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:26 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:26.713+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-05-14T17:17:57.000234+0000 front 2024-05-14T17:17:57.000203+0000 (oldest deadline 2024-05-14T17:18:21.700141+0000) 2024-05-14T17:18:27.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:27 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:27.535+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:27.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:27 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:27.535+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:27.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:27 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:27.535+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:27.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:27 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:27.535+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:27.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:27 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:27.535+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:27.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:27 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:27.535+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:28.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:27 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:27.761+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:28.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:27 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:27.761+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:28.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:27 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:27.761+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:28.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:27 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:27.761+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:28.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:27 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:27.761+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:28.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:27 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:27.761+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:28.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:27 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:27.761+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-05-14T17:17:57.000234+0000 front 2024-05-14T17:17:57.000203+0000 (oldest deadline 2024-05-14T17:18:21.700141+0000) 2024-05-14T17:18:28.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:28 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:28.565+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:28.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:28 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:28.565+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:28.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:28 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:28.565+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:28.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:28 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:28.565+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:28.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:28 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:28.565+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:28.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:28 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:28.565+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:29.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:28 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:28.767+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:29.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:28 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:28.767+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:29.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:28 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:28.767+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:29.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:28 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:28.767+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:29.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:28 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:28.767+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:29.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:28 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:28.767+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:29.251 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:28 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:28.767+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-05-14T17:17:57.000234+0000 front 2024-05-14T17:17:57.000203+0000 (oldest deadline 2024-05-14T17:18:21.700141+0000) 2024-05-14T17:18:29.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:29 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:29.599+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:29.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:29 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:29.599+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:29.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:29 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:29.599+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:29.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:29 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:29.599+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:29.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:29 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:29.599+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:29.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:29 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:29.599+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:29.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:29 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:29.599+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-14T17:18:06.847981+0000 front 2024-05-14T17:18:06.848109+0000 (oldest deadline 2024-05-14T17:18:29.147901+0000) 2024-05-14T17:18:30.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:29 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:29.771+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:30.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:29 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:29.771+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:30.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:29 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:29.771+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:30.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:29 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:29.771+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:30.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:29 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:29.771+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:30.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:29 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:29.771+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:30.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:29 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:29.771+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-05-14T17:17:57.000234+0000 front 2024-05-14T17:17:57.000203+0000 (oldest deadline 2024-05-14T17:18:21.700141+0000) 2024-05-14T17:18:30.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:30 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:30.612+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:30.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:30 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:30.612+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:30.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:30 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:30.612+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:30.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:30 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:30.612+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:30.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:30 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:30.612+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:30.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:30 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:30.612+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:30.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:30 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:30.612+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-14T17:18:06.847981+0000 front 2024-05-14T17:18:06.848109+0000 (oldest deadline 2024-05-14T17:18:29.147901+0000) 2024-05-14T17:18:31.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:30 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:30.801+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:08.990995+0000 front 2024-05-14T17:17:08.991153+0000 (oldest deadline 2024-05-14T17:17:30.091200+0000) 2024-05-14T17:18:31.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:30 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:30.801+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:16.794461+0000 front 2024-05-14T17:17:16.794568+0000 (oldest deadline 2024-05-14T17:17:42.094622+0000) 2024-05-14T17:18:31.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:30 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:30.801+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:23.195599+0000 front 2024-05-14T17:17:23.195651+0000 (oldest deadline 2024-05-14T17:17:48.495532+0000) 2024-05-14T17:18:31.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:30 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:30.801+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:28.996633+0000 front 2024-05-14T17:17:28.996665+0000 (oldest deadline 2024-05-14T17:17:56.597211+0000) 2024-05-14T17:18:31.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:30 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:30.801+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:41.198409+0000 front 2024-05-14T17:17:41.198434+0000 (oldest deadline 2024-05-14T17:18:03.498665+0000) 2024-05-14T17:18:31.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:30 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:30.801+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.799361+0000 front 2024-05-14T17:17:48.799411+0000 (oldest deadline 2024-05-14T17:18:14.099417+0000) 2024-05-14T17:18:31.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:30 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10[60832]: 2024-05-14T17:18:30.801+0000 7f869a87a700 -1 osd.10 129 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-05-14T17:17:57.000234+0000 front 2024-05-14T17:17:57.000203+0000 (oldest deadline 2024-05-14T17:18:21.700141+0000) 2024-05-14T17:18:31.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:31 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:31.579+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:31.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:31 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:31.579+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:31.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:31 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:31.579+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:31.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:31 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:31.579+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:31.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:31 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:31.579+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:31.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:31 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:31.579+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:31.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:31 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:31.579+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-14T17:18:06.847981+0000 front 2024-05-14T17:18:06.848109+0000 (oldest deadline 2024-05-14T17:18:29.147901+0000) 2024-05-14T17:18:32.061 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:31 smithi092 podman[78215]: 2024-05-14 17:18:31.796342003 +0000 UTC m=+5.690313193 container died 8ff65bc1ddcda186dc772cede3b040020f311c828165696d166c365556fdf764 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-14T17:18:32.499 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:32 smithi092 podman[78215]: 2024-05-14 17:18:32.186302328 +0000 UTC m=+6.080273515 container cleanup 8ff65bc1ddcda186dc772cede3b040020f311c828165696d166c365556fdf764 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-14T17:18:32.500 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:32 smithi092 bash[78215]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10 2024-05-14T17:18:32.928 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:32 smithi092 podman[78231]: 2024-05-14 17:18:32.554343965 +0000 UTC m=+0.755162128 container remove 8ff65bc1ddcda186dc772cede3b040020f311c828165696d166c365556fdf764 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:18:32.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:32 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:32.605+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:32.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:32 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:32.605+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:32.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:32 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:32.605+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:32.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:32 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:32.605+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:32.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:32 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:32.605+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:32.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:32 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:32.605+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:32.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:32 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:32.605+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-14T17:18:06.847981+0000 front 2024-05-14T17:18:06.848109+0000 (oldest deadline 2024-05-14T17:18:29.147901+0000) 2024-05-14T17:18:33.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:32 smithi092 podman[78323]: 2024-05-14 17:18:32.929767459 +0000 UTC m=+0.086130404 container create 4d201153d667271d9a1c899f3756e3be43260909417a0d2901a68c88fd7b15e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-14T17:18:33.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:32 smithi092 podman[78323]: 2024-05-14 17:18:32.861090452 +0000 UTC m=+0.017453433 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:18:33.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:33 smithi092 podman[78323]: 2024-05-14 17:18:33.173824447 +0000 UTC m=+0.330187402 container init 4d201153d667271d9a1c899f3756e3be43260909417a0d2901a68c88fd7b15e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-14T17:18:33.250 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:33 smithi092 podman[78323]: 2024-05-14 17:18:33.176899192 +0000 UTC m=+0.333262148 container start 4d201153d667271d9a1c899f3756e3be43260909417a0d2901a68c88fd7b15e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:18:33.750 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:33 smithi092 podman[78323]: 2024-05-14 17:18:33.257028505 +0000 UTC m=+0.413391460 container attach 4d201153d667271d9a1c899f3756e3be43260909417a0d2901a68c88fd7b15e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux ) 2024-05-14T17:18:33.750 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 14 17:18:33 smithi092 podman[78323]: 2024-05-14 17:18:33.475230108 +0000 UTC m=+0.631593067 container died 4d201153d667271d9a1c899f3756e3be43260909417a0d2901a68c88fd7b15e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-14T17:18:33.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:33 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:33.571+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:33.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:33 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:33.571+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:33.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:33 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:33.571+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:33.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:33 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:33.571+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:33.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:33 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:33.571+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:33.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:33 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:33.571+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:33.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:33 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:33.571+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-14T17:18:06.847981+0000 front 2024-05-14T17:18:06.848109+0000 (oldest deadline 2024-05-14T17:18:29.147901+0000) 2024-05-14T17:18:34.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.10.service' 2024-05-14T17:18:34.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:34.097 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-14T17:18:34.098 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-14T17:18:34.098 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.11 2024-05-14T17:18:34.401 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 systemd[1]: Stopping Ceph osd.11 for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:34.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:34.401+0000 7efdb3110700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:34.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:34.401+0000 7efdb3110700 -1 osd.11 129 *** Got signal Terminated *** 2024-05-14T17:18:34.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:34.401+0000 7efdb3110700 -1 osd.11 129 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:18:34.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:34.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:34.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:34.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:34.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:34.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:34.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:34.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:34.709 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:34.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:34.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:34.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:34.710 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:34 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:34.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-14T17:18:06.847981+0000 front 2024-05-14T17:18:06.848109+0000 (oldest deadline 2024-05-14T17:18:29.147901+0000) 2024-05-14T17:18:35.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:35 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:35.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:35.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:35 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:35.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:35.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:35 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:35.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:35.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:35 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:35.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:35.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:35 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:35.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:35.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:35 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:35.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:35.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:35 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:35.540+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-14T17:18:06.847981+0000 front 2024-05-14T17:18:06.848109+0000 (oldest deadline 2024-05-14T17:18:29.147901+0000) 2024-05-14T17:18:36.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:36 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:36.550+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:36.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:36 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:36.550+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:36.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:36 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:36.550+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:36.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:36 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:36.550+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:36.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:36 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:36.550+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:36.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:36 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:36.550+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:36.960 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:36 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:36.550+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-14T17:18:06.847981+0000 front 2024-05-14T17:18:06.848109+0000 (oldest deadline 2024-05-14T17:18:29.147901+0000) 2024-05-14T17:18:37.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:37 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:37.505+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:37.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:37 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:37.505+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:37.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:37 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:37.505+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:37.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:37 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:37.505+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:37.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:37 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:37.505+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:37.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:37 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:37.505+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:37.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:37 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:37.505+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-14T17:18:06.847981+0000 front 2024-05-14T17:18:06.848109+0000 (oldest deadline 2024-05-14T17:18:29.147901+0000) 2024-05-14T17:18:38.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:38.489+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-05-14T17:17:05.939582+0000 front 2024-05-14T17:17:05.939343+0000 (oldest deadline 2024-05-14T17:17:33.539957+0000) 2024-05-14T17:18:38.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:38.489+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-14T17:17:17.641217+0000 front 2024-05-14T17:17:17.641035+0000 (oldest deadline 2024-05-14T17:17:41.140975+0000) 2024-05-14T17:18:38.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:38.489+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-05-14T17:17:21.141252+0000 front 2024-05-14T17:17:21.141507+0000 (oldest deadline 2024-05-14T17:17:49.342042+0000) 2024-05-14T17:18:38.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:38.489+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-05-14T17:17:34.043239+0000 front 2024-05-14T17:17:34.043282+0000 (oldest deadline 2024-05-14T17:17:55.743344+0000) 2024-05-14T17:18:38.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:38.489+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-14T17:17:39.244557+0000 front 2024-05-14T17:17:39.244605+0000 (oldest deadline 2024-05-14T17:18:05.144408+0000) 2024-05-14T17:18:38.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:38.489+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-05-14T17:17:48.645590+0000 front 2024-05-14T17:17:48.645648+0000 (oldest deadline 2024-05-14T17:18:12.745701+0000) 2024-05-14T17:18:38.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:38.489+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-14T17:18:06.847981+0000 front 2024-05-14T17:18:06.848109+0000 (oldest deadline 2024-05-14T17:18:29.147901+0000) 2024-05-14T17:18:38.959 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:38 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11[61176]: 2024-05-14T17:18:38.489+0000 7efdaf704700 -1 osd.11 129 heartbeat_check: no reply from 172.21.15.203:6820 osd.8 since back 2024-05-14T17:18:14.249824+0000 front 2024-05-14T17:18:14.249855+0000 (oldest deadline 2024-05-14T17:18:37.749769+0000) 2024-05-14T17:18:40.167 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:39 smithi203 podman[79990]: 2024-05-14 17:18:39.91105076 +0000 UTC m=+5.613573834 container died c7e4472d8467021f48f7e98fa0b82cebb47b114d9f16be83e169ffd2c2b46553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-14T17:18:40.459 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:40 smithi203 podman[79990]: 2024-05-14 17:18:40.259842181 +0000 UTC m=+5.962365252 container cleanup c7e4472d8467021f48f7e98fa0b82cebb47b114d9f16be83e169ffd2c2b46553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-05-14T17:18:40.459 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:40 smithi203 bash[79990]: ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11 2024-05-14T17:18:40.877 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:40 smithi203 podman[80009]: 2024-05-14 17:18:40.544461845 +0000 UTC m=+0.633354099 container remove c7e4472d8467021f48f7e98fa0b82cebb47b114d9f16be83e169ffd2c2b46553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux ) 2024-05-14T17:18:41.180 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:40 smithi203 podman[80095]: 2024-05-14 17:18:40.780848944 +0000 UTC m=+0.017183762 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:18:41.180 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:40 smithi203 podman[80095]: 2024-05-14 17:18:40.895265613 +0000 UTC m=+0.131600425 container create 116f0a02436479e29dfa72e6382045852829b6d19a456eb3fdb23b7d788eb4f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:18:41.180 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:41 smithi203 podman[80095]: 2024-05-14 17:18:41.106395919 +0000 UTC m=+0.342730732 container init 116f0a02436479e29dfa72e6382045852829b6d19a456eb3fdb23b7d788eb4f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:18:41.180 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:41 smithi203 podman[80095]: 2024-05-14 17:18:41.109561242 +0000 UTC m=+0.345896060 container start 116f0a02436479e29dfa72e6382045852829b6d19a456eb3fdb23b7d788eb4f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514) 2024-05-14T17:18:41.180 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:41 smithi203 podman[80095]: 2024-05-14 17:18:41.181158724 +0000 UTC m=+0.417493539 container attach 116f0a02436479e29dfa72e6382045852829b6d19a456eb3fdb23b7d788eb4f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-14T17:18:41.459 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:41 smithi203 podman[80095]: 2024-05-14 17:18:41.405417369 +0000 UTC m=+0.641752185 container died 116f0a02436479e29dfa72e6382045852829b6d19a456eb3fdb23b7d788eb4f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, ceph=True) 2024-05-14T17:18:41.924 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.11.service' 2024-05-14T17:18:41.946 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:41 smithi203 podman[80121]: 2024-05-14 17:18:41.901234322 +0000 UTC m=+0.490484591 container remove 116f0a02436479e29dfa72e6382045852829b6d19a456eb3fdb23b7d788eb4f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:18:41.946 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:41 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.11.service: Deactivated successfully. 2024-05-14T17:18:41.947 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:41 smithi203 systemd[1]: Stopped Ceph osd.11 for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:18:41.947 INFO:journalctl@ceph.osd.11.smithi203.stdout:May 14 17:18:41 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@osd.11.service: Consumed 5min 24.829s CPU time. 2024-05-14T17:18:41.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:41.962 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-14T17:18:41.962 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-14T17:18:41.963 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.a 2024-05-14T17:18:42.263 INFO:journalctl@ceph.mds.a.smithi069.stdout:May 14 17:18:42 smithi069 systemd[1]: Stopping Ceph mds.a for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:42.605 INFO:journalctl@ceph.mds.a.smithi069.stdout:May 14 17:18:42 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-a[70134]: 2024-05-14T17:18:42.263+0000 7f84e45c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:42.605 INFO:journalctl@ceph.mds.a.smithi069.stdout:May 14 17:18:42 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-a[70134]: 2024-05-14T17:18:42.263+0000 7f84e45c7700 -1 mds.a *** got signal Terminated *** 2024-05-14T17:18:46.203 INFO:journalctl@ceph.mds.a.smithi069.stdout:May 14 17:18:45 smithi069 podman[112509]: 2024-05-14 17:18:45.918655277 +0000 UTC m=+3.763921043 container died 96a8766521a5184723704bd3005c153fb698e85edbd3a36deb86719c892f4d4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-14T17:18:46.574 INFO:journalctl@ceph.mds.a.smithi069.stdout:May 14 17:18:46 smithi069 podman[112509]: 2024-05-14 17:18:46.312984048 +0000 UTC m=+4.158249803 container cleanup 96a8766521a5184723704bd3005c153fb698e85edbd3a36deb86719c892f4d4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:18:46.574 INFO:journalctl@ceph.mds.a.smithi069.stdout:May 14 17:18:46 smithi069 bash[112509]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-a 2024-05-14T17:18:46.715 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.a.service' 2024-05-14T17:18:46.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:46.751 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-14T17:18:46.751 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-14T17:18:46.751 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.d 2024-05-14T17:18:47.203 INFO:journalctl@ceph.mds.d.smithi069.stdout:May 14 17:18:46 smithi069 systemd[1]: Stopping Ceph mds.d for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:47.203 INFO:journalctl@ceph.mds.d.smithi069.stdout:May 14 17:18:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-d[68883]: 2024-05-14T17:18:47.083+0000 7f8baa37e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:47.203 INFO:journalctl@ceph.mds.d.smithi069.stdout:May 14 17:18:47 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-d[68883]: 2024-05-14T17:18:47.083+0000 7f8baa37e700 -1 mds.d *** got signal Terminated *** 2024-05-14T17:18:48.841 INFO:journalctl@ceph.mds.d.smithi069.stdout:May 14 17:18:48 smithi069 podman[112641]: 2024-05-14 17:18:48.557853115 +0000 UTC m=+1.580051171 container died e8b2847e04895ebc5cfb1ab6d0b23f116ccc2542cd26c2ee670cf9719a4e53e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-14T17:18:49.203 INFO:journalctl@ceph.mds.d.smithi069.stdout:May 14 17:18:48 smithi069 podman[112641]: 2024-05-14 17:18:48.942020841 +0000 UTC m=+1.964218890 container cleanup e8b2847e04895ebc5cfb1ab6d0b23f116ccc2542cd26c2ee670cf9719a4e53e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-d, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-14T17:18:49.203 INFO:journalctl@ceph.mds.d.smithi069.stdout:May 14 17:18:48 smithi069 bash[112641]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-d 2024-05-14T17:18:49.372 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.d.service' 2024-05-14T17:18:49.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:49.407 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-14T17:18:49.407 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-14T17:18:49.407 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.g 2024-05-14T17:18:49.720 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 17:18:49 smithi069 systemd[1]: Stopping Ceph mds.g for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:50.203 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 17:18:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-g[69811]: 2024-05-14T17:18:49.720+0000 7fc8fef5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:50.203 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 17:18:49 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-g[69811]: 2024-05-14T17:18:49.720+0000 7fc8fef5d700 -1 mds.g *** got signal Terminated *** 2024-05-14T17:18:51.378 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 17:18:51 smithi069 podman[112764]: 2024-05-14 17:18:51.093405251 +0000 UTC m=+1.468070899 container died 5a6ffcf4a6db964709fbc6d6943b02105aeaba74252756101de7afb8efc008f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-14T17:18:51.703 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 17:18:51 smithi069 podman[112764]: 2024-05-14 17:18:51.454455773 +0000 UTC m=+1.829121420 container cleanup 5a6ffcf4a6db964709fbc6d6943b02105aeaba74252756101de7afb8efc008f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-14T17:18:51.703 INFO:journalctl@ceph.mds.g.smithi069.stdout:May 14 17:18:51 smithi069 bash[112764]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-g 2024-05-14T17:18:51.884 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.g.service' 2024-05-14T17:18:51.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:51.917 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-14T17:18:51.917 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-14T17:18:51.917 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.j 2024-05-14T17:18:52.240 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 17:18:51 smithi069 systemd[1]: Stopping Ceph mds.j for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:52.703 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 17:18:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-j[68434]: 2024-05-14T17:18:52.241+0000 7fee29088700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:52.703 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 17:18:52 smithi069 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-j[68434]: 2024-05-14T17:18:52.241+0000 7fee29088700 -1 mds.j *** got signal Terminated *** 2024-05-14T17:18:55.949 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 17:18:55 smithi069 podman[112884]: 2024-05-14 17:18:55.684938644 +0000 UTC m=+3.551052007 container died 3d50ce5a9d6e5b4f0b1d2be68ebd98984d8c41fab2b70902bfc992bd9c703d15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514) 2024-05-14T17:18:56.203 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 17:18:56 smithi069 podman[112884]: 2024-05-14 17:18:56.049921241 +0000 UTC m=+3.916034603 container cleanup 3d50ce5a9d6e5b4f0b1d2be68ebd98984d8c41fab2b70902bfc992bd9c703d15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-j, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-14T17:18:56.203 INFO:journalctl@ceph.mds.j.smithi069.stdout:May 14 17:18:56 smithi069 bash[112884]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-j 2024-05-14T17:18:56.432 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.j.service' 2024-05-14T17:18:56.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:56.464 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-14T17:18:56.465 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-14T17:18:56.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.b 2024-05-14T17:18:56.750 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 17:18:56 smithi092 systemd[1]: Stopping Ceph mds.b for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:57.019 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 17:18:56 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b[66051]: 2024-05-14T17:18:56.766+0000 7f5496ba2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:57.019 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 17:18:56 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b[66051]: 2024-05-14T17:18:56.766+0000 7f5496ba2700 -1 mds.b *** got signal Terminated *** 2024-05-14T17:19:01.050 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 17:19:00 smithi092 podman[78430]: 2024-05-14 17:19:00.782727657 +0000 UTC m=+4.110608178 container died 9aa688b58ad46af9ba55a4c8ee64015abcb49777e1d6681b3c33e3a266fdbb5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:01.366 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 17:19:01 smithi092 podman[78430]: 2024-05-14 17:19:01.133632157 +0000 UTC m=+4.461512670 container cleanup 9aa688b58ad46af9ba55a4c8ee64015abcb49777e1d6681b3c33e3a266fdbb5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:01.366 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 14 17:19:01 smithi092 bash[78430]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-b 2024-05-14T17:19:01.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.b.service' 2024-05-14T17:19:01.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:01.552 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-14T17:19:01.552 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-14T17:19:01.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.e 2024-05-14T17:19:01.999 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 17:19:01 smithi092 systemd[1]: Stopping Ceph mds.e for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:02.000 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 17:19:01 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e[65431]: 2024-05-14T17:19:01.878+0000 7f60c3839700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:02.000 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 17:19:01 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e[65431]: 2024-05-14T17:19:01.878+0000 7f60c3839700 -1 mds.e *** got signal Terminated *** 2024-05-14T17:19:05.249 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 17:19:05 smithi092 podman[78563]: 2024-05-14 17:19:05.001000967 +0000 UTC m=+3.229890905 container died a61898b8a3a9ee2d96f214dc23c5677c8f83df716d4d84bce35278b79c1a4e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-14T17:19:05.637 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 17:19:05 smithi092 podman[78563]: 2024-05-14 17:19:05.353560637 +0000 UTC m=+3.582450568 container cleanup a61898b8a3a9ee2d96f214dc23c5677c8f83df716d4d84bce35278b79c1a4e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:05.637 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 14 17:19:05 smithi092 bash[78563]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-e 2024-05-14T17:19:05.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.e.service' 2024-05-14T17:19:05.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:05.813 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-14T17:19:05.813 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-14T17:19:05.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.h 2024-05-14T17:19:06.249 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 17:19:05 smithi092 systemd[1]: Stopping Ceph mds.h for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:06.250 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 17:19:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h[65746]: 2024-05-14T17:19:06.122+0000 7fba5ef2b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:06.250 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 17:19:06 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h[65746]: 2024-05-14T17:19:06.122+0000 7fba5ef2b700 -1 mds.h *** got signal Terminated *** 2024-05-14T17:19:09.298 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 17:19:09 smithi092 podman[78686]: 2024-05-14 17:19:09.012889579 +0000 UTC m=+2.982107784 container died 545cf6bb8536cc64134fa7e309609785e88d53f43fc2a4eba4d4004c6540c9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-14T17:19:09.640 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 17:19:09 smithi092 podman[78686]: 2024-05-14 17:19:09.390477672 +0000 UTC m=+3.359695871 container cleanup 545cf6bb8536cc64134fa7e309609785e88d53f43fc2a4eba4d4004c6540c9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:09.640 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 14 17:19:09 smithi092 bash[78686]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-h 2024-05-14T17:19:09.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.h.service' 2024-05-14T17:19:09.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:09.804 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-14T17:19:09.805 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-14T17:19:09.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.k 2024-05-14T17:19:10.249 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 17:19:09 smithi092 systemd[1]: Stopping Ceph mds.k for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:10.250 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 17:19:10 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-k[65114]: 2024-05-14T17:19:10.126+0000 7f0692663700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:10.250 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 17:19:10 smithi092 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-k[65114]: 2024-05-14T17:19:10.126+0000 7f0692663700 -1 mds.k *** got signal Terminated *** 2024-05-14T17:19:14.409 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 17:19:14 smithi092 podman[78808]: 2024-05-14 17:19:14.139014186 +0000 UTC m=+4.105612057 container died 5ae9d798f7632a29ab96bf55e30d52af2c5242ca5dd1177ce3be26fa72cf358d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514) 2024-05-14T17:19:14.749 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 17:19:14 smithi092 podman[78808]: 2024-05-14 17:19:14.509787771 +0000 UTC m=+4.476385648 container cleanup 5ae9d798f7632a29ab96bf55e30d52af2c5242ca5dd1177ce3be26fa72cf358d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-k, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True) 2024-05-14T17:19:14.750 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 14 17:19:14 smithi092 bash[78808]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-k 2024-05-14T17:19:14.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.k.service' 2024-05-14T17:19:14.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:14.965 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-14T17:19:14.965 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-14T17:19:14.965 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c 2024-05-14T17:19:15.001 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.c.service' 2024-05-14T17:19:15.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:15.077 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-14T17:19:15.078 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-14T17:19:15.078 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.f 2024-05-14T17:19:15.399 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 17:19:15 smithi203 systemd[1]: Stopping Ceph mds.f for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:15.669 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 17:19:15 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f[65972]: 2024-05-14T17:19:15.399+0000 7f8875230700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:15.669 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 17:19:15 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f[65972]: 2024-05-14T17:19:15.399+0000 7f8875230700 -1 mds.f *** got signal Terminated *** 2024-05-14T17:19:17.432 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 17:19:17 smithi203 podman[80236]: 2024-05-14 17:19:17.018556946 +0000 UTC m=+1.710761726 container died 46634c2ae77ddef549b3f4a2ef519e9d187fb6dcb350741ad9653f387cd838a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-14T17:19:17.709 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 17:19:17 smithi203 podman[80236]: 2024-05-14 17:19:17.524983632 +0000 UTC m=+2.217188412 container cleanup 46634c2ae77ddef549b3f4a2ef519e9d187fb6dcb350741ad9653f387cd838a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:19:17.709 INFO:journalctl@ceph.mds.f.smithi203.stdout:May 14 17:19:17 smithi203 bash[80236]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-f 2024-05-14T17:19:17.878 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.f.service' 2024-05-14T17:19:17.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:17.912 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-14T17:19:17.912 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-14T17:19:17.912 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.i 2024-05-14T17:19:18.459 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 17:19:18 smithi203 systemd[1]: Stopping Ceph mds.i for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:18.459 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 17:19:18 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i[66586]: 2024-05-14T17:19:18.244+0000 7f9ac6aa6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:18.459 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 17:19:18 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i[66586]: 2024-05-14T17:19:18.244+0000 7f9ac6aa6700 -1 mds.i *** got signal Terminated *** 2024-05-14T17:19:21.335 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 17:19:21 smithi203 podman[80369]: 2024-05-14 17:19:21.013281995 +0000 UTC m=+2.871188274 container died 330076950df914426aa094ec224c454fad30ebc40b6cfdcc3867666c09d0028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:19:21.335 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 17:19:21 smithi203 podman[80369]: 2024-05-14 17:19:21.336907323 +0000 UTC m=+3.194813586 container cleanup 330076950df914426aa094ec224c454fad30ebc40b6cfdcc3867666c09d0028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-14T17:19:21.709 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 17:19:21 smithi203 bash[80369]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i 2024-05-14T17:19:21.709 INFO:journalctl@ceph.mds.i.smithi203.stdout:May 14 17:19:21 smithi203 podman[80382]: 2024-05-14 17:19:21.521306417 +0000 UTC m=+0.503151316 container remove 330076950df914426aa094ec224c454fad30ebc40b6cfdcc3867666c09d0028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:19:21.765 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.i.service' 2024-05-14T17:19:21.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:21.797 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-14T17:19:21.798 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-14T17:19:21.798 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.l 2024-05-14T17:19:22.098 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:19:21 smithi203 systemd[1]: Stopping Ceph mds.l for 356e185e-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:22.458 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:19:22 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-l[65668]: 2024-05-14T17:19:22.097+0000 7fa1d314d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:22.459 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:19:22 smithi203 ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-l[65668]: 2024-05-14T17:19:22.097+0000 7fa1d314d700 -1 mds.l *** got signal Terminated *** 2024-05-14T17:19:25.273 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:19:25 smithi203 podman[80488]: 2024-05-14 17:19:25.016554289 +0000 UTC m=+3.001394858 container died 1986f28d5d06384d8b2d649cfe18e39ac02cfca1aa18207a7264bbd038c773c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:25.524 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:19:25 smithi203 podman[80488]: 2024-05-14 17:19:25.340777375 +0000 UTC m=+3.325617938 container cleanup 1986f28d5d06384d8b2d649cfe18e39ac02cfca1aa18207a7264bbd038c773c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD) 2024-05-14T17:19:25.525 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:19:25 smithi203 bash[80488]: ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-l 2024-05-14T17:19:25.768 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.l.service' 2024-05-14T17:19:25.788 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:19:25 smithi203 podman[80502]: 2024-05-14 17:19:25.524980582 +0000 UTC m=+0.508463810 container remove 1986f28d5d06384d8b2d649cfe18e39ac02cfca1aa18207a7264bbd038c773c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-356e185e-1210-11ef-bc98-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T17:19:25.788 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:19:25 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.l.service: Deactivated successfully. 2024-05-14T17:19:25.788 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:19:25 smithi203 systemd[1]: Stopped Ceph mds.l for 356e185e-1210-11ef-bc98-c7b262605968. 2024-05-14T17:19:25.788 INFO:journalctl@ceph.mds.l.smithi203.stdout:May 14 17:19:25 smithi203 systemd[1]: ceph-356e185e-1210-11ef-bc98-c7b262605968@mds.l.service: Consumed 2.112s CPU time. 2024-05-14T17:19:25.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:25.799 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-14T17:19:25.799 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 356e185e-1210-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T17:19:25.990 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T17:19:28.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 356e185e-1210-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T17:19:28.577 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T17:19:31.189 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 356e185e-1210-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T17:19:31.377 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T17:19:33.693 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:19:33.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:19:33.757 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:19:33.782 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-14T17:19:33.784 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174/remote/smithi069/crash 2024-05-14T17:19:33.785 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/crash -- . 2024-05-14T17:19:33.828 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T17:19:33.828 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2024-05-14T17:19:33.830 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174/remote/smithi092/crash 2024-05-14T17:19:33.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/crash -- . 2024-05-14T17:19:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T17:19:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-05-14T17:19:33.872 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174/remote/smithi203/crash 2024-05-14T17:19:33.872 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/crash -- . 2024-05-14T17:19:33.906 INFO:teuthology.orchestra.run.smithi203.stderr:tar: /var/lib/ceph/356e185e-1210-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T17:19:33.906 INFO:teuthology.orchestra.run.smithi203.stderr:tar: Error is not recoverable: exiting now 2024-05-14T17:19:33.907 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-14T17:19:33.908 DEBUG:teuthology.orchestra.run.smithi069:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-14T17:19:33.945 INFO:teuthology.orchestra.run.smithi069.stdout:2024-05-14T17:10:29.924009+0000 mon.a (mon.0) 1439 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-14T17:19:33.945 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-14T17:19:33.945 INFO:tasks.cephadm:Compressing logs... 2024-05-14T17:19:33.946 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T17:19:33.988 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T17:19:33.990 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T17:19:34.013 INFO:teuthology.orchestra.run.smithi203.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-05-14T17:19:34.014 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-05-14T17:19:34.014 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.log 2024-05-14T17:19:34.014 INFO:teuthology.orchestra.run.smithi203.stderr:gzip/var/log/ceph/cephadm.log: -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-14T17:19:34.014 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-14T17:19:34.014 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T17:19:34.014 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-14T17:19:34.020 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-14T17:19:34.020 INFO:teuthology.orchestra.run.smithi203.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T17:19:34.020 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-14T17:19:34.020 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-14T17:19:34.020 INFO:teuthology.orchestra.run.smithi203.stderr:gzip/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.c.log: -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-14T17:19:34.020 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T17:19:34.020 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mon.c.log: /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.i.log: 90.6% 87.7% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T17:19:34.020 INFO:teuthology.orchestra.run.smithi203.stderr: -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T17:19:34.021 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T17:19:34.021 INFO:teuthology.orchestra.run.smithi069.stderr:: No such file or directory 2024-05-14T17:19:34.021 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.log 2024-05-14T17:19:34.021 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T17:19:34.022 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-14T17:19:34.022 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-14T17:19:34.022 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-14T17:19:34.022 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-14T17:19:34.022 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-14T17:19:34.022 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-14T17:19:34.022 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mgr.x.log: 90.4% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T17:19:34.022 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.6.log 87.7% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T17:19:34.022 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-05-14T17:19:34.022 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T17:19:34.028 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.6.log: 94.6% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-14T17:19:34.028 INFO:teuthology.orchestra.run.smithi069.stderr: 94.6% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-14T17:19:34.028 INFO:teuthology.orchestra.run.smithi069.stderr: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T17:19:34.028 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-14T17:19:34.028 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-volume.log 84.5% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T17:19:34.028 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-05-14T17:19:34.028 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-14T17:19:34.028 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-14T17:19:34.030 INFO:teuthology.orchestra.run.smithi203.stderr: 92.7% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-14T17:19:34.031 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:: No such file or directory 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.b.log: /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.h.log: 90.6% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T17:19:34.032 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.10.log: 87.7% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T17:19:34.033 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-14T17:19:34.033 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.cephadm.log: 84.4% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T17:19:34.033 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T17:19:34.033 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.e.log: 92.6% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-14T17:19:34.033 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-14T17:19:34.034 INFO:teuthology.orchestra.run.smithi069.stderr: 94.6% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-14T17:19:34.039 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-14T17:19:34.039 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-14T17:19:34.039 INFO:teuthology.orchestra.run.smithi203.stderr: 84.3% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T17:19:34.039 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T17:19:34.040 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.9.log: 91.1% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T17:19:34.041 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-14T17:19:34.051 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-14T17:19:34.053 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-14T17:19:34.056 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T17:19:34.079 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-volume.log: /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.l.log: 91.1% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T17:19:34.082 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-14T17:19:34.189 INFO:teuthology.orchestra.run.smithi203.stderr: 93.6% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-14T17:19:34.305 INFO:teuthology.orchestra.run.smithi092.stderr: 93.9% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-14T17:19:34.370 INFO:teuthology.orchestra.run.smithi203.stderr: 94.1% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-14T17:19:34.561 INFO:teuthology.orchestra.run.smithi203.stderr: 93.6% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-14T17:19:34.620 INFO:teuthology.orchestra.run.smithi069.stderr: 90.4% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-14T17:19:34.644 INFO:teuthology.orchestra.run.smithi203.stderr: 91.9% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-14T17:19:34.735 INFO:teuthology.orchestra.run.smithi092.stderr: 91.8% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-14T17:19:35.584 INFO:teuthology.orchestra.run.smithi069.stderr: 90.1% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-14T17:19:36.920 INFO:teuthology.orchestra.run.smithi203.stderr: 94.1% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-14T17:19:37.434 INFO:teuthology.orchestra.run.smithi092.stderr: 95.6% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-14T17:19:39.029 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-14T17:20:05.099 INFO:teuthology.orchestra.run.smithi069.stderr: 91.5% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-14T17:20:13.146 INFO:teuthology.orchestra.run.smithi203.stderr: 91.5% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-14T17:20:14.739 INFO:teuthology.orchestra.run.smithi092.stderr: 91.7% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-14T17:20:15.961 INFO:teuthology.orchestra.run.smithi069.stderr: 91.3% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-14T17:20:16.326 INFO:teuthology.orchestra.run.smithi203.stderr: 91.5% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-14T17:20:18.788 INFO:teuthology.orchestra.run.smithi203.stderr: 91.3% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-14T17:20:19.431 INFO:teuthology.orchestra.run.smithi203.stderr: 91.4% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-14T17:20:19.431 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-05-14T17:20:19.432 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m45.428s 2024-05-14T17:20:19.432 INFO:teuthology.orchestra.run.smithi203.stderr:user 2m49.979s 2024-05-14T17:20:19.433 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m5.409s 2024-05-14T17:20:21.717 INFO:teuthology.orchestra.run.smithi092.stderr: 91.4% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-14T17:20:26.901 INFO:teuthology.orchestra.run.smithi092.stderr: 91.3% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-14T17:20:27.341 INFO:teuthology.orchestra.run.smithi069.stderr: 91.2% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-14T17:20:30.475 INFO:teuthology.orchestra.run.smithi092.stderr: 91.3% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-14T17:20:30.477 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-05-14T17:20:30.477 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m56.472s 2024-05-14T17:20:30.477 INFO:teuthology.orchestra.run.smithi092.stderr:user 3m20.698s 2024-05-14T17:20:30.477 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m6.067s 2024-05-14T17:20:32.341 INFO:teuthology.orchestra.run.smithi069.stderr: 91.1% -- replaced with /var/log/ceph/356e185e-1210-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-14T17:20:32.343 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-05-14T17:20:32.343 INFO:teuthology.orchestra.run.smithi069.stderr:real 0m58.341s 2024-05-14T17:20:32.343 INFO:teuthology.orchestra.run.smithi069.stderr:user 3m0.764s 2024-05-14T17:20:32.343 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m5.701s 2024-05-14T17:20:32.343 INFO:tasks.cephadm:Archiving logs... 2024-05-14T17:20:32.344 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174/remote/smithi069/log 2024-05-14T17:20:32.346 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T17:21:04.807 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174/remote/smithi092/log 2024-05-14T17:21:04.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T17:21:39.026 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174/remote/smithi203/log 2024-05-14T17:21:39.027 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T17:22:08.299 INFO:tasks.cephadm:Removing cluster... 2024-05-14T17:22:08.299 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 356e185e-1210-11ef-bc98-c7b262605968 --force 2024-05-14T17:22:08.490 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T17:22:09.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 356e185e-1210-11ef-bc98-c7b262605968 --force 2024-05-14T17:22:09.219 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T17:22:09.750 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 356e185e-1210-11ef-bc98-c7b262605968 --force 2024-05-14T17:22:09.943 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 356e185e-1210-11ef-bc98-c7b262605968 2024-05-14T17:22:10.449 INFO:tasks.cephadm:Removing cephadm ... 2024-05-14T17:22:10.449 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T17:22:10.469 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T17:22:10.486 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T17:22:10.501 INFO:tasks.cephadm:Teardown complete 2024-05-14T17:22:10.501 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-14T17:22:10.512 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-14T17:22:10.512 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T17:22:10.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T17:22:10.528 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T17:22:10.631 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-14T17:22:10.641 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-14T17:22:10.641 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T17:22:10.644 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T17:22:10.646 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T17:22:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-05-14T17:22:10.658 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-05-14T17:22:10.659 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-05-14T17:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T17:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-05-14T17:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 +691us[ +687us] +/- 45ms 2024-05-14T17:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -1058us[-1062us] +/- 78ms 2024-05-14T17:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -230us[ -234us] +/- 69ms 2024-05-14T17:22:10.663 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T17:22:10.665 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T17:22:10.665 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-05-14T17:22:10.665 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +683us[ +682us] +/- 45ms 2024-05-14T17:22:10.665 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -1064us[-1065us] +/- 78ms 2024-05-14T17:22:10.665 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -243us[ -243us] +/- 69ms 2024-05-14T17:22:10.665 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T17:22:10.665 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T17:22:10.665 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-05-14T17:22:10.665 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +694us[ +694us] +/- 45ms 2024-05-14T17:22:10.665 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -1069us[-1068us] +/- 78ms 2024-05-14T17:22:10.666 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -229us[ -228us] +/- 69ms 2024-05-14T17:22:10.666 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T17:22:10.666 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-14T17:22:10.676 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-14T17:22:10.677 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-14T17:22:10.687 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T17:22:10.790 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T17:22:10.859 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T17:22:10.925 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T17:22:10.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:22:10.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T17:22:10.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:22:10.982 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T17:22:11.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:22:11.008 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-14T17:22:11.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-14T17:22:11.029 INFO:teuthology.task.internal:Duration was 3124.735104 seconds 2024-05-14T17:22:11.029 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-14T17:22:11.041 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-14T17:22:11.041 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T17:22:11.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T17:22:11.046 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T17:22:11.081 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T17:22:11.081 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T17:22:11.085 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T17:22:11.533 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-14T17:22:11.534 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2024-05-14T17:22:11.534 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T17:22:11.555 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-05-14T17:22:11.555 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T17:22:11.595 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-05-14T17:22:11.596 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T17:22:11.623 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-14T17:22:11.624 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T17:22:11.626 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T17:22:11.638 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T17:22:11.689 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-14T17:22:11.689 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T17:22:11.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T17:22:11.704 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T17:22:12.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-14T17:22:12.439 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-14T17:22:12.439 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T17:22:12.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T17:22:12.488 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T17:22:12.514 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-14T17:22:12.527 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:22:12.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:22:12.533 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:22:12.550 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2024-05-14T17:22:12.552 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-05-14T17:22:12.577 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-05-14T17:22:12.601 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:22:12.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:22:12.618 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:22:12.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:22:12.649 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:22:12.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:22:12.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-14T17:22:12.674 INFO:teuthology.task.internal:Transferring archived files... 2024-05-14T17:22:12.675 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174/remote/smithi069 2024-05-14T17:22:12.675 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T17:22:12.730 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174/remote/smithi092 2024-05-14T17:22:12.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T17:22:12.765 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174/remote/smithi203 2024-05-14T17:22:12.765 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T17:22:12.801 INFO:teuthology.task.internal:Removing archive directory... 2024-05-14T17:22:12.801 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T17:22:12.803 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T17:22:12.806 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T17:22:12.849 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-14T17:22:12.863 INFO:teuthology.task.internal:Not uploading archives. 2024-05-14T17:22:12.863 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-14T17:22:12.897 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-14T17:22:12.897 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T17:22:12.900 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T17:22:12.902 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T17:22:12.912 INFO:teuthology.orchestra.run.smithi069.stdout: 265307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 17:22 /home/ubuntu/cephtest 2024-05-14T17:22:12.914 INFO:teuthology.orchestra.run.smithi092.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 17:22 /home/ubuntu/cephtest 2024-05-14T17:22:12.917 INFO:teuthology.orchestra.run.smithi203.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 17:22 /home/ubuntu/cephtest 2024-05-14T17:22:12.918 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-14T17:22:12.929 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-14T17:22:12.955 INFO:teuthology.nuke:Checking targets against current locks 2024-05-14T17:22:12.980 DEBUG:teuthology.nuke:shortname: smithi069 2024-05-14T17:22:12.980 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T17:22:13.006 DEBUG:teuthology.nuke:shortname: smithi092 2024-05-14T17:22:13.006 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T17:22:13.011 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:22:02.942460', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJ90eUpCJ6wuZXp6MAOhv+w6thOMP7d5tfpHr2kJOllAejmcDjei6RR6cAWZtok72fh5dzotJjz18ss+bKNHaI='} 2024-05-14T17:22:13.026 DEBUG:teuthology.nuke:shortname: smithi203 2024-05-14T17:22:13.026 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T17:22:13.035 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:22:02.944237', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHiCDolLsnnAyQTGc2/+VkJf/cpW8ZyjHTchzfW7kEwYOZuaDq+ygUkRBxfqYoahbclH03H9cNOLguNfjx4NiMk='} 2024-05-14T17:22:13.049 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:22:02.945299', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZnvpHt//FMVuNIYvC3cHDn1f24SOomogUattDg1gTz01/KZ4UMHrZnCao3eXY99c3oxT7VKenGDLhB2+TU4dI='} 2024-05-14T17:22:13.058 INFO:teuthology.orchestra.console.smithi069:Power off 2024-05-14T17:22:13.058 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T17:22:13.099 INFO:teuthology.orchestra.console.smithi092:Power off 2024-05-14T17:22:13.099 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T17:22:13.129 INFO:teuthology.orchestra.console.smithi203:Power off 2024-05-14T17:22:13.130 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T17:22:13.137 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2024-05-14T17:22:13.138 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:22:13.148 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2024-05-14T17:22:13.149 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:22:13.158 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off 2024-05-14T17:22:13.181 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:22:13.193 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-05-14T17:22:13.193 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-05-14T17:22:13.210 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2024-05-14T17:22:17.194 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:22:17.202 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:22:17.211 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:22:17.304 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-05-14T17:22:17.311 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-05-14T17:22:17.321 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2024-05-14T17:22:21.306 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:22:21.314 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:22:21.322 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:22:21.416 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-05-14T17:22:21.424 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-05-14T17:22:21.431 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off 2024-05-14T17:22:21.517 INFO:teuthology.orchestra.console.smithi092:Power off completed 2024-05-14T17:22:21.532 INFO:teuthology.orchestra.console.smithi203:Power off completed 2024-05-14T17:22:25.428 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:22:25.538 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2024-05-14T17:22:25.639 INFO:teuthology.orchestra.console.smithi069:Power off completed 2024-05-14T17:22:25.744 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 3124.7351036071777 failure_reason: 'Command failed on smithi203 with status 22: ''sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 356e185e-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump''' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-05-14T17:22:25.744 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T17:22:25.834 INFO:teuthology.run:FAIL